Scale-invariance is a fundamental phenomenon in vision. It is widely accepted (often on naive or intuitive grounds) that visual information in natural images spans an entire range of scales, and the characteristics of the structures in this continuum are largely similar to one another.

A more careful analysis suggests that there are at least two major sources of multi-scale structure:

• Arbitrary viewing distance: an object may appear at any distance to the observer. As the information about absolute scale is lost in the imaging process, an object at half the distance looks as if having twice the dimensions. Figure 1 shows an example.
• Hierarchy of parts in a single object: objects in the natural world are themselves multi-scale as they consist of parts at various scales. For example, Figure 2 shows the boundary of a human body which has a hierarchy of parts: head, torso, arms, legs, fingers, hair, each with its own distinct scale. Many natural forms exhibit this fractal-like structure.

In recent years scale-invariant phenomenon have been extensively studied through the use of natural image statistics. Power laws of various kinds have been discovered, most noticeably in the power spectra and wavelet coefficients of natural images.

For this Berkeley Segmentation Dataset, it has been reported that the empirical distribution of region size roughly follows a power law with exponent ≈ 1. We find that for a boundary contour C, where C is the boundary between two adjacent regions, its length |C| follows a similar power law (see Figure 3).

Are the human-marked contours smooth? Suppose we travel along a contour C and estimate the tangent directions at each location t. Figure 4 shows the empirical distribution of the tangent change ΔT = T(t+1)-T(t). This distribution has the two characteristics commonly found in natural images: (1) it is sharply peaked around 0; and (2) it has a heavy tail.

A more interesting question is to study the statistical properties of contour C itself. Traditionally people make the Markov assumption about C: suppose the curvature κ is white noise, the tangent T is then a random walk; i.e., the distribution of T(t+1) only depends on T(t). This Markov model then leads to efficient computation using dynamic programming.

Figure 5: breaking up contours at high-curvature locations.

To check the validity of this model, we take a contour C and break it into segments S at high-curvature locations. Now, if the Markov assumption were true, then when we travel along a contour, at each step there is a constant probability that a high curvature event fires. Then the length of contour segment |S| is the waiting time between two high curvature events and it should have an exponential distribution. Figure 6 shows the empirical distribution, which is again a power law. Therefore we conclude that empirically the Markov assumption does not hold. Not so surprising, as the Markov model is not scale-invariant.

Is the scale-invariance soley due to arbitrariness in viewing distance? If so, when |C| is large (i.e. the object is large), we should expect that the contours are smooth and |S| should be large too. Figure 7 shows the distributions of |S|, conditioned on various ranges of |C|. There is no significant difference between these distributions, indicating that natural contours are fractal-like.

Similar power laws can also be found in the distribution of curvature.

Most existing vision algorithms ignore the scale-invariance of natural images and operate at a fixed scale (or a few selected ones). It is our belief that scale-invariance will become more and more important as we move to tackle more realistic vision problems.

One way to incorporate scale-invariance is to build a scale-invariant representation from bottom-up, and then operate on such a representation (see the CDT graphs).

Alternatively, instead of a Markov model we could use a multi-scale Markov model. Imagine a scale pyramid where the contour C becomes coarser and coarser as we subsample it. We can then develop a model P(T(t+1)|T(t),T⁽¹⁾(t+1),T⁽²⁾(t+1),...), where T^(k) is the tangent of C at scale k. We can still do dynamic programming with this model (coarse-to-fine). Emprirical studies show that, compared to the Markov model P(T(t+1)|T(t)), the additional terms provide a significant amount of extra information. Figure 8 shows some contours synthesized from the multi-scale Markov model, along with some from the first-order model.