CS 294-2, Grouping and Recognition (Prof. Jitendra Malik) Oct 4, 1999 Lecture #11 (What is texture?) DRAFT Notes by Cory Sharp

• Intro to Texture

• Modeling Texture

• Segmenting Texture

Difference of Windows

Quantification of Texture

• Texture Characterization

Julesz

Beck

Textons

Filters

Variation on Filters

1 Intro to Texture

Dealing with texture can be broken into three distinct problems:

• Segmentation

• Recognition

• Synthesis

Before confronting these problems, however, the notion of texture itself must be defined. The musician Louis Armstrong, when once asked for a definition of jazz, responded, “If you have to ask, you’ll never know.” And, so it is with texture.

Computers only speak in mathematics and algorithms and have no inherent notion of texture. And yet, our goal is for a computer to manipulate texture. Consequently, we cannot casually ponder the question, “What is texture?”, but we must answer it in explicit detail.

We’ll do so by bootstrapping the definition – taking baby steps to evolve the definition into something useful. This process is not so strange. In school, mathematical notions are presented logically and completely. But, in their infancy, properties of mathematical interest are iteratively understood, extracted, and developed. Only after an entire mathematical picture is complete is it presented in a compact, polished form with the necessary definitions first and logical conclusions after.

In this spirit of development, we begin with our first definition of texture:

Texture some parts repeated

‘Some parts’ refers to components in either the scene or image. This level of ambiguity is acceptable since repetition in the scene causes repetition in the image. With this definition, the three distinct texture problems can begin to be ellucidated:

Segmentation Texture segmentation is meaningful because different surfaces in the scene tend to have different textures in the image. For instance, see Figure 1.

Recognition Texture recognition is meaningful because different materials (e.g. silk, leather, etc.) in the scene have unique textures.

Synthesis Texture synthesis means to make something rendered look like silk, leather, etc.

α_{1} α_{2}  α_{3}

Figure 1: Image with distinct textures

2 Modeling Texture

We want a formal model to characterize texture. Ideally, we would like to be able to associate unique parameters with unique textures, which would make texture manipulation managable within a computer algorithm. The three texture problems can then be redefined with respect to texture parameters:

Segmentation is the process of estimating texture parameters in the image then segmenting the image with respect to the calculated parameters. For instance, in Figure 1, α_{1}, α_{2}, and α_{3} correspond to unique texture parameters for unique textures in the image.

Recognition of texture can be performed by a simple table look-up procedure, where calculated texture parameters in the image are compared to known texture parameters.

Synthesis of texture can be done by exploiting the knowledge of the texture model in conjunction with known texture parameters.

For instance, models of shape exist. A cylinder can be completely modeled using the parameters radius and height. With these parameters, we can create, place, and manipulate cylinders in a scene.

No such nice, complete model currently exists for texture. The best we can do is present individual solutions to each of the main three problems.

3 Segmenting Texture

In Figure 2, what should be segmented? What not? Without preknowledge of the image or kind of image, segmentation can be difficult for a computer, but simple for a human.

Figure 2: “Segment Me”

3.1 Difference of Windows

An overly simple method is to take a difference of windows. For instance, given two windows I_{1} and I_{2}, the difference between the windows is calculated to be the mean squared error (MSE) between them:

\frac{1}{N} ∑_{x,y} ||I_{1}(x, y) − I_{2}(x, y)||^{2}

For the image in Figure 2, the MSE gives high values for similar regions, such as the windows shown in Figure 3. For texture segmentation, we would like these to windows to give similar parameters. In this case, the MSE measure does anything but that. The MSE measure between windows is a bad criterion for segmentation and is totally inappropriate for texture classification.

I_{1}(x, y): I_{2}(x, y):

Figure 3: Windows taken for the same texture region.

3.2 Quantification of Texture

This introduces the subtelty of texture. Why do we assert that particular patches of texture are the “same”? The classification of texture must be appealing to a human, visual observer.

Can the idea of human perception and classification of texture be made precise? In Figure 4, humans can perceive the texture differences more easily for the image on the left than for the image on the right. However, the types of textures between both images are very similar in their geometric content. A computer algorithm should behave similarly by finding stronger segmentation in the left image than in the right image. Because, our goal is to create a mathematical model that follows human perception.

Figure 4: Human perception of triangle textures

Is it actually possible to define such parameters? To do so is a touchy endevour requiring finesse. Broadly, we desire a parameterization of texture that throws away qualities that are insignificant to human perception.

Researchers have conducted experiments that attempt to quantify how humans perceive similarity in texture. Figure 5 shows six such experimental texture pairs – the bracketed number gives the correct ordering to which humans could distinguish two distinct textures in each image. Notably, the response was found to be amazingly consistent across human subjects.

A question was raised in class discussion questioning the relevance of synthetic textures. In response, psychophysicists want to determine the core perception properties of human vision. Syntetic textures allow the testers to accutely vary parameters at will – leading to a more accurate understanding of human perception.

4 Texture Characterization

Two big names in texture characterization were discussed:

Julesz (1962-78) Theorized only first and second order statistics of textures discriminate textures in human vision.

Beck (1972) Studied first order statistics of stimulus features such as length, orientation, etc.

Figure 5: Six experimental textures

4.1 Julesz

First, Julesz experimented with dot densities. If dot densities are different, he correctly determined that the texture area is easily discrenible by humans. He further maintained that if the dot densities are the same, then the textures should be indiscrenible to humans. Dot densities are first order statistis. Julesz made similar assertions about second order statistics.

First Order Statistics involve only single points: density, consider the distribution of λ = the number of points per unit area.

Second Order Statistics involve pairs of points: arrows, pairs of points (base, tip), consider the distribution of δx = tip − base.

Julesz mantained that if the first and second order statics are the same, then texture differences are imperceptible to human vision. However, Julesz was wrong. It is possible to show counter examples in both cases.

With respect to countering the theory on first order statistics, consider Figure 6. In this figure, the left texture has uniform distance between dots, and the right texture has random distance between dots. In both cases, however, the density of the dots is the same. Even so, humans can still discern a clear difference in the textures.

Figure 6: Same dot density

4.2 Beck

Beck’s texture sample in Figure 7 counters Julesz theory with respect to second order statistics. That is, the first and second order statistics are the same, yet some of the texture is clearly discernable from the rest.

Figure 7: L-T-T as a group

Figure 8: L-T-T individually

In Figure 8, most subjects would say elements 1 and 2 are the most similar. In stark contrast, in Figure 7, most subjects would say the second and third regions are the most distinct.

Beck maintains that with respect to texture human vision does not perform sophisticated matches of shapes. Instead, human vision is only concerned with rough statistics: length, orientation, etc. To Beck, those are the only necessary texture elements.

4.3 Textons

In papers from 1981, 1983, and 1986, Julesz refined Beck’s notion into texton theory. It is founded in the idea that preattenctive vision detects differences in densities of textons.

Textons are elongated blobs with length, width, and orientation. Textons are to texture as phonemes are to speech.

The theory is that textons are the building blocks of human vision. Measurements can be made with respect to crossings and terminations of textons.

Crossings the intersection of two texton elements.

Terminators the ends of line segments.

Terminators are well validated in human vision. As proof, consider Figure 9 and Figure 10.

Figure 9: Human perception of terminators.

Figure 10: More human perception of terminators.

We would like a measure of the degree of discernability. For instance, refering back to Figure 4, we would expect the degree of discernability of the first image to be higher than that of the second image. This leads us in to the next section.

4.4 Filters

In the 1990’s, textons were displaced by “filters”.

Filters oriented gaussians of varying size.

Big papers on filters came out in 1989 by Fogel and Sagi and in 1990 by Malik and Perone.

The basic idea is this: given an image I and filters f_{1}, f_{2}, ..., f_{n}, generate filter responses R_{1} = I ∗ f_{1}, R_{2} = I ∗ f_{2}, ..., R_{n} = I ∗ f_{n}. Then, assert that textures differ if filter responses differ on some sort of average. Filtering can explain psychophysics tests and give similar results. The experimental results from filtering tests on the images in Figure 5 are given in Figure 11.

Figure 11: Experimental results from filtering the images in Figure 5.

Because the filters always respond to the degree of content, the degree of similarity between textures can be easily calculated. This is a significant advantage over binary comparisons of textures.

4.5 Variation on Filters

That linear filters always respond is a double-edged sword. The response of a filter is never uniform in a region of texture. After filtering, you have the responses R_{1}(x, y), R_{2}(x, y), ..., R_{3}(x, y). In the early days, the measure would be found by squaring, adding, and averaging each filter across a region.

From 1990 to 1995, this was the standard operating procedure for filters: square and average each filter on a region, or some other nonlinearity then average. Unfortunately, this gives undesirable variation in texture comparisons.

In 1995, Heegis and Berger proposed using histograms of filter outputs. That is, to quantize the filter responses to bins, then compute the similarity between textures. This gives much better results.