cs294-2, Grouping and Recognition (Prof. Jitendra Malik) Jan 25, 1999 Lecture #2 (Introduction) DRAFT Notes by Alex Berg

• Grouping (continuation from last lecture)

• Figure / Ground or Depth Ordering

• History

In the last lecture we had three related tasks, grouping, figure ground discrimination, and recognition.

1 Grouping (continued)

The idea is to use various cues to partition an image into groups. One motivation here is to organize the overwhelming data of an image into coherent simple groups. We attempt to motivate these cues in terms of image statistics.

Similarity Various features determine the similarity of two areas: brightness, texture, motion, and stereo disparity or depth. These were covered in the previous lecture.

Proximity Grouping by proximity is based on the idea that things which project to nearby points on the image are usually close to each other in the world. One could probably consider this another form of similarity.

Contour Continuation Here we concentrate on the contour or boundary of a group. In particular the following images motivate the concept of boundaries which are not apparent from changes in image intensity. In fact the apparent regions formed by these contours cold not be separated by any of the similarity clues above. These apparent boundaries are called subjective contours, illusory contours, imagined boundaries, or gradient less edges. Figures 1, 2, and 3 show examples of subjective contours and are taken from a paper by Kanisza.

We want to make up a story in terms of image statistics to explain this phenomenon. Basically one can imagine the images are the result of a white object occluding the dark disks and lines. Examining the features, we might argue that it is unlikely the apparent boundaries would line up so smoothly by chance, and therefore we “see” an occluding object that may or may not in actuality be present. Later on we will look at other cues for grouping and for figure/ground determination which might also help explain this phenomenon.

Closure The physical world consists of objects which are usually seen in images as closed contours. It therefore makes sense for there to be a preference for perceiving contours which close a region.

Figure 1: Illusory Contours Figure 2: Illusory Contours

Figure 3: Terminators and Illusory Contours

Figure 4: Symmetry

Symmetry and Parallelism Many shapes in the physical world are somewhat axisymmetric, most notably surfaces of revolution. As a result their outlines have a bilateral symmetry. Furthermore it is unlikely that there is such symmetry by chance. So we associate symmetric contours as in figure 4 and tend to associate the area between them as a region.

Familiar Configuration Faces are a good example. Dark regions for the eyes and mouth, and a shadow for the nose or nostril. We see faces everywhere, even the moon.

Before venturing onto figure/ground discrimination we take another look at the relationship between contours and regions. Many segmentation or grouping algorithms use one or the other, but in the end the two are more or less equivalent. Finding the contours that bound regions, or finding the regions inside contours, are really the same problem.

The reason we are interested in regions and contours in images is that interpretations about this proximal information (at the image plane) provides information about distal features (out in the world). In particular regions in the image correspond to surfaces of objects in the real world. This points out one relationship between grouping and figure/ground determination.

2 Figure / Ground

A single image contains a great deal of information about depth ordering. One way to see this is to have yourself blindfolded, dragged to a foreign place, and when you are unblindfolded carefully open one eye. You will probably be able to say a great deal... about the depth ordering of the objects you see, even without opening the other eye. We will focus on information which helps to determine depth ordering.

T-Junctions Actually other junctions can give you information about figure/ground relationships, but, as shown in figure 5, t-junctions often indicate which object is in front. At the same time the snake shows how this may not be the case. There the t-junctions result from texture on the figure, not occlusion.

Figure 5: Various junctions

Increasing Texture Gradient For textured objects, at an apparent boundary which does not result from occlusion the texture gets denser as it is foreshortened. This can give information about which object is not occluded and therefore closer. These same boundaries can result in t-junctions as shown before, and if the background color of the object matches that of the surface behind it, we get terminators.

Convexity What makes a closed contour look like the outline of a small object instead of a small hole in a very large object? Digression about some ancient idea that the night sky was in fact a very large dark sheet with small holes through to a bright place, causing people to see stars. In any event people seem to prefer explanations with fewer holes. We can operationalize this preference by locally assuming that the convex side of a curve is the foreground. Going back to the illusory contours example we see that this effect might be helping the illusion somewhat.

Symmetry By the same argument we used for grouping symmetry can be helpful. I included the following two examples in figure 6 for fun. In one the symmetry and convexity clues agree, and in the other they do not.

Occlusion Occlusion information from stereo can indicate which object is in front. Sequence of images of an object in motion can also give depth ordering information.

3 History

“I stand at the window and see a house, trees, sky. Theoretically I might say there were 327 brightnesses and nuances of color. Do I have ‘327’? No. I have sky, house, and trees.” Max Wertheimer

Figure 6: Convexity vs. Symmetry

Max Wertheimer worked on grouping around 1923 in Germany. He launched the gestalt movement whose motto was, “The whole is different than the sum of the parts.” Wolfgang Kohler, Kurt Kofka, and Hanz Wallach (aperture problem 1935) were also members of the gestalt movement. World War II caused them to emigrate from Germany to the United States. Their work was less mainstream there and as a result the gestalt movement ended. Recently there has been renewed interest in their work.

Egon Brunswick worked on relating probabilistic inference to the gestalt principles (probabilistic functionalism) around 1947- mid 1950’s. His work suffered from poor background in statistics and the lack of computers to do the large amount of computation necessary to analyze the statistics of images. He did manage to convince graduate students to analyze a few images to verify that proximity was in fact a good cue for grouping.

Edgar Rubin worked on the figure ground distinction in 1915 including the figure vase illusion.

Next lecture will look at grouping from similarity of brightness. Given image intensity as a function I(x, y) the goal is to divide it into piecewise smooth patches. These patches and the boundaries between them are the result. Three methods will be examined, maximum likelihood, Bayesian inference, and spectral graph theory.