CS 280, Computer Vision (Prof. Jitendra Malik) March 10, 1999 Lecture #14 (Occluding Contours) DRAFT Notes by Cory Sharp

• Intro to occlusion Properties of occluding contours Consequences of p · n_{s} = 0 Shape from silhouettes

• Identifying occlusion Texture Stereo

1 Intro to occlusion

In the scene in Figure 1, surface S1 is closer than and obscures a portion of surface S2. In the neighborhood of a T-junction as in Figure 2, the edge or limb with continuity is called either an occluding edge or occluding limb, respectively.

Figure 1: Occluding T-junctions

Figure 2: Occluding edges and limbs

Not all T-junctions are cues for occluding edges or limbs. For instance, the T-junction in Figure 3 is more likely the result of a reflectance discontinuity.

Figure 3: Non-occluding T-junction

1.1 Properties of occluding contours

Two significant properties of occluding contours are

1. Tangency

2. Conjucacy

Figure 4: Contour generator and occluding contour

Figure 4 illustrates the geometric relationship between a contour generator and the spherical projection of the corresponding occluding contour. Mathematical relationships between the surface, contour generator, and occluding contour are derived.

Let r(s) be the vector from c to a point on the 3D surface, where s is the arclength along the contour generator. Let r(s) be the distance from c to r(s). And, let p(s) be the projection of r(s) onto the viewsphere. Then, it holds that

r(s) = r(s)p(s).  (1)

Differentiating equation (1) with respect to s, we get

r_{s} = r_{s}p + rp_{s}.  (2)

Notice that r_{s} is the tangent direction on the contour generation, and p_{s} is the tangent direction on the projected contour.

If n is normal to the surface, it is clear that r_{s} is perpendicular to n. Also, due to tangency of occluding contours, p is perpendicular to n, as well. Now, take the dot product of both sides of equation (2) with respect to n:

r_{s} · n = r_{s}p · n + rp_{s} · n
0 = 0 + rp_{s} · n
p_{s} · n = 0  (3)

The conclusion of equation (3) is that n is perpendicular to p_{s}, the direction tangent to the projection of the occluding contour. This result, coupled with the tangency of the contour, necessarily implies

n = \frac{p × p_{s}}{||p × p_{s}||}  (4)

1.2 Consequences of p · n_{s} = 0

Differentiating p · n = 0 with respect to s and applying equation (3), we conclude

p · n_{s} = 0.  (5)

Let θ be the angle between ray p and r_{s} (the tangent to the contour generator). Equation (5) leads to the following consequences:

1. If ray p is along a principle direction, then the contour generator runs along the other principle direction.

2. At a parabolic point, the contour generator direction is parallel to the axis of the cylinder.

3. Ray lies along an asymptotic direction tangent to contour generator and ray are parallel. (fixme, notes are unclear)

Shadow boundaries act just like occluding contours. For instance, in Figure 5, the shadow boundary is parallel to the axis of the cylinder.

1.3 Shapes from silhouettes

Using the occludings contour of an object from a series of pictures from different directions, successsive bounds can be placed on the maximum space occupied by the object. Figure 6 illustrates this idea - a camera moves around an object, and each frame adds a constraint to the maximum volume in which the object lives.

The oct-tree is suggested as the core data structure in a possible algorithm for computing the “visual hull” of an object.

Figure 5: Shadow boundary as an occluding contour

Figure 6: Visual hull from silhouettes

2 Identifying occlusion

Four major cues can be used to identify occurences of occlusion:

1. Texture

2. Stereo

3. Motion

4. Junctions

2.1 Texture

The distortion of the pattern of texture can be a significant cue for determining to correct surface for line labeling. The texture pattern on the surface of an object in the neighborhood of a contour of that object will be an elongated version (in the direction of the contour) of the texture pattern proper. That is, texture near the occluding conture of surface tends to be parallel to the contour. This is demonstrated in Figure 7a.

Conversely, arbitrary texture not on the surface of the object tends to be biased toward being perpendicular to the occluding contour. This is demonstrated in Figure 7b.

Figure 7: Occluding contours from texture cues

2.2 Stereo

Disparity measurements from stereo vision can be used to determing occluding contours. In Figure 8, the object in the left eye is uniformly surrounded by pattern P_{1}. In the right eye, a pattern P_{2} not present in the image in the left eye can be detected around the object. As a result, it can be decided that the right side of the object is an occluding contour in the image of the right eye. This condition is referred to as half occlusion.

Occluding contours from stereo cues will be further embelished next lecture.

Figure 8: Occluding contours from stereo cues