\documentstyle[12pt,fullpage,psfig]{article}
%\input{psfig}

\newcommand{\lecturetitle}[4]{
  \noindent
  CS 280, Computer Vision (Prof. Jitendra Malik) \hfill #3\\
  Lecture \##1 (#2) \hfill {\bf DRAFT} Notes by #4\\[-.05in]
  \mbox{}\hrulefill \mbox{}\\}

\begin{document}

% MODIFY THE FOLLOWING TO INCLUDE THE LECTURE #, TOPIC, DATE, and YOUR NAME
\lecturetitle{12}{Epipolar geometry and shape from texture.}{March 3, 1999}{Dimitri Chernyak}

% INCLUDE AN OUTLINE OF THE CLASS TOPICS
\begin{itemize} 
  \item Epipolar Geometry.
\item Shape from Texture.
\end{itemize}

% THE NOTES WILL GO HERE

\section{Epipolar Geometry}
\subsection{Geometric aside. }

If vectors $\vec{a}$, $\vec{b}$ ,and $\vec{c}$ are coplanar, then:
\newline\newline

$\vec{a}\cdot(\vec{b}\otimes \vec{c}) = 0$
\newline\newline

or
\newline\newline

$\vec{a^{T}}\cdot(\hat{b}\cdot\vec{c}) = 0$
\newline\newline

where $\hat{b}$ is the skew-symmetric matrix associated with $\vec{b}$ .
\newline

Note that the magnitudes of the vectors don't make any difference. Only the directions matter. 

\subsection{Epipolar geometry } 

Consider the two cameras and the point $X$ in the outside world as shown on Figure \ref{fig1}. Points $C_{1}$ and $C_{2}$ are the focal points of the left and right cameras respectively, and $x$ and $x'$ are the projections of point $X$ onto the corresponding image planes.
\begin{figure*}[t]
\centerline{\hbox{
\psfig{file=fig1.eps,height=2.0in}}}
\vspace*{.25in}
\caption{Epipolar geometry}
\label{fig1}
\end{figure*}

Let $\vec{t}$ denote the translational component between the cameras and $R$, - the rotational one.


Plane formed by $C_{1}XC_{2}$ is called an {\em epipolar plane\/}. The lines of intersection of epipolar plane with the camera planes are called {\em epipolar lines\/}.


Projection of point $C_{1}$ onto the image plane of the right camera is point $E_{r}$, which is called an {\em epipole\/}. All epipolar lines pass through this epipole. Similarly, for the left camera there is a corresponding epipole $E_{l}$, which is projection of point $C_{2}$.


In a special case when the cameras are parallel ($R=I$ identity matrix), the epipolar lines are parallel, since the epipole for each camera is at infinity.


Lets choose the coordinate system of the left camera to be the coordinate system for the epipolar constrain with origin at point $C_{1}$. Vectors $\vec{x}$, $\vec{t}$ ,and $\vec{Rx'}$ are coplanar, so the following must hold:
\newline
\newline
$\vec{x}\cdot(\vec{t}\otimes \vec{Rx'}) = \vec{x^{T}}\hat{t}R\vec{x'} = \vec{x^{T}}E\vec{x'} = 0$,
\newline
\newline
where $E = \hat{t}R$ and $\hat{t} =  \left[\begin{array}{lcr} 0 & t3 & -t2 \\ -t3 & 0 & t1 \\ t2 & -t1 & 0 \end{array} \right] $
\newline
\newline
This is the epipolar constrain equation.


\subsection{Epipolar constrain }

Epipolar constrain is very useful for solving the correspondence problem for stereopsis. Let $A$ be a point in the image plane of the left camera. We can find the epipolar line $l$ passing through $A$ based on the relative positions of the two cameras. Using the epipolar constrain we can find the corresponding epipolar line $l'$ in the image plane of the right camera. Therefore, we can restrict the search for the corresponding point $A'$ to the line $l'$, i.e. we have to do one dimensional search instead of the full two dimensional one. This property is preserved under projective transformation because of the preserved co-planarity.

\section{Shape from texture}

\subsection{Basic task }

{\em Texture \/} is a spatially repeated pattern of elements. These can be regular (such as floor tiles) or stochastic (such as animal skin or field of flowers). Given a surface with repetitive or quasi-repetitive elements, texture elements would look different on a projective plane. Shape from texture uses these distortions to recover the actual surface geometry in the world.

\subsection{Shape from variations. }

Using apparent variations in texture, one can recover the surface using $F^{*}$ backprojection mapping. The problem is quite similar to recovering shape from motion or stereo by considering relative changes in appearance of elements. Table \ref{c_table} summarizes the similarities:
\newline

\begin{table}
\begin{tabular}{|c|c|c|}
\hline
\multicolumn{3}{|c|}{Shape From}
\\ \hline 
Motion & Stereo & Texture \\ \hline \hline 
Measure optical flow & Measure disparities & Measure texture\\ 
(apparent change & & distortion \\ 
 in position) & &   \\ \hline
Compute structure & Compute structure & Compute structure \\
from motion & from disparities & from texture  \\ 
& &  distortions \\ \hline
\end{tabular}
\vspace*{.25in}
\caption{3D shape recovery.}
\label{c_table}
\end{table}

\subsection{What is texure distortion? }
If the change between two nearby texture patches on the image plane is small, it is likely that these patches are nearby on the actual surface. Given a texture patch on the image plane we can fit texture distortion for each direction with some affine transformation.
What is being preserved? For regular textures it's the texture element; for stochastic texture it's the statistical properties.

\subsection{3D shape computation.}

In case of motion, translation, rotation and depth (Z-value) variations give rise to optical flow. In case of texture, slant, tilt, distance, surface normal, curvatures at point $\vec{p}$  give rise to affine texture distortions in different direction from $\vec{p}$.


% EXAMPLE OF A PostScript FIGURE
%\begin{figure}
%\centerline{\psfig{file=lec4.eps,height=3in}}
%\caption{Illustration}
%\label{surface}
%\end{figure}


%......

\end{document}

