\documentclass[12pt]{article}
\usepackage{fullpage}
\usepackage{psfig}
\usepackage{amsmath}
\usepackage{amssymb}
\usepackage{amsfonts}
\usepackage{latexsym}
\usepackage{floatflt}

\def\gpic#1{#1\vskip.25in\vskip.25in}

\newcommand{\lecturetitle}[4]{
  \noindent
  CS 294-2, Grouping and Recognition (Prof. Jitendra Malik) \hfill #3\\
  Lecture \##1 (#2) \hfill {\bf DRAFT} Notes by #4\\[-.05in]
  \mbox{}\hrulefill \mbox{}\\}

\begin{document}

% LECTURE #, TOPIC, DATE, and NAME
\lecturetitle{11}{What is texture?}{Oct 4, 1999}{Cory Sharp}

% OUTLINE OF THE CLASS TOPICS

\begin{itemize} 
  \item Intro to Texture
        \item Modeling Texture
        \item Segmenting Texture
                \subitem Difference of Windows
                \subitem Quantification of Texture
        \item Texture Characterization
                \subitem Julesz
                \subitem Beck
                \subitem Textons
                \subitem Filters
                \subitem Variation on Filters
\end{itemize}

% LECTURE NOTES 


\section{Intro to Texture}

Dealing with texture can be broken into three distinct problems:

\begin{itemize}
        \item Segmentation
        \item Recognition
        \item Synthesis
\end{itemize}

Before confronting these problems, however, the notion of texture itself
must be defined.  The musician Louis Armstrong, when once asked for a
definition of jazz, responded, ``If you have to ask, you'll never
know.''  And, so it is with texture.

Computers only speak in mathematics and algorithms and have no inherent
notion of texture.  And yet, our goal is for a computer to manipulate
texture.  Consequently, we cannot casually ponder the question, ``What
is texture?'', but we must answer it in explicit detail.

We'll do so by bootstrapping the definition -- taking baby steps to
evolve the definition into something useful.  This process is not so
strange.  In school, mathematical notions are presented logically and
completely.  But, in their infancy, properties of mathematical interest
are iteratively understood, extracted, and developed.  Only after an
entire mathematical picture is complete is it presented in a compact,
polished form with the necessary definitions first and logical
conclusions after.

In this spirit of development, we begin with our first definition of
texture:

\begin{description}
        \item[Texture] some parts repeated
\end{description}

`Some parts' refers to components in either the scene or image.  This
level of ambiguity is acceptable since repetition in the scene causes
repetition in the image.  With this definition, the three distinct
texture problems can begin to be ellucidated:

\begin{description}

        \item[Segmentation]

        Texture segmentation is meaningful because different surfaces in the
        scene tend to have different textures in the image.  For instance, see
        Figure~\ref{fTexture}.

        \item[Recognition]

        Texture recognition is meaningful because different materials (e.g.
                        silk, leather, etc.) in the scene have unique textures.

        \item[Synthesis]

        Texture synthesis means to make something rendered look like silk,
        leather, etc.

\end{description}

\begin{figure}[htb]
  \include{fTexture}
        \centerline{\box\graph}
        \caption{Image with distinct textures}
        \label{fTexture}
\end{figure}


\section{Modeling Texture}

We want a formal model to characterize texture.  Ideally, we would like
to be able to associate unique parameters with unique textures, which
would make texture manipulation managable within a computer algorithm.
The three texture problems can then be redefined with respect to texture
parameters:

\begin{description}

        \item[Segmentation]

        is the process of estimating texture parameters in the image then
        segmenting the image with respect to the calculated parameters.  For
        instance, in Figure~\ref{fTexture}, $\alpha_1$, $\alpha_2$, and
        $\alpha_3$ correspond to unique texture parameters for unique textures
        in the image.

        \item[Recognition]

        of texture can be performed by a simple table look-up procedure, where
        calculated texture parameters in the image are compared to known
        texture parameters.

        \item[Synthesis]

        of texture can be done by exploiting the knowledge of the texture
        model in conjunction with known texture parameters.

\end{description}

For instance, models of shape exist.  A cylinder can be completely
modeled using the parameters radius and height.  With these parameters,
we can create, place, and manipulate cylinders in a scene.

No such nice, complete model currently exists for texture.  The best we
can do is present individual solutions to each of the main three
problems.


\section{Segmenting Texture}

In Figure~\ref{fSegment}, what should be segmented?  What not?  Without
preknowledge of the image or kind of image, segmentation can be difficult
for a computer, but simple for a human.

\begin{figure}[htb]
        \centerline{\psfig{file=fSegment.eps}}
        \caption{``Segment Me''}
        \label{fSegment}
\end{figure}

\subsection{Difference of Windows}

An overly simple method is to take a difference of windows.  For
instance, given two windows $I_1$ and $I_2$, the difference between the
windows is calculated to be the mean squared error (MSE) between them:

\begin{equation}
        {1 \over N} \sum_{x,y} || I_1(x,y) - I_2(x,y) ||^2 \nonumber
\end{equation}

For the image in Figure~\ref{fSegment}, the MSE gives high values for
similar regions, such as the windows shown in
Figure~\ref{fSegmentWindows}.  For texture segmentation, we would like
these two windows to give similar parameters.  In this case, the MSE
measure does anything but that.  The MSE measure between windows is a
bad criterion for segmentation and is totally inappropriate for texture
classification.

\begin{figure}[htb]
        \centerline{
                $I_1(x,y)$: \psfig{file=fSegmentWA.eps}
                \hspace{.5in}
                $I_2(x,y)$: \psfig{file=fSegmentWB.eps} }
        \caption{Windows taken for the same texture region.}
        \label{fSegmentWindows}
\end{figure}

\subsection{Quantification of Texture}

This introduces the subtelty of texture.  Why do we assert that
particular patches of texture are the ``same''?  The classification of
texture must be appealing to a human, visual observer.

Can the idea of human perception and classification of texture be made
precise?  In Figure~\ref{fTriangleTexture}, humans can perceive the
texture differences more easily for the image on the left than for the
image on the right.  However, the types of textures between both images
are very similar in their geometric content.  A computer algorithm
should behave similarly by finding stronger segmentation in the left
image than in the right image.  Because, our goal is to create a
mathematical model that follows human perception.

\begin{figure}[htb]
        \centerline{ 
                \psfig{file=fArrowWide.eps} \hspace{1in} \psfig{file=fArrowThin.eps} }
        \caption{Human perception of triangle textures}
        \label{fTriangleTexture}
\end{figure}

Is it actually possible to define such parameters?  To do so is a touchy
endevour requiring finesse.  Broadly, we desire a parameterization of
texture that throws away qualities that are insignificant to human
perception.

Researchers have conducted experiments that attempt to quantify how
humans perceive similarity in texture.  Figure~\ref{fSixTextures} shows
six such experimental texture pairs -- the bracketed number gives the
correct ordering to which humans could distinguish two distinct textures
in each image.  Notably, the response was found to be amazingly
consistent across human subjects.

A question was raised in class discussion questioning the relevance of
synthetic textures.  In response, psychophysicists want to determine the
core perception properties of human vision.  Syntetic textures allow the
testers to accutely vary parameters at will -- leading to a more
accurate understanding of human perception.

\begin{figure}[htb]
        \centerline{\psfig{file=fSixTextures.eps}}
        \caption{Six experimental textures}
        \label{fSixTextures}
\end{figure}

\section{Texture Characterization}

Two big names in texture characterization were discussed:

\begin{description}

        \item[Julesz] (1962-78) Theorized only first and second order
        statistics of textures discriminate textures in human vision.

        \item[Beck] (1972) Studied first order statistics of stimulus features
        such as length, orientation, etc.

\end{description}

\subsection{Julesz}

First, Julesz experimented with dot densities.  If dot densities are
different, he correctly determined that the texture area is easily
discrenible by humans.  He further maintained that if the dot densities
are the same, then the textures should be indiscrenible to humans.  Dot
densities are first order statistis.  Julesz made similar assertions
about second order statistics.

\begin{description}

        \item[First Order Statistics] involve only single points: density,
        consider the distribution of $\lambda = $ the number of points per
        unit area.

        \item[Second Order Statistics] involve pairs of points: arrows, pairs
        of points ($base$, $tip$), consider the distribution of $\delta x =
        tip - base$.

\end{description}

Julesz mantained that if the first and second order statics are the same,
then texture differences are imperceptible to human vision.  However,
Julesz was {\bf wrong}.  It is possible to show counter examples in both
cases.

With respect to countering the theory on first order statistics,
consider Figure~\ref{fDotDensity}.  In this figure, the left texture has
uniform distance between dots, and the right texture has random distance
between dots.  In both cases, however, the density of the dots is the
same.  Even so, humans can still discern a clear difference in the
textures.

\begin{figure}[htb]
        \centerline{\psfig{file=fDotDensity.eps}}
        \caption{Same dot density}
        \label{fDotDensity}
\end{figure}

\subsection{Beck}

Beck's texture sample in Figure~\ref{fLTTImage} counters Julesz theory
with respect to second order statistics.  That is, the first and second
order statistics are the same, yet some of the texture is clearly
discernable from the rest.

\begin{figure}[htb]
        \centerline{\psfig{file=fLTTImage.eps}}
        \caption{L-T-T as a group}
        \label{fLTTImage}
\end{figure}

\begin{figure}[htb]
        \centerline{\psfig{file=fLTTElements.eps}}
        \caption{L-T-T individually}
        \label{fLTTElements}
\end{figure}

In Figure~\ref{fLTTElements}, most subjects would say elements 1 and 2
are the most similar.  In stark contrast, in Figure~\ref{fLTTImage}, most subjects
would say the second and third regions are the most distinct.

Beck maintains that with respect to texture human vision does not
perform sophisticated matches of shapes.  Instead, human vision is only
concerned with rough statistics: length, orientation, etc.  To Beck,
those are the only necessary texture elements.

\subsection{Textons}

In papers from 1981, 1983, and 1986, Julesz refined Beck's notion into
texton theory.  It is founded in the idea that preattenctive vision
detects differences in {\em densities of textons}.

\begin{description} 

        \item[Textons] are elongated blobs with length, width, and
        orientation.  Textons are to texture as phonemes are to speech.

\end{description}

The theory is that textons are the building blocks of human vision.
Measurements can be made with respect to crossings and terminations of
textons.

\begin{description} 

        \item[Crossings] the intersection of two texton elements.

        \item[Terminators] the ends of line segments.

\end{description}

Terminators are well validated in human vision.  As proof, consider
Figure~\ref{fTerminator} and Figure~\ref{fTerminatorE}.

\begin{figure}[htb]
        \centerline{\psfig{file=fTerminator.eps}}
        \caption{Human perception of terminators.}
        \label{fTerminator}
\end{figure}

\begin{figure}[htb]
        \centerline{\psfig{file=fTerminatorE.eps}}
        \caption{More human perception of terminators.}
        \label{fTerminatorE}
\end{figure}

We would like a measure of the {\em degree} of discernability.  For
instance, refering back to Figure~\ref{fTriangleTexture}, we would
expect the degree of discernability of the first image to be higher than
that of the second image.  This leads us in to the next section.


\subsection{Filters}

In the 1990's, textons were displaced by ``filters''.

\begin{description}
        \item[Filters] oriented gaussians of varying size.
\end{description}

Big papers on filters came out in 1989 by Fogel and Sagi and in 1990 by
Malik and Perone.

The basic idea is this: given an image $I$ and filters $f_1$, $f_2$,
..., $f_n$, generate filter responses $R_1 = I * f_1$, $R_2 = I * f_2$,
..., $R_n = I * f_n$.  Then, assert that textures differ if filter
responses differ on some sort of average.  Filtering can explain
psychophysics tests and give similar results.  The experimental results
from filtering tests on the images in Figure~\ref{fSixTextures} are
given in Figure~\ref{fDiscriminabilityTable}.

\begin{figure}[htb]
        \centerline{\psfig{file=fDiscriminabilityTable.eps}}
        \caption{Experimental results from filtering the images in Figure~\ref{fSixTextures}.}
        \label{fDiscriminabilityTable}
\end{figure}

Because the filters always respond to the degree of content, the degree
of similarity between textures can be easily calculated.  This is a
significant advantage over binary comparisons of textures.


\subsection{Variation on Filters}

That linear filters always respond is a double-edged sword.  The
response of a filter is never uniform in a region of texture.  After
filtering, you have the responses $R_1(x,y)$, $R_2(x,y)$, ...,
$R_n(x,y)$.  In the early days, the measure would be found by squaring,
adding, and averaging each filter across a region.

From 1990 to 1995, this was the standard operating procedure for
filters: square and average each filter on a region, or some other
nonlinearity then average.  Unfortunately, this gives undesirable
variation in texture comparisons.

In 1995, Heegis and Berger proposed using histograms of filter outputs.
That is, to quantize the filter responses to bins, then compute the
similarity between textures.  This gives much better results.

\end{document}

