Tamara Lee Berg and Alexander Christiansen Berg and John Alexander Edwards and Michael Randolph Maire and Ryan White and Yee-Whye Teh and Erik Learned-Miller and David Forsyth

EECS Department, University of California, Berkeley

Technical Report No. UCB/EECS-2007-12

January 12, 2007

http://www2.eecs.berkeley.edu/Pubs/TechRpts/2007/EECS-2007-12.pdf

We show that a large and realistic face data set can be built from news photographs and their associated captions. Our automatically constructed face data set consists of 30,281 face images, obtained by applying a face finder to approximately half a million captioned news images. The faces are labeled using image information from the photographs and word information extracted from the corresponding caption. This data set is more realistic than usual face recognition data sets, because it contains faces captured ``in the wild'' under a wide range of positions, poses, facial expressions, and illuminations. After faces are extracted from the images, and names with context are extracted from the associated caption, our system uses a clustering procedure to find the correspondence between faces and their associated names in the picture-caption pairs.

The context in which a name appears in a caption provides powerful cues as to whether it is depicted in the associated image. By incorporating simple natural language techniques, we are able to improve our name assignment significantly. We use two models of word context, a naive Bayes model and a maximum entropy model. Once our procedure is complete, we have an accurately labeled set of faces, an appearance model for each individual depicted, and a natural language model that can produce accurate results on captions in isolation.

---

BibTeX citation:

@techreport{Berg:EECS-2007-12,
    Author= {Berg, Tamara Lee and Berg, Alexander Christiansen and Edwards, John Alexander and Maire, Michael Randolph and White, Ryan and Teh, Yee-Whye and Learned-Miller, Erik and Forsyth, David},
    Title= {Names and Faces},
    Year= {2007},
    Month= {Jan},
    Url= {http://www2.eecs.berkeley.edu/Pubs/TechRpts/2007/EECS-2007-12.html},
    Number= {UCB/EECS-2007-12},
    Abstract= {We show that a large and realistic face data set can be built from news photographs and their associated captions. Our automatically constructed face data set consists of 30,281 face images, obtained by applying a face finder to approximately half a million captioned news images. The faces are labeled using image information from the photographs and word information extracted from the corresponding caption. This data set is more realistic than usual face recognition data sets, because it contains faces captured ``in the wild'' under a wide range of positions, poses, facial expressions, and illuminations. After faces are extracted from the images, and names with context are
extracted from the associated caption, our system uses a clustering procedure to find the correspondence between faces and their associated names in the picture-caption pairs.

The context in which a name appears in a caption provides powerful cues as to whether it is depicted in the associated image. By incorporating simple natural language techniques, we are able to improve our name assignment significantly. We use two models of word context, a naive Bayes model and a maximum entropy model. Once our procedure is complete, we have an accurately labeled set of faces, an appearance model for each individual depicted, and a natural language model that can produce accurate results on captions in isolation.},
}

EndNote citation:

%0 Report
%A Berg, Tamara Lee 
%A Berg, Alexander Christiansen 
%A Edwards, John Alexander 
%A Maire, Michael Randolph 
%A White, Ryan 
%A Teh, Yee-Whye 
%A Learned-Miller, Erik 
%A Forsyth, David 
%T Names and Faces
%I EECS Department, University of California, Berkeley
%D 2007
%8 January 12
%@ UCB/EECS-2007-12
%U http://www2.eecs.berkeley.edu/Pubs/TechRpts/2007/EECS-2007-12.html
%F Berg:EECS-2007-12