Beta processes, stick-breaking, and power laws
Tamara Broderick and Michael Jordan and Jim Pitman
EECS Department, University of California, Berkeley
Technical Report No. UCB/EECS-2011-125
December 8, 2011
http://www2.eecs.berkeley.edu/Pubs/TechRpts/2011/EECS-2011-125.pdf
The beta-Bernoulli process provides a Bayesian nonparametric prior for models involving collections of binary-valued features. A draw from the beta process yields an infinite collection of probabilities in the unit interval, and a draw from the Bernoulli process turns these into binary-valued features. Recent work has provided stick-breaking representations for the beta process analogous to the well-known stick-breaking representation for the Dirichlet process. We derive one such stick-breaking representation directly from the characterization of the beta process as a completely random measure. This approach motivates a three-parameter generalization of the beta process, and we study the power laws that can be obtained from this generalized beta process. We present a posterior inference algorithm for the beta-Bernoulli process that exploits the stick-breaking representation, and we present experimental results for a discrete factor-analysis model.
Advisors: Michael Jordan
BibTeX citation:
@mastersthesis{Broderick:EECS-2011-125, Author= {Broderick, Tamara and Jordan, Michael and Pitman, Jim}, Title= {Beta processes, stick-breaking, and power laws}, School= {EECS Department, University of California, Berkeley}, Year= {2011}, Month= {Dec}, Url= {http://www2.eecs.berkeley.edu/Pubs/TechRpts/2011/EECS-2011-125.html}, Number= {UCB/EECS-2011-125}, Abstract= {The beta-Bernoulli process provides a Bayesian nonparametric prior for models involving collections of binary-valued features. A draw from the beta process yields an infinite collection of probabilities in the unit interval, and a draw from the Bernoulli process turns these into binary-valued features. Recent work has provided stick-breaking representations for the beta process analogous to the well-known stick-breaking representation for the Dirichlet process. We derive one such stick-breaking representation directly from the characterization of the beta process as a completely random measure. This approach motivates a three-parameter generalization of the beta process, and we study the power laws that can be obtained from this generalized beta process. We present a posterior inference algorithm for the beta-Bernoulli process that exploits the stick-breaking representation, and we present experimental results for a discrete factor-analysis model.}, }
EndNote citation:
%0 Thesis %A Broderick, Tamara %A Jordan, Michael %A Pitman, Jim %T Beta processes, stick-breaking, and power laws %I EECS Department, University of California, Berkeley %D 2011 %8 December 8 %@ UCB/EECS-2011-125 %U http://www2.eecs.berkeley.edu/Pubs/TechRpts/2011/EECS-2011-125.html %F Broderick:EECS-2011-125