An Introduction to Variational Methods for Graphical Models

Michael I. Jordan, Zoubin Ghahramani, Tommi S. Jaakkola and Lawrence K. Saul

EECS Department
University of California, Berkeley
Technical Report No. UCB/CSD-98-980
January 1998

http://www2.eecs.berkeley.edu/Pubs/TechRpts/1998/CSD-98-980.pdf

This paper presents a tutorial introduction to the use of variational methods for inference and learning in graphical models. We present a number of examples of graphical models, including the QMR-DT database, the sigmoid belief network, the Boltzmann machine, and several variants of hidden Markov models, in which it is infeasible to run exact inference algorithms. We then introduce variational methods, showing how upper and lower bounds can be found for local probabilities, and discussing methods for extending these bounds to bounds on global probabilities of interest. Finally we return to the examples and demonstrate how variational algorithms can be formulated in each case.


BibTeX citation:

@techreport{Jordan:CSD-98-980,
    Author = {Jordan, Michael I. and Ghahramani, Zoubin and Jaakkola, Tommi S. and Saul, Lawrence K.},
    Title = {An Introduction to Variational Methods for Graphical Models},
    Institution = {EECS Department, University of California, Berkeley},
    Year = {1998},
    Month = {Jan},
    URL = {http://www2.eecs.berkeley.edu/Pubs/TechRpts/1998/5425.html},
    Number = {UCB/CSD-98-980},
    Abstract = {This paper presents a tutorial introduction to the use of variational methods for inference and learning in graphical models. We present a number of examples of graphical models, including the QMR-DT database, the sigmoid belief network, the Boltzmann machine, and several variants of hidden Markov models, in which it is infeasible to run exact inference algorithms. We then introduce variational methods, showing how upper and lower bounds can be found for local probabilities, and discussing methods for extending these bounds to bounds on global probabilities of interest. Finally we return to the examples and demonstrate how variational algorithms can be formulated in each case.}
}

EndNote citation:

%0 Report
%A Jordan, Michael I.
%A Ghahramani, Zoubin
%A Jaakkola, Tommi S.
%A Saul, Lawrence K.
%T An Introduction to Variational Methods for Graphical Models
%I EECS Department, University of California, Berkeley
%D 1998
%@ UCB/CSD-98-980
%U http://www2.eecs.berkeley.edu/Pubs/TechRpts/1998/5425.html
%F Jordan:CSD-98-980