2001 AdvancedMeanFieldMethods

• (Opper & Saad, 2001) ⇒ Manfred Opper, David Saad. (2001). “Advanced Mean Field Methods: theory and practice.” MIT Press. ISBN:0-262-15054-9

Subject Headings: Belief Propagation, Boltzmann Machine, Gibbs Free Energy, Hidden Markov Model, Markov Network, Mean Field Approximation, Mean Field Algorithm, Mean Field Theory.

Cited By

~181 http://scholar.google.com/scholar?cites=17580972986797604693

Quotes

Book overview

• A major problem in modern probabilistic modeling is the huge computational complexity involved in typical calculations with multivariate probability distributions when the number of random variables is large. Because exact computations are infeasible in such cases and Monte Carlo sampling techniques may reach their limits, there is a need for methods that allow for efficient approximate computations. One of the simplest approximations is based on the mean field method, which has a long history in statistical physics. The method is widely used, particularly in the growing field of graphical models.
• Researchers from disciplines such as statistical physics, computer science, and mathematical statistics are studying ways to improve this and related methods and are exploring novel application areas. Leading approaches include the variational approach, which goes beyond factorizable distributions to achieve systematic improvements; the Tap (Thouless-Anderson-Palmer) approach, which incorporates correlations by including effective reaction terms in the mean field theory; and the more general methods of graphical models.

Foreword p viii, by Michael I. Jordan

• The links between statistical physics and the information sciences - including computer science, statistics, and communication theory - have grown strong in recent years, as the need of applications have increasingly led researchers in the information sciences towards the study of large-scale, highly-complex probabilistic systems that are reminiscent of models in statistical physics. One useful link is the class of Markov Chain Monte Carlo (MCMC) methods, sampling-based algorithms whose roots lie in the simulation of gases and condensed matter, but whose appealing generality and simplicity of implementation have sparked new applications throughout the information sciences. Another source of links, currently undergoing rapid development, is the class of mean-field methods that are the topic of this book. Mean-field methods aim to solve many of the same problems as are addressed by MCMC methods, but do so using different conceptual and mathematical tools. Mean-field methods are deterministic methods, making use of tools such as Taylor expansion and convex relaxations to approximate or bound quantities of interest. While the analysis of MCMC methods reposes on the theory of Markov chains and stochastic matrices, mean-field methods make links to optimization theory and perturbation theory.
• Underlying much of the heightened interest in these links between statistical physical and the information sciences is the development (in the latter field) of a general framework for associating joint probability distributions with graphs, and for exploiting the structure of the graph in the computation of marginal probabilities and expectations.
  • From Naive Mean Field Theory to the TAP Equations p.7
  • An Idiosyncratic Journey Beyond Mean Field Theory p.21
  • Mean Field Theory for Graphical Models p.37
  • The TAP Approach to Intensive and Extensive p.51
  • TAP For Parity Check Error Correcting Codes p.67
  • Adaptive TAP Equations p.85
  • From Dynamics p.99
  • Saddlepoint Methods for Intractable Graphical p.119
  • Graphical Models and Variational Methods p.161
  • Some Examples of Recursive Variational p.179
  • Tractable Approximate Belief Propagation p.197
  • The Attenuated MaxProduct Algorithm p.213
  • Comparing the Mean Field Method and Belief p.29
  • Information Geometry of MeanField Approximation p.259

References

,

	Author	volume	Date Value	title	type	journal	titleUrl	doi	note	year
2001 AdvancedMeanFieldMethods	Manfred Opper David Saad			Advanced Mean Field Methods: theory and practice			http://books.google.com/books?id=cuOX8sCDeNAC			2001