Mean-field theory of Boltzmann machine learning

Toshiyuki Tanaka
Phys. Rev. E 58, 2302 – Published 1 August 1998
PDFExport Citation

Abstract

I present a mean-field theory for Boltzmann machine learning, derived by employing Thouless-Anderson-Palmer free energy formalism to a full extent. Using the Plefka expansion an extended theory that takes higher-order correction to mean-field free energy formalism into consideration is presented, from which the mean-field approximation of general orders, along with the linear response correction, are derived by truncating the Plefka expansion up to desired orders. A theoretical foundation for an effective trick of using “diagonal weights,” introduced by Kappen and Rodríguez, is also given. Because of the finite system size and a lack of scaling assumptions on interaction coefficients, the truncated free energy formalism cannot provide an exact description in the case of Boltzmann machines. Accuracies of mean-field approximations of several orders are compared by computer simulations.

  • Received 11 December 1997

DOI:https://doi.org/10.1103/PhysRevE.58.2302

©1998 American Physical Society

Authors & Affiliations

Toshiyuki Tanaka

  • Graduate School of Electrical Engineering, Tokyo Metropolitan University, 1-1 Minami Osawa, Hachioji-shi, Tokyo 192-0397 Japan

References (Subscription Required)

Click to Expand
Issue

Vol. 58, Iss. 2 — August 1998

Reuse & Permissions
Access Options
Author publication services for translation and copyediting assistance advertisement

Authorization Required


×
×

Images

×

Sign up to receive regular email alerts from Physical Review E

Log In

Cancel
×

Search


Article Lookup

Paste a citation or DOI

Enter a citation
×