Abstract
The principles of Bayesian reasoning are reviewed and applied to problems of inference from data sampled from Poisson, Gaussian and Cauchy distributions. Probability distributions (priors and likelihoods) are assigned in appropriate hypothesis spaces using the Maximum Entropy Principle, and then manipulated via Bayes’ Theorem. Bayesian hypothesis testing requires careful consideration of the prior ranges of any parameters involved, and this leads to a quantitive statement of Occam’s Razor. As an example of this general principle we offer a solution to an important problem in regression analysis; determining the optimal number of parameters to use when fitting graphical data with a set of basis functions.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Preview
Unable to display preview. Download preview PDF.
References
Birkinshaw, M., Gull, S.F. & Hardebeck, H. (1984). Nature, 309, 34–35.
Brans, C. & Dicke, R.H. (1961). Phys. Rev., 124, 925.
Cox, R.P. (1946). Probability, Frequency and Reasonable Expectation. Am. Jour. Phys. 17, 1–13.
Gull, S.F. & Skilling, J. (1984). Maximum entropy method in image processing. IEE Proc., 131(F), 646–659.
Jaynes, E.T. (1968). Prior probabilities. Reprinted in E.T. Jaynes: Papers on Probability, Statistics and Statistical Physics, ed. R. Rosenkrantz, 1983 Dordrecht: Reidel.
Jaynes, E.T. (1976). Confidence intervals versus Bayesian Intervals. Reprinted in E.T. Jaynes: Papers on Probability, Statistics and Statistical Physics, ed. R. Rosenkrantz, 1983. Dordrecht: Reidel.
Jaynes, E.T. (1986). Bayesian Methods — an Introductory Tutorial. In Maximum Entropy and Bayesian Methods in Applied Statistics. ed. J.H. Justice. Cambridge University Press.
Jeffreys, H. (1939). Theory of Probability, Oxford University Press. Later editions 1948, 1961,1983.
Shore, J.E. & Johnson, R.W. (1980). Axiomatic derivation of the principle of maximum entropy and the principle of minimum cross-entropy. IEEE Trans. Info. Theory, IT-26, 26–39 and IT-29, 942–943.
Skilling, J. (1986). The Axioms of Maximum Entropy. Presented at 1986 Maximum Entropy conference, Seattle, Washington (this volume).
Skilling, J. & Gull, S.F. (1984). The entropy of an image. SIAM Amer. Math. Soc. proc. Appl. Math., 14, 167–189
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 1988 Kluwer Academic Publishers
About this chapter
Cite this chapter
Gull, S.F. (1988). Bayesian Inductive Inference and Maximum Entropy. In: Erickson, G.J., Smith, C.R. (eds) Maximum-Entropy and Bayesian Methods in Science and Engineering. Fundamental Theories of Physics, vol 31-32. Springer, Dordrecht. https://doi.org/10.1007/978-94-009-3049-0_4
Download citation
DOI: https://doi.org/10.1007/978-94-009-3049-0_4
Publisher Name: Springer, Dordrecht
Print ISBN: 978-94-010-7871-9
Online ISBN: 978-94-009-3049-0
eBook Packages: Springer Book Archive