Abstract
Bayesian probability theory does not define a probability as a frequency of occurrence; rather it defines it as a reasonable degree of belief. Because it does not define a probability as a frequency of occurrence, it is possible to assign probabilities to propositions such as “The probability that the frequency had value ω when the data were taken,” or “The probability that hypothesis x is a better description of the data than hypothesis y.” Problems of the first type are parameter estimation problems, they implicitly assume the correct model. Problems of the second type are more general, they are model selections problems and do not assume the model. Both types of problems are straight forward applications of the rules of Bayesian probability theory. This paper is a tutorial on parameter estimation. The basic rules for manipulating and assigning probabilities are given and an example, the estimation of a single stationary sinusoidal frequency, is worked in detail. This example is sufficiently complex as to illustrate all of the points of principle that must be faced in more realistic problems, yet sufficiently simple that anyone with a background in calculus can follow it. Additionally, the model selection problem is discussed and it is shown that parameter estimation calculation is essentially the first step in the more general model selection calculation.
This is a preview of subscription content, log in via an institution.
Buying options
Tax calculation will be finalised at checkout
Purchases are for personal use only
Learn about institutional subscriptionsPreview
Unable to display preview. Download preview PDF.
References
Laplace, P. S., A Philosophical Essay on Probabilities, unabridged and unaltered reprint of Truscott and Emory translation, Dover Publications, Inc., New York, 1951, original publication data 1814.
Jeffreys, H., Theory of Probability, Oxford University Press, London, 1939; Later editions, 1948, 1961.
Jaynes, E. T., “How Does the Brain do Plausible Reasoning?” unpublished Stanford University Microwave Laboratory Report No. 421 (1957); reprinted in Maximum-Entropy and Bayesian Methods in Science and Engineering 1, pp. 1–24, G. J. Erickson and C. R. Smith Eds., 1988.
Shannon, C. E., “A Mathematical Theory of Communication,” Bell Syst. Tech. J. 27, pp. 379–423 (1948).
Abel, N. H., Crelle’s Jour., Bd. 1 (1826).
Cox, R. T., “Probability, Frequency, and Reasonable Expectations,” Amer. J. Phys. 14,pp. 1–13 (1946).
Tribus, M., Rational Descriptions, Decisions and Designs, Pergamon Press, Oxford, 1969.
Zellner, A., An Introduction to Bayesian Inference in Econometrics, John Wiley and Sons, New York, 1971; Second edition 1987.
Bayes, Rev. T., “An Essay Toward Solving a Problem in the Doctrine of Chances,” Philos. Trans. R. Soc. London 53, pp. 370–418 (1763); reprinted in Biometrika 45, pp. 293–315 (1958), and Facsimiles of Two Papers by Bayes, with commentary by W. Edwards Deming, New York, Hafner, 1963.
Jaynes, E. T., “Prior Probabilities,” IEEE Transactions on Systems Science and Cybernetics, SSC-4, pp. 227–241 (1968); reprinted in [13].
Bretthorst, G. Larry, “Bayesian Spectrum Analysis and Parameter Estimation,” in Lecture Notes in Statistics 48, Springer-Verlag, New York, New York, 1988.
Jaynes, E. T., “Marginalization and Prior Probabilities,” in Bayesian Analysis in Econometrics and Statistics, A. Zellner, ed., North-Holland Publishing Company, Amsterdam, 1980; reprinted in [13].
Jaynes, E. T., Papers on Probability, Statistics and Statistical Physics, a reprint collection, D. Reidel, Dordrecht the Netherlands, 1983; second edition Kluwer Academic Publishers, Dordrecht the Netherlands, 1989.
Jaynes, E. T., “Where Do We Stand On Maximum Entropy?” in The Maximum Entropy Formalism, R. D. Levine and M. Tribus Eds.,pp. 15–118, Cambridge: MIT Press, 1978; Reprinted in [13].
Schuster, A., “The Periodogram and its Optical Analogy,” Proc. R. Soc. London 77, pp. 136 (1905).
Bretthorst, G. Larry, Bayesian Spectrum Analysis and Parameter Estimation, Ph.D. thesis, Washington University, St. Louis, MO., available from University Microfilms Inc., Ann Arbor Mich. 1987; an “Excerpts from Bayesian Spectrum Analysis and Parameter Estimation,” is printed in Maximum-Entropy and Bayesian Methods in Science and Engineering 1, pp. 75–145, G. J. Erickson and C. R. Smith Eds., Kluwer Academic Publishers, Dordrecht the Netherlands, 1988.
Bretthorst, G. Larry, “Bayesian Spectrum Analysis on Quadrature NMR Data with Noise Correlations,” Maximum Entropy and Bayesian Methods, pp. 261–274, J. Skilling, ed., Kluwer Academic Publishers, Dordrecht the Netherlands, 1989.
Bretthorst, G. Larry and C. Ray Smith, “Bayesian Analysis of Signals from Closely-Spaced Objects,” Infrared Systems and Components III, pp 93.104, Robert L. Caswell ed., SPIE Vol. 1050, 1989.
Jaynes, E. T., “Bayesian Spectrum and Chirp Analysis,” in Maximum Entropy and Bayesian Spectral Analysis and Estimation Problems, pp. 1–37, C. Ray Smith and G. J. Erickson, Eds., Kluwer Academic Publishers, Dordrecht the Netherlands, 1987.
Lord Rayleigh, Philos. Mag. 5, p. 261 (1879).
Blackman, R. B. and J. W. Tukey, The Measurement of Power Spectra, Dover Publications, Inc., New York, 1959.
Tukey, J. W., several conversations with E. T. Jaynes, in the period 1980–1983.
Marple, S. L., Digital Spectral Analysis with Applications, Prentice-Hall, New Jersey, 1987.
Waldmeier, M., The Sunspot Activity in the Years 1610–1960, Schulthes, Zurich, 1961.
Nyquist, H., “Certain Topics in Telegraph Transmission Theory,” Trans. AIEE, pp. 617 (1928).
Nyquist, H., “Certain Factors Affecting Telegraph Speed,” Bell Sys. Tech. J. 3, pp. 324 (1924).
Bretthorst, G. Larry, “Bayesian Model Selection: Examples Relevant to NMR,” Maximum Entropy and Bayesian Methods, J. Skilling, ed., pp. 377–388, Kluwer Academic Publishers, Dordrecht the Netherlands, 1989.
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 1990 Kluwer Academic Publishers
About this chapter
Cite this chapter
Bretthorst, G.L. (1990). An Introduction to Parameter Estimation Using Bayesian Probability Theory. In: Fougère, P.F. (eds) Maximum Entropy and Bayesian Methods. Fundamental Theories of Physics, vol 39. Springer, Dordrecht. https://doi.org/10.1007/978-94-009-0683-9_5
Download citation
DOI: https://doi.org/10.1007/978-94-009-0683-9_5
Publisher Name: Springer, Dordrecht
Print ISBN: 978-94-010-6792-8
Online ISBN: 978-94-009-0683-9
eBook Packages: Springer Book Archive