Abstract
In this paper we analyze the problem of learning and updating of uncertainty in Dirichlet models, where updating refers to determining the conditional distribution of a single variable when some evidence is known. We first obtain the most general family of prior-posterior distributions which is conjugate to a Dirichlet likelihood and we identify those hyperparameters that are influenced by data values. Next, we describe some methods to assess the prior hyperparameters and we give a numerical method to estimate the Dirichlet parameters in a Bayesian context, based on the posterior mode. We also give formulas for updating uncertainty by determining the conditional probabilities of single variables when the values of other variables are known. A time series approach is presented for dealing with the cases in which samples are not identically distributed, that is, the Dirichlet parameters change from sample to sample. This typically occurs when the population is observed at different times. Finally, two examples are given that illustrate the learning and updating processes and the time series approach.
Article PDF
Similar content being viewed by others
Avoid common mistakes on your manuscript.
References
Arnold, B. C., Castillo, E., & Sarabia, J. M. (1993). Conjugate Exponential Family Priors for Exponential Family Likelihoods. Statistics, 25, 71-77.
Arnold, B. C., Castillo, E., & Sarabia, J. M. (1994). Priors with Convenient Posteriors, (Technical Report No. 94-10). Santander, Spain: University of Cantabria.
Bouckaert, R. (1994). Properties of Bayesian Belief Networks Learning Algorithms. Proceedings of the Tenth Conference on Uncertainty in Artificial Intelligence (pp. 102-109). San Francisco, CA: Morgan Kaufmann.
Castillo, E., & Alvarez, E. (1991). Experts Systems: Uncertainty and Learning. London: Computational Mechanics Publications and Elsevier Applied Science.
Castillo, E., Gutiérrez, J. M., & Hadi, A. S. (1995). Symbolic Propagation in Discrete and Continuous Bayesian Network Models. In V. Keranen & P. Mitic (Eds.), Mathematics with Vision: Proceedings of the First International Mathematica Symposium. Computational Mechanics Publications.
Castillo, E., Gutiérrez, J. M., & Hadi, A. S. (1996a). Parametric Structure of Probabilities in Bayesian Networks. C. Froidevaux & J. Kohlas (Eds.), Lecture Notes in Artificial Intelligence: Symbolic and Quantitative Approaches to Reasoning and Uncertainty. New York: Springer-Verlag.
Castillo, E., Gutiérrez, J. M., & Hadi, A. S. (1996b). Expert Systems and Probabilistic Network Models. New York: Springer-Verlag.
Cooper, G. F. (1990). The Computational Complexity of Probabilistic Inference Using Bayesian Belief Networks. Artificial Intelligence, 42, 393-405.
Cooper, G. F., & Herskovits, E. (1992). A Bayesian Method for the Induction of Probabilistic Networks from Data. Machine Learning, 9, 309-348.
DeGroot, M. H. (1986). Probability and Statistics, Second Editions. Menlo Park, CA: Addison Wesley.
Geiger, D., & Heckerman, D. (1994). Learning Gaussian Networks. Proceedings of the Tenth Conference on Uncertainty in Artificial Intelligence (pp. 235-243). San Francisco, CA: Morgan Kaufmann.
Geiger, D., & Heckerman, D. (1995). A Characterization of the Dirichlet Distribution with Application to Learning Bayesian Networks. Proceedings of the Eleventh Conference on Uncertainty in Artificial Intelligence (pp. 196-207). Montreal: Morgan Kaufmann.
Good, I. J. (1976). On the Application of Symmetric Dirichlet Distributions and Their Mixtures to Contingency Tables. The Annals of Statistics, 4, 1159-1189.
Heckerman, D. (1990). An Empirical Comparison of Three Inference Methods. In R. D. Shachter, T. S. Levitt, L. N. Kanal, & J. F. Lemmer (Eds.), Uncertainty in Artificial Intelligence 4, Amsterdam: Elsevier Science Publishers.
Heckerman, D., Geiger, D., & Chickering, D. M. (1994). Learning Bayesian Networks: The Combination of Knowledge and Statistical Data. Proceedings of the Tenth Conference on Uncertainty in Artificial Intelligence (pp. 293-301). San Francisco, CA: Morgan Kaufmann.
Klieter, G. (1992). Bayesian Diagnosis in Expert Systems. Proceedings of the AIJ92.
Lauritzen, S. L., & Spiegelhalter, D. J. (1988). Local Computations with Probabilities on Graphical Structures and their Application to Expert Systems. Journal of the Royal Statistical Society, Series B, 50, 157-224.
Musick, R. (1993). Maintaining Inference Distributions in Belief Nets. Proceedings of the Eighth Conference on Uncertainty in Artificial Intelligence.
Musick, R. (1994). Belief Network Induction. Doctoral Dissertation, Computer Science Division, University of California, Berkeley.
Neapolitan R., & Kenevan, J. (1991). Investigation of Variances in Belief Networks. Proceedings of the Seventh Conference on Uncertainty in Artificial Intelligence.
Pearl, J. (1986a). A Constraint-Propagation Approach to Probabilistic Reasoning. In L. N. Kanal & J. F. Lemmer (Eds.), Uncertainty in Artificial Intelligence, Amsterdam: Elsevier Science Publishers.
Pearl, J. (1986b). Fusion, Propagation and Structuring in Belief Networks. Artificial Intelligence, 29, 241-288.
Poland, W. B. (1994). Decision Analysis with Continuous and Discrete Variables: A Mixture Distribution Approach. Doctoral Dissertation, Department of Engineering Economic Systems, Stanford University, Stanford, CA.
Press, W. H., Teukolsky, S. A., Vetterling, W. T., & Flannery B. P. (1992). Numerical Recipes in C: The Art of Scientific Computing. London: Cambridge University Press.
Shachter, R. D., Andersen, S. K., & Szolovits, P. (1994). Global Conditioning for Probabilistic Inference in Belief Networks. Proceedings of the Tenth Conference on Uncertainty in Artificial Intelligence (pp. 514-522). San Francisco, CA: Morgan Kaufmann.
Wilks, S. S. (1962). Mathematical Statistics, New York: John Wiley & Sons.
Author information
Authors and Affiliations
Rights and permissions
About this article
Cite this article
Castillo, E., Hadi, A.S. & Solares, C. Learning and Updating of Uncertainty in Dirichlet Models. Machine Learning 26, 43–63 (1997). https://doi.org/10.1023/A:1007372016040
Issue Date:
DOI: https://doi.org/10.1023/A:1007372016040