David M. Blei
Michael I. Jordan
ABSTRACT
We consider the problem of modeling annotated data---data with multiple types where the instance of one type (such as a caption) serves as a description of the other type (such as an image). We describe three hierarchical probabilistic mixture models which aim to describe such data, culminating in correspondence latent Dirichlet allocation, a latent variable model that is effective at modeling the joint distribution of both types and the conditional distribution of the annotation given the primary type. We conduct experiments on the Corel database of images and captions, assessing performance in terms of held-out likelihood, automatic annotation, and text-based image retrieval.
- H. Attias. A variational Bayesian framework for graphical models. In Advances in Neural Information Processing Systems 12, 2000.Google Scholar
Digital Library
- K. Barnard, P. Duygulu, N. de Freitas, D. Forsyth, D. Blei, and M. Jordan. Matching words and pictures. Journal of Machine Learning Research, 3:1107--1135, 2003. Google ScholarDigital Library
- D. Blei, A. Ng, and M. Jordan. Latent Dirichlet allocation. Journal of Machine Learning Research, 3:993--1022, January 2003. Google ScholarDigital Library
- D. Cohn and T. Hofmann. The missing link-A probabilistic model of document content and hypertext connectivity. In Advances in Neural Information Processing Systems 13, 2001.Google Scholar
- A. Goodrum. Image information retrieval: An overview of current research. Informing Science, 3(2):63--67, 2000.Google ScholarCross Ref
- J. Jeon, V. Lavrenko, and R. Manmatha. Automatic image annotation and retrieval using cross-media relevance models. In ACM SIGIR 2003, July 2003. Google ScholarDigital Library
- M. Jordan, Z. Ghahramani, T. Jaakkola, and L. Saul. Introduction to variational methods for graphical models. Machine Learning, 37:183--233, 1999. Google ScholarDigital Library
- C. Meghini, F. Sebastiani, and U. Straccia. A model of multimedia information retrieval. Journal of the ACM (JACM), 48(5):909--970, 2001. Google ScholarDigital Library
- C. Morris. Parametric empirical Bayes inference: Theory and applications. Journal of the American Statistical Association, 78(381):47--65, 1983.Google ScholarCross Ref
- M. Naphade and T. Huang. A probabilistic framework for semantic video indexing, filtering and retrieval. IEEE Transactions on Multimedia, 3(1):141--151, March 2001. Google ScholarDigital Library
- J. Ponte and B. Croft. A language modeling approach to information retrieval. In ACM SIGIR 1998, pages 275--281. Google ScholarDigital Library
- J. Shi and J. Malik. Normalized cuts and image segmentation. IEEE Transactions on Pattern Analysis and Machine Intelligence, 22(9):888--905, 2000. Google ScholarDigital Library
- B. Taskar, E. Segal, and D. Koller. Probabilistic clustering in relational data. In IJCAI-01, pages 870--876, 2001. Google ScholarDigital Library
Index Terms
- Modeling annotated data
Recommendations
Empirical Bayes inference for the Weibull model
In this study, the theory of statistical kernel density estimation has been applied for deriving non-parametric kernel prior to the empirical Bayes which frees the Bayesian inference from subjectivity that has worried some statisticians. For comparing ...
The segmented and annotated IAPR TC-12 benchmark
Automatic image annotation (AIA), a highly popular topic in the field of information retrieval research, has experienced significant progress within the last decade. Yet, the lack of a standardized evaluation platform tailored to the needs of AIA, has ...
Local Bayesian image restoration using variational methods and Gamma-normal distributions
ICIP'09: Proceedings of the 16th IEEE international conference on Image processingIn this paper we present a new Bayesian methodology for the restoration of blurred and noisy images. Bayesian methods rely on image priors that encapsulate prior image knowledge and avoid the ill-posedness of image restoration problems. We use a ...
Comments