Marcel Worring
ABSTRACT
We propose a multimedia analytics solution for getting insight in image collections by extending the powerful method of pivot tables, found in the ubiquitous spreadsheets, to multimedia. Our proposed solution is designed by considering the characteristics of multimedia data as well as insight and provides integral access to visual content through concept detection results, tags, geolocation, and other metadata. We present a set of scenarios of using the pivot tables for a collection of images, tags, and metadata from Flickr. User experiments have been instrumental in realizing the final design presented in this paper. The accompanying video shows the solution in action.
- R. Burtner, S. Bohn, and D. Payne. Interactive visual comparison of multimedia data through type-specific views. In SPIE, Visualization and data analysis, 2013.Google Scholar
- E. Chi, J. Riedl, P. Barry, and J. Konstan. Principles for information visualization spreadsheets. Computer Graphics and Applications, July/August 1998. Google ScholarDigital Library
- N. Chinchor, J. Thomas, P. C. Wong, M. Christel, and W. Ribarsky. Multimedia analysis + visual analytics = multimedia analytics. Computer Graphics and Applications, IEEE, 30(5):52--60, 2010. Google ScholarDigital Library
- M. Christel. Automated Metadata in Multimedia Information Systems: Creation, Refinement, Use in Surrogates, and Evaluation. Morgan and Claypool Publishers, 2009. Google ScholarDigital Library
- O. de Rooij, M. Worring, and J. J. van Wijk. Mediatable: Interactive categorization of multimedia collections. IEEE Computer Graphics and Applications, 30(5):42--51, 2010. Google ScholarDigital Library
- M. Dörk, D. Gruen, C. Williamson, and S. Carpendale. A visual backchannel for large-scale events. IEEE Trans. on Visualization and Computer Graphics, 16(6):1129--1138, 2010. Google ScholarDigital Library
- M. Dörk, H. Riche, R. Gonzalo, and S. Dumais. Pivotpaths: strolling through faceted information spaces. IEEE Transactions on Visualization and Computer Graphics, 2012. Google ScholarDigital Library
- D. Fisher, S. Drucker, R. Fernandez, and S. Ruble. Visualizations everywhere: A multiplatform infrastructure for linked visualizations. IEEE Transactions on Visualization and Computer Graphics, 16(6), 2010.Google ScholarDigital Library
- A. Girgensohn, F. Shipman, T. Turner, and L. Wilcox. Flexible access to photo libraries via time, place, tags, and visual features. In Proceedings of JCDL, 2010. Google ScholarDigital Library
- S. Gratzl, A. Lex, N. Gehlenborg, H. Pfister, and M. Streit. Lineup: Visual analysis of multi-attribute rankings. IEEE Trans. Vis. Comput. Graph., 19(12):2277--2286, 2013. Google ScholarDigital Library
- J. Gray, S. Chaudhuri, A. Bosworth, A. Layman, D. Reichart, and M. Venkatrao. Data cube: a relational aggregation operator generalizing group-by, cross-tab, and sub-totals. In Data Mining and Knowledge Discovery, 1997. Google ScholarDigital Library
- B. Höferlin, R. Netzel, M. Höferlin, D. Weiskopf, and G. Heidemann. Inter-active learning of ad-hoc classifiers for video visual analytics. In IEEE Conference on Visual Analytics Science and Technology (VAST), 2012, pages 23--32, 2012. Google ScholarDigital Library
- B. Jelen and M. Alexander. Pivot Table Data Chrunching. Que Publishing, 2005. Google ScholarDigital Library
- P. Joia, F. Paulovich, J. C. D. Coimbra, and L. Nonato. Local affine multidimensional projection. IEEE Transactions on Visualization and Computer Graphics, 17(12), 2011. Google ScholarDigital Library
- S. Kandel, E. Abelson, H. Garcia-Molina, A. Paepcke, and M. Theobald. Photospread: A spreadsheet for managing photos. In Proceedings of SIGCHI, 2008. Google ScholarDigital Library
- D. Keim, G. Andrienko, J.-D. Fekete, C. Gorg, J. Kohlhammer, and G. Melancon. Visual analytics : Definition, process, and challenges. In Information Visualization, LNCS 4960, 2008. Google ScholarDigital Library
- H. Luo, J. Fan, S. Satoh, J. Yang, and W. Ribarsky. Integrating multi-modal content analysis and hyperbolic visualization for large-scale news video retrieval and exploration. Signal Processing: Image Communication, 23(7), 2008. Google ScholarDigital Library
- S. MacNeil and N. Elmqvist. Visualization mosaics for multivariate visual exploration. Computer Graphics Forum, 2013. Google ScholarDigital Library
- G. P. Nguyen and M. Worring. Interactive access to large image collections using similarity-based visualization. Journal of Visual Languages and Computing, 19(2):203--224, 2008. Google ScholarDigital Library
- C. North. Toward measuring visualization insight. IEEE Comput. Graph. Appl., 26(3):6--9, 2006. Google ScholarDigital Library
- D.-S. Ryu, W.-K. Chung, and H.-G. Cho. Photoland: a new image layout system using spatio-temporal information in digital photos. In Proceedings of the 2010 ACM Symposium on Applied Computing, SAC '10, pages 1884--1891, 2010. Google ScholarDigital Library
- K. Schoeffmann, D. Ahlström, and L. Böszörmenyi. 3D storyboards for interactive visual search. In ICME, 2012. Google ScholarDigital Library
- C. G. M. Snoek and et.al. Mediamill at TRECVID 2013: Searching concepts, objects, instances and events in video. In Proceedings of TRECVID Workshop, Gaithersburg, USA, 2013.Google Scholar
- C. Stolte, D. Tang, and P. Hanrahan. Polaris: a system for query, analysis, and visualization of multidimensional relations databases. IEEE Transactions on Visualization and Computer Graphics, 8(1), 2002. Google ScholarDigital Library
- G. Tómasson, H. Sigurthórsson, B. Jónsson, and L. Amsaleg. Photocube: effective and efficient multi-dimensional browsing of personal photo collections. In Proceedings of the ICMR, 2011. Google ScholarDigital Library
- C. Wang, J. Reese, H. Zhang, J. Tao, Y. Gu, J. Ma, and R. Nemiroff. Similarity-based visualization of large image collections. Information Visualization, 6, 2013.Google Scholar
- C. Ware. Visual Thinking for Design. Morgan Kaufmann, 2008. Google ScholarDigital Library
- M. Worring and D. C. Koelma. Multimedia pivot tables. In Proceedings of Visual Analytics Science and Technology (VAST), 2013.Google Scholar
- M. Worring, P. Sajda, S. Santini, D. A. Shamma, A. F. Smeaton, and Q. Yang. Where is the user in multimedia retrieval? IEEE Multimedia, 19:6--10, 2012. Google ScholarDigital Library
- J. Yang, J. Fan, D. Hubball, Y. Gao, H. Luo, W. Ribarsky, and W. M. Semantic image browser: Bridging information visualization with automated intelligent image analysis. In IEEE Symposium on Visual Analytics Science and Technology, 2006.Google ScholarCross Ref
- J. Zahalka and M. Worring. Towards interactive, intelligent, and integrated multimedia analytics. In IEEE Conference on Visual Analytics Science and Technology, 2014.Google ScholarCross Ref
- E. Zavesky, S.-F. Chang, and C.-C. Yang. Visual islands: Intuitive browsing of visual search results. In Proceedings of the 2008 International Conference on Content-based Image and Video Retrieval, pages 617--626, 2008. Google ScholarDigital Library
Index Terms
- Insight in Image Collections by Multimedia Pivot Tables
Recommendations
A multimedia analytics framework for browsing image collections in digital forensics
MM '12: Proceedings of the 20th ACM international conference on MultimediaSearching through large collections of images to find patterns of use or to find sets of relevant items is difficult, especially when the information to consider is not only the content of the images itself, but also the associated metadata. Multimedia ...
Multimedia Pivot Tables for Multimedia Analytics on Image Collections
We propose a multimedia analytics solution for getting insight into image collections by extending the powerful analytic capabilities of pivot tables, found in the ubiquitous spreadsheets, to multimedia. We formalize the concept of multimedia pivot ...
Multimedia Analysis + Visual Analytics = Multimedia Analytics
To deal with the extent and variety of digital media, researchers are combining multimedia analysis and visual analytics to form the new field of multimedia analytics. This article gives some historical background, discusses surveys of related research, ...
Comments