Skip to main content
Erschienen in: International Journal of Computer Vision 3/2016

01.02.2016

Image Search with Selective Match Kernels: Aggregation Across Single and Multiple Images

verfasst von: Giorgos Tolias, Yannis Avrithis, Hervé Jégou

Erschienen in: International Journal of Computer Vision | Ausgabe 3/2016

Einloggen

Aktivieren Sie unsere intelligente Suche, um passende Fachinhalte oder Patente zu finden.

search-config
loading …

Abstract

This paper considers a family of metrics to compare images based on their local descriptors. It encompasses the vector or locally aggregated descriptors descriptor and matching techniques such as hamming embedding. Making the bridge between these approaches leads us to propose a match kernel that takes the best of existing techniques by combining an aggregation procedure with a selective match kernel. The representation underpinning this kernel is approximated, providing a large scale image search both precise and scalable, as shown by our experiments on several benchmarks. We show that the same aggregation procedure, originally applied per image, can effectively operate on groups of similar features found across multiple images. This method implicitly performs feature set augmentation, while enjoying savings in memory requirements at the same time. Finally, the proposed method is shown effective for place recognition, outperforming state of the art methods on a large scale landmark recognition benchmark.

Sie haben noch keine Lizenz? Dann Informieren Sie sich jetzt über unsere Produkte:

Springer Professional "Wirtschaft+Technik"

Online-Abonnement

Mit Springer Professional "Wirtschaft+Technik" erhalten Sie Zugriff auf:

  • über 102.000 Bücher
  • über 537 Zeitschriften

aus folgenden Fachgebieten:

  • Automobil + Motoren
  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Elektrotechnik + Elektronik
  • Energie + Nachhaltigkeit
  • Finance + Banking
  • Management + Führung
  • Marketing + Vertrieb
  • Maschinenbau + Werkstoffe
  • Versicherung + Risiko

Jetzt Wissensvorsprung sichern!

Springer Professional "Wirtschaft"

Online-Abonnement

Mit Springer Professional "Wirtschaft" erhalten Sie Zugriff auf:

  • über 67.000 Bücher
  • über 340 Zeitschriften

aus folgenden Fachgebieten:

  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Finance + Banking
  • Management + Führung
  • Marketing + Vertrieb
  • Versicherung + Risiko




Jetzt Wissensvorsprung sichern!

Springer Professional "Technik"

Online-Abonnement

Mit Springer Professional "Technik" erhalten Sie Zugriff auf:

  • über 67.000 Bücher
  • über 390 Zeitschriften

aus folgenden Fachgebieten:

  • Automobil + Motoren
  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Elektrotechnik + Elektronik
  • Energie + Nachhaltigkeit
  • Maschinenbau + Werkstoffe




 

Jetzt Wissensvorsprung sichern!

Fußnoten
1
This is in contrast to our previous work (Tolias et al. 2013), where we have combined ASMK\(^\star \) with the geometry-based variant.
 
Literatur
Zurück zum Zitat Arandjelovic, R., & Zisserman, A. (2012). Three things everyone should know to improve object retrieval. In CVPR. Arandjelovic, R., & Zisserman, A. (2012). Three things everyone should know to improve object retrieval. In CVPR.
Zurück zum Zitat Arandjelović, R., & Zisserman, A. (2013). All about VLAD. In CVPR. Arandjelović, R., & Zisserman, A. (2013). All about VLAD. In CVPR.
Zurück zum Zitat Arandjelović, R., & Zisserman, A. (2014). DisLocation: Scalable descriptor distinctiveness for location recognition. In ACCV. Arandjelović, R., & Zisserman, A. (2014). DisLocation: Scalable descriptor distinctiveness for location recognition. In ACCV.
Zurück zum Zitat Avrithis, Y., Kalantidis, Y., Tolias, G., & Spyrou, E. (2010). Retrieving landmark and non-landmark images from community photo collections. In ACM Multimedia. Avrithis, Y., Kalantidis, Y., Tolias, G., & Spyrou, E. (2010). Retrieving landmark and non-landmark images from community photo collections. In ACM Multimedia.
Zurück zum Zitat Bo, L., & Sminchisescu, C. (2009). Efficient match kernel between sets of features for visual recognition. In NIPS. Bo, L., & Sminchisescu, C. (2009). Efficient match kernel between sets of features for visual recognition. In NIPS.
Zurück zum Zitat Boureau, Y., Bach, F., Lecun, Y., & Ponce, J. (2010). Learning mid-level features for recognition. In cvpr. Boureau, Y., Bach, F., Lecun, Y., & Ponce, J. (2010). Learning mid-level features for recognition. In cvpr.
Zurück zum Zitat Charikar, M. (2002). Similarity estimation techniques from rounding algorithms. In ACM Symposium on Theory of Computing. Charikar, M. (2002). Similarity estimation techniques from rounding algorithms. In ACM Symposium on Theory of Computing.
Zurück zum Zitat Chen, D. M., Baatz, G., Koser, K., Tsai, S. S., Vedantham, R., Pylvanainen, T., Roimela, K., Chen, X., Bach, J., Pollefeys, M., Girod, B., & Grzeszczuk, R. (2011). City-scale landmark identification on mobile devices. In CVPR. Chen, D. M., Baatz, G., Koser, K., Tsai, S. S., Vedantham, R., Pylvanainen, T., Roimela, K., Chen, X., Bach, J., Pollefeys, M., Girod, B., & Grzeszczuk, R. (2011). City-scale landmark identification on mobile devices. In CVPR.
Zurück zum Zitat Chum, O., Mikulik, A., Perdoch, M., & Matas, J. (2011). Total recall II: Query expansion revisited. In CVPR. Chum, O., Mikulik, A., Perdoch, M., & Matas, J. (2011). Total recall II: Query expansion revisited. In CVPR.
Zurück zum Zitat Chum, O., Philbin, J., Sivic, J., Isard, M., & Zisserman, A. (2007) Total recall: Automatic query expansion with a generative feature model for object retrieval. In ICCV. Chum, O., Philbin, J., Sivic, J., Isard, M., & Zisserman, A. (2007) Total recall: Automatic query expansion with a generative feature model for object retrieval. In ICCV.
Zurück zum Zitat Csurka, G., Dance, C., Fan, L., Willamowski, J., & Bray, C. (2004). Visual categorization with bags of keypoints. In ECCV Workshop Statistical Learning in Computer Vision. Csurka, G., Dance, C., Fan, L., Willamowski, J., & Bray, C. (2004). Visual categorization with bags of keypoints. In ECCV Workshop Statistical Learning in Computer Vision.
Zurück zum Zitat Danfeng, Q., Gammeter, S., Bossard, L., Quack, T., & Gool, L. V. (2011). Hello neighbor: Accurate object retrieval with k-reciprocal nearest neighbors. In CVPR. Danfeng, Q., Gammeter, S., Bossard, L., Quack, T., & Gool, L. V. (2011). Hello neighbor: Accurate object retrieval with k-reciprocal nearest neighbors. In CVPR.
Zurück zum Zitat Delhumeau, J., Gosselin, P.H., Jégou, H., & Pérez, P. (2013). Revisiting the vlad image representation. In ACM Multimedia. Delhumeau, J., Gosselin, P.H., Jégou, H., & Pérez, P. (2013). Revisiting the vlad image representation. In ACM Multimedia.
Zurück zum Zitat Delvinioti, A., Jégou, H., Amsaleg, L., & Houle, M. E. (2014). Image retrieval with reciprocal and shared nearest neighbors. In VISAPP. Delvinioti, A., Jégou, H., Amsaleg, L., & Houle, M. E. (2014). Image retrieval with reciprocal and shared nearest neighbors. In VISAPP.
Zurück zum Zitat Hays, J., & Efros, A. A. (2008). Im2gps: estimating geographic information from a single image. In CVPR. Hays, J., & Efros, A. A. (2008). Im2gps: estimating geographic information from a single image. In CVPR.
Zurück zum Zitat Jain, M., Benmokhtar, R., Gros, P., & Jégou, H. (2012). Hamming embedding similarity-based image classification. In ICMR. Jain, M., Benmokhtar, R., Gros, P., & Jégou, H. (2012). Hamming embedding similarity-based image classification. In ICMR.
Zurück zum Zitat Jain, M., Jégou, H., & Gros, P. (2011). Asymmetric hamming embedding: Taking the best of our bits for large scale image search. In ACM Multimedia. Jain, M., Jégou, H., & Gros, P. (2011). Asymmetric hamming embedding: Taking the best of our bits for large scale image search. In ACM Multimedia.
Zurück zum Zitat Jégou, H., Douze, M., & Schmid, C. (2008). Hamming embedding and weak geometric consistency for large scale image search. In ECCV. Jégou, H., Douze, M., & Schmid, C. (2008). Hamming embedding and weak geometric consistency for large scale image search. In ECCV.
Zurück zum Zitat Jégou, H., Douze, M., & Schmid, C. (2009). On the burstiness of visual elements. In CVPR. Jégou, H., Douze, M., & Schmid, C. (2009). On the burstiness of visual elements. In CVPR.
Zurück zum Zitat Jégou, H., Douze, M., Schmid, C., & Pérez, P. (2010). Aggregating local descriptors into a compact image representation. In CVPR. Jégou, H., Douze, M., Schmid, C., & Pérez, P. (2010). Aggregating local descriptors into a compact image representation. In CVPR.
Zurück zum Zitat Ji, R., Duan, L., Chen, J., Yao, H., Yuan, J., Rui, Y., & Gao, W. (2012). Location discriminative vocabulary coding for mobile landmark search. IJCV, 1–25. Ji, R., Duan, L., Chen, J., Yao, H., Yuan, J., Rui, Y., & Gao, W. (2012). Location discriminative vocabulary coding for mobile landmark search. IJCV, 1–25.
Zurück zum Zitat Johns, E., & Yang, G. Z. (2011). From images to scenes: Compressing an image cluster into a single scene model for place recognition. In ICCV. Johns, E., & Yang, G. Z. (2011). From images to scenes: Compressing an image cluster into a single scene model for place recognition. In ICCV.
Zurück zum Zitat Kalantidis, Y., & Avrithis, Y. (2014). Locally optimized product quantization for approximate nearest neighbor search. In: CVPR, Columbus, Ohio. Kalantidis, Y., & Avrithis, Y. (2014). Locally optimized product quantization for approximate nearest neighbor search. In: CVPR, Columbus, Ohio.
Zurück zum Zitat Knopp, J., Sivic, J., & Pajdla, T. (2010). Avoiding confusing features in place recognition. In ECCV. Knopp, J., Sivic, J., & Pajdla, T. (2010). Avoiding confusing features in place recognition. In ECCV.
Zurück zum Zitat Li, Y., Crandall, D. J., & Huttenlocher, D. P. (2009). Landmark classification in large-scale image collections. In ICCV. Li, Y., Crandall, D. J., & Huttenlocher, D. P. (2009). Landmark classification in large-scale image collections. In ICCV.
Zurück zum Zitat Lowe, D. (2004). Distinctive image features from scale-invariant keypoints. IJCV, 60(2), 91–110.CrossRef Lowe, D. (2004). Distinctive image features from scale-invariant keypoints. IJCV, 60(2), 91–110.CrossRef
Zurück zum Zitat Mikolajczyk, K., & Schmid, C. (2005). A performance evaluation of local descriptors. IEEE Transactions on Pattern Analysis and Machine Intelligence, 27(10), 1615–1630.CrossRef Mikolajczyk, K., & Schmid, C. (2005). A performance evaluation of local descriptors. IEEE Transactions on Pattern Analysis and Machine Intelligence, 27(10), 1615–1630.CrossRef
Zurück zum Zitat Mikulik, A., Perdoch, M., Chum, O., & Matas, J. (2013). Learning vocabularies over a fine quantization. IJCV, 103(1), 163–175.MathSciNetCrossRef Mikulik, A., Perdoch, M., Chum, O., & Matas, J. (2013). Learning vocabularies over a fine quantization. IJCV, 103(1), 163–175.MathSciNetCrossRef
Zurück zum Zitat Nistér, D., & Stewénius, H. (2006). Scalable recognition with a vocabulary tree. In CVPR (pp. 2161–2168). Nistér, D., & Stewénius, H. (2006). Scalable recognition with a vocabulary tree. In CVPR (pp. 2161–2168).
Zurück zum Zitat Perdoch, M., Chum, O., Matas, J. (2009). Efficient representation of local geometry for large scale object retrieval. In CVPR. Perdoch, M., Chum, O., Matas, J. (2009). Efficient representation of local geometry for large scale object retrieval. In CVPR.
Zurück zum Zitat Perronnin, F., & Dance, C. R. (2007). Fisher kernels on visual vocabularies for image categorization. In CVPR. Perronnin, F., & Dance, C. R. (2007). Fisher kernels on visual vocabularies for image categorization. In CVPR.
Zurück zum Zitat Perronnin, F., Liu, Y., Sanchez, J., & Poirier, H. (2010). Large-scale image retrieval with compressed Fisher vectors. In CVPR. Perronnin, F., Liu, Y., Sanchez, J., & Poirier, H. (2010). Large-scale image retrieval with compressed Fisher vectors. In CVPR.
Zurück zum Zitat Perronnin, F., Sánchez, J., Mensink, T. (2010). Improving the fisher kernel for large-scale image classification. In ECCV. Perronnin, F., Sánchez, J., Mensink, T. (2010). Improving the fisher kernel for large-scale image classification. In ECCV.
Zurück zum Zitat Philbin, J., Chum, O., Isard, M., Sivic, J., Zisserman, A. (2007). Object retrieval with large vocabularies and fast spatial matching. In CVPR. Philbin, J., Chum, O., Isard, M., Sivic, J., Zisserman, A. (2007). Object retrieval with large vocabularies and fast spatial matching. In CVPR.
Zurück zum Zitat Philbin, J., Chum, O., Isard, M., Sivic, J., Zisserman, A. (2008). Lost in quantization: Improving particular object retrieval in large scale image databases. In CVPR. Philbin, J., Chum, O., Isard, M., Sivic, J., Zisserman, A. (2008). Lost in quantization: Improving particular object retrieval in large scale image databases. In CVPR.
Zurück zum Zitat Qin, D., Wengert, C., Van Gool, L. (2013). Query adaptive similarity for large scale object retrieval. In CVPR. Qin, D., Wengert, C., Van Gool, L. (2013). Query adaptive similarity for large scale object retrieval. In CVPR.
Zurück zum Zitat Salton, G., & Buckley, C. (1988). Term-weighting approaches in automatic text retrieval. Information Processing & Management, 24(5), 513–523. Salton, G., & Buckley, C. (1988). Term-weighting approaches in automatic text retrieval. Information Processing & Management, 24(5), 513–523.
Zurück zum Zitat Schindler, G., Brown, M., & Szeliski, R. (2007). City-scale location recognition. In CVPR. Schindler, G., Brown, M., & Szeliski, R. (2007). City-scale location recognition. In CVPR.
Zurück zum Zitat Shen, X., Lin, Z., Brandt, J., Avidan, S., & Wu, Y. (2012). Object retrieval and localization with spatially-constrained similarity measure and k-nn re-ranking. In CVPR. Shen, X., Lin, Z., Brandt, J., Avidan, S., & Wu, Y. (2012). Object retrieval and localization with spatially-constrained similarity measure and k-nn re-ranking. In CVPR.
Zurück zum Zitat Sivic, J., Zisserman, A. (2003). Video Google: A text retrieval approach to object matching in videos. In ICCV. Sivic, J., Zisserman, A. (2003). Video Google: A text retrieval approach to object matching in videos. In ICCV.
Zurück zum Zitat Tao, R., Gavves, E., Snoek, C. G., & Smeulders, A. W. (2014). Locality in generic instance search from one example. In CVPR. Tao, R., Gavves, E., Snoek, C. G., & Smeulders, A. W. (2014). Locality in generic instance search from one example. In CVPR.
Zurück zum Zitat Tolias, G., & Avrithis, Y. (2011). Speeded-up, relaxed spatial matching. In ICCV. Tolias, G., & Avrithis, Y. (2011). Speeded-up, relaxed spatial matching. In ICCV.
Zurück zum Zitat Tolias, G., Avrithis, Y., & Jégou, H. (2013). To aggregate or not to aggregate: selective match kernels for image search. In ICCV. Tolias, G., Avrithis, Y., & Jégou, H. (2013). To aggregate or not to aggregate: selective match kernels for image search. In ICCV.
Zurück zum Zitat Tolias, G., & Jégou, H. (2014). Visual query expansion with or without geometry: Refining local descriptors by feature aggregation. Pattern Recognition. Tolias, G., & Jégou, H. (2014). Visual query expansion with or without geometry: Refining local descriptors by feature aggregation. Pattern Recognition.
Zurück zum Zitat Torii, A., Sivic, J., Pajdla, T., & Okutomi, M. (2013). Visual place recognition with repetitive structures. In CVPR. Torii, A., Sivic, J., Pajdla, T., & Okutomi, M. (2013). Visual place recognition with repetitive structures. In CVPR.
Zurück zum Zitat Torralba, A., Fergus, R., & Weiss, Y. (2008). Small codes and large databases for recognition. In CVPR. Torralba, A., Fergus, R., & Weiss, Y. (2008). Small codes and large databases for recognition. In CVPR.
Zurück zum Zitat Turcot, P., & Lowe, D. G. (2009). Better matching with fewer features: The selection of useful features in large database recognition problems. In CVPR. Turcot, P., & Lowe, D. G. (2009). Better matching with fewer features: The selection of useful features in large database recognition problems. In CVPR.
Zurück zum Zitat Wang, J., Yang, J., K. Yu, F. L., Huang, T., & Gong, Y. (2010). Locality-constrained linear coding for image classification. In CVPR. Wang, J., Yang, J., K. Yu, F. L., Huang, T., & Gong, Y. (2010). Locality-constrained linear coding for image classification. In CVPR.
Zurück zum Zitat Wu, Z., Ke, Q., Isard, M., & Sun, J. (2009). Bundling features for large scale partial-duplicate web image search. In CVPR (pp. 25–32). Wu, Z., Ke, Q., Isard, M., & Sun, J. (2009). Bundling features for large scale partial-duplicate web image search. In CVPR (pp. 25–32).
Zurück zum Zitat Zhang, S., Yang, M., Cour, T., Yu, K., & Metaxas, D. N. (2012). Query specific fusion for image retrieval. In ECCV. Zhang, S., Yang, M., Cour, T., Yu, K., & Metaxas, D. N. (2012). Query specific fusion for image retrieval. In ECCV.
Metadaten
Titel
Image Search with Selective Match Kernels: Aggregation Across Single and Multiple Images
verfasst von
Giorgos Tolias
Yannis Avrithis
Hervé Jégou
Publikationsdatum
01.02.2016
Verlag
Springer US
Erschienen in
International Journal of Computer Vision / Ausgabe 3/2016
Print ISSN: 0920-5691
Elektronische ISSN: 1573-1405
DOI
https://doi.org/10.1007/s11263-015-0810-4

Weitere Artikel der Ausgabe 3/2016

International Journal of Computer Vision 3/2016 Zur Ausgabe