Skip to main content

2016 | OriginalPaper | Buchkapitel

Retrieving Images by Multiple Samples via Fusing Deep Features

verfasst von : Kecai Wu, Xueliang Liu, Jie Shao, Richang Hong, Tao Yang

Erschienen in: Advances in Multimedia Information Processing - PCM 2016

Verlag: Springer International Publishing

Aktivieren Sie unsere intelligente Suche, um passende Fachinhalte oder Patente zu finden.

search-config
loading …

Abstract

Most existing image retrieval systems search similar images on a given single input, while querying based on multiple images is not a trivial. In this paper, we describe a novel image retrieval paradigm that users could input two images as query to search the images that include the content of the two input images-synchronously. In our solution, the deep CNN feature is extracted from each single query image and then fused as the query feature. Due to the role of the two query images is different and changeable, we propose the FWC (Feature weighting by Clustering), a novel algorithm to weight the two query features. All the CNN features in the whole dataset are clustered and the weight of each query is obtained by the distance to the mutual nearest cluster. The effectiveness of our algorithm is evaluated in PASCAL VOC2007 and Microsoft COCO datasets.

Sie haben noch keine Lizenz? Dann Informieren Sie sich jetzt über unsere Produkte:

Springer Professional "Wirtschaft+Technik"

Online-Abonnement

Mit Springer Professional "Wirtschaft+Technik" erhalten Sie Zugriff auf:

  • über 102.000 Bücher
  • über 537 Zeitschriften

aus folgenden Fachgebieten:

  • Automobil + Motoren
  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Elektrotechnik + Elektronik
  • Energie + Nachhaltigkeit
  • Finance + Banking
  • Management + Führung
  • Marketing + Vertrieb
  • Maschinenbau + Werkstoffe
  • Versicherung + Risiko

Jetzt Wissensvorsprung sichern!

Springer Professional "Technik"

Online-Abonnement

Mit Springer Professional "Technik" erhalten Sie Zugriff auf:

  • über 67.000 Bücher
  • über 390 Zeitschriften

aus folgenden Fachgebieten:

  • Automobil + Motoren
  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Elektrotechnik + Elektronik
  • Energie + Nachhaltigkeit
  • Maschinenbau + Werkstoffe




 

Jetzt Wissensvorsprung sichern!

Springer Professional "Wirtschaft"

Online-Abonnement

Mit Springer Professional "Wirtschaft" erhalten Sie Zugriff auf:

  • über 67.000 Bücher
  • über 340 Zeitschriften

aus folgenden Fachgebieten:

  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Finance + Banking
  • Management + Führung
  • Marketing + Vertrieb
  • Versicherung + Risiko




Jetzt Wissensvorsprung sichern!

Literatur
1.
Zurück zum Zitat Chiang, J.L.: Knowledge-based principal component analysis for image fusion. Appl. Math. 8(1L), 223–230 (2014) Chiang, J.L.: Knowledge-based principal component analysis for image fusion. Appl. Math. 8(1L), 223–230 (2014)
2.
Zurück zum Zitat Elkan, C.: Using the triangle inequality to accelerate k-means. In: ICML, vol. 3, pp. 147–153 (2003) Elkan, C.: Using the triangle inequality to accelerate k-means. In: ICML, vol. 3, pp. 147–153 (2003)
3.
Zurück zum Zitat Fernando, B., Tuytelaars, T.: Mining multiple queries for image retrieval: on-the-fly learning of an object-specific mid-level representation. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2544–2551 (2013) Fernando, B., Tuytelaars, T.: Mining multiple queries for image retrieval: on-the-fly learning of an object-specific mid-level representation. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2544–2551 (2013)
4.
Zurück zum Zitat Fu, Y., Cao, L., Guo, G., et al.: Multiple feature fusion by subspace learning. In: Proceedings of the 2008 International Conference on Content-Based Image and Video Retrieval, pp. 127–134. ACM (2008) Fu, Y., Cao, L., Guo, G., et al.: Multiple feature fusion by subspace learning. In: Proceedings of the 2008 International Conference on Content-Based Image and Video Retrieval, pp. 127–134. ACM (2008)
5.
Zurück zum Zitat Lowe, D.G.: Distinctive image features from scale-invariant keypoints. Int. J. Comput. Vision 60(2), 91–110 (2004)CrossRef Lowe, D.G.: Distinctive image features from scale-invariant keypoints. Int. J. Comput. Vision 60(2), 91–110 (2004)CrossRef
6.
Zurück zum Zitat Ghodrati, A., Jia, X., Pedersoli, M., et al.: Swap retrieval: retrieving images of cats when the query shows a dog. In: Proceedings of the 5th ACM on International Conference on Multimedia Retrieval, pp. 395–402. ACM (2015) Ghodrati, A., Jia, X., Pedersoli, M., et al.: Swap retrieval: retrieving images of cats when the query shows a dog. In: Proceedings of the 5th ACM on International Conference on Multimedia Retrieval, pp. 395–402. ACM (2015)
7.
Zurück zum Zitat Gawande, U., Zaveri, M., Kapur, A.: A novel algorithm for feature level fusion using SVM classifier for multibiometrics-based person identification. Appl. Comput. Intell. Soft Comput. 2013, 9 (2013)CrossRef Gawande, U., Zaveri, M., Kapur, A.: A novel algorithm for feature level fusion using SVM classifier for multibiometrics-based person identification. Appl. Comput. Intell. Soft Comput. 2013, 9 (2013)CrossRef
8.
Zurück zum Zitat Girshick, R., Donahue, J., Darrell, T., et al.: Rich feature hierarchies for accurate object detection and semantic segmentation. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp. 580–587 (2014) Girshick, R., Donahue, J., Darrell, T., et al.: Rich feature hierarchies for accurate object detection and semantic segmentation. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp. 580–587 (2014)
9.
Zurück zum Zitat Hariharan, B., Arbeláez, P., Girshick, R., et al.: Hypercolumns for object segmentation and fine-grained localization. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 447–456 (2015) Hariharan, B., Arbeláez, P., Girshick, R., et al.: Hypercolumns for object segmentation and fine-grained localization. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 447–456 (2015)
10.
Zurück zum Zitat Jia, Y., Shelhamer, E., Donahue, J., et al.: Caffe: Convolutional architecture for fast feature embedding. In: Proceedings of the 22nd ACM International Conference on Multimedia, pp. 675–678. ACM (2014) Jia, Y., Shelhamer, E., Donahue, J., et al.: Caffe: Convolutional architecture for fast feature embedding. In: Proceedings of the 22nd ACM International Conference on Multimedia, pp. 675–678. ACM (2014)
11.
Zurück zum Zitat Krizhevsky, A., Sutskever, I., Hinton, G.E.: ImageNet classification with deep convolutional neural networks. In: Advances in Neural Information Processing Systems, pp. 1097–1105 (2012) Krizhevsky, A., Sutskever, I., Hinton, G.E.: ImageNet classification with deep convolutional neural networks. In: Advances in Neural Information Processing Systems, pp. 1097–1105 (2012)
12.
Zurück zum Zitat Makadia, A.: Feature tracking for wide-baseline image retrieval. In: Daniilidis, K., Maragos, P., Paragios, N. (eds.) ECCV 2010, Part V. LNCS, vol. 6315, pp. 310–323. Springer, Heidelberg (2010). doi:10.1007/978-3-642-15555-0_23 CrossRef Makadia, A.: Feature tracking for wide-baseline image retrieval. In: Daniilidis, K., Maragos, P., Paragios, N. (eds.) ECCV 2010, Part V. LNCS, vol. 6315, pp. 310–323. Springer, Heidelberg (2010). doi:10.​1007/​978-3-642-15555-0_​23 CrossRef
13.
Zurück zum Zitat Nister, D., Stewenius, H.: Scalable recognition with a vocabulary tree. In: 2006 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR 2006), vol. 2, pp. 2161–2168. IEEE (2006) Nister, D., Stewenius, H.: Scalable recognition with a vocabulary tree. In: 2006 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR 2006), vol. 2, pp. 2161–2168. IEEE (2006)
14.
Zurück zum Zitat Razavian, A.S., Azizpour, H., Sullivan, J., et al.: CNN features off-the-shelf: an astounding baseline for recognition. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition Workshops, pp. 806–813 (2014) Razavian, A.S., Azizpour, H., Sullivan, J., et al.: CNN features off-the-shelf: an astounding baseline for recognition. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition Workshops, pp. 806–813 (2014)
15.
Zurück zum Zitat Vaca-Castano, G., Shah, M.: Semantic image search from multiple query images. In: Proceedings of the 23rd ACM International Conference on Multimedia, pp. 887–890. ACM (2015) Vaca-Castano, G., Shah, M.: Semantic image search from multiple query images. In: Proceedings of the 23rd ACM International Conference on Multimedia, pp. 887–890. ACM (2015)
16.
Zurück zum Zitat Zhang, S., Yang, M., Cour, T., Yu, K., Metaxas, D.N.: Query specific fusion for image retrieval. In: Fitzgibbon, A., Lazebnik, S., Perona, P., Sato, Y., Schmid, C. (eds.) ECCV 2012, Part II. LNCS, vol. 7573, pp. 660–673. Springer, Heidelberg (2012). doi:10.1007/978-3-642-33709-3_47 CrossRef Zhang, S., Yang, M., Cour, T., Yu, K., Metaxas, D.N.: Query specific fusion for image retrieval. In: Fitzgibbon, A., Lazebnik, S., Perona, P., Sato, Y., Schmid, C. (eds.) ECCV 2012, Part II. LNCS, vol. 7573, pp. 660–673. Springer, Heidelberg (2012). doi:10.​1007/​978-3-642-33709-3_​47 CrossRef
17.
Zurück zum Zitat Yang, X., Qian, X., Xue, Y.: Scalable mobile image retrieval by exploring contextual saliency. IEEE Trans. Image Proc. 24(6), 1709–1721 (2015)MathSciNetCrossRef Yang, X., Qian, X., Xue, Y.: Scalable mobile image retrieval by exploring contextual saliency. IEEE Trans. Image Proc. 24(6), 1709–1721 (2015)MathSciNetCrossRef
18.
Zurück zum Zitat Liu, X., Wang, M., Yin, B.-C., Huet, B., Li, X.: Event-based media enrichment using an adaptive probabilistic hypergraph model. IEEE Trans. Cybern. 45(11), 2461–2471 (2015)CrossRef Liu, X., Wang, M., Yin, B.-C., Huet, B., Li, X.: Event-based media enrichment using an adaptive probabilistic hypergraph model. IEEE Trans. Cybern. 45(11), 2461–2471 (2015)CrossRef
19.
Zurück zum Zitat Wang, M., Li, W., Liu, D., Ni, B., Shen, J., Yan, S.: Facilitating image search with a scalable and compact semantic mapping. IEEE Trans. Cybern. 45(8), 1561–1574 (2015)CrossRef Wang, M., Li, W., Liu, D., Ni, B., Shen, J., Yan, S.: Facilitating image search with a scalable and compact semantic mapping. IEEE Trans. Cybern. 45(8), 1561–1574 (2015)CrossRef
20.
Zurück zum Zitat Wang, M., Li, G., Lu, Z., Gao, Y., Chua, T.-S.: When Amazon meets Google: product visualization by exploring multiple information sources. ACM Trans. Internet Technol. 12(4), 1–17 (2013). Article 12CrossRef Wang, M., Li, G., Lu, Z., Gao, Y., Chua, T.-S.: When Amazon meets Google: product visualization by exploring multiple information sources. ACM Trans. Internet Technol. 12(4), 1–17 (2013). Article 12CrossRef
21.
Zurück zum Zitat Wang, M., Gao, Y., Ke, L., Rui, Y.: View-based discriminative probabilistic modeling for 3D object retrieval and recognition. IEEE Trans. Image Proces. 22(4), 1395–1407 (2013)MathSciNetCrossRef Wang, M., Gao, Y., Ke, L., Rui, Y.: View-based discriminative probabilistic modeling for 3D object retrieval and recognition. IEEE Trans. Image Proces. 22(4), 1395–1407 (2013)MathSciNetCrossRef
Metadaten
Titel
Retrieving Images by Multiple Samples via Fusing Deep Features
verfasst von
Kecai Wu
Xueliang Liu
Jie Shao
Richang Hong
Tao Yang
Copyright-Jahr
2016
DOI
https://doi.org/10.1007/978-3-319-48890-5_22

Neuer Inhalt