Skip to main content

2018 | OriginalPaper | Buchkapitel

Semantic Image Retrieval by Uniting Deep Neural Networks and Cognitive Architectures

verfasst von : Alexey Potapov, Innokentii Zhdanov, Oleg Scherbakov, Nikolai Skorobogatko, Hugo Latapie, Enzo Fenoglio

Erschienen in: Artificial General Intelligence

Verlag: Springer International Publishing

Aktivieren Sie unsere intelligente Suche, um passende Fachinhalte oder Patente zu finden.

search-config
loading …

Abstract

Image and video retrieval by their semantic content has been an important and challenging task for years, because it ultimately requires bridging the symbolic/subsymbolic gap. Recent successes in deep learning enabled detection of objects belonging to many classes greatly outperforming traditional computer vision techniques. However, deep learning solutions capable of executing retrieval queries are still not available. We propose a hybrid solution consisting of a deep neural network for object detection and a cognitive architecture for query execution. Specifically, we use YOLOv2 and OpenCog. Queries allowing the retrieval of video frames containing objects of specified classes and specified spatial arrangement are implemented.

Sie haben noch keine Lizenz? Dann Informieren Sie sich jetzt über unsere Produkte:

Springer Professional "Wirtschaft+Technik"

Online-Abonnement

Mit Springer Professional "Wirtschaft+Technik" erhalten Sie Zugriff auf:

  • über 102.000 Bücher
  • über 537 Zeitschriften

aus folgenden Fachgebieten:

  • Automobil + Motoren
  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Elektrotechnik + Elektronik
  • Energie + Nachhaltigkeit
  • Finance + Banking
  • Management + Führung
  • Marketing + Vertrieb
  • Maschinenbau + Werkstoffe
  • Versicherung + Risiko

Jetzt Wissensvorsprung sichern!

Springer Professional "Technik"

Online-Abonnement

Mit Springer Professional "Technik" erhalten Sie Zugriff auf:

  • über 67.000 Bücher
  • über 390 Zeitschriften

aus folgenden Fachgebieten:

  • Automobil + Motoren
  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Elektrotechnik + Elektronik
  • Energie + Nachhaltigkeit
  • Maschinenbau + Werkstoffe




 

Jetzt Wissensvorsprung sichern!

Springer Professional "Wirtschaft"

Online-Abonnement

Mit Springer Professional "Wirtschaft" erhalten Sie Zugriff auf:

  • über 67.000 Bücher
  • über 340 Zeitschriften

aus folgenden Fachgebieten:

  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Finance + Banking
  • Management + Führung
  • Marketing + Vertrieb
  • Versicherung + Risiko




Jetzt Wissensvorsprung sichern!

Literatur
1.
Zurück zum Zitat Smeulders, A.W., Worring, M., Santini, S., Gupta, A., Jain, R.: Content-based image retrieval at the end of the early years. IEEE Trans. Pattern Anal. Mach. Intell. 22(12), 1349–1380 (2000)CrossRef Smeulders, A.W., Worring, M., Santini, S., Gupta, A., Jain, R.: Content-based image retrieval at the end of the early years. IEEE Trans. Pattern Anal. Mach. Intell. 22(12), 1349–1380 (2000)CrossRef
2.
Zurück zum Zitat Thanga Ramya, S., Rangarajan, P.: Knowledge based methods for video data retrieval. Int. J. Comput. Sci. Inf. Technol. (IJCSIT) 3(5), 165–172 (2011) Thanga Ramya, S., Rangarajan, P.: Knowledge based methods for video data retrieval. Int. J. Comput. Sci. Inf. Technol. (IJCSIT) 3(5), 165–172 (2011)
3.
Zurück zum Zitat Manzoor, U., Balubaid, M.A., Zafar, B., Umar, H., Khan, M.Sh.: Semantic image retrieval: an ontology based approach. (IJARAI) Int. J. Adv. Res. Artif. Intell. 4(4) (2015) Manzoor, U., Balubaid, M.A., Zafar, B., Umar, H., Khan, M.Sh.: Semantic image retrieval: an ontology based approach. (IJARAI) Int. J. Adv. Res. Artif. Intell. 4(4) (2015)
6.
Zurück zum Zitat Liu, H., Wang, R., Shan, Sh., Chen, X.: Deep supervised hashing for fast image retrieval. In: IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 2064–2072 (2016) Liu, H., Wang, R., Shan, Sh., Chen, X.: Deep supervised hashing for fast image retrieval. In: IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 2064–2072 (2016)
7.
Zurück zum Zitat Varga, D., Szirányi, T.: Fast content-based image retrieval using convolutional neural network and hash function. In: IEEE International Conference on Systems, Man, and Cybernetics (SMC) (2016) Varga, D., Szirányi, T.: Fast content-based image retrieval using convolutional neural network and hash function. In: IEEE International Conference on Systems, Man, and Cybernetics (SMC) (2016)
8.
Zurück zum Zitat Karpathy, A., Li, F.-F.: Deep visual-semantic alignments for generating image descriptions. CoRR, abs/1412.2306 (2014) Karpathy, A., Li, F.-F.: Deep visual-semantic alignments for generating image descriptions. CoRR, abs/1412.2306 (2014)
10.
Zurück zum Zitat Quinn, M., Conser, E., Witte, J., Mitchell, M.: Semantic image retrieval via active grounding of visual situations. arXiv:1711.00088 [cs.CV] (2017) Quinn, M., Conser, E., Witte, J., Mitchell, M.: Semantic image retrieval via active grounding of visual situations. arXiv:​1711.​00088 [cs.CV] (2017)
11.
Zurück zum Zitat Celebi, E., Alpkocak, A.: Semantic image retrieval and auto-annotation by converting keyword space to image space. In: 12th International IEEE Multi-Media Modelling Conference Proceedings (2006) Celebi, E., Alpkocak, A.: Semantic image retrieval and auto-annotation by converting keyword space to image space. In: 12th International IEEE Multi-Media Modelling Conference Proceedings (2006)
12.
Zurück zum Zitat Singh, N., Dubey, S., Dixit, P., Gupta, J.: Semantic image retrieval using multiple features. In: Meghanathan, N., et al. (eds.) SIPM, FCST, ITCA, WSE, ACSIT, CS & IT 06, pp. 277–284 (2012) Singh, N., Dubey, S., Dixit, P., Gupta, J.: Semantic image retrieval using multiple features. In: Meghanathan, N., et al. (eds.) SIPM, FCST, ITCA, WSE, ACSIT, CS & IT 06, pp. 277–284 (2012)
14.
Zurück zum Zitat Subramanyam, V., RallabandiSett, R.: Knowledge-based image retrieval system. Knowl. Based Syst. 21(2), 89–100 (2008)CrossRef Subramanyam, V., RallabandiSett, R.: Knowledge-based image retrieval system. Knowl. Based Syst. 21(2), 89–100 (2008)CrossRef
16.
Zurück zum Zitat Goertzel, B., Pennachin, C., Geisweiller, G.: Engineering General Intelligence, Part 2: A Path to Advanced AGI via Embodied Learning and Cognitive Synergy. Atlantis Publishing Corporation (2014) Goertzel, B., Pennachin, C., Geisweiller, G.: Engineering General Intelligence, Part 2: A Path to Advanced AGI via Embodied Learning and Cognitive Synergy. Atlantis Publishing Corporation (2014)
18.
Zurück zum Zitat Johnson, J., et al.: Image retrieval using scene graphs. In: IEEE Conference on Computer Vision and Pattern Recognition (CVPR) (2015) Johnson, J., et al.: Image retrieval using scene graphs. In: IEEE Conference on Computer Vision and Pattern Recognition (CVPR) (2015)
19.
Zurück zum Zitat Redmon, J., Divvala, S., Girshick, R., Farhadi, A.: You only look once: unified, real-time object detection. arXiv:1506.02640 (2015) Redmon, J., Divvala, S., Girshick, R., Farhadi, A.: You only look once: unified, real-time object detection. arXiv:​1506.​02640 (2015)
20.
Zurück zum Zitat Li, Y., He, K., Sun, J., et al.: R-FCN: object detection via region based fully convolutional networks. In: Advances in Neural Information Processing Systems, pp. 379–387 (2016) Li, Y., He, K., Sun, J., et al.: R-FCN: object detection via region based fully convolutional networks. In: Advances in Neural Information Processing Systems, pp. 379–387 (2016)
21.
Zurück zum Zitat Hofstadter, D.R., Mitchell, M.: The copycat project: a model of mental fluidity and analogy-making. In: Holyoak, K., Barnden, J. (eds.) Advances in Connectionist and Neural Computation Theory. Ablex Publishing Corporation (1994) Hofstadter, D.R., Mitchell, M.: The copycat project: a model of mental fluidity and analogy-making. In: Holyoak, K., Barnden, J. (eds.) Advances in Connectionist and Neural Computation Theory. Ablex Publishing Corporation (1994)
Metadaten
Titel
Semantic Image Retrieval by Uniting Deep Neural Networks and Cognitive Architectures
verfasst von
Alexey Potapov
Innokentii Zhdanov
Oleg Scherbakov
Nikolai Skorobogatko
Hugo Latapie
Enzo Fenoglio
Copyright-Jahr
2018
DOI
https://doi.org/10.1007/978-3-319-97676-1_19