Skip to main content
Disciplines
Automotive Business IT + Informatics Construction + Real Estate Electrical Engineering + Electronics Energy + Sustainability Insurance + Risk Finance + Banking Management + Leadership Marketing + Sales Mechanical Engineering + Materials
Events
DE
EN
Books
Journals
Topic Page
Marketing
Start single access now
Access for companies
EXTENDED SEARCH
Log in
Top
Published in:
Skip Act Vectors: Integrating Dialogue Context into Sentence Embeddings Read chapter Read first chapter

2021 | OriginalPaper | Chapter

Measuring Catastrophic Forgetting in Visual Question Answering

Authors : Claudio Greco, Barbara Plank, Raquel Fernández, Raffaella Bernardi

Published in: Increasing Naturalness and Flexibility in Spoken Dialogue Interaction

Publisher: Springer Singapore

Log in

Activate our intelligent search to find suitable subject content or patents.

search-config
loading …

Abstract

Catastrophic forgetting is a ubiquitous problem for the current generation of Artificial Neural Networks: When a network is asked to learn multiple tasks in a sequence, it fails dramatically as it tends to forget past knowledge. Little is known on how far multimodal conversational agents suffer from this phenomenon. In this paper, we study the problem of catastrophic forgetting in Visual Question Answering (VQA) and propose experiments in which we analyze pairs of tasks based on CLEVR, a dataset requiring different skills which involve visual or linguistic knowledge. Our results show that dramatic forgetting is at place in VQA, calling for studies on how multimodal models can be enhanced with continual learning methods.

Dont have a licence yet? Then find out more about our products and how to get one now:

Springer Professional "Wirtschaft+Technik"

Online-Abonnement

Mit Springer Professional "Wirtschaft+Technik" erhalten Sie Zugriff auf:

  • über 102.000 Bücher
  • über 537 Zeitschriften

aus folgenden Fachgebieten:

  • Automobil + Motoren
  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Elektrotechnik + Elektronik
  • Energie + Nachhaltigkeit
  • Finance + Banking
  • Management + Führung
  • Marketing + Vertrieb
  • Maschinenbau + Werkstoffe
  • Versicherung + Risiko

Jetzt Wissensvorsprung sichern!

inform now

Springer Professional "Technik"

Online-Abonnement

Mit Springer Professional "Technik" erhalten Sie Zugriff auf:

  • über 67.000 Bücher
  • über 390 Zeitschriften

aus folgenden Fachgebieten:

  • Automobil + Motoren
  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Elektrotechnik + Elektronik
  • Energie + Nachhaltigkeit
  • Maschinenbau + Werkstoffe




 

Jetzt Wissensvorsprung sichern!

inform now
previous chapter Incremental Improvement of a Question Answering System by Re-ranking Answer Candidates Using Machine Learning
next chapter Position Paper: Brain Signal-Based Dialogue Systems
Footnotes
1
In Experiment 1, Task A and B are trained on different amounts of data (see Sect. 3). We have trained the model on Task A with the same amount of data as Task B obtaining very similar performance: 0.566.
 
Literature
1.
Chaudhry A, Dokania PK, Ajanthan T, Torr P (2018) Understanding forgetting and intransigence. In: ECCV, Riemannian walk for incremental learning
2.
Das A, Kottur S, Gupta K, Singh A, Yadav D, Moura JMF, Parikh D, Batra D (2017) Visual dialog. In: CVPR
3.
Díaz-Rodríguez N, Lomonaco V, Filliat D, Maltoni D (2018) Don’t forget, there is more than forgetting: new metrics for continual learning. In: Workshop on continual learning, NeurIPS
4.
Johnson J, Hariharan B, van der Maaten L, Fei-Fei L, Lawrence Zitnick C, Girshick R (2017) CLEVR: a diagnostic dataset for compositional language and elementary visual reasoning. In: CVPR
5.
6.
Kirkpatrick J, Pascanu R, Rabinowitz N, Veness J, Desjardins G, Rusu AA, Milan K, Quan J, Ramalho T, Grabska-Barwinska A et al (2017) Overcoming catastrophic forgetting in neural networks. PNAS
7.
8.
McClelland JL, McNaughton BL, O’reilly RC (1995) Why there are complementary learning systems in the hippocampus and neocortex: insights from the successes and failures of connectionist models of learning and memory. Psychol Rev 102(3)
9.
Moradlou S, Ginzburg J (2014) Learning to understand questions. In: SemDial
10.
Perez E, Strub F, De Vries H, Dumoulin V (2018) Visual reasoning with a general conditioning layer. In: AAAI, Film
11.
Ring M (1997) CHILD: a first step towards continual learning. Mach Learning 28(1)
12.
Yang Z, He X, Gao J, Deng L, Smola A (2016) Stacked attention networks for image question answering. In: CVPR
13.
Zamir AR, Sax A, Shen W, Guibas L, Malik J, Savarese S (2018) Disentangling task transfer learning. In: CVPR, Taskonomy
14.
Zenke F, Poole B, Ganguli S (2017) Continual learning through synaptic intelligence. In: ICML
Metadata
Title
Measuring Catastrophic Forgetting in Visual Question Answering
Authors
Claudio Greco
Barbara Plank
Raquel Fernández
Raffaella Bernardi
Copyright Year
2021
Publisher
Springer Singapore
Book
Increasing Naturalness and Flexibility in Spoken Dialogue Interaction

Print ISBN: 978-981-15-9322-2

Electronic ISBN: 978-981-15-9323-9

Copyright Year: 2021

DOI
https://doi.org/10.1007/978-981-15-9323-9_35