Skip to main content
Erschienen in: Neural Processing Letters 1/2023

10.06.2022

Convolutional Shrinkage Neural Networks Based Model-Agnostic Meta-Learning for Few-Shot Learning

verfasst von: Yunpeng He, Chuanzhi Zang, Peng Zeng, Qingwei Dong, Ding Liu, Yuqi Liu

Erschienen in: Neural Processing Letters | Ausgabe 1/2023

Einloggen

Aktivieren Sie unsere intelligente Suche, um passende Fachinhalte oder Patente zu finden.

search-config
loading …

Abstract

Meta Learning (ML) has the ability to quickly learn from a small number of samples, and has become an important research field after reinforcement learning. However, the complexity of sample features severely reduces the performance of few-shot learning, and proper feature selection plays a vital role in the performance of neural networks. To address this problem, this article draws up a new type of convolutional neural network with an attention mechanism, namely, convolutional shrinkage neural networks (CSNNs), using the characteristics of negligible noise to obtain a good optimization parameter model. Moreover, soft thresholding is inserted into the network architectures as nonlinear transformation layers to eliminate nonessential features. In addition, considering that it is difficult to set appropriate values for the thresholds, the developed convolutional shrinkage neural networks integrates some specialized neural networks into trainable modules to automatically set the thresholds. To illustrate the effectiveness of the proposed method, the model-agnostic meta-learning method is considered for testing. The results show that the improved method can significantly improve the accuracy of few-shot images classification and enhance the generalization performance.

Sie haben noch keine Lizenz? Dann Informieren Sie sich jetzt über unsere Produkte:

Springer Professional "Wirtschaft+Technik"

Online-Abonnement

Mit Springer Professional "Wirtschaft+Technik" erhalten Sie Zugriff auf:

  • über 102.000 Bücher
  • über 537 Zeitschriften

aus folgenden Fachgebieten:

  • Automobil + Motoren
  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Elektrotechnik + Elektronik
  • Energie + Nachhaltigkeit
  • Finance + Banking
  • Management + Führung
  • Marketing + Vertrieb
  • Maschinenbau + Werkstoffe
  • Versicherung + Risiko

Jetzt Wissensvorsprung sichern!

Springer Professional "Technik"

Online-Abonnement

Mit Springer Professional "Technik" erhalten Sie Zugriff auf:

  • über 67.000 Bücher
  • über 390 Zeitschriften

aus folgenden Fachgebieten:

  • Automobil + Motoren
  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Elektrotechnik + Elektronik
  • Energie + Nachhaltigkeit
  • Maschinenbau + Werkstoffe




 

Jetzt Wissensvorsprung sichern!

Springer Professional "Wirtschaft"

Online-Abonnement

Mit Springer Professional "Wirtschaft" erhalten Sie Zugriff auf:

  • über 67.000 Bücher
  • über 340 Zeitschriften

aus folgenden Fachgebieten:

  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Finance + Banking
  • Management + Führung
  • Marketing + Vertrieb
  • Versicherung + Risiko




Jetzt Wissensvorsprung sichern!

Literatur
1.
Zurück zum Zitat Tang WX, Li B, Barni M et al (2021) An automatic cost learning framework for image steganography using deep reinforcement learning. IEEE Trans Inf Forensics Secur 16:952–967CrossRef Tang WX, Li B, Barni M et al (2021) An automatic cost learning framework for image steganography using deep reinforcement learning. IEEE Trans Inf Forensics Secur 16:952–967CrossRef
2.
Zurück zum Zitat Choi Y, Lee K, Oh S (2019) Distributional deep reinforcement learning with a mixture of Gaussians. In: Proceedings - IEEE International Conference on Robotics and Automation 2019-May: p 9791–9797 Choi Y, Lee K, Oh S (2019) Distributional deep reinforcement learning with a mixture of Gaussians. In: Proceedings - IEEE International Conference on Robotics and Automation 2019-May: p 9791–9797
3.
Zurück zum Zitat Zoph B, Vasudevan V, Shlens J et al (2018) Learning Transferable Architectures for Scalable Image Recognition. In: Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition, p 8697–8710 Zoph B, Vasudevan V, Shlens J et al (2018) Learning Transferable Architectures for Scalable Image Recognition. In: Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition, p 8697–8710
4.
Zurück zum Zitat Baker B, Gupta O, Naik N et al (2017) Designing neural network architectures using reinforcement learning. In: 5th International Conference on Learning Representations, ICLR 2017 - Conference Track Proceedings Baker B, Gupta O, Naik N et al (2017) Designing neural network architectures using reinforcement learning. In: 5th International Conference on Learning Representations, ICLR 2017 - Conference Track Proceedings
5.
Zurück zum Zitat Zoph B, Le QV (2017) Neural architecture search with reinforcement learning. In: 5th International Conference on Learning Representations, ICLR 2017 - Conference Track Proceedings Zoph B, Le QV (2017) Neural architecture search with reinforcement learning. In: 5th International Conference on Learning Representations, ICLR 2017 - Conference Track Proceedings
6.
Zurück zum Zitat Pham H, Guan M Y, Zoph B et al (2018) Efficient Neural Architecture Search via parameter Sharing. In: 35th International Conference on Machine Learning, ICML 2018, vol 9, p 6522–6531 Pham H, Guan M Y, Zoph B et al (2018) Efficient Neural Architecture Search via parameter Sharing. In: 35th International Conference on Machine Learning, ICML 2018, vol 9, p 6522–6531
7.
Zurück zum Zitat Pang G, Shen C, Cao L et al (2021) Deep Learning for Anomaly Detection: A Review. ACM Comput Surv 54(2):1–38 CrossRef Pang G, Shen C, Cao L et al (2021) Deep Learning for Anomaly Detection: A Review. ACM Comput Surv 54(2):1–38 CrossRef
8.
Zurück zum Zitat Leake D, Crandall D (2020) On Bringing Case-Based Reasoning Methodology to Deep Learning. Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics), 12311 LNAI: 343–348 Leake D, Crandall D (2020) On Bringing Case-Based Reasoning Methodology to Deep Learning. Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics), 12311 LNAI: 343–348
9.
Zurück zum Zitat Lu H, Jin L, Luo X et al (2019) RNN for Solving Perturbed Time-Varying Underdetermined Linear System With Double Bound Limits on Residual Errors and State Variables. IEEE Trans Ind Inform 15(11):5931–5942CrossRef Lu H, Jin L, Luo X et al (2019) RNN for Solving Perturbed Time-Varying Underdetermined Linear System With Double Bound Limits on Residual Errors and State Variables. IEEE Trans Ind Inform 15(11):5931–5942CrossRef
10.
Zurück zum Zitat Hong Y, Niu L, Zhang J et al (2020) Matchinggan: Matching-Based Few-Shot Image Generation. IEEE Int Conf Multimed Expo (ICME) 2020:1–6 Hong Y, Niu L, Zhang J et al (2020) Matchinggan: Matching-Based Few-Shot Image Generation. IEEE Int Conf Multimed Expo (ICME) 2020:1–6
11.
Zurück zum Zitat Lake B, Salakhutdinov R, Tenenbaum J (2015) Human-level concept learning through probabilistic program induction. Science 350(6266):1332–1338MathSciNetCrossRefMATH Lake B, Salakhutdinov R, Tenenbaum J (2015) Human-level concept learning through probabilistic program induction. Science 350(6266):1332–1338MathSciNetCrossRefMATH
12.
Zurück zum Zitat Wang J, Zhai Y (2020) Prototypical Siamese Networks for Few-shot Learning. In: 2020 IEEE 10th International Conference on Electronics Information and Emergency Communication (ICEIEC), p 178–181 Wang J, Zhai Y (2020) Prototypical Siamese Networks for Few-shot Learning. In: 2020 IEEE 10th International Conference on Electronics Information and Emergency Communication (ICEIEC), p 178–181
13.
Zurück zum Zitat Das D, Lee CSG (2020) A Two-Stage Approach to Few-Shot Learning for Image Recognition. IEEE Trans Image Process 29:3336–3350CrossRefMATH Das D, Lee CSG (2020) A Two-Stage Approach to Few-Shot Learning for Image Recognition. IEEE Trans Image Process 29:3336–3350CrossRefMATH
14.
Zurück zum Zitat Ramalho T, Garnelo M (2019) Adaptive Posterior Learning: few-shot learning with a surprise-based memory module. In: 7th International Conference on Learning Representations, ICLR Ramalho T, Garnelo M (2019) Adaptive Posterior Learning: few-shot learning with a surprise-based memory module. In: 7th International Conference on Learning Representations, ICLR
15.
Zurück zum Zitat Finn C, Abbeel P, Levine S (2017) Model-agnostic meta-learning for fast adaptation of deep networks. In: 34th International Conference on Machine Learning, ICML 2017, vol 3, p 1856–1868 Finn C, Abbeel P, Levine S (2017) Model-agnostic meta-learning for fast adaptation of deep networks. In: 34th International Conference on Machine Learning, ICML 2017, vol 3, p 1856–1868
16.
Zurück zum Zitat Antoniou A, Storkey A, Edwards H (2019) How to train your MAML. In: 7th International Conference on Learning Representations, ICLR Antoniou A, Storkey A, Edwards H (2019) How to train your MAML. In: 7th International Conference on Learning Representations, ICLR
17.
Zurück zum Zitat Liu Y, Lee J, Park M et al (2019) Learning to propagate labels: Transductive propagation network for few-shot learning. In: 7th International Conference on Learning Representations, ICLR Liu Y, Lee J, Park M et al (2019) Learning to propagate labels: Transductive propagation network for few-shot learning. In: 7th International Conference on Learning Representations, ICLR
18.
Zurück zum Zitat Yao H, Wei Y, Huang J et al (2019) Hierarchically structured meta-learning. In: 36th International Conference on Machine Learning, ICML, 2019-June, p 12189–12209 Yao H, Wei Y, Huang J et al (2019) Hierarchically structured meta-learning. In: 36th International Conference on Machine Learning, ICML, 2019-June, p 12189–12209
20.
Zurück zum Zitat Ravi S, Larochelle H (2017) Optimization as a model for few-shot learning. In: 5th International Conference on Learning Representations, ICLR 2017 - Conference Track Proceedings Ravi S, Larochelle H (2017) Optimization as a model for few-shot learning. In: 5th International Conference on Learning Representations, ICLR 2017 - Conference Track Proceedings
21.
Zurück zum Zitat Santoro A, Bartunov S, Botvinick M et al (2016) Meta-Learning with Memory-Augmented Neural Networks. In: 33rd International Conference on Machine Learning, ICML 2016, vol 4, p 2740–2751 Santoro A, Bartunov S, Botvinick M et al (2016) Meta-Learning with Memory-Augmented Neural Networks. In: 33rd International Conference on Machine Learning, ICML 2016, vol 4, p 2740–2751
22.
Zurück zum Zitat Finn C, Rajeswaran A, Kakade S et al (2019) Online meta-learning. In: 36th International Conference on Machine Learning, ICML 2019, 2019-June, p 3398–3410 Finn C, Rajeswaran A, Kakade S et al (2019) Online meta-learning. In: 36th International Conference on Machine Learning, ICML 2019, 2019-June, p 3398–3410
23.
Zurück zum Zitat Munkhdalai T, Yu H (2017) Meta networks. In: 34th International Conference on Machine Learning, ICML 2017, vol 5, p 3933–3943 Munkhdalai T, Yu H (2017) Meta networks. In: 34th International Conference on Machine Learning, ICML 2017, vol 5, p 3933–3943
24.
Zurück zum Zitat Lee K, Maji S, Ravichandran A et al (2019) Meta-learning with differentiable convex optimization. In: Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition, p 10649-10657 Lee K, Maji S, Ravichandran A et al (2019) Meta-learning with differentiable convex optimization. In: Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition, p 10649-10657
25.
Zurück zum Zitat Xue T, Yu H (2020) Model-Agnostic Metalearning-Based Text-Driven Visual Navigation Model for Unfamiliar Tasks. IEEE Access 8:166742–166752CrossRef Xue T, Yu H (2020) Model-Agnostic Metalearning-Based Text-Driven Visual Navigation Model for Unfamiliar Tasks. IEEE Access 8:166742–166752CrossRef
26.
Zurück zum Zitat Isogawa K, Ida T, Shiodera T et al (2018) Deep shrinkage convolutional neural network for adaptive noise reduction. IEEE Signal Process Lett 25:224–228CrossRef Isogawa K, Ida T, Shiodera T et al (2018) Deep shrinkage convolutional neural network for adaptive noise reduction. IEEE Signal Process Lett 25:224–228CrossRef
27.
Zurück zum Zitat Zhao M, Zhong S, Fu X et al (2020) Deep Residual Shrinkage Networks for Fault Diagnosis. IEEE Trans Industr Inf 16:4681–4690CrossRef Zhao M, Zhong S, Fu X et al (2020) Deep Residual Shrinkage Networks for Fault Diagnosis. IEEE Trans Industr Inf 16:4681–4690CrossRef
28.
Zurück zum Zitat Hu J, Shen L, Albanie S et al (2020) Squeeze-and-Excitation Networks. IEEE Trans Pattern Anal Mach Intell 42:2011–2023CrossRef Hu J, Shen L, Albanie S et al (2020) Squeeze-and-Excitation Networks. IEEE Trans Pattern Anal Mach Intell 42:2011–2023CrossRef
29.
Zurück zum Zitat Tan M, Chen B, Pang R et al (2019) Mnasnet: Platform-aware neural architecture search for mobile. In: Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition, 2019-June, p 2815–2823 Tan M, Chen B, Pang R et al (2019) Mnasnet: Platform-aware neural architecture search for mobile. In: Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition, 2019-June, p 2815–2823
30.
Zurück zum Zitat Vaswani A, Shazeer N, Parmar N et al (2017) Attention is all you need. Advances in Neural Information Processing Systems, 2017-December: 5999-6009 Vaswani A, Shazeer N, Parmar N et al (2017) Attention is all you need. Advances in Neural Information Processing Systems, 2017-December: 5999-6009
31.
Zurück zum Zitat Gu J, Wang Z, Kuen J et al (2018) Recent Advances in Convolutional Neural Networks. Pattern Recogn 77:354–377CrossRef Gu J, Wang Z, Kuen J et al (2018) Recent Advances in Convolutional Neural Networks. Pattern Recogn 77:354–377CrossRef
32.
Zurück zum Zitat Zhao ZQ, Zheng P, Xu S-T et al (2019) Object Detection with Deep Learning: A Review. IEEE Trans Neural Netw Learn Syst 30:3212–3232CrossRef Zhao ZQ, Zheng P, Xu S-T et al (2019) Object Detection with Deep Learning: A Review. IEEE Trans Neural Netw Learn Syst 30:3212–3232CrossRef
33.
Zurück zum Zitat Yu SD, Liu LL, Wang ZY et al (2019) Transferring deep neural networks for the differentiation of mammographic breast lesions. Sci China Tech Sci 62:441–447CrossRef Yu SD, Liu LL, Wang ZY et al (2019) Transferring deep neural networks for the differentiation of mammographic breast lesions. Sci China Tech Sci 62:441–447CrossRef
35.
Zurück zum Zitat Munkhdalai T, Yuan X, Mehri S et al (2018) Rapid adaptation with conditionallyshifted neurons. In: 35th International Conference on Machine Learning, ICML, vol 8, p 5898–5909 Munkhdalai T, Yuan X, Mehri S et al (2018) Rapid adaptation with conditionallyshifted neurons. In: 35th International Conference on Machine Learning, ICML, vol 8, p 5898–5909
36.
Zurück zum Zitat Das D, LeeC (2020) A Two-Stage Approach to Few-Shot Learning for Image Recognition. IEEE Trans Image Process 29:3336–3350CrossRefMATH Das D, LeeC (2020) A Two-Stage Approach to Few-Shot Learning for Image Recognition. IEEE Trans Image Process 29:3336–3350CrossRefMATH
37.
Zurück zum Zitat Vinyals O, Blundell C, Lillicrap T et al (2016) Matching networks for one shot learning. In: 30th conference on neural information processing systems (NIPS), vol 29 Vinyals O, Blundell C, Lillicrap T et al (2016) Matching networks for one shot learning. In: 30th conference on neural information processing systems (NIPS), vol 29
38.
Zurück zum Zitat Wang R, Zhang X, Liu C (2021) Meta-Prototypical Learning for Domain-Agnostic Few-Shot Recognition. IEEE Transactions on Neural Networks and Learning Systems, 1–7 Wang R, Zhang X, Liu C (2021) Meta-Prototypical Learning for Domain-Agnostic Few-Shot Recognition. IEEE Transactions on Neural Networks and Learning Systems, 1–7
39.
Zurück zum Zitat Lee H, Lee H, Na D et al (2020) Learning to Balance: Bayesian Meta-Learning for Imbalanced and Out-of-distribution Tasks. arXiv:1905.12917, 1–15 Lee H, Lee H, Na D et al (2020) Learning to Balance: Bayesian Meta-Learning for Imbalanced and Out-of-distribution Tasks. arXiv:​1905.​12917, 1–15
40.
Zurück zum Zitat Luo X, Zhou M, Shang M et al (2016) A Novel Approach to Extracting Non-Negative Latent Factors From Non-Negative Big Sparse Matrices. IEEE Access 4:2649–2655CrossRef Luo X, Zhou M, Shang M et al (2016) A Novel Approach to Extracting Non-Negative Latent Factors From Non-Negative Big Sparse Matrices. IEEE Access 4:2649–2655CrossRef
41.
Zurück zum Zitat Luo X, Zhou M, Li S et al (2021) Algorithms of Unconstrained Non-Negative Latent Factor Analysis for Recommender Systems. IEEE Trans Big Data 7(1):227–240CrossRef Luo X, Zhou M, Li S et al (2021) Algorithms of Unconstrained Non-Negative Latent Factor Analysis for Recommender Systems. IEEE Trans Big Data 7(1):227–240CrossRef
Metadaten
Titel
Convolutional Shrinkage Neural Networks Based Model-Agnostic Meta-Learning for Few-Shot Learning
verfasst von
Yunpeng He
Chuanzhi Zang
Peng Zeng
Qingwei Dong
Ding Liu
Yuqi Liu
Publikationsdatum
10.06.2022
Verlag
Springer US
Erschienen in
Neural Processing Letters / Ausgabe 1/2023
Print ISSN: 1370-4621
Elektronische ISSN: 1573-773X
DOI
https://doi.org/10.1007/s11063-022-10894-7

Weitere Artikel der Ausgabe 1/2023

Neural Processing Letters 1/2023 Zur Ausgabe

Neuer Inhalt