Skip to main content
Fachgebiete
Automobil + Motoren Bauwesen + Immobilien Business IT + Informatik Elektrotechnik + Elektronik Energie + Nachhaltigkeit Finance + Banking Management + Führung Marketing + Vertrieb Maschinenbau + Werkstoffe Versicherung + Risiko
DE
EN
Themenseiten
Organisationspsychologie Projektmanagement Marketing Smart Manufacturing
Bücher
Zeitschriften
Weitere Formate
Podcasts Webinare Technik Webinare Wirtschaft Kongresse Veranstaltungskalender Awards
Jetzt Einzelzugang starten
Zugang für Unternehmen
Referenzkunden
MTZ-Fachkongress

Antriebe und Energiesysteme von morgen


Austausch von Automobil- und Energiewirtschaft

Am 14./15. Mai geht es in Chemnitz um die Mobilitätswende durch Elektro- und Wasserstofffahrzeuge.
Erweiterte Suche
Anmelden
nach oben
Pattern Analysis and Applications
Erschienen in: Pattern Analysis and Applications 2/2024

01.06.2024 | Original Article

The limitations of differentiable architecture search

verfasst von: Lacharme Guillaume, Cardot Hubert, Lente Christophe, Monmarche Nicolas

Erschienen in: Pattern Analysis and Applications | Ausgabe 2/2024

Einloggen

Aktivieren Sie unsere intelligente Suche, um passende Fachinhalte oder Patente zu finden.

search-config
loading …

Abstract

In this paper, we will provide a detailed explanation of the limitations behind differentiable architecture search (DARTS). Algorithms based on the DARTS paradigm tend to converge towards degenerate solutions. A degenerate solution corresponds to an architecture with a shallow graph containing mainly skip connections. We have identified 6 sources of errors that could explain this phenomenon. Some of these errors can only be partially eliminated. Therefore, we will propose an innovative solution to remove degenerate solutions from the search space. We will demonstrate the validity of our approach through experiments conducted on the CIFAR10 and CIFAR100 databases. Our code is available at the following link: https://​scm.​univ-tours.​fr/​projetspublics/​lifat/​darts_​ibpria_​sparcity
Vorheriger Artikel Focalize K-NN: an imputation algorithm for time series datasets
Nächster Artikel High-recall calibration monitoring for stereo cameras

Sie haben noch keine Lizenz? Dann Informieren Sie sich jetzt über unsere Produkte:

Springer Professional "Wirtschaft+Technik"

Online-Abonnement

Mit Springer Professional "Wirtschaft+Technik" erhalten Sie Zugriff auf:

  • über 102.000 Bücher
  • über 537 Zeitschriften

aus folgenden Fachgebieten:

  • Automobil + Motoren
  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Elektrotechnik + Elektronik
  • Energie + Nachhaltigkeit
  • Finance + Banking
  • Management + Führung
  • Marketing + Vertrieb
  • Maschinenbau + Werkstoffe
  • Versicherung + Risiko

Jetzt Wissensvorsprung sichern!

Jetzt informieren

Springer Professional "Technik"

Online-Abonnement

Mit Springer Professional "Technik" erhalten Sie Zugriff auf:

  • über 67.000 Bücher
  • über 390 Zeitschriften

aus folgenden Fachgebieten:

  • Automobil + Motoren
  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Elektrotechnik + Elektronik
  • Energie + Nachhaltigkeit
  • Maschinenbau + Werkstoffe




 

Jetzt Wissensvorsprung sichern!

Jetzt informieren

Springer Professional "Wirtschaft"

Online-Abonnement

Mit Springer Professional "Wirtschaft" erhalten Sie Zugriff auf:

  • über 67.000 Bücher
  • über 340 Zeitschriften

aus folgenden Fachgebieten:

  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Finance + Banking
  • Management + Führung
  • Marketing + Vertrieb
  • Versicherung + Risiko




Jetzt Wissensvorsprung sichern!

Jetzt informieren
Anhänge
Nur mit Berechtigung zugänglich
Literatur
1.
Ang A, Ma J, Liu N, Huang K, Wang Y (2021) Fast projection onto the capped simplex with applications to sparse regression in bioinformatics. Neural Inf Process Syst (NeurIPS)
2.
Balles L, Hennig P (2018) Dissecting adam: the sign, magnitude and variance of stochastic gradients. In: International conference on machine learning (PMLR)
3.
Choe H, Na B, Mok J, Yoon S (2021) Variance-stationary differentiable NAS. In: British machine vision conference (BMVC)
4.
Franceschi L, Donini M, Frasconi P, Pontil M (2017) Forward and reverse gradient-based hyperparameter optimization. In: International conference on machine learning (ICML)
5.
Fu J, Luo H, Feng J, Low KH, Chua TS (2016) DrMAD: distilling reverse-mode automatic differentiation for optimizing hyperparameters of deep neural networks. In: International joint conference on artificial intelligence (IJCAI)
6.
Gu YC, Wang LJ, Liu Y, Yang Y, Wu YH, Lu SP, Cheng MM (2021) DOTS: decoupling operation and topology in differentiable architecture search. In: Conference on computer vision and pattern recognition (CVPR)
7.
He C, Ye H, Shen L, Zhang T (2020) MiLeNAS: efficient neural architecture search via mixed-level reformulation. In: Conference on computer vision and pattern recognition (CVPR)
8.
Hong W, Li G, Zhang W, Tang R, Wang Y, Li Z, Yu Y (2020) DropNAS: grouped operation dropout for differentiable architecture search. In: International joint conference on artificial intelligence (IJCAI)
9.
Hou P, Jin Y, Chen Y (2021) Single-DARTS: towards stable architecture search. In: IEEE/CVF international conference on computer vision workshops (ICCVW)
10.
Kendall MG (1938) A new measure of rank correlation. Biometrika
11.
Kingma DP, Ba JL (2015) ADAM: a method for stochastic optimization. In: International conference on learning representations (ICLR)
12.
Lacharme G, Cardot H, Lenté C, Monmarché N (2023) DARTS with degeneracy correction. In: Iberian conference on pattern recognition and image analysis (IbPRIA)
13.
Lee HB, Lee H, Shin J, Yang E, Hospedales TM, Hwang SJ (2022) online hyperparameter meta-learning with hypergradient distillation. In: International conference on learning representations (ICLR)
14.
Li L, Jamieson K, DeSalvo G, Rostamizadeh A, Talwalkar A (2018) Hyperband: a novel bandit-based approach to hyperparameter optimization. J Mach Learn Res
15.
Li G, Qian G, Delgadillo IC, Muller M, Thabet A, Ghanem B (2020) SGAS: sequential greedy architecture search. In: Conference on computer vision and pattern recognition (CVPR)
16.
Liang H, Zhang S, Sun J, He X, Huang W, Zhuang K, Li Z (2021) DARTS+: improved differentiable architecture search with early stopping. http://arxiv.org/abs/1909.06035
17.
Lin M, Wang P, Sun Z, Chen H, Sun X, Qian Q, Li H, Jin R (2021) Zen-NAS: a zero-shot NAS for high-performance image recognition. In: International conference on computer vision (ICCV)
18.
Liu H, Simonyan K, Yang Y (2019) Darts: differentiable architecture search. In: International conference on learning representations (ICLR)
19.
Lorraine J, Vicol P, Duvenaud D (2020) Optimizing millions of hyperparameters by implicit differentiation
20.
Loshchilov I, Hutter F (2019) Decoupled weight decay regularization. In: International conference on learning representations (ICLR)
21.
Luketina J, Berglund M, Greff K, Raiko T (2016) Scalable gradient-based tuning of continuous regularization hyperparameters. In: International conference on machine learning (ICML)
22.
Metz L, Maheswaranathan N, Sun R, Daniel Freeman C, Poole B, Sohl-Dickstein J (2020) Using a thousand optimization tasks to learn hyperparameter search strategies Neural Inf Process Syst (NeurIPS)
23.
Real E, Aggarwal A, Huang Y, Le QV (2019) Regularized evolution for image classifier architecture search. In: Conference on artificial intelligence (AAAI)
24.
Vicol P, Lorraine JP, Pedregosa F, Duvenaud D, Grosse RB (2022) On implicit bias in overparameterized bilevel optimization. In: International conference on machine learning (ICML)
25.
Vicol P, Metz L, Sohl-Dickstein J (2021) Persistent unbiased gradient estimation in unrolled computation graphs with persistent evolution strategies
26.
Wei T, Wang C, Rui Y, Chen CW (2016) Network morphism. In: Proceedings of machine learning research (PMLR)
27.
Wu Y, Ren M, Liao R, Grosse R (2018) Understanding short-horizon bias in stochastic meta-optimizations. In: International conference on learning representations (ICLR)
28.
Yibo Y, Hongyang L, Shan Y, Fei W, Chen Q, Zhouchen L (2020) ISTA-NAS: efficient and consistent neural architecture search by sparse coding. In: Proceedings of the 34th international conference on neural information processing systems (NeurIPS)
29.
Zhang M, Su S, Pan S, Chang X, Abbasnejad E, Haffari R (2021) iDARTS: differentiable architecture search with stochastic implicit gradients. In: International conference on machine learning (ICML)
30.
Zhou P, Xiong C, Socher R, Hoi SCH (2020) Theory-inspired path-regularized differential network architecture search. Neural Inf Process Syst (NeurIPS)
31.
Zoph B, Vasudevan V, Shlens J, Le QV (2019) Learning transferable architectures for scalable image recognition. In: Conference on computer vision and pattern recognition (CVPR)
Metadaten
Titel
The limitations of differentiable architecture search
verfasst von
Lacharme Guillaume
Cardot Hubert
Lente Christophe
Monmarche Nicolas
Publikationsdatum
01.06.2024
Verlag
Springer London
Erschienen in
Pattern Analysis and Applications / Ausgabe 2/2024
Print ISSN: 1433-7541
Elektronische ISSN: 1433-755X
DOI
https://doi.org/10.1007/s10044-024-01260-5

Weitere Artikel der Ausgabe 2/2024

Pattern Analysis and Applications 2/2024 Zur Ausgabe

Original Article

Proxemics-net++: classification of human interactions in still images

Theoretical Advances

A novel Venus’ visible image processing neoteric workflow for improved planetary surface feature analysis

Original Paper

CABF-YOLO: a precise and efficient deep learning method for defect detection on strip steel surface

Original Article

Domain-free fire detection using the spatial–temporal attention transform of the YOLO backbone

Theoretical Advances

A weakly supervised end-to-end framework for semantic segmentation of cancerous area in whole slide image

Original Article

Understanding the limitations of self-supervised learning for tabular anomaly detection

Premium Partner

    Bildnachweise