Skip to main content
Top
Published in: Neural Computing and Applications 5/2023

13-09-2021 | S.I. : Deep Geospatial Data Understanding

Unsupervised active learning with loss prediction

Authors: Chuanbing Wan, Fusheng Jin, Zhuang Qiao, Weiwei Zhang, Ye Yuan

Published in: Neural Computing and Applications | Issue 5/2023

Log in

Activate our intelligent search to find suitable subject content or patents.

search-config
loading …

Abstract

Active learning is an effective technique to reduce the cost of labeling data by selecting the most beneficial samples. Most existing active learning methods use linear models to select the most representative points to approximate other points. However, they only select samples from the perspective of informativeness or representativeness and cannot model the nonlinearity of data well. In this paper, we propose a novel unsupervised active learning method with a loss prediction module, called UALL. Specifically, UALL uses a deep neural network to model the nonlinearity of data and considers simultaneously the representativeness, informativeness, and diversity, three essential criteria in active learning. Furthermore, we introduce an autoencoder and a loss prediction module to evaluate the representativeness and informativeness and combine K-means and simple calculations to measure the diversity. We compare with the state-of-the-art on eight publicly available datasets from different fields, and the experimental results demonstrate the effectiveness of our method.

Dont have a licence yet? Then find out more about our products and how to get one now:

Springer Professional "Wirtschaft"

Online-Abonnement

Mit Springer Professional "Wirtschaft" erhalten Sie Zugriff auf:

  • über 67.000 Bücher
  • über 340 Zeitschriften

aus folgenden Fachgebieten:

  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Finance + Banking
  • Management + Führung
  • Marketing + Vertrieb
  • Versicherung + Risiko




Jetzt Wissensvorsprung sichern!

Springer Professional "Technik"

Online-Abonnement

Mit Springer Professional "Technik" erhalten Sie Zugriff auf:

  • über 67.000 Bücher
  • über 390 Zeitschriften

aus folgenden Fachgebieten:

  • Automobil + Motoren
  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Elektrotechnik + Elektronik
  • Energie + Nachhaltigkeit
  • Maschinenbau + Werkstoffe




 

Jetzt Wissensvorsprung sichern!

Springer Professional "Wirtschaft+Technik"

Online-Abonnement

Mit Springer Professional "Wirtschaft+Technik" erhalten Sie Zugriff auf:

  • über 102.000 Bücher
  • über 537 Zeitschriften

aus folgenden Fachgebieten:

  • Automobil + Motoren
  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Elektrotechnik + Elektronik
  • Energie + Nachhaltigkeit
  • Finance + Banking
  • Management + Führung
  • Marketing + Vertrieb
  • Maschinenbau + Werkstoffe
  • Versicherung + Risiko

Jetzt Wissensvorsprung sichern!

Footnotes
1
More information about the datasets can be found at https://​archive.​ics.​uci.​edu/​.
 
Literature
1.
go back to reference Krizhevsky A, Sutskever I, Hinton GE (2012) Imagenet classification with deep convolutional neural networks. Adv Neural Inf Process Syst 25:1097 Krizhevsky A, Sutskever I, Hinton GE (2012) Imagenet classification with deep convolutional neural networks. Adv Neural Inf Process Syst 25:1097
4.
go back to reference He Z, Chen C, Bu J, Wang C, Zhang L, Cai D, He X (2012) Document summarization based on data reconstruction. In: Twenty-sixth AAAI conference on artificial intelligence He Z, Chen C, Bu J, Wang C, Zhang L, Cai D, He X (2012) Document summarization based on data reconstruction. In: Twenty-sixth AAAI conference on artificial intelligence
5.
go back to reference Cai JJ, Tang J, Chen QG, Hu Y, Wang X, Huang SJ (2019) Multi-view active learning for video recommendation. In: IJCAI, pp 2053–2059 Cai JJ, Tang J, Chen QG, Hu Y, Wang X, Huang SJ (2019) Multi-view active learning for video recommendation. In: IJCAI, pp 2053–2059
6.
go back to reference Balcan MF, Broder A, Zhang T (2007) Margin based active learning. In: International conference on computational learning theory. Springer, pp 35–50 Balcan MF, Broder A, Zhang T (2007) Margin based active learning. In: International conference on computational learning theory. Springer, pp 35–50
7.
go back to reference Lewis DD, Catlett J (1994) Heterogeneous uncertainty sampling for supervised learning. In: Machine learning proceedings 1994. Elsevier, pp 148–156 Lewis DD, Catlett J (1994) Heterogeneous uncertainty sampling for supervised learning. In: Machine learning proceedings 1994. Elsevier, pp 148–156
8.
go back to reference Freund Y, Seung HS, Shamir E, Tishby N (1997) Selective sampling using the query by committee algorithm. Mach Learn 28(2):133CrossRefMATH Freund Y, Seung HS, Shamir E, Tishby N (1997) Selective sampling using the query by committee algorithm. Mach Learn 28(2):133CrossRefMATH
9.
go back to reference Seung HS, Opper M, Sompolinsky H (1992) Query by committee. In: Proceedings of the fifth annual workshop on computational learning theory, pp 287–294 Seung HS, Opper M, Sompolinsky H (1992) Query by committee. In: Proceedings of the fifth annual workshop on computational learning theory, pp 287–294
10.
go back to reference Lindley DV (1956) On a measure of the information provided by an experiment. Ann Math Stat 27(4):986–1005 Lindley DV (1956) On a measure of the information provided by an experiment. Ann Math Stat 27(4):986–1005
11.
go back to reference Roy N, McCallum A (2001) Toward optimal active learning through Monte Carlo estimation of error reduction. ICML, Williamstown, pp 441–448 Roy N, McCallum A (2001) Toward optimal active learning through Monte Carlo estimation of error reduction. ICML, Williamstown, pp 441–448
12.
go back to reference Yu K, Bi J, Tresp V (2006) Active learning via transductive experimental design. In: Proceedings of the 23rd international conference on machine learning, pp 1081–1088 Yu K, Bi J, Tresp V (2006) Active learning via transductive experimental design. In: Proceedings of the 23rd international conference on machine learning, pp 1081–1088
13.
go back to reference Nguyen HT, Smeulders A (2004) Active learning using pre-clustering. In: Proceedings of the twenty-first international conference on machine learning, p 79 Nguyen HT, Smeulders A (2004) Active learning using pre-clustering. In: Proceedings of the twenty-first international conference on machine learning, p 79
14.
go back to reference Nie F, Wang H, Huang H, Ding C (2013) Early active learning via robust representation and structured sparsity. In: Twenty-third international joint conference on artificial intelligence Nie F, Wang H, Huang H, Ding C (2013) Early active learning via robust representation and structured sparsity. In: Twenty-third international joint conference on artificial intelligence
15.
go back to reference Hu Y, Zhang D, Jin Z, Cai D, He X (2013) Active learning via neighborhood reconstruction. In: Proceedings of the twenty-third international joint conference on artificial intelligence, Citeseer, 2013, pp 1415–1421 Hu Y, Zhang D, Jin Z, Cai D, He X (2013) Active learning via neighborhood reconstruction. In: Proceedings of the twenty-third international joint conference on artificial intelligence, Citeseer, 2013, pp 1415–1421
16.
go back to reference Cai D, He X (2011) Manifold adaptive experimental design for text categorization. IEEE Trans Knowl Data Eng 24(4):707CrossRef Cai D, He X (2011) Manifold adaptive experimental design for text categorization. IEEE Trans Knowl Data Eng 24(4):707CrossRef
18.
go back to reference Yoo D, Kweon IS (2019) Learning loss for active learning. In: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp 93–102 Yoo D, Kweon IS (2019) Learning loss for active learning. In: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp 93–102
19.
go back to reference Wu D (2018) Pool-based sequential active learning for regression. IEEE Trans Neural Netw Learn Syst 30(5):1348CrossRef Wu D (2018) Pool-based sequential active learning for regression. IEEE Trans Neural Netw Learn Syst 30(5):1348CrossRef
20.
go back to reference Yu K, Zhu S, Xu W, Gong Y (2008) TrNon-greedy active learning for text categorization using convex ansductive experimental design. In: Proceedings of the 31st annual international ACM SIGIR conference on research and development in information retrieval, pp 635–642 Yu K, Zhu S, Xu W, Gong Y (2008) TrNon-greedy active learning for text categorization using convex ansductive experimental design. In: Proceedings of the 31st annual international ACM SIGIR conference on research and development in information retrieval, pp 635–642
21.
go back to reference Shi L, Shen YD (2003) Diversifying convex transductive experimental design for active learning. In: IJCAI (2016), pp 1997–2003 Shi L, Shen YD (2003) Diversifying convex transductive experimental design for active learning. In: IJCAI (2016), pp 1997–2003
22.
go back to reference Zhu F, Fan B, Zhu X, Wang Y, Xiang S, Pan C (2014) 10,000+ times accelerated robust subset selection (ARSS). arXiv preprint arXiv:1409.3660 Zhu F, Fan B, Zhu X, Wang Y, Xiang S, Pan C (2014) 10,000+ times accelerated robust subset selection (ARSS). arXiv preprint arXiv:​1409.​3660
23.
go back to reference Rowes ST (2000) Nonlinear dimensionality reduction by locally linear embedding. Science 290:232 Rowes ST (2000) Nonlinear dimensionality reduction by locally linear embedding. Science 290:232
24.
go back to reference Zhang L, Chen C, Bu J, Cai D, He X, Huang TS (2026) Active learning based on locally linear reconstruction. IEEE Trans Pattern Anal Mach Intell 33(10):2026CrossRef Zhang L, Chen C, Bu J, Cai D, He X, Huang TS (2026) Active learning based on locally linear reconstruction. IEEE Trans Pattern Anal Mach Intell 33(10):2026CrossRef
25.
go back to reference Li Q, Xi Shi, Zhou L, Bao Z, Guo Z (2017) Active learning via local structure reconstruction. Pattern Recognit Lett 92:81CrossRef Li Q, Xi Shi, Zhou L, Bao Z, Guo Z (2017) Active learning via local structure reconstruction. Pattern Recognit Lett 92:81CrossRef
26.
go back to reference Inatsu Y, Sugita D, Toyoura K, Takeuchi I (2020) Active learning for enumerating local minima based on Gaussian process derivatives. Neural Comput 32(10):2032CrossRefMATH Inatsu Y, Sugita D, Toyoura K, Takeuchi I (2020) Active learning for enumerating local minima based on Gaussian process derivatives. Neural Comput 32(10):2032CrossRefMATH
27.
go back to reference Li C, Wang X, Dong W, Yan J, Liu Q, Zha H (2018) Joint active learning with feature selection via cur matrix decomposition. IEEE Trans Pattern Anal Mach Intell 41(6):1382CrossRef Li C, Wang X, Dong W, Yan J, Liu Q, Zha H (2018) Joint active learning with feature selection via cur matrix decomposition. IEEE Trans Pattern Anal Mach Intell 41(6):1382CrossRef
28.
go back to reference Natarajan BK (1995) Sparse approximate solutions to linear systems. SIAM J Comput 24(2):227CrossRefMATH Natarajan BK (1995) Sparse approximate solutions to linear systems. SIAM J Comput 24(2):227CrossRefMATH
29.
go back to reference Brinker K (2003) Incorporating diversity in active learning with support vector machines. In: Proceedings of the 20th international conference on machine learning (ICML-03), pp 59–66 Brinker K (2003) Incorporating diversity in active learning with support vector machines. In: Proceedings of the 20th international conference on machine learning (ICML-03), pp 59–66
Metadata
Title
Unsupervised active learning with loss prediction
Authors
Chuanbing Wan
Fusheng Jin
Zhuang Qiao
Weiwei Zhang
Ye Yuan
Publication date
13-09-2021
Publisher
Springer London
Published in
Neural Computing and Applications / Issue 5/2023
Print ISSN: 0941-0643
Electronic ISSN: 1433-3058
DOI
https://doi.org/10.1007/s00521-021-06480-y

Other articles of this Issue 5/2023

Neural Computing and Applications 5/2023 Go to the issue

Premium Partner