Skip to main content
Disciplines
Automotive Business IT + Informatics Construction + Real Estate Electrical Engineering + Electronics Energy + Sustainability Insurance + Risk Finance + Banking Management + Leadership Marketing + Sales Mechanical Engineering + Materials
Events
DE
EN
Books
Journals
Topic Page
Marketing
Start single access now
Access for companies
EXTENDED SEARCH
Log in
Top
Published in:
‘If Only I Would Have Done that…’: A Controlled Adaptive Network Model for Learning by Counterfactual Thinking Read chapter Read first chapter

2021 | OriginalPaper | Chapter

A Survey of Methods for Detection and Correction of Noisy Labels in Time Series Data

Authors : Gentry Atkinson, Vangelis Metsis

Published in: Artificial Intelligence Applications and Innovations

Publisher: Springer International Publishing

Log in

Activate our intelligent search to find suitable subject content or patents.

search-config
loading …

Abstract

Mislabeled data in large datasets can quickly degrade the performance of machine learning models. There is a substantial base of work on how to identify and correct instances in data with incorrect annotations. However, time series data pose unique challenges that often are not accounted for in label noise detecting platforms. This paper reviews the body of literature concerning label noise and methods of dealing with it, with a focus on applicability to time series data. Time series data visualization and feature extraction techniques used in the denoising process are also discussed.

Dont have a licence yet? Then find out more about our products and how to get one now:

Springer Professional "Wirtschaft+Technik"

Online-Abonnement

Mit Springer Professional "Wirtschaft+Technik" erhalten Sie Zugriff auf:

  • über 102.000 Bücher
  • über 537 Zeitschriften

aus folgenden Fachgebieten:

  • Automobil + Motoren
  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Elektrotechnik + Elektronik
  • Energie + Nachhaltigkeit
  • Finance + Banking
  • Management + Führung
  • Marketing + Vertrieb
  • Maschinenbau + Werkstoffe
  • Versicherung + Risiko

Jetzt Wissensvorsprung sichern!

inform now

Springer Professional "Technik"

Online-Abonnement

Mit Springer Professional "Technik" erhalten Sie Zugriff auf:

  • über 67.000 Bücher
  • über 390 Zeitschriften

aus folgenden Fachgebieten:

  • Automobil + Motoren
  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Elektrotechnik + Elektronik
  • Energie + Nachhaltigkeit
  • Maschinenbau + Werkstoffe




 

Jetzt Wissensvorsprung sichern!

inform now

Springer Professional "Wirtschaft"

Online-Abonnement

Mit Springer Professional "Wirtschaft" erhalten Sie Zugriff auf:

  • über 67.000 Bücher
  • über 340 Zeitschriften

aus folgenden Fachgebieten:

  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Finance + Banking
  • Management + Führung
  • Marketing + Vertrieb
  • Versicherung + Risiko




Jetzt Wissensvorsprung sichern!

inform now
previous chapter BEMS in the Era of Internet of Energy: A Review
next chapter An Automated Tool to Support an Intelligence Learner Management System Using Learning Analytics and Machine Learning
Literature
1.
2.
Aha, D.W., Kibler, D., Albert, M.K.: Instance-based learning algorithms. Mach. Learn. 6(1), 37–66 (1991)
3.
Aigner, W., Kainz, C., Ma, R., Miksch, S.: Bertin was right: an empirical evaluation of indexing to compare multivariate time-series data using line plots. In: Computer Graphics Forum, vol. 30, pp. 215–228. Wiley Online Library (2011)
4.
Aigner, W., Miksch, S., Müller, W., Schumann, H., Tominski, C.: Visual methods for analyzing time-oriented data. IEEE Trans. Vis. Comput. Graph. 14(1), 47–60 (2007)CrossRef
5.
Almeida, M., Zhuang, Y., Ding, W., Crouter, S.E., Chen, P.: Mitigating class-boundary label uncertainty to reduce both model bias and variance. ACM Trans. Knowl. Disc. Data (TKDD) 15(2), 1–18 (2021)CrossRef
6.
Atkinson, G., Metsis, V.: Identifying label noise in time-series datasets. In: Adjunct Proceedings of the 2020 ACM International Joint Conference on Pervasive and Ubiquitous Computing and Proceedings of the 2020 ACM International Symposium on Wearable Computers, pp. 238–243 (2020)
7.
Atkinson, G., Metsis, V.: TSAR: a time series assisted relabeling tool for reducing label noise. In: 14th PErvasive Technologies Related to Assistive Environments Conference (2021)
8.
Bauer, E., Kohavi, R.: An empirical comparison of voting classification algorithms: bagging, boosting, and variants. Mach. Learn. 36(1), 105–139 (1999)CrossRef
9.
10.
Bertin, J.: Semiology of graphics; diagrams networks maps. Technical report (1983)
11.
Bingham, E., Gionis, A., Haiminen, N., Hiisilä, H., Mannila, H., Terzi, E.: Segmentation and dimensionality reduction. In: Proceedings of the 2006 SIAM International Conference on Data Mining, pp. 372–383. SIAM (2006)
12.
Birjandtalab, J., Pouyan, M.B., Nourani, M.: Nonlinear dimension reduction for EEG-based epileptic seizure detection. In: 2016 IEEE-EMBS International Conference on Biomedical and Health Informatics (BHI), pp. 595–598. IEEE (2016)
13.
Boeva, V., Lundberg, L., Angelova, M., Kohstall, J.: Cluster validation measures for label noise filtering. In: 2018 International Conference on Intelligent Systems (IS), pp. 109–116. IEEE (2018)
14.
Bootkrajang, J., Chaijaruwanich, J.: Towards instance-dependent label noise-tolerant classification: a probabilistic approach. Pattern Anal. Appl. 23(1), 95–111 (2020)MathSciNetCrossRef
15.
Bootkrajang, J., Kabán, A.: Multi-class classification in the presence of labelling errors. In: ESANN, pp. 345–350. Citeseer (2011)
16.
17.
Bootkrajang, J., Kabán, A.: Classification of mislabelled microarrays using robust sparse logistic regression. Bioinformatics 29(7), 870–877 (2013)CrossRef
18.
Brodley, C.E., Friedl, M.A.: Identifying mislabeled training data. J. Artif. intell. Res. 11, 131–167 (1999)CrossRef
19.
20.
Cannings, T.I., Fan, Y., Samworth, R.J.: Classification with imperfect training labels. Biometrika 107(2), 311–330 (2020)MathSciNetCrossRef
21.
Cheng, Y., Church, G.M.: Biclustering of expression data. In: ISMB, vol. 8, pp. 93–103 (2000)
22.
Chung, F.L., Fu, T.C., Luk, R., Ng, V., et al.: Flexible time series pattern matching based on perceptually important points (2001)
23.
24.
de França, F.O., Coelho, A.L.: A biclustering approach for classification with mislabeled data. Exp. Syst. Appl. 42(12), 5065–5075 (2015)CrossRef
25.
Frénay, B., Kabán, A., et al.: A comprehensive introduction to label noise. In: ESANN. Citeseer (2014)
26.
Frénay, B., Verleysen, M.: Classification in the presence of label noise: a survey. IEEE Trans. Neural Netw. Learn. Syst. 25(5), 845–869 (2013)CrossRef
27.
Fu, T., Chung, F., Ng, C.: Financial time series segmentation based on specialized binary tree representation. In: DMIN 2006, pp. 26–29 (2006)
28.
Ghoniem, M., Shurkhovetskyy, G., Bahey, A., Otjacques, B.: VAFLE: visual analytics of firewall log events. In: Visualization and Data Analysis 2014, vol. 9017, p. 901704. International Society for Optics and Photonics (2014)
29.
Gschwandtner, T., et al..: Timecleanser: a visual analytics approach for data cleansing of time-oriented data. In: Proceedings of the 14th International Conference on Knowledge Technologies and Data-Driven Business, pp. 1–8 (2014)
30.
Guan, D., Yuan, W.: A survey of mislabeled training data detection techniques for pattern classification. IETE Tech. Rev. 30(6), 524–530 (2013)CrossRef
31.
Guan, D., Yuan, W., Ma, T., Lee, S.: Detecting potential labeling errors for bioinformatics by multiple voting. Knowl. Based Syst. 66, 28–35 (2014)CrossRef
32.
Hinton, G., Roweis, S.T.: Stochastic neighbor embedding. In: NIPS, vol. 15, pp. 833–840. Citeseer (2002)
33.
Höppner, F.: Time series abstraction methods-a survey. Informatik bewegt: Informatik 2002–32. Jahrestagung der Gesellschaft für Informatik ev (GI) (2002)
34.
Jaromczyk, J.W., Toussaint, G.T.: Relative neighborhood graphs and their relatives. Proc. IEEE 80(9), 1502–1517 (1992)CrossRef
35.
36.
Khoshgoftaar, T.M., Van Hulse, J., Napolitano, A.: Comparing boosting and bagging techniques with noisy and imbalanced data. IEEE Trans. Syst. Man Cybern. Part A Syst. Hum. 41(3), 552–568 (2010)CrossRef
37.
Kramer, M.A.: Nonlinear principal component analysis using autoassociative neural networks. AIChE J. 37(2), 233–243 (1991)CrossRef
38.
Li, Y., Cui, W.: Identifying the mislabeled training samples of ECG signals using machine learning. Biomed. Signal Process. Control 47, 168–176 (2019)CrossRef
39.
Müller, N.M., Markert, K.: Identifying mislabeled instances in classification datasets. In: 2019 International Joint Conference on Neural Networks (IJCNN), pp. 1–8. IEEE (2019)
40.
Nettleton, D.F., Orriols-Puig, A., Fornells, A.: A study of the effect of different types of noise on the precision of supervised learning techniques. Artif. Intell. Rev. 33(4), 275–306 (2010)CrossRef
41.
Nicholson, B., Zhang, J., Sheng, V.S., Wang, Z.: Label noise correction methods. In: 2015 IEEE International Conference on Data Science and Advanced Analytics (DSAA), pp. 1–9. IEEE (2015)
42.
Parzen, E., et al.: An approach to time series analysis. Annals of Math. Stat. 32(4), 951–989 (1961)CrossRef
43.
Pechenizkiy, M., Tsymbal, A., Puuronen, S., Pechenizkiy, O.: Class noise and supervised learning in medical domains: the effect of feature extraction. In: 19th IEEE Symposium on Computer-Based Medical Systems, CBMS 2006, pp. 708–713. IEEE (2006)
44.
Rädsch, T., Eckhardt, S., Leiser, F., Pandl, K.D., Thiebes, S., Sunyaev, A.: What your radiologist might be missing: using machine learning to identify mislabeled instances of x-ray images. In: Proceedings of the 54th Hawaii International Conference on System Sciences (HICSS)
45.
Sánchez, J.S., Pla, F., Ferri, F.J.: Prototype selection for the nearest neighbour rule through proximity graphs. Pattern Recogn. Lett. 18(6), 507–513 (1997)CrossRef
46.
Schafer, J.L., Graham, J.W.: Missing data: our view of the state of the art. Psychol. Meth. 7(2), 147 (2002)CrossRef
47.
Schapire, R.E., Freund, Y., Bartlett, P., Lee, W.S., et al.: Boosting the margin: a new explanation for the effectiveness of voting methods. Ann. Stat. 26(5), 1651–1686 (1998)MathSciNetMATH
48.
Sheng, V.S., Provost, F., Ipeirotis, P.G.: Get another label? Improving data quality and data mining using multiple, noisy labelers. In: Proceedings of the 14th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, pp. 614–622 (2008)
49.
Shurkhovetskyy, G., Andrienko, N., Andrienko, G., Fuchs, G.: Data abstraction for visualizing large time series. In: Computer Graphics Forum, vol. 37, pp. 125–144. Wiley Online Library (2018)
50.
Silva, S.F., Catarci, T.: Visualization of linear time-oriented data: a survey. In: Proceedings of the 1st International Conference on Web Information Systems Engineering, vol. 1, pp. 310–319. IEEE (2000)
51.
Steiger, M., et al.: Visual analysis of time-series similarities for anomaly detection in sensor networks. In: Computer Graphics Forum, vol. 33, pp. 401–410. Wiley Online Library (2014)
52.
53.
Teng, C.M.: Correcting noisy data. In: ICML, pp. 239–248. Citeseer (1999)
54.
Thulasidasan, S., Bhattacharya, T., Bilmes, J., Chennupati, G., Mohd-Yusof, J.: Combating label noise in deep learning using abstention. arXiv preprint arXiv:​1905.​10964 (2019)
55.
Tomek, I., et al.: An experiment with the edited nearest-nieghbor rule (1976)
56.
Tüceryan, M., Chorzempa, T.: Relative sensitivity of a family of closest-point graphs in computer vision applications. Pattern Recogn. 24(5), 361–373 (1991)CrossRef
57.
Venkataraman, S., Metaxas, D., Fradkin, D., Kulikowski, C., Muchnik, I.: Distinguishing mislabeled data from correctly labeled data in classifier design. In: 16th IEEE International Conference on Tools with Artificial Intelligence, pp. 668–672. IEEE (2004)
58.
Wilson, D.L.: Asymptotic properties of nearest neighbor rules using edited data. IEEE Trans. Syst. Man Cybern. 3, 408–421 (1972)MathSciNetCrossRef
59.
Xiao, T., Xia, T., Yang, Y., Huang, C., Wang, X.: Learning from massive noisy labeled data for image classification. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 2691–2699 (2015)
60.
Xu, Z., Zhang, R., Kotagiri, R., Parampalli, U.: An adaptive algorithm for online time series segmentation with error bound guarantee. In: Proceedings of the 15th International Conference on Extending Database Technology, pp. 192–203 (2012)
61.
Yang, K., Shahabi, C.: A PCA-based similarity measure for multivariate time series. In: Proceedings of the 2nd ACM International Workshop on Multimedia Databases, pp. 65–74 (2004)
62.
Yuan, Y., Xun, G., Suo, Q., Jia, K., Zhang, A.: Wave2Vec: learning deep representations for biosignals. In: 2017 IEEE International Conference on Data Mining (ICDM), pp. 1159–1164. IEEE (2017)
63.
Yuan, Y., Xun, G., Suo, Q., Jia, K., Zhang, A.: Wave2Vec: deep representation learning for clinical temporal data. Neurocomputing 324, 31–42 (2019)CrossRef
64.
Zeng, X., Martinez, T.R.: An algorithm for correcting mislabeled data. Intell. Data Anal. 5(6), 491–502 (2001)CrossRef
65.
Zeni, M., Zhang, W., Bignotti, E., Passerini, A., Giunchiglia, F.: Fixing mislabeling by human annotators leveraging conflict resolution and prior knowledge. In: Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies, vol. 3, no. 1, pp. 1–23 (2019)
66.
Zhang, H., Ho, T.B., Zhang, Y., Lin, M.S.: Unsupervised feature extraction for time series clustering using orthogonal wavelet transform. Informatica 30(3), 305–319 (2006)MathSciNetMATH
67.
Zhang, Z., Jiang, J., Wang, H.: A new segmentation algorithm to stock time series based on pip approach. In: 2007 International Conference on Wireless Communications, Networking and Mobile Computing, pp. 5609–5612. IEEE (2007)
68.
Zhao, L., Sukthankar, G., Sukthankar, R.: Incremental relabeling for active learning with noisy crowdsourced annotations. In: 2011 IEEE 3rd International Conference on Privacy, Security, Risk and Trust and 2011 IEEE 3rd International Conference on Social Computing, pp. 728–733. IEEE (2011)
69.
Zhu, X., Wu, X.: Class noise vs. attribute noise: a quantitative study. Artif. Intell. Rev. 22(3), 177–210 (2004)CrossRef
70.
Zhu, X., Wu, X., Chen, Q.: Eliminating class noise in large datasets. In: Proceedings of the 20th International Conference on Machine Learning, ICML 2003, pp. 920–927 (2003)
71.
Zhu, X., Zhang, P., Lin, X., Shi, Y.: Active learning from stream data using optimal weight classifier ensemble. IEEE Trans. Syst. Man Cybern. Part B (Cybern.) 40(6), 1607–1621 (2010)CrossRef
Metadata
Title
A Survey of Methods for Detection and Correction of Noisy Labels in Time Series Data
Authors
Gentry Atkinson
Vangelis Metsis
Copyright Year
2021
Book
Artificial Intelligence Applications and Innovations

Print ISBN: 978-3-030-79149-0

Electronic ISBN: 978-3-030-79150-6

Copyright Year: 2021

DOI
https://doi.org/10.1007/978-3-030-79150-6_38

Premium Partner

    Image Credits