Skip to main content
Erschienen in:
Buchtitelbild

2016 | OriginalPaper | Buchkapitel

Improving Automatic Classifiers Through Interaction

verfasst von : Silvia Acid, Luis M. de Campos

Erschienen in: Artificial Intelligence and Soft Computing

Verlag: Springer International Publishing

Aktivieren Sie unsere intelligente Suche, um passende Fachinhalte oder Patente zu finden.

search-config
loading …

Abstract

We consider a scenario where an automatic classifier has been built, but it sometimes decides to ask the correct label of an instance to an oracle, instead of accepting its own prediction. This interactive classifier only knows with certainty the labels provided by the oracle. Our proposal is to use this information to dynamically improve the behavior of the classifier, either increasing its accuracy when it is being used autonomously or reducing the number of queries to the oracle. We have tested our proposal by using twenty data sets and two adaptive classifiers from the Massive Online Analysis (MOA) open source framework for data stream mining.

Sie haben noch keine Lizenz? Dann Informieren Sie sich jetzt über unsere Produkte:

Springer Professional "Wirtschaft+Technik"

Online-Abonnement

Mit Springer Professional "Wirtschaft+Technik" erhalten Sie Zugriff auf:

  • über 102.000 Bücher
  • über 537 Zeitschriften

aus folgenden Fachgebieten:

  • Automobil + Motoren
  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Elektrotechnik + Elektronik
  • Energie + Nachhaltigkeit
  • Finance + Banking
  • Management + Führung
  • Marketing + Vertrieb
  • Maschinenbau + Werkstoffe
  • Versicherung + Risiko

Jetzt Wissensvorsprung sichern!

Springer Professional "Technik"

Online-Abonnement

Mit Springer Professional "Technik" erhalten Sie Zugriff auf:

  • über 67.000 Bücher
  • über 390 Zeitschriften

aus folgenden Fachgebieten:

  • Automobil + Motoren
  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Elektrotechnik + Elektronik
  • Energie + Nachhaltigkeit
  • Maschinenbau + Werkstoffe




 

Jetzt Wissensvorsprung sichern!

Springer Professional "Wirtschaft"

Online-Abonnement

Mit Springer Professional "Wirtschaft" erhalten Sie Zugriff auf:

  • über 67.000 Bücher
  • über 340 Zeitschriften

aus folgenden Fachgebieten:

  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Finance + Banking
  • Management + Führung
  • Marketing + Vertrieb
  • Versicherung + Risiko




Jetzt Wissensvorsprung sichern!

Fußnoten
1
The evaluation should not be based only on its predictive accuracy but should also take into account the cost of the human intervention.
 
2
In [1], it was shown that with values of \(\beta \) lesser than or equal to 0.5, the interactive classifiers systematically outperform their non-interactive counterparts.
 
3
In this paper we assume that the process is stationary and no concept drift occurs. The case of non-stationary problems, where a concept drift can modify the probability distribution, will be considered in future research.
 
4
By lack of space, we omit the proof of this result.
 
5
The new instances are supposed to come from the same data distribution.
 
6
Without taking into account those instances labeled by the oracle, \(n_i\).
 
Literatur
1.
Zurück zum Zitat Acid, S., de Campos, L.M., Fernández, M.: Evaluation methods and strategies for the interactive use of classifiers. Int. J. Hum.-Comput. Stud. 70(5), 321–331 (2012)CrossRef Acid, S., de Campos, L.M., Fernández, M.: Evaluation methods and strategies for the interactive use of classifiers. Int. J. Hum.-Comput. Stud. 70(5), 321–331 (2012)CrossRef
2.
Zurück zum Zitat Agrawal, R., Imielinski, T., Swami, A.: Database mining: a performance perspective. IEEE Trans. Knowl. Data Eng. 5(6), 914–925 (1993)CrossRef Agrawal, R., Imielinski, T., Swami, A.: Database mining: a performance perspective. IEEE Trans. Knowl. Data Eng. 5(6), 914–925 (1993)CrossRef
3.
Zurück zum Zitat Auer, P., Cesa-Bianchi, N., Gentile, C.: Adaptive and self-confident on-line learning algorithms. J. Comput. Syst. Sci. 64, 48–75 (2002)MathSciNetCrossRefMATH Auer, P., Cesa-Bianchi, N., Gentile, C.: Adaptive and self-confident on-line learning algorithms. J. Comput. Syst. Sci. 64, 48–75 (2002)MathSciNetCrossRefMATH
4.
Zurück zum Zitat Barlet, P.L., Wegkamp, M.H.: Classification with a reject option using a hinge loss. J. Mach. Learn. Res. 9, 1823–1840 (2008)MathSciNetMATH Barlet, P.L., Wegkamp, M.H.: Classification with a reject option using a hinge loss. J. Mach. Learn. Res. 9, 1823–1840 (2008)MathSciNetMATH
5.
Zurück zum Zitat Bifet, A., Holmes, G., Kirkby, R., Pfahringer, B.: MOA: massive online analysis. J. Mach. Learn. Res. 11, 1601–1604 (2010) Bifet, A., Holmes, G., Kirkby, R., Pfahringer, B.: MOA: massive online analysis. J. Mach. Learn. Res. 11, 1601–1604 (2010)
6.
Zurück zum Zitat Fu, Y., Zhu, X., Li, B.: A survey on instance selection for active lerning. Knowl. Inf. Syst. 35, 249–283 (2013)CrossRef Fu, Y., Zhu, X., Li, B.: A survey on instance selection for active lerning. Knowl. Inf. Syst. 35, 249–283 (2013)CrossRef
7.
Zurück zum Zitat Fumera, G., Roli, F., Giacinto, G.: Reject option with multiple thresholds. Pattern Recogn. 33(12), 2099–2101 (2000)CrossRefMATH Fumera, G., Roli, F., Giacinto, G.: Reject option with multiple thresholds. Pattern Recogn. 33(12), 2099–2101 (2000)CrossRefMATH
8.
Zurück zum Zitat Gama, J.: Knowledge Discovery from Data Streams. Chapman and Hall/CRC, Boca Raton (2010)CrossRefMATH Gama, J.: Knowledge Discovery from Data Streams. Chapman and Hall/CRC, Boca Raton (2010)CrossRefMATH
9.
Zurück zum Zitat Gama, J.: A survey on learning from data streams: current and future trends. Prog. Artif. Intell. 1, 45–55 (2012)CrossRef Gama, J.: A survey on learning from data streams: current and future trends. Prog. Artif. Intell. 1, 45–55 (2012)CrossRef
10.
Zurück zum Zitat Hulten, G., Spencer, L., Domingos, P.: Mining time-changing data streams. In: KDD-01, pp. 97–106. ACM Press (2001) Hulten, G., Spencer, L., Domingos, P.: Mining time-changing data streams. In: KDD-01, pp. 97–106. ACM Press (2001)
11.
Zurück zum Zitat Lewis, D., Gale, W.: A sequential algorithm for training text classifiers. In: Proceedings of the ACM SIGIR Conference, pp. 3–12 (1994) Lewis, D., Gale, W.: A sequential algorithm for training text classifiers. In: Proceedings of the ACM SIGIR Conference, pp. 3–12 (1994)
12.
Zurück zum Zitat Settles, B.: Active learning literature survey. Computer Sciences Technical Report 1648, University of Wisconsin-Madison (2009) Settles, B.: Active learning literature survey. Computer Sciences Technical Report 1648, University of Wisconsin-Madison (2009)
13.
Zurück zum Zitat Shalev-Shwartz, S.: Online learning and online convex optimization. Found. Trends Mach. Learn. 4(2), 107–194 (2012)CrossRefMATH Shalev-Shwartz, S.: Online learning and online convex optimization. Found. Trends Mach. Learn. 4(2), 107–194 (2012)CrossRefMATH
14.
Zurück zum Zitat van Rijsbergen, C.J.: Foundation of evaluation. J. Documentation 30(4), 365–373 (1974)CrossRef van Rijsbergen, C.J.: Foundation of evaluation. J. Documentation 30(4), 365–373 (1974)CrossRef
Metadaten
Titel
Improving Automatic Classifiers Through Interaction
verfasst von
Silvia Acid
Luis M. de Campos
Copyright-Jahr
2016
DOI
https://doi.org/10.1007/978-3-319-39384-1_1