Skip to main content
Disciplines
Automotive Business IT + Informatics Construction + Real Estate Electrical Engineering + Electronics Energy + Sustainability Insurance + Risk Finance + Banking Management + Leadership Marketing + Sales Mechanical Engineering + Materials
Events
DE
EN
Books
Journals
Topic Page
Marketing
Start single access now
Access for companies
EXTENDED SEARCH
Log in
Top
Published in:
Addressing Local Class Imbalance in Balanced Datasets with Dynamic Impurity Decision Trees Read chapter Read first chapter

2018 | OriginalPaper | Chapter

Preference-Based Reinforcement Learning Using Dyad Ranking

Authors : Dirk Schäfer, Eyke Hüllermeier

Published in: Discovery Science

Publisher: Springer International Publishing

Log in

Activate our intelligent search to find suitable subject content or patents.

search-config
loading …

Abstract

Preference-based reinforcement learning has recently been introduced as a generalization of conventional reinformcement learning. Instead of numerical rewards, which are often difficult to specify, the former assumes weaker feedback in the form of qualitative preferences between states or trajectories. A specific realization of preference-based reinforcement learning is approximate policy iteration using label ranking. We propose an extension of this method, in which label ranking is replaced by so-called dyad ranking. The main advantage of this extension is the ability of dyad ranking to learn from feature descriptions of actions, which are often available in reinforcement learning. Several simulation studies are conducted to confirm the usefulness of the approach.

Dont have a licence yet? Then find out more about our products and how to get one now:

Springer Professional "Wirtschaft+Technik"

Online-Abonnement

Mit Springer Professional "Wirtschaft+Technik" erhalten Sie Zugriff auf:

  • über 102.000 Bücher
  • über 537 Zeitschriften

aus folgenden Fachgebieten:

  • Automobil + Motoren
  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Elektrotechnik + Elektronik
  • Energie + Nachhaltigkeit
  • Finance + Banking
  • Management + Führung
  • Marketing + Vertrieb
  • Maschinenbau + Werkstoffe
  • Versicherung + Risiko

Jetzt Wissensvorsprung sichern!

inform now

Springer Professional "Technik"

Online-Abonnement

Mit Springer Professional "Technik" erhalten Sie Zugriff auf:

  • über 67.000 Bücher
  • über 390 Zeitschriften

aus folgenden Fachgebieten:

  • Automobil + Motoren
  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Elektrotechnik + Elektronik
  • Energie + Nachhaltigkeit
  • Maschinenbau + Werkstoffe




 

Jetzt Wissensvorsprung sichern!

inform now

Springer Professional "Wirtschaft"

Online-Abonnement

Mit Springer Professional "Wirtschaft" erhalten Sie Zugriff auf:

  • über 67.000 Bücher
  • über 340 Zeitschriften

aus folgenden Fachgebieten:

  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Finance + Banking
  • Management + Führung
  • Marketing + Vertrieb
  • Versicherung + Risiko




Jetzt Wissensvorsprung sichern!

inform now
previous chapter Predicting Rice Phenotypes with Meta-learning
next chapter COBRASTS: A New Approach to Semi-supervised Clustering of Time Series
Footnotes
1
Note that the number of actions is not fixed per rollout and rather depends on the quality of the current policy. This includes the case that rollouts can stop prematurely before the maximal trajectory length L is reached.
 
2
Throughout all experiments we used the RPC method in conjunction with logistic regression.
 
Literature
1.
Akrour, R., Schoenauer, M., Sebag, M.: Preference-based policy learning. In: Proceedings of ECML/PKDD-2011, Athens, Greece (2011)
2.
Brazdil, P., Giraud-Carrier, C.G.: Metalearning and algorithm selection: progress, state of the art and introduction to the 2018 special issue. Mach. Learn. 107(1), 1–14 (2018)MathSciNetCrossRef
3.
Cheng, W., Fürnkranz, J., Hüllermeier, E., Park, S.H.: Preference-based policy iteration: leveraging preference learning for reinforcement learning. In: Proceedings of ECML/PKDD-2011, Athens, Greece (2011)
4.
Dimitrakakis, C., Lagoudakis, M.G.: Rollout sampling approximate policy iteration. Mach. Learn. 72(3), 157–171 (2008)CrossRef
5.
Fürnkranz, J., Hüllermeier, E., Cheng, W., Park, S.H.: Preference-based reinforcement learning: a formal framework and a policy iteration algorithm. Mach. Learn. 89(1–2), 123–156 (2012)MathSciNetCrossRef
6.
Gonzalez, R.C., Woods, R.E.: Digital Image Processing, 2nd edn. Prentice Hall, Englewood Cliffs (2002)
7.
Hüllermeier, E., Fürnkranz, J., Cheng, W., Brinker, K.: Label ranking by learning pairwise preferences. Artif. Intell. 172, 1897–1917 (2008)MathSciNetCrossRef
8.
Lagoudakis, M., Parr, R.: Reinforcement learning as classification: leveraging modern classifiers. In: Proceedings of ICML, 20th International Conference on Machine Learning, vol. 20, pp. 424–431. AAAI Press (2003)
9.
Schäfer, D., Hüllermeier, E.: Plackett-Luce networks for dyad ranking. In: Workshop LWDA, Lernen, Wissen, Daten, Analysen, Potsdam, Germany (2016)
10.
11.
Schäfer, D., Hüllermeier, E.: Dyad ranking using Plackett-Luce models based on joint feature representations. Mach. Learn. (2018)
12.
Settles, B.: Active learning literature survey. Technical Report 1648, University of Wisconsin-Madison (2008)
13.
Sutton, R.S.: Learning to predict by the methods of temporal differences. Mach. Learn. 3(1), 9–44 (1988)
14.
Sutton, R.S., Barto, A.G.: Reinforcement Learning: An Introduction. MIT Press, Cambridge (1998)
15.
Tsochantaridis, I., Joachims, T., Hofmann, T., Altun, Y.: Large margin methods for structured and interdependent output variables. J. Mach. Learn. Res. 6, 1453–1484 (2005)MathSciNetMATH
16.
Vembu, S., Gärtner, T.: Label ranking: a survey. In: Fürnkranz, J., Hüllermeier, E., (eds.) Preference Learning. Springer (2010)
17.
Wang, Z., Bovik, A.C., Sheikh, H.R., Simoncelli, E.P.: Image quality assessment: from error visibility to structural similarity. IEEE Trans. Image Process. 13(4), 600–612 (2004)CrossRef
18.
Watkins, C.J., Dayan, P.: Q-learning. Mach. Learn. 8(3), 272–292 (1992)MATH
19.
Wirth, C., Akrour, R., Neumann, G., Fürnkranz, J.: A survey of preference-based reinforcement learning methods. J. Mach. Learn. Res. 18, 136:1–136:46 (2017)
20.
Xiao, H., Rasul, K., Vollgraf, R.: Fashion-MNIST: a novel image dataset for benchmarking machine learning algorithms (2017), arXiv:​1708.​07747
21.
Zhao, Y., Kosorok, M.R., Zeng, D.: Reinforcement learning design for cancer clinical trials. Stat. Med. 28(15), 1982–1998 (2009)MathSciNetCrossRef
Metadata
Title
Preference-Based Reinforcement Learning Using Dyad Ranking
Authors
Dirk Schäfer
Eyke Hüllermeier
Copyright Year
2018
Book
Discovery Science

Print ISBN: 978-3-030-01770-5

Electronic ISBN: 978-3-030-01771-2

Copyright Year: 2018

DOI
https://doi.org/10.1007/978-3-030-01771-2_11

Premium Partner

    Image Credits