Skip to main content
Erschienen in: Group Decision and Negotiation 6/2019

20.09.2019

Lie on the Fly: Strategic Voting in an Iterative Preference Elicitation Process

verfasst von: Lihi Dery, Svetlana Obraztsova, Zinovi Rabinovich, Meir Kalech

Erschienen in: Group Decision and Negotiation | Ausgabe 6/2019

Einloggen

Aktivieren Sie unsere intelligente Suche, um passende Fachinhalte oder Patente zu finden.

search-config
loading …

Abstract

A voting center is in charge of collecting and aggregating voter preferences. In an iterative process, the center sends comparison queries to voters, requesting them to submit their preference between two items. Voters might discuss the candidates among themselves, figuring out during the elicitation process which candidates stand a chance of winning and which do not. Consequently, strategic voters might attempt to manipulate by deviating from their true preferences and instead submit a different response in order to attempt to maximize their profit. We provide a practical algorithm for strategic voters which computes the best manipulative vote and maximizes the voter’s selfish outcome when such a vote exists. We also provide a careful voting center which is aware of the possible manipulations and avoids manipulative queries when possible. In an empirical study on four real world domains, we show that in practice manipulation occurs in a low percentage of settings and has a low impact on the final outcome. The careful voting center reduces manipulation even further, thus allowing for a non-distorted group decision process to take place.We thus provide a core technology study of a voting process that can be adopted in opinion or information aggregation systems and in crowdsourcing applications, e.g., peer grading in massive open online courses.

Sie haben noch keine Lizenz? Dann Informieren Sie sich jetzt über unsere Produkte:

Springer Professional "Wirtschaft+Technik"

Online-Abonnement

Mit Springer Professional "Wirtschaft+Technik" erhalten Sie Zugriff auf:

  • über 102.000 Bücher
  • über 537 Zeitschriften

aus folgenden Fachgebieten:

  • Automobil + Motoren
  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Elektrotechnik + Elektronik
  • Energie + Nachhaltigkeit
  • Finance + Banking
  • Management + Führung
  • Marketing + Vertrieb
  • Maschinenbau + Werkstoffe
  • Versicherung + Risiko

Jetzt Wissensvorsprung sichern!

Springer Professional "Wirtschaft"

Online-Abonnement

Mit Springer Professional "Wirtschaft" erhalten Sie Zugriff auf:

  • über 67.000 Bücher
  • über 340 Zeitschriften

aus folgenden Fachgebieten:

  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Finance + Banking
  • Management + Führung
  • Marketing + Vertrieb
  • Versicherung + Risiko




Jetzt Wissensvorsprung sichern!

Anhänge
Nur mit Berechtigung zugänglich
Fußnoten
1
Notice that \(CL({{\mathbf {Q}}})\) is a composition of \(CL(Q_i)\), i.e. transitive closures of individual voter response sets.
 
2
Notice that such a pair always exists.
 
Literatur
Zurück zum Zitat Aziz H, Brill M, Fischer F, Harrenstein P, Lang J, Seedig HG (2015) Possible and necessary winners of partial tournaments. J Artif Intell Res 54:493–534CrossRef Aziz H, Brill M, Fischer F, Harrenstein P, Lang J, Seedig HG (2015) Possible and necessary winners of partial tournaments. J Artif Intell Res 54:493–534CrossRef
Zurück zum Zitat Aziz H, Lev O, Mattei N, Rosenschein JS, Walsh T (2016) Strategyproof peer selection: mechanisms, analyses, and experiments. In: AAAI, pp 397–403 Aziz H, Lev O, Mattei N, Rosenschein JS, Walsh T (2016) Strategyproof peer selection: mechanisms, analyses, and experiments. In: AAAI, pp 397–403
Zurück zum Zitat Balakrishnan S, Chopra S (2012) Two of a kind or the ratings game? Adaptive pairwise preferences and latent factor models. Front Comput Sci 6(2):197–208 Balakrishnan S, Chopra S (2012) Two of a kind or the ratings game? Adaptive pairwise preferences and latent factor models. Front Comput Sci 6(2):197–208
Zurück zum Zitat Bannikova M, Naamani-Dery L, Obraztsova S, Rabinovich Z, Rosenschein JS (2016) Between fairness and a mistrial: consensus under a deadline. In: The 10th workshop on advances in preference handling (MPref), at the 25th international joint conference on artificial intelligence (IJCAI-2016) Bannikova M, Naamani-Dery L, Obraztsova S, Rabinovich Z, Rosenschein JS (2016) Between fairness and a mistrial: consensus under a deadline. In: The 10th workshop on advances in preference handling (MPref), at the 25th international joint conference on artificial intelligence (IJCAI-2016)
Zurück zum Zitat Betzler N, Hemmann S, Niedermeier R (2009) A multivariate complexity analysis of determining possible winners given incomplete votes. In: Proceedings of the international joint conference on artifical intelligence (IJCAI), vol 9, pp 53–58 Betzler N, Hemmann S, Niedermeier R (2009) A multivariate complexity analysis of determining possible winners given incomplete votes. In: Proceedings of the international joint conference on artifical intelligence (IJCAI), vol 9, pp 53–58
Zurück zum Zitat Branzei S, Caragiannis I, Morgenstern J, Procaccia AD (2013) How bad is selfish voting? In: Proceedings of the AAAI conference on artificial intelligence, pp 138–144 Branzei S, Caragiannis I, Morgenstern J, Procaccia AD (2013) How bad is selfish voting? In: Proceedings of the AAAI conference on artificial intelligence, pp 138–144
Zurück zum Zitat Bredereck R, Faliszewski P, Niedermeier R, Talmon N (2016) Large-scale election campaigns: combinatorial shift bribery. J Artif Intell Res 55:603–652CrossRef Bredereck R, Faliszewski P, Niedermeier R, Talmon N (2016) Large-scale election campaigns: combinatorial shift bribery. J Artif Intell Res 55:603–652CrossRef
Zurück zum Zitat Brewka G (1991) Nonmonotonic reasoning: logical foundations of commonsense. Cambridge University Press, Cambridge Brewka G (1991) Nonmonotonic reasoning: logical foundations of commonsense. Cambridge University Press, Cambridge
Zurück zum Zitat Brewka G, Dix J, Konolige K (1997) Non-monotonic reasoning: an overview. CSLI Publications, Stanford Brewka G, Dix J, Konolige K (1997) Non-monotonic reasoning: an overview. CSLI Publications, Stanford
Zurück zum Zitat Brill M, Conitzer V (2015) Strategic voting and strategic candidacy. In: Proceedings of the AAAI conference on artificial intelligence, vol 15, pp 819–826 Brill M, Conitzer V (2015) Strategic voting and strategic candidacy. In: Proceedings of the AAAI conference on artificial intelligence, vol 15, pp 819–826
Zurück zum Zitat Chen YL, Cheng LC, Huang PH (2013) Mining consensus preference graphs from users’ ranking data. Decis Support Syst 54(2):1055–1064CrossRef Chen YL, Cheng LC, Huang PH (2013) Mining consensus preference graphs from users’ ranking data. Decis Support Syst 54(2):1055–1064CrossRef
Zurück zum Zitat Conitzer V, Walsh T, Xia L (2011) Dominating manipulations in voting with partial information. In: Proceedings of the AAAI conference on artificial intelligence, pp 638–643 Conitzer V, Walsh T, Xia L (2011) Dominating manipulations in voting with partial information. In: Proceedings of the AAAI conference on artificial intelligence, pp 638–643
Zurück zum Zitat Damart S, Dias LC, Mousseau V (2007) Supporting groups in sorting decisions: methodology and use of a multi-criteria aggregation/disaggregation DSS. Decis Support Syst 43(4):1464–1475CrossRef Damart S, Dias LC, Mousseau V (2007) Supporting groups in sorting decisions: methodology and use of a multi-criteria aggregation/disaggregation DSS. Decis Support Syst 43(4):1464–1475CrossRef
Zurück zum Zitat Dasgupta A, Ghosh A (2013) Crowdsourced judgement elicitation with endogenous proficiency. In: Proceedings of the 22nd international conference on World Wide Web. ACM, New York, pp 319–330 Dasgupta A, Ghosh A (2013) Crowdsourced judgement elicitation with endogenous proficiency. In: Proceedings of the 22nd international conference on World Wide Web. ACM, New York, pp 319–330
Zurück zum Zitat de Borda JC (1781) Mémoire sur les élections au scrutin. Histoire de l’Academie Royale des Sciences de Borda JC (1781) Mémoire sur les élections au scrutin. Histoire de l’Academie Royale des Sciences
Zurück zum Zitat Ding N, Lin F (2013) Voting with partial information: What questions to ask? In: Proceedings of the 12th international conference on autonomous agents and multiagent systems (AAMAS), pp 1237–1238 Ding N, Lin F (2013) Voting with partial information: What questions to ask? In: Proceedings of the 12th international conference on autonomous agents and multiagent systems (AAMAS), pp 1237–1238
Zurück zum Zitat Endriss U, Obraztsova S, Polukarov M, Rosenschein JS (2016) Strategic voting with incomplete information. In: Proceedings of the international joint conference on artifical intelligence (IJCAI), pp 236–242 Endriss U, Obraztsova S, Polukarov M, Rosenschein JS (2016) Strategic voting with incomplete information. In: Proceedings of the international joint conference on artifical intelligence (IJCAI), pp 236–242
Zurück zum Zitat Farquharson R (1969) Theory of voting. Yale University Press, Yale Farquharson R (1969) Theory of voting. Yale University Press, Yale
Zurück zum Zitat Gibbard A (1973) Manipulation of voting schemes. Econometrica 41(4):587–602CrossRef Gibbard A (1973) Manipulation of voting schemes. Econometrica 41(4):587–602CrossRef
Zurück zum Zitat Grandi U, Loreggia A, Rossi F, Venable KB, Walsh T (2013) Restricted manipulation in iterative voting: condorcet efficiency and Borda score. In: International conference on algorithmic decision theory (ADT). Springer, Berlin, pp 181–192CrossRef Grandi U, Loreggia A, Rossi F, Venable KB, Walsh T (2013) Restricted manipulation in iterative voting: condorcet efficiency and Borda score. In: International conference on algorithmic decision theory (ADT). Springer, Berlin, pp 181–192CrossRef
Zurück zum Zitat Jiang J, An B, Jiang Y, Lin D (2017) Context-aware reliable crowdsourcing in social networks. IEEE Trans Syst Man Cybern Syst 99:1–16CrossRef Jiang J, An B, Jiang Y, Lin D (2017) Context-aware reliable crowdsourcing in social networks. IEEE Trans Syst Man Cybern Syst 99:1–16CrossRef
Zurück zum Zitat Kamishima T, Kazawa H, Akaho S (2005) Supervised ordering—an empirical survey. In: Proceedings of the 5th IEEE international conference on data mining (ICDM). IEEE Computer Society, New York, pp 673–676 Kamishima T, Kazawa H, Akaho S (2005) Supervised ordering—an empirical survey. In: Proceedings of the 5th IEEE international conference on data mining (ICDM). IEEE Computer Society, New York, pp 673–676
Zurück zum Zitat Kendall MG (1938) A new measure of rank correlation. Biometrika 30(1–2):81–93CrossRef Kendall MG (1938) A new measure of rank correlation. Biometrika 30(1–2):81–93CrossRef
Zurück zum Zitat Konczak K, Lang J (2005) Voting procedures with incomplete preferences. In: Proceeding of the IJCAI-05 multidisciplinary workshop on advances in preference handling, vol 20, Citeseer Konczak K, Lang J (2005) Voting procedures with incomplete preferences. In: Proceeding of the IJCAI-05 multidisciplinary workshop on advances in preference handling, vol 20, Citeseer
Zurück zum Zitat Kukushkin NS (2011) Acyclicity of improvements in finite game forms. Int J Game Theory 40(1):147–177 CrossRef Kukushkin NS (2011) Acyclicity of improvements in finite game forms. Int J Game Theory 40(1):147–177 CrossRef
Zurück zum Zitat Laffont JJ (1987) Incentives and the allocation of public goods. In: Auerbach AJ, Feldstein M (eds) Handbook of public economics, vol 2. Elsevier, London, pp 537–569 (Chap 10) Laffont JJ (1987) Incentives and the allocation of public goods. In: Auerbach AJ, Feldstein M (eds) Handbook of public economics, vol 2. Elsevier, London, pp 537–569 (Chap 10)
Zurück zum Zitat Lev O, Rosenschein JS (2012) Convergence of iterative voting. In: Proceedings of the 11th international conference on autonomous agents and multiagent systems-volume 2, international foundation for autonomous agents and multiagent systems, pp 611–618 Lev O, Rosenschein JS (2012) Convergence of iterative voting. In: Proceedings of the 11th international conference on autonomous agents and multiagent systems-volume 2, international foundation for autonomous agents and multiagent systems, pp 611–618
Zurück zum Zitat Lev O, Rosenschein JS (2016) Convergence of iterative scoring rules. J Artif Intell Res 57:573–591 CrossRef Lev O, Rosenschein JS (2016) Convergence of iterative scoring rules. J Artif Intell Res 57:573–591 CrossRef
Zurück zum Zitat Lu T, Boutilier C (2011) Robust approximation and incremental elicitation in voting protocols. In: Proceedings of the international joint conference on artifical intelligence (IJCAI), pp 287–293 Lu T, Boutilier C (2011) Robust approximation and incremental elicitation in voting protocols. In: Proceedings of the international joint conference on artifical intelligence (IJCAI), pp 287–293
Zurück zum Zitat Lu T, Boutilier C (2013) Multi-winner social choice with incomplete preferences. In: Proceedings of the international joint conference on artifical intelligence (IJCAI), pp 263–270 Lu T, Boutilier C (2013) Multi-winner social choice with incomplete preferences. In: Proceedings of the international joint conference on artifical intelligence (IJCAI), pp 263–270
Zurück zum Zitat Mattei N, Walsh T (2013) Preflib: A library of preference data http://preflib.org. In: Proceedings of the 3rd international conference on algorithmic decision theory (ADT 2013). Lecture notes in artificial intelligence. Springer, BerlinCrossRef Mattei N, Walsh T (2013) Preflib: A library of preference data http://preflib.org. In: Proceedings of the 3rd international conference on algorithmic decision theory (ADT 2013). Lecture notes in artificial intelligence. Springer, BerlinCrossRef
Zurück zum Zitat Meir R, Polukarov M, Rosenschein JS, Jennings NR (2010) Convergence to equilibria of plurality voting. In: Proceedings of the AAAI conference on artificial intelligence, pp 823–828 Meir R, Polukarov M, Rosenschein JS, Jennings NR (2010) Convergence to equilibria of plurality voting. In: Proceedings of the AAAI conference on artificial intelligence, pp 823–828
Zurück zum Zitat Meir R, Lev O, Rosenschein JS (2014) A local-dominance theory of voting equilibria. In: Proceedings of the 15th ACM EC, pp 313–330 Meir R, Lev O, Rosenschein JS (2014) A local-dominance theory of voting equilibria. In: Proceedings of the 15th ACM EC, pp 313–330
Zurück zum Zitat Miller GA (1956) The magical number seven, plus or minus two: some limits on our capacity for processing information. Psychol Rev 63(2):81CrossRef Miller GA (1956) The magical number seven, plus or minus two: some limits on our capacity for processing information. Psychol Rev 63(2):81CrossRef
Zurück zum Zitat Naamani-Dery L, Golan I, Kalech M, Rokach L (2014a) Preference elicitation for group decisions. In: Group decision and negotiation conference 2014, Toulouse, France, June 10–13, 2014. Proceedings, pp 193–200 Naamani-Dery L, Golan I, Kalech M, Rokach L (2014a) Preference elicitation for group decisions. In: Group decision and negotiation conference 2014, Toulouse, France, June 10–13, 2014. Proceedings, pp 193–200
Zurück zum Zitat Naamani-Dery L, Golan I, Kalech M, Rokach L (2015a) Preference elicitation for group decisions using the borda voting rule. Group Decis Negot 41(6):1015–1033CrossRef Naamani-Dery L, Golan I, Kalech M, Rokach L (2015a) Preference elicitation for group decisions using the borda voting rule. Group Decis Negot 41(6):1015–1033CrossRef
Zurück zum Zitat Naamani-Dery L, Obraztsova S, Rabinovich Z, Kalech M (2015b) Lie on the fly: iterative voting center with manipulative voters. In: Yang Q, Wooldridge M (eds) Proceedings of the 24th international joint conference on artificial intelligence, IJCAI 2015, Buenos Aires, Argentina, July 25–31, 2015. AAAI Press, New York, pp 2033–2039. http://ijcai.org/Abstract/15/288 Naamani-Dery L, Obraztsova S, Rabinovich Z, Kalech M (2015b) Lie on the fly: iterative voting center with manipulative voters. In: Yang Q, Wooldridge M (eds) Proceedings of the 24th international joint conference on artificial intelligence, IJCAI 2015, Buenos Aires, Argentina, July 25–31, 2015. AAAI Press, New York, pp 2033–2039. http://​ijcai.​org/​Abstract/​15/​288
Zurück zum Zitat Naamani-Dery L, Kalech M, Rokach L, Shapira B (2016) Reducing preference elicitation in group decision making. Expert Syst Appl 61:246–261CrossRef Naamani-Dery L, Kalech M, Rokach L, Shapira B (2016) Reducing preference elicitation in group decision making. Expert Syst Appl 61:246–261CrossRef
Zurück zum Zitat Obraztsova S, Rabinovich Z, Madunts A (2014) Faustian dynamics in Sarkar’s social cycle. In: Proceedings of the 21st European conference on artificial intelligence (ECAI-2014), pp 1071–1072 Obraztsova S, Rabinovich Z, Madunts A (2014) Faustian dynamics in Sarkar’s social cycle. In: Proceedings of the 21st European conference on artificial intelligence (ECAI-2014), pp 1071–1072
Zurück zum Zitat Obraztsova S, Elkind E, Polukarov M, Rabinovich Z (2015a) Strategic candidacy games with lazy candidates. In: Proceedings of the 24th international joint conference on artificial intelligence (IJCAI-2015), pp 610–616 Obraztsova S, Elkind E, Polukarov M, Rabinovich Z (2015a) Strategic candidacy games with lazy candidates. In: Proceedings of the 24th international joint conference on artificial intelligence (IJCAI-2015), pp 610–616
Zurück zum Zitat Obraztsova S, Markakis E, Polukarov M, Rabinovich Z, Jennings NR (2015b) On the convergence of iterative voting: How restrictive should restricted dynamics be? In: Proceedings of the AAAI conference on artificial intelligence, pp 993–999 Obraztsova S, Markakis E, Polukarov M, Rabinovich Z, Jennings NR (2015b) On the convergence of iterative voting: How restrictive should restricted dynamics be? In: Proceedings of the AAAI conference on artificial intelligence, pp 993–999
Zurück zum Zitat Obraztsova S, Lev O, Polukarov M, Rabinovich Z, Rosenschein JS (2016) Non-myopic voting dynamics: an optimistic approach. In: The 10th workshop on advances in preference handling (MPref), at the 25th international joint conference on artificial intelligence (IJCAI-2016) Obraztsova S, Lev O, Polukarov M, Rabinovich Z, Rosenschein JS (2016) Non-myopic voting dynamics: an optimistic approach. In: The 10th workshop on advances in preference handling (MPref), at the 25th international joint conference on artificial intelligence (IJCAI-2016)
Zurück zum Zitat Pini MS, Rossi F, Venable KB, Walsh T (2007) Incompleteness and incomparability in preference aggregation. In: Proceedings of the international joint conference on artifical intelligence (IJCAI), pp 1464–1469 Pini MS, Rossi F, Venable KB, Walsh T (2007) Incompleteness and incomparability in preference aggregation. In: Proceedings of the international joint conference on artifical intelligence (IJCAI), pp 1464–1469
Zurück zum Zitat Polukarov M, Obraztsova S, Rabinovich Z, Kruglyi A, Jennings NR (2015) Convergence to equilibria in strategic candidacy. In: Proceedings of the 24th international joint conference on artificial intelligence (IJCAI-2015), pp 624–630 Polukarov M, Obraztsova S, Rabinovich Z, Kruglyi A, Jennings NR (2015) Convergence to equilibria in strategic candidacy. In: Proceedings of the 24th international joint conference on artificial intelligence (IJCAI-2015), pp 624–630
Zurück zum Zitat Rabinovich Z, Obraztsova S, Lev O, Markakis E, Rosenschein JS (2015) Analysis of equilibria in iterative voting schemes. In: Proceedings of the AAAI conference on artificial intelligence, pp 1007–1013 Rabinovich Z, Obraztsova S, Lev O, Markakis E, Rosenschein JS (2015) Analysis of equilibria in iterative voting schemes. In: Proceedings of the AAAI conference on artificial intelligence, pp 1007–1013
Zurück zum Zitat Reijngoud A, Endriss U (2012) Voter response to iterated poll information. In: Proceedings of the 11th international conference on autonomous agents and multiagent systems (AAMAS), pp 635–644 Reijngoud A, Endriss U (2012) Voter response to iterated poll information. In: Proceedings of the 11th international conference on autonomous agents and multiagent systems (AAMAS), pp 635–644
Zurück zum Zitat Reyhani R, Wilson M (2012) Best reply dynamics for scoring rules. In: Proceedings of the European conference on artificial intelligence (ECAI), pp 672–677 Reyhani R, Wilson M (2012) Best reply dynamics for scoring rules. In: Proceedings of the European conference on artificial intelligence (ECAI), pp 672–677
Zurück zum Zitat Roughgarden, T., & Schrijvers, O. (2017, December). Online prediction with selfish experts. In Proceedings of the 31st international conference on neural information processing systems, Curran Associates Inc., pp 1300–1310 Roughgarden, T., & Schrijvers, O. (2017, December). Online prediction with selfish experts. In Proceedings of the 31st international conference on neural information processing systems, Curran Associates Inc., pp 1300–1310
Zurück zum Zitat Satterthwaite MA (1975) Strategy-proofness and Arrow’s conditions: existence and correspondence theorems for voting procedures and social welfare functions. J Econ Theory 10(2):187–217CrossRef Satterthwaite MA (1975) Strategy-proofness and Arrow’s conditions: existence and correspondence theorems for voting procedures and social welfare functions. J Econ Theory 10(2):187–217CrossRef
Zurück zum Zitat Walsh T (2007) Uncertainty in preference elicitation and aggregation. In: Proceedings of the AAAI conference on artificial intelligence, pp 3–8 Walsh T (2007) Uncertainty in preference elicitation and aggregation. In: Proceedings of the AAAI conference on artificial intelligence, pp 3–8
Zurück zum Zitat Walsh T (2011) Where are the hard manipulation problems? J Artif Intell Res 42:1–29 Walsh T (2011) Where are the hard manipulation problems? J Artif Intell Res 42:1–29
Zurück zum Zitat Xia L, Conitzer V (2011) A maximum likelihood approach towards aggregating partial orders. In: Proceedings of the international joint conference on artifical intelligence (IJCAI), pp 446–451 Xia L, Conitzer V (2011) A maximum likelihood approach towards aggregating partial orders. In: Proceedings of the international joint conference on artifical intelligence (IJCAI), pp 446–451
Zurück zum Zitat Yuen MC, King I, Leung KS (2011) A survey of crowdsourcing systems. In: Privacy, security, risk and trust (PASSAT) and 2011 IEEE 3rd international conference on social computing (SocialCom), 2011 IEEE 3rd international conference on. IEEE, New York, pp 766–773 Yuen MC, King I, Leung KS (2011) A survey of crowdsourcing systems. In: Privacy, security, risk and trust (PASSAT) and 2011 IEEE 3rd international conference on social computing (SocialCom), 2011 IEEE 3rd international conference on. IEEE, New York, pp 766–773
Zurück zum Zitat Zou J, Meir R, Parkes D (2015) Strategic voting behavior in Doodle polls. In: Proceedings of the 18th ACM conference on computer-supported cooperative work and social computing (CSCW), pp 464–472 Zou J, Meir R, Parkes D (2015) Strategic voting behavior in Doodle polls. In: Proceedings of the 18th ACM conference on computer-supported cooperative work and social computing (CSCW), pp 464–472
Metadaten
Titel
Lie on the Fly: Strategic Voting in an Iterative Preference Elicitation Process
verfasst von
Lihi Dery
Svetlana Obraztsova
Zinovi Rabinovich
Meir Kalech
Publikationsdatum
20.09.2019
Verlag
Springer Netherlands
Erschienen in
Group Decision and Negotiation / Ausgabe 6/2019
Print ISSN: 0926-2644
Elektronische ISSN: 1572-9907
DOI
https://doi.org/10.1007/s10726-019-09637-2

Weitere Artikel der Ausgabe 6/2019

Group Decision and Negotiation 6/2019 Zur Ausgabe

Premium Partner