Skip to main content

2017 | OriginalPaper | Buchkapitel

Auction Based Mechanisms for Dynamic Task Assignments in Expert Crowdsourcing

Aktivieren Sie unsere intelligente Suche, um passende Fachinhalte oder Patente zu finden.

search-config
loading …

Abstract

Crowdsourcing marketplaces link large populations of workers to an even larger number of tasks. Thus, it is necessary to have mechanisms for matching workers with interesting and suitable tasks. Earlier work has addressed the problem of finding optimal workers for a given set of tasks. However, workers also have preferences and will stay with a platform only if it gives them interesting tasks. We therefore analyze several matching mechanisms that take into account workers’ preferences as well. We propose that the workers pay premiums to get preferred matches and auction-based models where preferences are expressed through variations of the payment for a task. We analyze the properties of two matching different mechanisms: Split Dynamic VCG (SDV) and e-Auction. We compare both the mechanisms with Arrival Priority Serial Dictatorship (APSD) empirically for efficiency.

Sie haben noch keine Lizenz? Dann Informieren Sie sich jetzt über unsere Produkte:

Springer Professional "Wirtschaft+Technik"

Online-Abonnement

Mit Springer Professional "Wirtschaft+Technik" erhalten Sie Zugriff auf:

  • über 102.000 Bücher
  • über 537 Zeitschriften

aus folgenden Fachgebieten:

  • Automobil + Motoren
  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Elektrotechnik + Elektronik
  • Energie + Nachhaltigkeit
  • Finance + Banking
  • Management + Führung
  • Marketing + Vertrieb
  • Maschinenbau + Werkstoffe
  • Versicherung + Risiko

Jetzt Wissensvorsprung sichern!

Springer Professional "Technik"

Online-Abonnement

Mit Springer Professional "Technik" erhalten Sie Zugriff auf:

  • über 67.000 Bücher
  • über 390 Zeitschriften

aus folgenden Fachgebieten:

  • Automobil + Motoren
  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Elektrotechnik + Elektronik
  • Energie + Nachhaltigkeit
  • Maschinenbau + Werkstoffe




 

Jetzt Wissensvorsprung sichern!

Springer Professional "Wirtschaft"

Online-Abonnement

Mit Springer Professional "Wirtschaft" erhalten Sie Zugriff auf:

  • über 67.000 Bücher
  • über 340 Zeitschriften

aus folgenden Fachgebieten:

  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Finance + Banking
  • Management + Führung
  • Marketing + Vertrieb
  • Versicherung + Risiko




Jetzt Wissensvorsprung sichern!

Fußnoten
1
The workers has to be logged into the system for his availability and hence cannot report early arrival or late departure.
 
3
It should be noted that, the settings of expert crowdsourcing are different than microtasking where the workers finish the task quickly and move on to a next task immediately.
 
4
\(\mu \) takes \(\mathbf {b_j}\)s, \(arr_j,dep_j\) as inputs and produces a bipartite matching. However to simplify notation, we just refer to \(\mu \) as a bipartite matching.
 
5
Note that this is upper bound on competitive ratio.
 
7
Note that we are referring to expert crowdsourcing tasks and not the microtasks. Hence, such rewards are feasible.
 
Literatur
1.
Zurück zum Zitat Babaioff, M., Immorlica, N., Kempe, D., Kleinberg, R.: Online auctions and generalized secretary problems. ACM SIGecom Exchanges 7(2), 7 (2008)CrossRef Babaioff, M., Immorlica, N., Kempe, D., Kleinberg, R.: Online auctions and generalized secretary problems. ACM SIGecom Exchanges 7(2), 7 (2008)CrossRef
2.
Zurück zum Zitat Bhat, S., Nath, S., Zoeter, O., Gujar, S., Narahari, Y., Dance, C.: A mechanism to optimally balance cost and quality of labeling tasks outsourced to strategic agents. In: Thirtheenth International Conference on Autonomous Agents and Multiagent Systems, pp. 917–924 (2014) Bhat, S., Nath, S., Zoeter, O., Gujar, S., Narahari, Y., Dance, C.: A mechanism to optimally balance cost and quality of labeling tasks outsourced to strategic agents. In: Thirtheenth International Conference on Autonomous Agents and Multiagent Systems, pp. 917–924 (2014)
3.
Zurück zum Zitat Bigham, J.P., Jayant, C., Ji, H., Little, G., Miller, A., Miller, R.C., Miller, R., Tatarowicz, A., White, B., White, S., et al.: Vizwiz: nearly real-time answers to visual questions. In: Proceedings of the 23nd Annual ACM Symposium on User Interface Software and Technology, pp. 333–342. ACM (2010) Bigham, J.P., Jayant, C., Ji, H., Little, G., Miller, A., Miller, R.C., Miller, R., Tatarowicz, A., White, B., White, S., et al.: Vizwiz: nearly real-time answers to visual questions. In: Proceedings of the 23nd Annual ACM Symposium on User Interface Software and Technology, pp. 333–342. ACM (2010)
4.
Zurück zum Zitat Chen, X., Lin, Q., Zhou, D.: Optimistic knowledge gradient policy for optimal budget allocation in crowdsourcing. In: Proceedings of the 30th International Conference on Machine Learning (ICML-2013), pp. 64–72 (2013) Chen, X., Lin, Q., Zhou, D.: Optimistic knowledge gradient policy for optimal budget allocation in crowdsourcing. In: Proceedings of the 30th International Conference on Machine Learning (ICML-2013), pp. 64–72 (2013)
5.
Zurück zum Zitat Difallah, D.E., Demartini, G., Cudré-Mauroux, P.: Pick-a-crowd: tell me what you like, and i’ll tell you what to do. In: Proceedings of the 22nd International Conference on World Wide Web, pp. 367–374 (2013) Difallah, D.E., Demartini, G., Cudré-Mauroux, P.: Pick-a-crowd: tell me what you like, and i’ll tell you what to do. In: Proceedings of the 22nd International Conference on World Wide Web, pp. 367–374 (2013)
6.
Zurück zum Zitat Garg, D., Narahari, Y., Gujar, S.: Foundations of mechanism design: a tutorial - part 1: key concepts and classical results. In: Sadhana - Indian Academy Proceedings in Engineering Sciences, vol. 33(Part 2), pp. 83–130, April 2008 Garg, D., Narahari, Y., Gujar, S.: Foundations of mechanism design: a tutorial - part 1: key concepts and classical results. In: Sadhana - Indian Academy Proceedings in Engineering Sciences, vol. 33(Part 2), pp. 83–130, April 2008
7.
Zurück zum Zitat Garg, D., Narahari, Y., Gujar, S.: Foundations of mechanism design: a tutorial - part 2: advanced concepts and results. In: Sadhana - Indian Academy Proceedings in Engineering Sciences, vol. 33(Part 2), pp. 131–174, April 2008 Garg, D., Narahari, Y., Gujar, S.: Foundations of mechanism design: a tutorial - part 2: advanced concepts and results. In: Sadhana - Indian Academy Proceedings in Engineering Sciences, vol. 33(Part 2), pp. 131–174, April 2008
8.
Zurück zum Zitat Goel, G., Nikzad, A., Singla, A.: Allocating tasks to workers with matching constraints: truthful mechanisms for crowdsourcing markets. In: Proceedings of the Companion Publication of the 23rd International Conference on World Wide Web Companion, pp. 279–280 (2014) Goel, G., Nikzad, A., Singla, A.: Allocating tasks to workers with matching constraints: truthful mechanisms for crowdsourcing markets. In: Proceedings of the Companion Publication of the 23rd International Conference on World Wide Web Companion, pp. 279–280 (2014)
9.
Zurück zum Zitat Gujar, S., Faltings, B.: Dynamic task assignments: an online two sided matching approach. In: 3rd International Workshop on Matching Under Preferences, MATCHUP (2015) Gujar, S., Faltings, B.: Dynamic task assignments: an online two sided matching approach. In: 3rd International Workshop on Matching Under Preferences, MATCHUP (2015)
10.
Zurück zum Zitat Gujar, S., Faltings, B.: Online auctions for dynamic assignment: theory and empirical evaluation. In: ECAI 2016–22nd European Conference on Artificial Intelligence, 29 August-2 September 2016, The Hague, The Netherlands - Including Prestigious Applications of Artificial Intelligence (PAIS 2016), pp. 1035–1043 (2016) Gujar, S., Faltings, B.: Online auctions for dynamic assignment: theory and empirical evaluation. In: ECAI 2016–22nd European Conference on Artificial Intelligence, 29 August-2 September 2016, The Hague, The Netherlands - Including Prestigious Applications of Artificial Intelligence (PAIS 2016), pp. 1035–1043 (2016)
11.
Zurück zum Zitat Ho, C.J., Jabbari, S., Vaughan, J.W.: Adaptive task assignment for crowdsourced classification. In: Proceedings of the 30th International Conference on Machine Learning (ICML-2013), pp. 534–542 (2013) Ho, C.J., Jabbari, S., Vaughan, J.W.: Adaptive task assignment for crowdsourced classification. In: Proceedings of the 30th International Conference on Machine Learning (ICML-2013), pp. 534–542 (2013)
12.
Zurück zum Zitat Ho, C.J., Vaughan, J.W.: Online task assignment in crowdsourcing markets. In: AAAI (2012) Ho, C.J., Vaughan, J.W.: Online task assignment in crowdsourcing markets. In: AAAI (2012)
13.
Zurück zum Zitat Howe, J.: The rise of crowdsourcing. Wired Mag. 14(6), 1–4 (2006) Howe, J.: The rise of crowdsourcing. Wired Mag. 14(6), 1–4 (2006)
14.
Zurück zum Zitat Huang, E., Zhang, H., Parkes, D.C., Gajos, K.Z., Chen, Y.: Toward automatic task design: a progress report. In: Proceedings of the ACM SIGKDD Workshop on Human Computation, pp. 77–85. ACM (2010) Huang, E., Zhang, H., Parkes, D.C., Gajos, K.Z., Chen, Y.: Toward automatic task design: a progress report. In: Proceedings of the ACM SIGKDD Workshop on Human Computation, pp. 77–85. ACM (2010)
15.
Zurück zum Zitat Ipeirotis, P.G., Provost, F., Wang, J.: Quality management on Amazon mechanical turk. In: Proceedings of the ACM SIGKDD Workshop on Human Computation, HCOMP 2010, pp. 64–67. ACM, New York (2010) Ipeirotis, P.G., Provost, F., Wang, J.: Quality management on Amazon mechanical turk. In: Proceedings of the ACM SIGKDD Workshop on Human Computation, HCOMP 2010, pp. 64–67. ACM, New York (2010)
16.
Zurück zum Zitat Jain, S., Gujar, S., Zoeter, O., Narahari, Y.: A quality assuring multi-armed bandit crowdsourcing mechanism with incentive compatible learning. In: Thirtheenth International Conference on Autonomous Agents and Multiagent Systems, pp. 1609–1610 (2014) Jain, S., Gujar, S., Zoeter, O., Narahari, Y.: A quality assuring multi-armed bandit crowdsourcing mechanism with incentive compatible learning. In: Thirtheenth International Conference on Autonomous Agents and Multiagent Systems, pp. 1609–1610 (2014)
17.
Zurück zum Zitat Karger, D.R., Oh, S., Shah, D.: Budget-optimal crowdsourcing using low-rank matrix approximations. In: 49th Annual Allerton Conference on Communication, Control, and Computing (Allerton), pp. 284–291. IEEE (2011) Karger, D.R., Oh, S., Shah, D.: Budget-optimal crowdsourcing using low-rank matrix approximations. In: 49th Annual Allerton Conference on Communication, Control, and Computing (Allerton), pp. 284–291. IEEE (2011)
18.
Zurück zum Zitat Kittur, A., Chi, E.H., Suh, B.: Crowdsourcing user studies with mechanical turk. In: Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, CHI 2008, pp. 453–456. ACM, New York (2008) Kittur, A., Chi, E.H., Suh, B.: Crowdsourcing user studies with mechanical turk. In: Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, CHI 2008, pp. 453–456. ACM, New York (2008)
19.
Zurück zum Zitat Parkes, D.C.: Online mechanisms (2007) Parkes, D.C.: Online mechanisms (2007)
20.
Zurück zum Zitat Raykar, V.C., Yu, S., Zhao, L.H., Valadez, G.H., Florin, C., Bogoni, L., Moy, L.: Learning from crowds. J. Mach. Learn. Res. 11, 1297–1322 (2010)MathSciNet Raykar, V.C., Yu, S., Zhao, L.H., Valadez, G.H., Florin, C., Bogoni, L., Moy, L.: Learning from crowds. J. Mach. Learn. Res. 11, 1297–1322 (2010)MathSciNet
21.
Zurück zum Zitat Sönmez, T., Ünver, M.U.: Matching, allocation, and exchange of discrete resources. Handbook Soc. Econ. 1, 781–852 (2011)CrossRef Sönmez, T., Ünver, M.U.: Matching, allocation, and exchange of discrete resources. Handbook Soc. Econ. 1, 781–852 (2011)CrossRef
22.
Zurück zum Zitat Tran-Thanh, L., Stein, S., Rogers, A., Jennings, N.R.: Efficient crowdsourcing of unknown experts using multi-armed bandits. In: European Conference on Artificial Intelligence, pp. 768–773 (2012) Tran-Thanh, L., Stein, S., Rogers, A., Jennings, N.R.: Efficient crowdsourcing of unknown experts using multi-armed bandits. In: European Conference on Artificial Intelligence, pp. 768–773 (2012)
23.
Zurück zum Zitat Zou, J.Y., Gujar, S., Parkes, D.C.: Tolerable manipulability in dynamic assignment without money. In: AAAI (2010) Zou, J.Y., Gujar, S., Parkes, D.C.: Tolerable manipulability in dynamic assignment without money. In: AAAI (2010)
Metadaten
Titel
Auction Based Mechanisms for Dynamic Task Assignments in Expert Crowdsourcing
verfasst von
Sujit Gujar
Boi Faltings
Copyright-Jahr
2017
DOI
https://doi.org/10.1007/978-3-319-54229-4_4