Skip to main content
Erschienen in: The VLDB Journal 1/2019

26.07.2018 | Regular Paper

Optimized group formation for solving collaborative tasks

verfasst von: Habibur Rahman, Senjuti Basu Roy, Saravanan Thirumuruganathan, Sihem Amer-Yahia, Gautam Das

Erschienen in: The VLDB Journal | Ausgabe 1/2019

Einloggen

Aktivieren Sie unsere intelligente Suche, um passende Fachinhalte oder Patente zu finden.

search-config
loading …

Abstract

Many popular applications, such as collaborative document editing, sentence translation, or citizen science, resort to collaborative crowdsourcing, a special form of human-based computing, where, crowd workers with appropriate skills and expertise are required to form groups to solve complex tasks. While there has been extensive research on workers’ task assignment for traditional microtask-based crowdsourcing, they often ignore the critical aspect of collaboration. Central to any collaborative crowdsourcing process is the aspect of solving collaborative tasks that requires successful collaboration among the workers. Our formalism considers two main collaboration-related factors—affinity and upper critical mass—appropriately adapted from organizational science and social theories. Our contributions are threefold. First, we formalize the notion of collaboration among crowd workers and propose a comprehensive optimization model for task assignment in a collaborative crowdsourcing environment. Next, we study the hardness of the task assignment optimization problem and propose a series of efficient exact and approximation algorithms with provable theoretical guarantees. Finally, we present a detailed set of experimental results stemming from two real-world collaborative crowdsourcing application using Amazon Mechanical Turk.

Sie haben noch keine Lizenz? Dann Informieren Sie sich jetzt über unsere Produkte:

Springer Professional "Wirtschaft+Technik"

Online-Abonnement

Mit Springer Professional "Wirtschaft+Technik" erhalten Sie Zugriff auf:

  • über 102.000 Bücher
  • über 537 Zeitschriften

aus folgenden Fachgebieten:

  • Automobil + Motoren
  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Elektrotechnik + Elektronik
  • Energie + Nachhaltigkeit
  • Finance + Banking
  • Management + Führung
  • Marketing + Vertrieb
  • Maschinenbau + Werkstoffe
  • Versicherung + Risiko

Jetzt Wissensvorsprung sichern!

Springer Professional "Technik"

Online-Abonnement

Mit Springer Professional "Technik" erhalten Sie Zugriff auf:

  • über 67.000 Bücher
  • über 390 Zeitschriften

aus folgenden Fachgebieten:

  • Automobil + Motoren
  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Elektrotechnik + Elektronik
  • Energie + Nachhaltigkeit
  • Maschinenbau + Werkstoffe




 

Jetzt Wissensvorsprung sichern!

Springer Professional "Wirtschaft"

Online-Abonnement

Mit Springer Professional "Wirtschaft" erhalten Sie Zugriff auf:

  • über 67.000 Bücher
  • über 340 Zeitschriften

aus folgenden Fachgebieten:

  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Finance + Banking
  • Management + Führung
  • Marketing + Vertrieb
  • Versicherung + Risiko




Jetzt Wissensvorsprung sichern!

Fußnoten
1
This work is the extension of our paper [57]. We extend our previous work by providing (i) an additional technique for task assignment referred to as Cons-cost-K-ApprxGrp, (ii) detail proofs of our algorithms and (iii) additional experiments on both real and synthetic data
 
2
Notice that posing affinity as a constraint does not fully exploit the effect of “group synergy.”
 
3
Star graph is a tree on v nodes with one node having degree \(v-1\) and other \(v-1\) nodes with degree 1.
 
4
Without triangle inequality assumption, no theoretical guarantee could be ensured [59].
 
Literatur
1.
Zurück zum Zitat Ahmad, S., Battle, A., Malkani, Z., Kamvar, S.: The Jabberwocky programming environment for structured social computing. In: Proceedings of the 24th Annual ACM Symposium on User Interface Software and Technology, pp. 53–64. ACM (2011) Ahmad, S., Battle, A., Malkani, Z., Kamvar, S.: The Jabberwocky programming environment for structured social computing. In: Proceedings of the 24th Annual ACM Symposium on User Interface Software and Technology, pp. 53–64. ACM (2011)
2.
Zurück zum Zitat Amer-Yahia, S., Basu Roy, S.: From complex object exploration to complex crowdsourcing. In: Proceedings of the 24th International Conference on World Wide Web, pp. 1531–1532. ACM (2015) Amer-Yahia, S., Basu Roy, S.: From complex object exploration to complex crowdsourcing. In: Proceedings of the 24th International Conference on World Wide Web, pp. 1531–1532. ACM (2015)
3.
Zurück zum Zitat Anagnostopoulos, A., Becchetti, L., Castillo, C., Gionis, A., Leonardi, S.: Power in unity: forming teams in large-scale community systems. In: CIKM ’10 (2010) Anagnostopoulos, A., Becchetti, L., Castillo, C., Gionis, A., Leonardi, S.: Power in unity: forming teams in large-scale community systems. In: CIKM ’10 (2010)
4.
Zurück zum Zitat Anagnostopoulos, A., Becchetti, L., Castillo, C., Gionis, A., Leonardi, S.: Online team formation in social networks. In: WWW ’12 (2012) Anagnostopoulos, A., Becchetti, L., Castillo, C., Gionis, A., Leonardi, S.: Online team formation in social networks. In: WWW ’12 (2012)
5.
Zurück zum Zitat Andres, H.P.: Team cognition using collaborative technology: a behavioral analysis. J. Manage. Psychol. 28, 38–54 (2013)CrossRef Andres, H.P.: Team cognition using collaborative technology: a behavioral analysis. J. Manage. Psychol. 28, 38–54 (2013)CrossRef
6.
Zurück zum Zitat Bernstein, M.S., Little, G., Miller, R.C., Hartmann, B., Ackerman, M.S., Karger, D.R., Crowell, D., Panovich, K.: Soylent: a word processor with a crowd inside. In: Proceedings of the 23nd Annual ACM Symposium on User Interface Software and Technology, pp. 313–322. ACM (2010) Bernstein, M.S., Little, G., Miller, R.C., Hartmann, B., Ackerman, M.S., Karger, D.R., Crowell, D., Panovich, K.: Soylent: a word processor with a crowd inside. In: Proceedings of the 23nd Annual ACM Symposium on User Interface Software and Technology, pp. 313–322. ACM (2010)
7.
Zurück zum Zitat Bertsekas, D.P.: Nonlinear Programming. Athena Scientific, Belmont (1999)MATH Bertsekas, D.P.: Nonlinear Programming. Athena Scientific, Belmont (1999)MATH
8.
Zurück zum Zitat Bisschop, J.: AIMMS - optimization modeling. Lulu.com (2006). ISBN:1411698991 Bisschop, J.: AIMMS - optimization modeling. Lulu.com (2006). ISBN:1411698991
9.
Zurück zum Zitat Boudreau, K., Gaule, P., Lakhani, K.R., Riedl, C., Woolley, A.W.: From crowds to collaborators: initiating effort and catalyzing interactions among online creative workers (2014) Boudreau, K., Gaule, P., Lakhani, K.R., Riedl, C., Woolley, A.W.: From crowds to collaborators: initiating effort and catalyzing interactions among online creative workers (2014)
10.
Zurück zum Zitat Chai, K., Potdar, V., Dillon, T.: Content quality assessment related frameworks for social media. In: ICCSA(2009) Chai, K., Potdar, V., Dillon, T.: Content quality assessment related frameworks for social media. In: ICCSA(2009)
11.
Zurück zum Zitat Chen, D.L., Dolan, W.B.: Building a persistent workforce on mechanical turk for multilingual data collection. In: HCOMP (2011) Chen, D.L., Dolan, W.B.: Building a persistent workforce on mechanical turk for multilingual data collection. In: HCOMP (2011)
12.
Zurück zum Zitat Chhabra, M., Das, S., Szymanski, B.: Team formation in social networks. In: Computer and Information Sciences III, pp. 291–299. Springer (2013) Chhabra, M., Das, S., Szymanski, B.: Team formation in social networks. In: Computer and Information Sciences III, pp. 291–299. Springer (2013)
13.
Zurück zum Zitat Daniela Retelny, A.T. M.B., Robaszkiewicz, S.: Expert crowdsourcing with flash teams. In: CrowdConf 2013 Poster Daniela Retelny, A.T. M.B., Robaszkiewicz, S.: Expert crowdsourcing with flash teams. In: CrowdConf 2013 Poster
14.
Zurück zum Zitat Difallah, D.E., Catasta, M., Demartini, G., Ipeirotis, P.G., Cudré-Mauroux, P.: The dynamics of micro-task crowdsourcing: the case of Amazon MTurk. In: Proceedings of the 24th International Conference on World Wide Web, pp. 238–247. ACM (2015) Difallah, D.E., Catasta, M., Demartini, G., Ipeirotis, P.G., Cudré-Mauroux, P.: The dynamics of micro-task crowdsourcing: the case of Amazon MTurk. In: Proceedings of the 24th International Conference on World Wide Web, pp. 238–247. ACM (2015)
15.
Zurück zum Zitat Downs, J.S., Holbrook, M.B., Sheng, S., Cranor, L.F.: Are your participants gaming the system? Screening mechanical turk workers. In: Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, CHI ’10, pp. 2399–2402. ACM, New York, NY (2010). https://doi.org/10.1145/1753326.1753688 Downs, J.S., Holbrook, M.B., Sheng, S., Cranor, L.F.: Are your participants gaming the system? Screening mechanical turk workers. In: Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, CHI ’10, pp. 2399–2402. ACM, New York, NY (2010). https://​doi.​org/​10.​1145/​1753326.​1753688
16.
Zurück zum Zitat Duh, K., Hirao, T., Kimura, A., Ishiguro, K., Iwata, T., Yeung, C.-M.A.: Creating stories: Social curation of twitter messages. In: ICWSM (2012) Duh, K., Hirao, T., Kimura, A., Ishiguro, K., Iwata, T., Yeung, C.-M.A.: Creating stories: Social curation of twitter messages. In: ICWSM (2012)
17.
Zurück zum Zitat Gajewar, A., Sarma, A.D.: Multi-skill collaborative teams based on densest subgraphs. In: Proceedings of the 2012 SIAM International Conference on Data Mining, pp. 165–176 (2012) Gajewar, A., Sarma, A.D.: Multi-skill collaborative teams based on densest subgraphs. In: Proceedings of the 2012 SIAM International Conference on Data Mining, pp. 165–176 (2012)
18.
Zurück zum Zitat Gao, D., Tong, Y., She, J., Song, T., Chen, L., Xu, K.: Top-k team recommendation in spatial crowdsourcing. In: International Conference on Web-Age Information Management, pp. 191–204. Springer (2016) Gao, D., Tong, Y., She, J., Song, T., Chen, L., Xu, K.: Top-k team recommendation in spatial crowdsourcing. In: International Conference on Web-Age Information Management, pp. 191–204. Springer (2016)
19.
Zurück zum Zitat Gao, D., Tong, Y., She, J., Song, T., Chen, L., Xu, K.: Top-k team recommendation and its variants in spatial crowdsourcing. Data Sci. Eng. 2(2), 136–150 (2017)CrossRef Gao, D., Tong, Y., She, J., Song, T., Chen, L., Xu, K.: Top-k team recommendation and its variants in spatial crowdsourcing. Data Sci. Eng. 2(2), 136–150 (2017)CrossRef
20.
Zurück zum Zitat Garey, M.R., Johnson, D.S.: Computers and Intractability: A Guide to the Theory of NP-Completeness. W.H. Freeman and Company, New York (1979)MATH Garey, M.R., Johnson, D.S.: Computers and Intractability: A Guide to the Theory of NP-Completeness. W.H. Freeman and Company, New York (1979)MATH
21.
22.
Zurück zum Zitat Guo, S., Parameswaran, A.G., Garcia-Molina, H.: So who won? Dynamic max discovery with the crowd. In: SIGMOD Conference, pp. 385–396 (2012) Guo, S., Parameswaran, A.G., Garcia-Molina, H.: So who won? Dynamic max discovery with the crowd. In: SIGMOD Conference, pp. 385–396 (2012)
24.
Zurück zum Zitat Hertel, G.: Synergetic effects in working teams. J. Manage. Psychol. 26, 176–184 (2011)CrossRef Hertel, G.: Synergetic effects in working teams. J. Manage. Psychol. 26, 176–184 (2011)CrossRef
25.
Zurück zum Zitat Hffmeier, J., Hertel, G.: When the whole is more than the sum of its parts: group motivation gains in the wild. J. Exp. Soc. Psychol. 47(2), 455–459 (2011)CrossRef Hffmeier, J., Hertel, G.: When the whole is more than the sum of its parts: group motivation gains in the wild. J. Exp. Soc. Psychol. 47(2), 455–459 (2011)CrossRef
26.
Zurück zum Zitat Ikeda, K., Morishima, A., Rahman, H., Roy, S.B., Thirumuruganathan, S., Amer-Yahia, S., Das, G.: Collaborative crowdsourcing with Crowd4U. Proc. VLDB Endow. 9(13), 1497–1500 (2016)CrossRef Ikeda, K., Morishima, A., Rahman, H., Roy, S.B., Thirumuruganathan, S., Amer-Yahia, S., Das, G.: Collaborative crowdsourcing with Crowd4U. Proc. VLDB Endow. 9(13), 1497–1500 (2016)CrossRef
27.
Zurück zum Zitat Jøsang, A., Ismail, R., Boyd, C.: A survey of trust and reputation systems for online service provision. Decis. Support Syst. 43(2), 618–644 (2007)CrossRef Jøsang, A., Ismail, R., Boyd, C.: A survey of trust and reputation systems for online service provision. Decis. Support Syst. 43(2), 618–644 (2007)CrossRef
28.
Zurück zum Zitat Kaplan, H., Lotosh, I., Milo, T., Novgorodov, S.: Answering planning queries with the crowd. PVLDB 6(9), 697–708 (2013) Kaplan, H., Lotosh, I., Milo, T., Novgorodov, S.: Answering planning queries with the crowd. PVLDB 6(9), 697–708 (2013)
29.
Zurück zum Zitat Krumke, S., et. al.: Compact location problems. Comput. Sci. (1996) Krumke, S., et. al.: Compact location problems. Comput. Sci. (1996)
30.
Zurück zum Zitat Kargar, M., An, A.: Discovering top-k teams of experts with/without a leader in social networks. In: CIKM ’11 (2011) Kargar, M., An, A.: Discovering top-k teams of experts with/without a leader in social networks. In: CIKM ’11 (2011)
31.
Zurück zum Zitat Kargar, M., An, A., Zihayat, M.: Efficient bi-objective team formation in social networks. In: Flach, P., Bie, T., Cristianini, N. (eds.) Machine Learning and Knowledge Discovery in Databases, Lecture Notes in Computer Science, vol. 7524, pp. 483–498. Springer, Berlin (2012)CrossRef Kargar, M., An, A., Zihayat, M.: Efficient bi-objective team formation in social networks. In: Flach, P., Bie, T., Cristianini, N. (eds.) Machine Learning and Knowledge Discovery in Databases, Lecture Notes in Computer Science, vol. 7524, pp. 483–498. Springer, Berlin (2012)CrossRef
32.
Zurück zum Zitat Kargar, M., Zihayat, M., An, A.: Finding affordable and collaborative teams from a network of experts In: Proceedings of the 2013 SIAM International Conference on Data Mining, pp. 587–595 (2013) Kargar, M., Zihayat, M., An, A.: Finding affordable and collaborative teams from a network of experts In: Proceedings of the 2013 SIAM International Conference on Data Mining, pp. 587–595 (2013)
33.
Zurück zum Zitat Karpinski, M.: Approximability of the minimum bisection problem: an algorithmic challenge. In: Mathematical Foundations of Computer Science (2002) Karpinski, M.: Approximability of the minimum bisection problem: an algorithmic challenge. In: Mathematical Foundations of Computer Science (2002)
34.
Zurück zum Zitat Katz, D., Kahn, R.L.: The social psychology of organizations, vol. 2. Wiley, New York (1978) Katz, D., Kahn, R.L.: The social psychology of organizations, vol. 2. Wiley, New York (1978)
35.
Zurück zum Zitat Kenna, R., Berche, B.: Managing research quality: critical mass and optimal academic research group size. IMA J. Manage. Math. 23, 195–207 (2012)MathSciNetCrossRef Kenna, R., Berche, B.: Managing research quality: critical mass and optimal academic research group size. IMA J. Manage. Math. 23, 195–207 (2012)MathSciNetCrossRef
36.
Zurück zum Zitat Kim, J., Agrawala, M., Bernstein, M.S.: Mosaic: designing online creative communities for sharing works-in-progress. In: Design Thinking Research, pp. 105–129. Springer (2018) Kim, J., Agrawala, M., Bernstein, M.S.: Mosaic: designing online creative communities for sharing works-in-progress. In: Design Thinking Research, pp. 105–129. Springer (2018)
37.
Zurück zum Zitat Kim, J., Sterman, S., Cohen, A.A.B., Bernstein, M.S.: Mechanical novel: crowdsourcing complex work through reflection and revision. In: Design Thinking Research, pp. 79–104. Springer (2018) Kim, J., Sterman, S., Cohen, A.A.B., Bernstein, M.S.: Mechanical novel: crowdsourcing complex work through reflection and revision. In: Design Thinking Research, pp. 79–104. Springer (2018)
38.
Zurück zum Zitat Kittur, A., Kraut, R.E.: Harnessing the wisdom of crowds in Wikipedia: quality through coordination. In: Proceedings of the 2008 ACM Conference on Computer Supported Cooperative Work, CSCW ’08, pp. 37–46. ACM, New York, NY, USA (2008) Kittur, A., Kraut, R.E.: Harnessing the wisdom of crowds in Wikipedia: quality through coordination. In: Proceedings of the 2008 ACM Conference on Computer Supported Cooperative Work, CSCW ’08, pp. 37–46. ACM, New York, NY, USA (2008)
39.
Zurück zum Zitat Kittur, A., Nickerson, J.V., Bernstein, M., Gerber, E., Shaw, A., Zimmerman, J., Lease, M., Horton, J.: The future of crowd work. In: CSCW ’13 (2013) Kittur, A., Nickerson, J.V., Bernstein, M., Gerber, E., Shaw, A., Zimmerman, J., Lease, M., Horton, J.: The future of crowd work. In: CSCW ’13 (2013)
40.
Zurück zum Zitat Kittur, A., Smus, B., Khamkar, S., Kraut, R.E.: Crowdforge: Crowdsourcing complex work. In: UIST (2011) Kittur, A., Smus, B., Khamkar, S., Kraut, R.E.: Crowdforge: Crowdsourcing complex work. In: UIST (2011)
41.
Zurück zum Zitat Kittur, A., Suh, B., Pendleton, B.A., Chi, E.H.: He says, she says: conflict and coordination in Wikipedia. In: Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, pp. 453–462. ACM (2007) Kittur, A., Suh, B., Pendleton, B.A., Chi, E.H.: He says, she says: conflict and coordination in Wikipedia. In: Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, pp. 453–462. ACM (2007)
42.
Zurück zum Zitat Klakow, D., Peters, J.: Testing the correlation of word error rate and perplexity. Speech Commun. 38(1), 19–28 (2002)CrossRefMATH Klakow, D., Peters, J.: Testing the correlation of word error rate and perplexity. Speech Commun. 38(1), 19–28 (2002)CrossRefMATH
43.
Zurück zum Zitat Kulkarni, A., Can, M., Hartmann, B.: Collaboratively crowdsourcing workflows with turkomatic. In: Proceedings of the ACM 2012 Conference on Computer Supported Cooperative Work, pp. 1003–1012. ACM (2012) Kulkarni, A., Can, M., Hartmann, B.: Collaboratively crowdsourcing workflows with turkomatic. In: Proceedings of the ACM 2012 Conference on Computer Supported Cooperative Work, pp. 1003–1012. ACM (2012)
44.
Zurück zum Zitat Lappas, T., Liu, K., Terzi. E.: Finding a team of experts in social networks. In: Proceedings of the 15th ACM SIGKDD international conference on Knowledge discovery and data mining, pp. 467–476 (2009) Lappas, T., Liu, K., Terzi. E.: Finding a team of experts in social networks. In: Proceedings of the 15th ACM SIGKDD international conference on Knowledge discovery and data mining, pp. 467–476 (2009)
45.
Zurück zum Zitat Lappas, T., Liu, K., Terzi, E.: Finding a team of experts in social networks. In: SIGKDD, pp. 467–476 (2009) Lappas, T., Liu, K., Terzi, E.: Finding a team of experts in social networks. In: SIGKDD, pp. 467–476 (2009)
46.
Zurück zum Zitat Lasecki, W.S., Murray, K.I., White, S., Miller, R.C., Bigham, J.P.: Real-time crowd control of existing interfaces. In: Proceedings of the 24th Annual ACM Symposium on User Interface Software and Technology, UIST ’11, pp. 23–32. ACM, New York, NY, USA (2011) Lasecki, W.S., Murray, K.I., White, S., Miller, R.C., Bigham, J.P.: Real-time crowd control of existing interfaces. In: Proceedings of the 24th Annual ACM Symposium on User Interface Software and Technology, UIST ’11, pp. 23–32. ACM, New York, NY, USA (2011)
48.
Zurück zum Zitat Li, G., Wang, J., Zheng, Y., Franklin, M.J.: Crowdsourced data management: a survey. IEEE Trans. Knowl. Data Eng. 28(9), 2296–2319 (2016)CrossRef Li, G., Wang, J., Zheng, Y., Franklin, M.J.: Crowdsourced data management: a survey. IEEE Trans. Knowl. Data Eng. 28(9), 2296–2319 (2016)CrossRef
49.
Zurück zum Zitat Li, G., Zheng, Y., Fan, J., Wang, J., Cheng, R.: Crowdsourced data management: overview and challenges. In: Proceedings of the 2017 ACM International Conference on Management of Data, pp. 1711–1716. ACM (2017) Li, G., Zheng, Y., Fan, J., Wang, J., Cheng, R.: Crowdsourced data management: overview and challenges. In: Proceedings of the 2017 ACM International Conference on Management of Data, pp. 1711–1716. ACM (2017)
50.
Zurück zum Zitat Little, G., Chilton, L.B., Goldman, M., Miller, R.C.: Turkit: human computation algorithms on mechanical turk. In: Proceedings of the 23nd Annual ACM Symposium on User Interface Software and Technology, pp. 57–66. ACM (2010) Little, G., Chilton, L.B., Goldman, M., Miller, R.C.: Turkit: human computation algorithms on mechanical turk. In: Proceedings of the 23nd Annual ACM Symposium on User Interface Software and Technology, pp. 57–66. ACM (2010)
51.
Zurück zum Zitat Majumder, A., Datta, S., Naidu, K.: Capacitated team formation problem on social networks. In: Proceedings of the 18th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, KDD ’12, pp. 1005–1013. ACM, New York, NY, USA (2012) Majumder, A., Datta, S., Naidu, K.: Capacitated team formation problem on social networks. In: Proceedings of the 18th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, KDD ’12, pp. 1005–1013. ACM, New York, NY, USA (2012)
52.
Zurück zum Zitat Marcus, A., Parameswaran, A., et al.: Crowdsourced data management: industry and academic perspectives. Found. Trends Databases 6(1–2), 1–161 (2015)CrossRef Marcus, A., Parameswaran, A., et al.: Crowdsourced data management: industry and academic perspectives. Found. Trends Databases 6(1–2), 1–161 (2015)CrossRef
53.
Zurück zum Zitat Marwell, G., Oliver, P.E., Prahl, R.: Social networks and collective action: a theory of the critical mass. Am. J. Sociol. 94, 502–534 (1988)CrossRef Marwell, G., Oliver, P.E., Prahl, R.: Social networks and collective action: a theory of the critical mass. Am. J. Sociol. 94, 502–534 (1988)CrossRef
54.
Zurück zum Zitat Myers, I.B., McCaulley, M.H.: Myers–Briggs Type Indicator: MBTI. Consulting Psychologists Press, Palo Alto (1988) Myers, I.B., McCaulley, M.H.: Myers–Briggs Type Indicator: MBTI. Consulting Psychologists Press, Palo Alto (1988)
55.
Zurück zum Zitat Ojha, P., Talukdar, P.: Quality estimation of workers in collaborative crowdsourcing using group testing. In: Fourth AAAI Conference on Human Computation and Crowdsourcing (2016) Ojha, P., Talukdar, P.: Quality estimation of workers in collaborative crowdsourcing using group testing. In: Fourth AAAI Conference on Human Computation and Crowdsourcing (2016)
56.
Zurück zum Zitat Ravi, S.S., et. al.: Facility dispersion problems: heuristics and special cases. In: WADS (1991) Ravi, S.S., et. al.: Facility dispersion problems: heuristics and special cases. In: WADS (1991)
57.
Zurück zum Zitat Rahman, H., Roy, S.B., Thirumuruganathan, S., Amer-Yahia, S., Das, G.: Task assignment optimization in collaborative crowdsourcing. In: 2015 IEEE International Conference on Data Mining (ICDM), pp. 949–954. IEEE (2015) Rahman, H., Roy, S.B., Thirumuruganathan, S., Amer-Yahia, S., Das, G.: Task assignment optimization in collaborative crowdsourcing. In: 2015 IEEE International Conference on Data Mining (ICDM), pp. 949–954. IEEE (2015)
58.
Zurück zum Zitat Rahman, H., Thirumuruganathan, S., Roy, S.B., Amer-Yahia, S., Das, G.: Worker skill estimation in team-based tasks. Proc. VLDB Endow. 8(11), 1142–1153 (2015)CrossRef Rahman, H., Thirumuruganathan, S., Roy, S.B., Amer-Yahia, S., Das, G.: Worker skill estimation in team-based tasks. Proc. VLDB Endow. 8(11), 1142–1153 (2015)CrossRef
59.
Zurück zum Zitat Rosenkrantz, D.J., Tayi, G.K., Ravi, S.S.: Facility dispersion problems under capacity and cost constraints. J. Comb. Optim. 4, 7–33 (2000)MathSciNetCrossRefMATH Rosenkrantz, D.J., Tayi, G.K., Ravi, S.S.: Facility dispersion problems under capacity and cost constraints. J. Comb. Optim. 4, 7–33 (2000)MathSciNetCrossRefMATH
60.
Zurück zum Zitat Roy, S.B., Lykourentzou, I., Thirumuruganathan, S., Amer-Yahia, S., Das, G.: Crowds, not drones: modeling human factors in interactive crowdsourcing. In: DBCrowd (2013) Roy, S.B., Lykourentzou, I., Thirumuruganathan, S., Amer-Yahia, S., Das, G.: Crowds, not drones: modeling human factors in interactive crowdsourcing. In: DBCrowd (2013)
61.
Zurück zum Zitat Roy, S.B., Lykourentzou, I., Thirumuruganathan, S., Amer-Yahia, S., Das, G.: Optimization in knowledge-intensive crowdsourcing. CoRR, arXiv:1401.1302 (2014) Roy, S.B., Lykourentzou, I., Thirumuruganathan, S., Amer-Yahia, S., Das, G.: Optimization in knowledge-intensive crowdsourcing. CoRR, arXiv:​1401.​1302 (2014)
62.
Zurück zum Zitat Salehi, N., McCabe, A., Valentine, M., Bernstein, M.: Huddler: convening stable and familiar crowd teams despite unpredictable availability. ArXiv preprint arXiv:1610.08216 (2016) Salehi, N., McCabe, A., Valentine, M., Bernstein, M.: Huddler: convening stable and familiar crowd teams despite unpredictable availability. ArXiv preprint arXiv:​1610.​08216 (2016)
63.
Zurück zum Zitat Schrijver, A.: Theory of Linear and Integer Programming. Wiley, New York (1986)MATH Schrijver, A.: Theory of Linear and Integer Programming. Wiley, New York (1986)MATH
64.
Zurück zum Zitat Snehalkumar, G., Chhibber, N., Sehgal, V., Ballav, A., Mullings, C., Nasser, A., Richmond-Fuller, A., Gilbee, A., Gamage, D., Whiting, M., et al.: Prototype tasks: improving crowdsourcing results through rapid, iterative task design. ArXiv preprint arXiv:1707.05645 (2017) Snehalkumar, G., Chhibber, N., Sehgal, V., Ballav, A., Mullings, C., Nasser, A., Richmond-Fuller, A., Gilbee, A., Gamage, D., Whiting, M., et al.: Prototype tasks: improving crowdsourcing results through rapid, iterative task design. ArXiv preprint arXiv:​1707.​05645 (2017)
65.
Zurück zum Zitat Solomon, J., Wash, R.: Critical mass of what? Exploring community growth in wikiprojects. In: ICWSM (2014) Solomon, J., Wash, R.: Critical mass of what? Exploring community growth in wikiprojects. In: ICWSM (2014)
66.
Zurück zum Zitat Surowiecki, J.: The wisdom of crowds: why the many are smarter than the few and how collective wisdom shapes business. Economies, Societies and Nations (2004) Surowiecki, J.: The wisdom of crowds: why the many are smarter than the few and how collective wisdom shapes business. Economies, Societies and Nations (2004)
67.
Zurück zum Zitat Tong, Y., She, J., Ding, B., Wang, L., Chen, L.: Online mobile micro-task allocation in spatial crowdsourcing. In: 2016 IEEE 32nd International Conference on Data Engineering (ICDE), pp. 49–60. IEEE (2016) Tong, Y., She, J., Ding, B., Wang, L., Chen, L.: Online mobile micro-task allocation in spatial crowdsourcing. In: 2016 IEEE 32nd International Conference on Data Engineering (ICDE), pp. 49–60. IEEE (2016)
68.
Zurück zum Zitat Valentine, M.A., Retelny, D., To, A., Rahmati, N., Doshi, T., Bernstein, M.S.: Flash organizations: crowdsourcing complex work by structuring crowds as organizations. In: Proceedings of the 2017 CHI Conference on Human Factors in Computing Systems, pp. 3523–3537. ACM (2017) Valentine, M.A., Retelny, D., To, A., Rahmati, N., Doshi, T., Bernstein, M.S.: Flash organizations: crowdsourcing complex work by structuring crowds as organizations. In: Proceedings of the 2017 CHI Conference on Human Factors in Computing Systems, pp. 3523–3537. ACM (2017)
69.
Zurück zum Zitat van Dongen, S., Enright, A.J.: Metric distances derived from cosine similarity and Pearson and Spearman correlations. CoRR arXiv:1208.3145 (2012) van Dongen, S., Enright, A.J.: Metric distances derived from cosine similarity and Pearson and Spearman correlations. CoRR arXiv:​1208.​3145 (2012)
70.
Zurück zum Zitat Yan, R., Gao, M., Pavlick, E., Callison-Burch, C.: Are two heads better than one? Crowdsourced translation via a two-step collaboration of non-professional translators and editors In: Proceedings of the 52nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), vol 1, pp. 1134–1144 (2014) Yan, R., Gao, M., Pavlick, E., Callison-Burch, C.: Are two heads better than one? Crowdsourced translation via a two-step collaboration of non-professional translators and editors In: Proceedings of the 52nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), vol 1, pp. 1134–1144 (2014)
Metadaten
Titel
Optimized group formation for solving collaborative tasks
verfasst von
Habibur Rahman
Senjuti Basu Roy
Saravanan Thirumuruganathan
Sihem Amer-Yahia
Gautam Das
Publikationsdatum
26.07.2018
Verlag
Springer Berlin Heidelberg
Erschienen in
The VLDB Journal / Ausgabe 1/2019
Print ISSN: 1066-8888
Elektronische ISSN: 0949-877X
DOI
https://doi.org/10.1007/s00778-018-0516-7

Weitere Artikel der Ausgabe 1/2019

The VLDB Journal 1/2019 Zur Ausgabe

Premium Partner