Skip to main content
Erschienen in: World Wide Web 6/2017

27.01.2017

A context-aware approach for trustworthy worker selection in social crowd

verfasst von: Yang Zhao, Guanfeng Liu, Kai Zheng, An Liu, Zhixu Li, Xiaofang Zhou

Erschienen in: World Wide Web | Ausgabe 6/2017

Einloggen

Aktivieren Sie unsere intelligente Suche, um passende Fachinhalte oder Patente zu finden.

search-config
loading …

Abstract

Crowdsourcing applications like Amazon Mechanical Turk (AMT) make it possible to address many difficult tasks (e.g., image tagging and sentiment analysis) on the internet and make full use of the wisdom of crowd, where worker quality is one of the most crucial issues for the task owners. Thus, a challenging problem is how to effectively and efficiently select the high quality workers, so that the tasks online can be accomplished successfully under a certain budget. The existing methods on the crowd worker selection problem mainly based on the quality measurement of the crowd workers, those who have to register on the crowdsourcing platforms. With the connect of the OSNs and the crowdsourcing applications, the social contexts like social relationships and social trust between participants and social positions of participants can assist requestors to select one or a group of trustworthy crowdsourcing workers. In this paper, we first present a contextual social network structure and a concept of Strong Social Component (SSC), which emblems a group of workers who have high social contexts values. Then, we propose a novel index for SSC, and a new efficient and effective algorithm C-AWSA to find trustworthy workers, who can complete the tasks with high quality. The results of our experiments conducted on four real OSN datasets illustrate that the superiority of our method in trustworthy worker selection.

Sie haben noch keine Lizenz? Dann Informieren Sie sich jetzt über unsere Produkte:

Springer Professional "Wirtschaft"

Online-Abonnement

Mit Springer Professional "Wirtschaft" erhalten Sie Zugriff auf:

  • über 67.000 Bücher
  • über 340 Zeitschriften

aus folgenden Fachgebieten:

  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Finance + Banking
  • Management + Führung
  • Marketing + Vertrieb
  • Versicherung + Risiko




Jetzt Wissensvorsprung sichern!

Springer Professional "Wirtschaft+Technik"

Online-Abonnement

Mit Springer Professional "Wirtschaft+Technik" erhalten Sie Zugriff auf:

  • über 102.000 Bücher
  • über 537 Zeitschriften

aus folgenden Fachgebieten:

  • Automobil + Motoren
  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Elektrotechnik + Elektronik
  • Energie + Nachhaltigkeit
  • Finance + Banking
  • Management + Führung
  • Marketing + Vertrieb
  • Maschinenbau + Werkstoffe
  • Versicherung + Risiko

Jetzt Wissensvorsprung sichern!

Springer Professional "Technik"

Online-Abonnement

Mit Springer Professional "Technik" erhalten Sie Zugriff auf:

  • über 67.000 Bücher
  • über 390 Zeitschriften

aus folgenden Fachgebieten:

  • Automobil + Motoren
  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Elektrotechnik + Elektronik
  • Energie + Nachhaltigkeit
  • Maschinenbau + Werkstoffe




 

Jetzt Wissensvorsprung sichern!

Literatur
1.
Zurück zum Zitat Adler, P.S.: Market, hierarchy, and trust: the knowledge economy and the future of capitalism. Organ. Sci. 12(2), 215–234 (2001)CrossRef Adler, P.S.: Market, hierarchy, and trust: the knowledge economy and the future of capitalism. Organ. Sci. 12(2), 215–234 (2001)CrossRef
2.
Zurück zum Zitat Berger, P.L., Luckmann, T.: The social construction of reality: A treatise in the sociology of knowledge, Penguin (1991). no. 10 Berger, P.L., Luckmann, T.: The social construction of reality: A treatise in the sociology of knowledge, Penguin (1991). no. 10
3.
Zurück zum Zitat Biggs, N., Lloyd, E., Wilson, R.: Graph Theory. Oxford University Press (1986) Biggs, N., Lloyd, E., Wilson, R.: Graph Theory. Oxford University Press (1986)
4.
Zurück zum Zitat Brabham, D.C.: Crowdsourcing as a model for problem solving an introduction and cases. Convergence: the International Journal of Research Into New Media Technologies 14(1), 75–90 (2008)CrossRef Brabham, D.C.: Crowdsourcing as a model for problem solving an introduction and cases. Convergence: the International Journal of Research Into New Media Technologies 14(1), 75–90 (2008)CrossRef
5.
Zurück zum Zitat Cao, C.C., She, J., Tong, Y., Chen, L.: Whom to ask?: jury selection for decision making tasks on micro-blog services. Proceedings of the VLDB Endowment 5 (11), 1495–1506 (2012)CrossRef Cao, C.C., She, J., Tong, Y., Chen, L.: Whom to ask?: jury selection for decision making tasks on micro-blog services. Proceedings of the VLDB Endowment 5 (11), 1495–1506 (2012)CrossRef
6.
Zurück zum Zitat Cao, C.C., Tong, Y., Chen, L., Jagadish, H.: Wisemarket: a new paradigm for managing wisdom of online social users. In: Proceedings of the 19th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, pp 455–463. ACM (2013) Cao, C.C., Tong, Y., Chen, L., Jagadish, H.: Wisemarket: a new paradigm for managing wisdom of online social users. In: Proceedings of the 19th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, pp 455–463. ACM (2013)
7.
Zurück zum Zitat Carpenter, B.: A hierarchical bayesian model of crowdsourced relevance coding. In: TREC (2011) Carpenter, B.: A hierarchical bayesian model of crowdsourced relevance coding. In: TREC (2011)
8.
Zurück zum Zitat Dempster, A.P., Laird, N.M., Rubin, D.B.: Maximum likelihood from incomplete data via the em algorithm. J. R. Stat. Soc. Ser. B Methodol., 1–38 (1977) Dempster, A.P., Laird, N.M., Rubin, D.B.: Maximum likelihood from incomplete data via the em algorithm. J. R. Stat. Soc. Ser. B Methodol., 1–38 (1977)
9.
Zurück zum Zitat Feng, A., Franklin, M., Kossmann, D., Kraska, T., Madden, S. R., Ramesh, S., Wang, A., Xin, R. : Crowddb: Query processing with the vldb crowd (2011) Feng, A., Franklin, M., Kossmann, D., Kraska, T., Madden, S. R., Ramesh, S., Wang, A., Xin, R. : Crowddb: Query processing with the vldb crowd (2011)
10.
Zurück zum Zitat Forlines, C., Miller, S., Guelcher, L., Bruzzi, R.: Crowdsourcing the future: predictions made with a social network. In: Proceedings of the 32nd Annual ACM Conference on Human Factors in Computing Systems, pp 3655–3664. ACM (2014) Forlines, C., Miller, S., Guelcher, L., Bruzzi, R.: Crowdsourcing the future: predictions made with a social network. In: Proceedings of the 32nd Annual ACM Conference on Human Factors in Computing Systems, pp 3655–3664. ACM (2014)
11.
Zurück zum Zitat Gao, J., Liu, X., Ooi, B.C., Wang, H., Chen, G.: An online cost sensitive decision-making method in crowdsourcing systems. In: Proceedings of the 2013 ACM SIGMOD International Conference on Management of Data, pp 217–228. ACM (2013) Gao, J., Liu, X., Ooi, B.C., Wang, H., Chen, G.: An online cost sensitive decision-making method in crowdsourcing systems. In: Proceedings of the 2013 ACM SIGMOD International Conference on Management of Data, pp 217–228. ACM (2013)
12.
Zurück zum Zitat Gentle, J.E., Härdle, W.K., Mori, Y.: Handbook of computational statistics: concepts and methods. Springer Science & Business Media (2012) Gentle, J.E., Härdle, W.K., Mori, Y.: Handbook of computational statistics: concepts and methods. Springer Science & Business Media (2012)
13.
Zurück zum Zitat Gupta, M.R., Chen, Y.: Theory and use of the EM algorithm. Now Publishers Inc (2011) Gupta, M.R., Chen, Y.: Theory and use of the EM algorithm. Now Publishers Inc (2011)
14.
Zurück zum Zitat Hirth, M., Scheuring, S., Hossfeld, T., Schwartz, C., Tran-Gia, P.: Predicting result quality in crowdsourcing using application layer monitoring. In: 2014 IEEE Fifth International Conference on Communications and Electronics (ICCE), pp 510–515. IEEE (2014) Hirth, M., Scheuring, S., Hossfeld, T., Schwartz, C., Tran-Gia, P.: Predicting result quality in crowdsourcing using application layer monitoring. In: 2014 IEEE Fifth International Conference on Communications and Electronics (ICCE), pp 510–515. IEEE (2014)
15.
Zurück zum Zitat Howe, J.: The rise of crowdsourcing. Wired Mag. 14(6), 1–4 (2006) Howe, J.: The rise of crowdsourcing. Wired Mag. 14(6), 1–4 (2006)
16.
Zurück zum Zitat Hung, N.Q.V., Tam, N.T., Miklós, Z., Aberer, K.: On leveraging crowdsourcing techniques for schema matching networks. In: Database Systems for Advanced Applications, pp 139–154. Springer (2013) Hung, N.Q.V., Tam, N.T., Miklós, Z., Aberer, K.: On leveraging crowdsourcing techniques for schema matching networks. In: Database Systems for Advanced Applications, pp 139–154. Springer (2013)
17.
Zurück zum Zitat Ipeirotis, P.G., Provost, F., Wang, J.: Quality management on amazon mechanical turk. In: Proceedings of the ACM SIGKDD Workshop on Human Computation, pp 64–67. ACM (2010) Ipeirotis, P.G., Provost, F., Wang, J.: Quality management on amazon mechanical turk. In: Proceedings of the ACM SIGKDD Workshop on Human Computation, pp 64–67. ACM (2010)
18.
Zurück zum Zitat Kelley, H.H., Berscheid, E., Christensen, A., Harvey, J.H., Huston, T.L., Levinger, G., McClintock, E., Peplau, L.A., Peterson, D.R.: Analyzing close relationships. Close Relationships 20, 67 (1983) Kelley, H.H., Berscheid, E., Christensen, A., Harvey, J.H., Huston, T.L., Levinger, G., McClintock, E., Peplau, L.A., Peterson, D.R.: Analyzing close relationships. Close Relationships 20, 67 (1983)
19.
Zurück zum Zitat Korkmaz, T., Krunz, M.: Multi-constrained optimal path selection. In: INFOCOM 2001. Twentieth Annual Joint Conference of the IEEE Computer and Communications Societies. Proceedings, vol. 2, pp 834–843. IEEE (2001) Korkmaz, T., Krunz, M.: Multi-constrained optimal path selection. In: INFOCOM 2001. Twentieth Annual Joint Conference of the IEEE Computer and Communications Societies. Proceedings, vol. 2, pp 834–843. IEEE (2001)
20.
Zurück zum Zitat Li, H., Liu, Q.: Cheaper and better: Selecting good workers for crowdsourcing (2015). arXiv preprint arXiv:1502.00725 Li, H., Liu, Q.: Cheaper and better: Selecting good workers for crowdsourcing (2015). arXiv preprint arXiv:1502.​00725
21.
Zurück zum Zitat Li, L., Wang, Y., Lim, E.-P.: Trust-oriented composite service selection and discovery. In: Service-Oriented Computing, pp 50–67. Springer (2009) Li, L., Wang, Y., Lim, E.-P.: Trust-oriented composite service selection and discovery. In: Service-Oriented Computing, pp 50–67. Springer (2009)
22.
Zurück zum Zitat Li, H., Zhao, B., Fuxman, A.: The wisdom of minority: discovering and targeting the right group of workers for crowdsourcing. In: Proceedings of the 23rd international conference on World Wide Web. International World Wide Web Conferences Steering Committee, pp 165–176 (2014) Li, H., Zhao, B., Fuxman, A.: The wisdom of minority: discovering and targeting the right group of workers for crowdsourcing. In: Proceedings of the 23rd international conference on World Wide Web. International World Wide Web Conferences Steering Committee, pp 165–176 (2014)
23.
Zurück zum Zitat Liu, G., Wang, Y., Orgun, M.A., Lim, E.-P.: A heuristic algorithm for trust-oriented service provider selection in complex social networks. In: 2010 IEEE International Conference on Services Computing (SCC), pp 130–137. IEEE (2010) Liu, G., Wang, Y., Orgun, M.A., Lim, E.-P.: A heuristic algorithm for trust-oriented service provider selection in complex social networks. In: 2010 IEEE International Conference on Services Computing (SCC), pp 130–137. IEEE (2010)
24.
Zurück zum Zitat Liu, G., Wang, Y., Orgun, M.A., et al.: Optimal social trust path selection in complex social networks. In: AAAI, vol. 10, pp 1397–1398 (2010) Liu, G., Wang, Y., Orgun, M.A., et al.: Optimal social trust path selection in complex social networks. In: AAAI, vol. 10, pp 1397–1398 (2010)
25.
Zurück zum Zitat Liu, X., Lu, M., Ooi, B.C., Shen, Y., Wu, S., Zhang, M.: Cdas: a crowdsourcing data analytics system. Proceedings of the VLDB Endowment 5(10), 1040–1051 (2012) Liu, X., Lu, M., Ooi, B.C., Shen, Y., Wu, S., Zhang, M.: Cdas: a crowdsourcing data analytics system. Proceedings of the VLDB Endowment 5(10), 1040–1051 (2012)
26.
Zurück zum Zitat Liu, Q., Ihler, A.T., Steyvers, M.: Scoring workers in crowdsourcing: How many control questions are enough?. In: Advances in Neural Information Processing Systems, pp 1914–1922 (2013) Liu, Q., Ihler, A.T., Steyvers, M.: Scoring workers in crowdsourcing: How many control questions are enough?. In: Advances in Neural Information Processing Systems, pp 1914–1922 (2013)
27.
Zurück zum Zitat Liu, G., Liu, A., Wang, Y., Li, L.: An efficient multiple trust paths finding algorithm for trustworthy service provider selection in real-time online social network environments. In: 2014 IEEE International Conference on Web Services (ICWS), pp 121–128. IEEE (2014) Liu, G., Liu, A., Wang, Y., Li, L.: An efficient multiple trust paths finding algorithm for trustworthy service provider selection in real-time online social network environments. In: 2014 IEEE International Conference on Web Services (ICWS), pp 121–128. IEEE (2014)
28.
Zurück zum Zitat Malone, T.W., Laubacher, R., Dellarocas, C.: Harnessing crowds: mapping the genome of collective intelligence (2009) Malone, T.W., Laubacher, R., Dellarocas, C.: Harnessing crowds: mapping the genome of collective intelligence (2009)
29.
Zurück zum Zitat Mansell, R., Collins, B.: Trust and crime in information societies. Edward Elgar Publishing (2005) Mansell, R., Collins, B.: Trust and crime in information societies. Edward Elgar Publishing (2005)
30.
Zurück zum Zitat Marcus, A., Wu, E., Karger, D., Madden, S., Miller, R.: Human-powered sorts and joins. Proceedings of the VLDB Endowment 5(1), 13–24 (2011)CrossRef Marcus, A., Wu, E., Karger, D., Madden, S., Miller, R.: Human-powered sorts and joins. Proceedings of the VLDB Endowment 5(1), 13–24 (2011)CrossRef
31.
Zurück zum Zitat Miller, R.: Intimate relationships. McGraw-Hill Higher Education (2007) Miller, R.: Intimate relationships. McGraw-Hill Higher Education (2007)
32.
Zurück zum Zitat Mislove, A., Marcon, M., Gummadi, P.K., Druschel, P., Bhattacharjee, B.: Measurement and analysis of online social networks. In: Internet Measurement Conference, pp 29–42 (2007) Mislove, A., Marcon, M., Gummadi, P.K., Druschel, P., Bhattacharjee, B.: Measurement and analysis of online social networks. In: Internet Measurement Conference, pp 29–42 (2007)
33.
Zurück zum Zitat Parameswaran, A.G., Garcia-Molina, H., Park, H., Polyzotis, N., Ramesh, A., Widom, J.: Crowdscreen: Algorithms for filtering data with humans. In: Proceedings of the 2012 ACM SIGMOD International Conference on Management of Data, pp 361–372. ACM (2012) Parameswaran, A.G., Garcia-Molina, H., Park, H., Polyzotis, N., Ramesh, A., Widom, J.: Crowdscreen: Algorithms for filtering data with humans. In: Proceedings of the 2012 ACM SIGMOD International Conference on Management of Data, pp 361–372. ACM (2012)
34.
Zurück zum Zitat Park, H., Garcia-Molina, H., Pang, R., Polyzotis, N., Parameswaran, A., Widom, J.: Deco: A system for declarative crowdsourcing. Proceedings of the VLDB Endowment 5(12), 1990–1993 (2012)CrossRef Park, H., Garcia-Molina, H., Pang, R., Polyzotis, N., Parameswaran, A., Widom, J.: Deco: A system for declarative crowdsourcing. Proceedings of the VLDB Endowment 5(12), 1990–1993 (2012)CrossRef
35.
Zurück zum Zitat Sun, Z., Wang, H., Wang, H., Shao, B., Li, J.: Efficient subgraph matching on billion node graphs. In: VLDB’12, pp 788–799 Sun, Z., Wang, H., Wang, H., Shao, B., Li, J.: Efficient subgraph matching on billion node graphs. In: VLDB’12, pp 788–799
36.
Zurück zum Zitat Tang, J., Zhang, J., Yan, L., Li, J., Zhang, L., Su, Z.: Arnetminer: Extraction and mining of academic social networks. In: KDD’08, pp 990–998 Tang, J., Zhang, J., Yan, L., Li, J., Zhang, L., Su, Z.: Arnetminer: Extraction and mining of academic social networks. In: KDD’08, pp 990–998
37.
Zurück zum Zitat Von Ahn, L., Maurer, B., McMillen, C., Abraham, D., Blum, M.: Recaptcha: Human-based character recognition via Web security measures. Science 321 (5895), 1465–1468 (2008)MathSciNetCrossRefMATH Von Ahn, L., Maurer, B., McMillen, C., Abraham, D., Blum, M.: Recaptcha: Human-based character recognition via Web security measures. Science 321 (5895), 1465–1468 (2008)MathSciNetCrossRefMATH
38.
Zurück zum Zitat Wang, Y., Varadharajan, V.: Role-based recommendation and trust evaluation. In: E-Commerce Technology and the 4th IEEE International Conference on Enterprise Computing, E-Commerce, and E-Services, 2007. The 9th IEEE International Conference on CEC/EEE 2007, pp 278–288. IEEE (2007) Wang, Y., Varadharajan, V.: Role-based recommendation and trust evaluation. In: E-Commerce Technology and the 4th IEEE International Conference on Enterprise Computing, E-Commerce, and E-Services, 2007. The 9th IEEE International Conference on CEC/EEE 2007, pp 278–288. IEEE (2007)
39.
Zurück zum Zitat Wang, J., Kraska, T., Franklin, M.J., Feng, J.: Crowder: Crowdsourcing entity resolution. Proceedings of the VLDB Endowment 5(11), 1483–1494 (2012)CrossRef Wang, J., Kraska, T., Franklin, M.J., Feng, J.: Crowder: Crowdsourcing entity resolution. Proceedings of the VLDB Endowment 5(11), 1483–1494 (2012)CrossRef
40.
Zurück zum Zitat Whang, S.E., Lofgren, P., Garcia-Molina, H.: Question selection for crowd entity resolution. Proceedings of the VLDB Endowment 6(6), 349–360 (2013)CrossRef Whang, S.E., Lofgren, P., Garcia-Molina, H.: Question selection for crowd entity resolution. Proceedings of the VLDB Endowment 6(6), 349–360 (2013)CrossRef
41.
Zurück zum Zitat Wang, J., Li, G., Kraska, T., Franklin, M.J., Feng, J.: Leveraging transitive relations for crowdsourced joins. In: Proceedings of the 2013 ACM SIGMOD International Conference on Management of Data, pp 229–240. ACM (2013) Wang, J., Li, G., Kraska, T., Franklin, M.J., Feng, J.: Leveraging transitive relations for crowdsourced joins. In: Proceedings of the 2013 ACM SIGMOD International Conference on Management of Data, pp 229–240. ACM (2013)
42.
Zurück zum Zitat Yang, X.S., Cheng, R., Mo, L., Kao, B., Cheung, D.W.: On incentive-based tagging. In: 2013 IEEE 29th International Conference on Data Engineering (ICDE), pp 685–696. IEEE (2013) Yang, X.S., Cheng, R., Mo, L., Kao, B., Cheung, D.W.: On incentive-based tagging. In: 2013 IEEE 29th International Conference on Data Engineering (ICDE), pp 685–696. IEEE (2013)
43.
Zurück zum Zitat Yang, S., Zhang, J., Chen, I.: Web 2.0 services for identifying communities of practice. In: SCC’07, pp 130–137 Yang, S., Zhang, J., Chen, I.: Web 2.0 services for identifying communities of practice. In: SCC’07, pp 130–137
44.
Zurück zum Zitat Ye, B., Wang, Y., Liu, L.: Crowd trust: a context-aware trust model for worker selection in crowdsourcing environments. In: 2015 IEEE International Conference on Web Services (ICWS), pp 121–128. IEEE (2015) Ye, B., Wang, Y., Liu, L.: Crowd trust: a context-aware trust model for worker selection in crowdsourcing environments. In: 2015 IEEE International Conference on Web Services (ICWS), pp 121–128. IEEE (2015)
45.
Zurück zum Zitat Zhang, C.J., Chen, L., Jagadish, H., Cao, C.C.: Reducing uncertainty of schema matching via crowdsourcing. Proceedings of the VLDB Endowment 6(9), 757–768 (2013)CrossRef Zhang, C.J., Chen, L., Jagadish, H., Cao, C.C.: Reducing uncertainty of schema matching via crowdsourcing. Proceedings of the VLDB Endowment 6(9), 757–768 (2013)CrossRef
46.
Zurück zum Zitat Zhao, Z., Wei, F., Zhou, M., Chen, W., Ng, W.: Crowd-selection query processing in crowdsourcing databases: A task-driven approach. In: International Conference on Extending Database Technology (EDBT 2015), pp. 397–408, Brussels (2015) Zhao, Z., Wei, F., Zhou, M., Chen, W., Ng, W.: Crowd-selection query processing in crowdsourcing databases: A task-driven approach. In: International Conference on Extending Database Technology (EDBT 2015), pp. 397–408, Brussels (2015)
47.
Zurück zum Zitat Zheng, Y., Cheng, R., Maniu, S., Mo, L.: On optimality of jury selection in crowdsourcing. In: 18th International Conference on Extending Database Technology (EDBT 2015), Brussels (2015) Zheng, Y., Cheng, R., Maniu, S., Mo, L.: On optimality of jury selection in crowdsourcing. In: 18th International Conference on Extending Database Technology (EDBT 2015), Brussels (2015)
48.
Zurück zum Zitat Zhu, Y., Qin, L., Xu, J.X., Cheng, H.: Finding top-k similar graphs in graph databases. In: EDBT’12, pp 456–467 Zhu, Y., Qin, L., Xu, J.X., Cheng, H.: Finding top-k similar graphs in graph databases. In: EDBT’12, pp 456–467
Metadaten
Titel
A context-aware approach for trustworthy worker selection in social crowd
verfasst von
Yang Zhao
Guanfeng Liu
Kai Zheng
An Liu
Zhixu Li
Xiaofang Zhou
Publikationsdatum
27.01.2017
Verlag
Springer US
Erschienen in
World Wide Web / Ausgabe 6/2017
Print ISSN: 1386-145X
Elektronische ISSN: 1573-1413
DOI
https://doi.org/10.1007/s11280-016-0429-6

Weitere Artikel der Ausgabe 6/2017

World Wide Web 6/2017 Zur Ausgabe

Premium Partner