nach oben

Erschienen in:

2018 | OriginalPaper | Buchkapitel

Mechanical Novel: Crowdsourcing Complex Work Through Reflection and Revision

verfasst von : Joy Kim, Sarah Sterman, Allegra Argent Beal Cohen, Michael S. Bernstein

Erschienen in: Design Thinking Research

Verlag: Springer International Publishing

Einloggen

Aktivieren Sie unsere intelligente Suche, um passende Fachinhalte oder Patente zu finden.

search-config

KI-gestützte Suche

Aus

Abstract

Crowdsourcing systems accomplish large tasks with scale and speed by breaking work down into independent parts. However, many types of complex creative work, such as fiction writing, have remained out of reach for crowds because work is tightly interdependent: changing one part of a story may trigger changes to the overall plot and vice versa. Taking inspiration from how expert authors write, we propose a technique for achieving interdependent complex goals with crowds. With this technique, the crowd loops between reflection, to select a high-level goal, and revision, to decompose that goal into low-level, actionable tasks. We embody this approach in Mechanical Novel, a system that crowdsources short fiction stories on Amazon Mechanical Turk. In a field experiment, Mechanical Novel resulted in higher-quality stories than an iterative crowdsourcing workflow. Our findings suggest that orienting crowd work around high-level goals may enable workers to coordinate their effort to accomplish complex work.

Sie haben noch keine Lizenz? Dann Informieren Sie sich jetzt über unsere Produkte:

Springer Professional "Wirtschaft+Technik"

Online-Abonnement

Mit Springer Professional "Wirtschaft+Technik" erhalten Sie Zugriff auf:

über 102.000 Bücher
über 537 Zeitschriften

aus folgenden Fachgebieten:

Automobil + Motoren
Bauwesen + Immobilien
Business IT + Informatik
Elektrotechnik + Elektronik
Energie + Nachhaltigkeit
Finance + Banking
Management + Führung
Marketing + Vertrieb
Maschinenbau + Werkstoffe
Versicherung + Risiko

Jetzt Wissensvorsprung sichern!

Jetzt informieren

Springer Professional "Technik"

Online-Abonnement

Mit Springer Professional "Technik" erhalten Sie Zugriff auf:

über 67.000 Bücher
über 390 Zeitschriften

aus folgenden Fachgebieten:

Automobil + Motoren
Bauwesen + Immobilien
Business IT + Informatik
Elektrotechnik + Elektronik
Energie + Nachhaltigkeit
Maschinenbau + Werkstoffe

Jetzt Wissensvorsprung sichern!

Jetzt informieren

Springer Professional "Wirtschaft"

Online-Abonnement

Mit Springer Professional "Wirtschaft" erhalten Sie Zugriff auf:

über 67.000 Bücher
über 340 Zeitschriften

aus folgenden Fachgebieten:

Bauwesen + Immobilien
Business IT + Informatik
Finance + Banking
Management + Führung
Marketing + Vertrieb
Versicherung + Risiko

Jetzt Wissensvorsprung sichern!

Jetzt informieren

Vorheriges Kapitel Breaks with a Purpose

Nächstes Kapitel Mosaic: Designing Online Creative Communities for Sharing Works-in-Progress

Nur mit Berechtigung zugänglich

This story length struck a balance between being long enough to make it difficult to coordinate work and short enough to complete in a reasonable amount of time on Mechanical Turk.

The federal minimum wage at the time of this writing was $7.25.

Alexander, C., Ishikawa, S., & Silverstein, M. (1977). A pattern language (Vol. 2). Oxford: Oxford University Press.

André, P., Kittur, A., & Dow, S. P. (2014). Crowd synthesis: Extracting categories and clusters from complex data. In Proceedings of the 17th ACM Conference on Computer Supported Cooperative Work & Social Computing, CSCW ’14 (pp. 989–998). New York, NY: ACM. doi:10.1145/2531602.2531653. http://doi.acm.org/10.1145/2531602.2531653

Bayles, D., Orland, T., & Morey, A. (2012). Art & fear. Old Saybrook: Tantor Media, Incorporated.

Bernstein, M. S., Little, G., Miller, R. C., Hartmann, B., Ackerman, M. S., Karger, D. R., Crowell, D., & Panovich, K. (2010). Soylent: A word processor with a crowd inside. In Proceedings of the 23Nd Annual ACM Symposium on User Interface Software and Technology, UIST ’10 (pp. 313–322). New York, NY: ACM. doi:10.1145/1866029.1866078. http://doi.acm.org/10.1145/1866029.1866078 CrossRef

Bigham, J. P., Bernstein, M. S., & Adar, E. (2015). Human-computer interaction and collective intelligence. In T. W. Malone & M. S. Bernstein (Eds.), Collective intelligence handbook. Cambridge: MIT Press.

Burroway, J. (2003). Imaginative writing: The elements of craft. New York: Longman

Chan, J., Dang, S., Kremer, P., Guo, L., & Dow, S. (2014). Ideagens: A social ideation system for guided crowd brainstorming. In Second AAAI Conference on Human Computation and Crowdsourcing.

Chilton, L. B., Little, G., Edge, D., Weld, D. S., & Landay, J. A. (2013). Cascade: Crowdsourcing taxonomy creation. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, CHI ’13 (pp. 1999–2008). New York, NY: ACM. doi:10.1145/2470654.2466265. http://doi.acm.org/10.1145/2470654.2466265

d.school, S. (2016). Design method: I like, i wish, what if. http://dschool.stanford.edu/wp-content/themes/dschool/method-cards/i-like-i-wish-what-if.pdf

Flower, L., & Hayes, J. R. (1981). A cognitive process theory of writing. College Composition and Communication, 32(4), 365–387.CrossRef

Foldingstory. (2016). http://foldingstory.com/

Hahn, N., Chang, J., Kim, J. E., & Kittur, A. (2016). The knowledge accelerator: Big picture thinking in small pieces. In Proceedings of the 2016 CHI Conference on Human Factors in Computing Systems, CHI ’16 (pp. 2258–2270). New York, NY: ACM. doi:10.1145/2858036.2858364. http://doi.acm.org/10.1145/2858036.2858364

Hill, B. M., & Monroy-Hernández, A. (2013). The cost of collaboration for code and art: Evidence from a remixing community. In Proceedings of the 2013 Conference on Computer Supported Cooperative Work, CSCW ’13 (pp. 1035–1046). New York, NY: ACM. doi:10.1145/2441776.2441893. http://doi.acm.org/10.1145/2441776.2441893

Jansson, D. G., & Smith, S. M. (1991). Design fixation. Design Studies, 12(1), 3–11.CrossRef

Kim, H. C. E., & Eklundh, K. S. (2001). Reviewing practices in collaborative writing. Computer Supported Cooperative Work, 10(2), 247–259. doi:10.1023/A:1011229212323. http://dx.doi.org/10.1023/A:1011229212323 CrossRef

Kim, J., Cheng, J., & Bernstein, M. S. (2014). Ensemble: Exploring complementary strengths of leaders and crowds in creative collaboration. In Proceedings of the 17th ACM Conference on Computer Supported Cooperative Work & Social Computing, CSCW ’14 (pp. 745–755). New York, NY: ACM. doi:10.1145/2531602.2531638. http://doi.acm.org/10.1145/2531602.2531638

Kim, J., & Monroy-Hernández, A. (2016). Storia: Summarizing social media content based on narrative theory using crowdsourcing. In Proceedings of the 19th ACM Conference on Computer-Supported Cooperative Work & Social Computing, CSCW ’16 (pp. 1018–1027). New York: ACM. doi:10.1145/2818048.2820072. http://doi.acm.org/10.1145/2818048.2820072

Kittur, A., Smus, B., Khamkar, S., & Kraut, R. E. (2011). Crowdforge: Crowdsourcing complex work. In Proceedings of the 24th Annual ACM Symposium on User Interface Software and Technology, UIST ’11 (pp. 43–52). New York, NY: ACM. doi:10.1145/2047196.2047202. http://doi.acm.org/10.1145/2047196.2047202

Koblin, A. (2010). The Johnny cash project. http://www.thejohnnycashproject.com/

Kulkarni, A., Can, M., & Hartmann, B. (2012). Collaboratively crowdsourcing workflows with turkomatic. In Proceedings of the ACM 2012 Conference on Computer Supported Cooperative Work, CSCW ’12 (pp. 1003–1012). New York, NY: ACM. doi:10.1145/2145204.2145354. http://doi.acm.org/10.1145/2145204.2145354

Lasecki, W., Miller, C., Sadilek, A., Abumoussa, A., Borrello, D., Kushalnagar, R., & Bigham, J. (2012). Real-time captioning by groups of non-experts. In Proceedings of the 25th Annual ACM Symposium on User Interface Software and Technology, UIST ’12 (pp. 23–34). New York, NY: ACM. doi:10.1145/2380116.2380122. http://doi.acm.org/10.1145/2380116.2380122 CrossRef

Lasecki, W. S., Kim, J., Rafter, N., Sen, O., Bigham, J. P., & Bernstein, M. S. (2015). Apparition: Crowdsourced user interfaces that come to life as you sketch them. In Proceedings of the 33rd Annual ACM Conference on Human Factors in Computing Systems, CHI ’15 (pp. 1925–1934). New York, NY: ACM. doi:10.1145/2702123.2702565. http://doi.acm.org/10.1145/2702123.2702565

Lasecki, W. S., Wesley, R., Nichols, J., Kulkarni, A., Allen, J. F., & Bigham, J. P. (2013). Chorus: A crowd-powered conversational assistant. In Proceedings of the 26th Annual ACM Symposium on User Interface Software and Technology, UIST ’13 (pp. 151–162). New York, NY: ACM. doi:10.1145/2501988.2502057. http://doi.acm.org/10.1145/2501988.2502057 CrossRef

Little, G., Chilton, L. B., Goldman, M., & Miller, R. C. (2010a). Exploring iterative and parallel human computation processes. In Proceedings of the ACM SIGKDD Workshop on Human Computation, HCOMP ’10 (pp. 68–76). New York, NY: ACM. doi:10.1145/1837885.1837907. http://doi.acm.org/10.1145/1837885.1837907

Little, G., Chilton, L. B., Goldman, M., & Miller, R. C. (2010b). Turkit: Human computation algorithms on mechanical turk. In Proceedings of the 23nd Annual ACM Symposium on User Interface Software and Technology, UIST ’10 (pp. 57–66). New York, NY: ACM. doi:10.1145/1866029.1866040. http://doi.acm.org/10.1145/1866029.1866040

Luther, K., Fiesler, C., & Bruckman, A. (2013). Redistributing leadership in online creative collaboration. In Proceedings of the 2013 Conference on Computer Supported Cooperative Work, CSCW ’13 (pp. 1007–1022). New York, NY: ACM. doi:10.1145/2441776.2441891. http://doi.acm.org/10.1145/2441776.2441891

Mason, B., & Thomas, S. (2008). A million penguins research report. Leicester: Institute of Creative Technologies, De Montfort University.

Nebeling, M., To, A., Guo, A., de Freitas, A. A., Teevan, J., Dow, S. P., & Bigham, J. P. (2016). Wearwrite: Crowd-assisted writing from smart watches. In Proceedings of the 2016 CHI Conference on Human Factors in Computing Systems, CHI ’16 (pp. 3834–3846). New York, NY: ACM. doi:10.1145/2858036.2858169. http://doi.acm.org/10.1145/2858036.2858169

Noël, S., & Robert, J. M. (2004). Empirical study on collaborative writing: What do co-authors do, use, and like? Computer Supported Cooperative Work, 13(1), 63–89. doi:10.1023/B:COSU.0000014876.96003.be. http://dx.doi.org/10.1023/B:COSU.0000014876.96003.be CrossRef

Pecher, D., & Zwaan, R. A. (2005). Grounding cognition: The role of perception and action in memory, language, and thinking. Cambridge: Cambridge University Press.CrossRef

Retelny, D., Robaszkiewicz, S., To, A., Lasecki, W. S., Patel, J., Rahmati, N., Doshi, T., Valentine, M., & Bernstein, M. S. (2014). Expert crowdsourcing with flash teams. In Proceedings of the 27th Annual ACM Symposium on User Interface Software and Technology, UIST ’14 (pp. 75–85). New York, NY: ACM. doi:10.1145/2642918.2647409. http://doi.acm.org/10.1145/2642918.2647409

Schön, D. A. (1983). The reflective practitioner. London: Temple Smith.

Sharples, M. (1999). An account of writing as creative design. Abingdon: Psychology Press.

Simpson, R., Page, K. R., & De Roure, D. (2014). Zooniverse: Observing the world’s largest citizen science platform. In Proceedings of the 23rd International Conference on World Wide Web, WWW ’14 Companion (pp. 1049–1054). New York, NY: ACM. doi:10.1145/2567948.2579215. http://doi.acm.org/10.1145/2567948.2579215 CrossRef

Teevan, J., Iqbal, S. T., & von Veh, C. (2016). Supporting collaborative writing with microtasks. In Proceedings of the 2016 CHI Conference on Human Factors in Computing Systems, CHI ’16 (pp. 2657–2668). New York, NY: ACM. doi:10.1145/2858036.2858108. http://doi.acm.org/10.1145/2858036.2858108

Thompson, J. D. (1967). Organizations in action: Social science bases of administrative theory. Piscataway: Transaction Publishers.

Verroios, V., & Bernstein, M. S. (2014). Context trees: Crowdsourcing global understanding from local views. In Proceedings of the Second AAAI Conference on Human Computation and Crowdsourcing. http://ilpubs.stanford.edu:8090/1105/

WeAreDynamo: Guidelines for academic requesters. http://wiki.wearedynamo.org/index.php/Guidelines_for_Academic_Requesters Accessed May 15, 2014.

Xu, A., Huang, S. W., & Bailey, B. (2014). Voyant: Generating structured feedback on visual designs using a crowd of non-experts. In Proceedings of the 17th ACM Conference on Computer Supported Cooperative Work & Social Computing, CSCW ’14 (pp. 1433–1444). New York, NY: ACM. doi:10.1145/2531602.2531604. http://doi.acm.org/10.1145/2531602.2531604

Yu, L., & Nickerson, J. V. (2011). Cooks or cobblers?: Crowd creativity through combination. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, CHI ’11 (pp. 1393–1402). New York, NY: ACM. doi:10.1145/1978942.1979147. http://doi.acm.org/10.1145/1978942.1979147

Zhang, H., Law, E., Miller, R., Gajos, K., Parkes, D., & Horvitz, E. (2012). Human computation tasks with global constraints. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, CHI ’12 (pp. 217–226). New York, NY: ACM. doi:10.1145/2207676.2207708. http://doi.acm.org/10.1145/2207676.2207708

Titel: Mechanical Novel: Crowdsourcing Complex Work Through Reflection and Revision
verfasst von: Joy Kim
Sarah Sterman
Allegra Argent Beal Cohen
Michael S. Bernstein
Verlag: Springer International Publishing
Buch: Design Thinking Research
Print ISBN: 978-3-319-60966-9

Electronic ISBN: 978-3-319-60967-6

Copyright-Jahr: 2018
DOI: https://doi.org/10.1007/978-3-319-60967-6_5

Springer Professional

Abstract

Bitte loggen Sie sich ein, um Zugang zu Ihrer Lizenz zu erhalten.

Sie haben noch keine Lizenz? Dann Informieren Sie sich jetzt über unsere Produkte:

Springer Professional "Wirtschaft+Technik"

Springer Professional "Technik"

Springer Professional "Wirtschaft"