Handbook of Human Computation

The current state-of-the art in Human Computing all too often involves large batches of some mundane but nevertheless computationally intractable problem (find the blue dot; read the words; fit the puzzle pieces); and is undertaken by a developer who realizes that large numbers of people might by various means be induced to each perform modest numbers of these tasks before getting bored and moving on to something else. And if enough people can do enough of these tasks useful things can be accomplished.

Crowdsourcing has emerged as a method to draw on the intellectual skills of large numbers of people. The text reviews such past work, and then asks questions and makes general proposals about moving towards more powerful means of large scale collaboration: moving from Crowdsourcing to “Distributed Thinking”.

Modern crowdsourcing, as a form of processing information, has many precedents in history. It was commonly found in times when labor was inexpensive and employers desired flexible organizational structures. It was commonly used during the Great Depression and perhaps had its greatest influence in the Mathematical Tables Project. However, the theoretical underpinnings of crowdsourcing can be found a century earlier, at the start of the Victorian Age, in the writings of Charles Babbage.

In this chapter we describe how ant colonies are complex systems capable of computation, and we describe the manner in which ants use local information and behavior to produce robust and adaptive colonies. While there are key differences between ant colonies and collections of human agents, the nascent field of human computation can learn from the myriad strategies that ants have evolved for successful cooperation. The cooperative behaviors of ants reflect not just the particular physiology of these insects, but also more general principles for cooperative computation.

The model of the brain—particularly the human brain—as a computer is widespread in the modern age. In keeping with most analogies by which complex systems behavior has been understood, this model has provided some useful conceptualizations of brain processing while leaving unanswered the emergent property of the mind. This chapter will explore the similarities and differences in the potential power of both neural and human communication systems to solve complex problems by working with the framework of complex dynamic systems analysis. To this end, the chapter will explore the basic computational elements of each of these two diverse systems and the modes of communication available to each.

Information processing systems composed of groups of humans may exhibit modes of dysfunction that correspond to psychopathology observed in individuals. Thus, clinical models normally applied to individuals are considered as candidate models for understanding psychosis and neurosis in distributed systems. In the first part, Matthew Blumberg considers dysfunction at the interaction level in the context of schizophrenia, and in the second part, Pietro Michelucci examines dysfunction at the neurological level in the context of obsessive-compulsive disorder.

In this chapter, concepts related to information and computation are reviewed in the context of human computation. A brief introduction to information theory and different types of computation is given. Two examples of human computation systems, online social networks and Wikipedia, are used to illustrate how these can be described and compared in terms of information and computation.

Epistemology is the branch of philosophy dealing with knowledge. It provides a base for developing a scientific definition of knowledge that is still missing today.

Knowledge is understood here as an individual model of an external world that enables decisions for goal-orientated actions. Individual and social processes involved in generating such knowledge are illustrated using a cybernetic approach. The possible contributions of computers are shown.

Finally human computation is placed in that framework: It is an endeavor subordinated to the core problem of social epistemology, i.e. to make individuals pursue shared goal-values.

This chapter seeks to characterize the conceptual space of human computation by defining key terminology within an evolving taxonomy.

One of the backbones of human society has been finding ways to organize human labor to achieve desired outcomes. The advent of computing has allowed us to bring to bear the ideas and tools of computing to this task, giving rise to what we are now calling “human computation.” Unlike mechanical computers, which are sufficiently developed and formalized that we can write down on paper an abstract representation of an algorithm and have reasonable expectations about its behavior, human computation bottoms out at fallible, unpredictable people, and, at least at present, no amount of talking or theorizing replaces the need to see what happens when you pull people together in some new way in service of some human-computation-based effort. We’re still in the early years of human computation, and our growing understanding of the field is occurring by people building real systems with real people achieving real outcomes.

Human computation methods involving the use of real-time social media data have been used successfully to support humanitarian efforts for disaster-affected communities. Through an examination of various case studies, this chapter describes specific crowdsourcing methodologies applied to disaster relief, with attention to the challenges, benefits, and outcomes. Furthermore, consideration is given to potential methods that might combine more effectively the roles of machines and humans, such as adaptive systems, gamification, and high volume analytic techniques. A “call to action” concludes the chapter, endorsing a policy by which existing volume limitations on social media data access are suspended temporarily for humanitarian aid organizations during emergent crises.

Historically, the ability of people diagnosed with medical conditions to meet and organize has been restricted to real-world support groups and exchanges of support. The pre-web Internet digitized these interactions to allow patients to communicate from the comfort of their homes and a time of their choosing. More recently, the era of the social network, “big data”, and ubiquitous electronic devices have combined with the patient empowerment movement to create a number of opportunities in human computation. Today, patients with serious illnesses are sharing their medical data online whether it is their genetic profile, treatments, symptoms, outcomes, or treatment evaluations. They are not only contributing to research by donating their data, they are mobilizing to conduct their own research in ways that can circumvent and even outpace the traditional medical establishment. Most significantly for the existing system, patients have shown that through their distributed human computation they can predict the outcome of clinical trials, one of the most expensive parts of the healthcare research process that can cost anywhere from $300 m to $1.2b to run and which could mean the balance of power will shift from patients as “subjects” to truly becoming informed and empowered participants. In this chapter we consider the history of the online patient movement, recent advances in distributed data collection and analysis by those with serious illness, the benefits accruing in the “virtuous circle” of condition-orientated human computation, the burgeoning availability of continuous sensor networks through smartphones, and briefly consider some of the risks and limitations of current approaches.

In this chapter, we will analyze a number of essential knowledge engineering activities that, for technical or principled reasons, can hardly be optimally executed through automatic processing approaches, thus remaining heavily reliant on human intervention. Human computation methods can be applied to this field in order to overcome these limitations in terms of accuracy, while still being able to fully take advantage of the scalability and performance of machine-driven capabilities. For each activity, we will explain how this symbiosis can be achieved by giving a short overview of the state of the art and several examples of systems and applications such as games-with-a-purpose, microtask crowdsourcing projects, and community-driven collaborative initiatives that showcase the benefits of the general idea.

The increasing volume and variety of scientific data sets has produced a need for serious engagement with human computation. These citizen science efforts, which include disciplines as diverse as astronomy and zoology, are reviewed with a particular focus on Galaxy Zoo and the Zooniverse platform that grew from it. The key advantages of this approach—scalability, serendipity and the ability to inform machine learning—are demonstrated and the likely motivations of citizen scientists discussed. As datasets continue to grow in size, we argue that an increased focus on efficiency will be needed, but such an approach needs to carefully account for the likely effect on both motivation and on opportunities for learning.

“Citizen science” refers to the emerging practice in which individuals in the community, often en masse, partner with researchers to assist with data collection, analysis or interpretation. Such partnerships benefit researchers through access to data at a scale not possible for individuals or small teams. To date, the benefits to the citizen scientists have been less apparent, although some have argued that participation increases critical thinking and appreciation for science methodologies. The present chapter reports a case study in which 12-year-old citizen scientists contributed to a major research investigation of evapotranspiration and, in turn, deepened their own understanding of the water cycle.

In the latest evolution of the Internet, human networks are becoming functionalized through collective collaboration frameworks. Questions are now being addressed as never before, by leveraging the easy digital accessibility of crowds to supplement the limitations of machine computation. This is especially relevant in the case of visual analytics where human intuition remains beyond the scope of existing computer object recognition algorithms. Distributing the effort over a massive network of humans not only succeeds in expanding the capacity of human based analytical power, but if set up appropriately, can also provide a statistical basis to pool human perceptive knowledge when identifying the unknown. Here we describe the impacts of this capacity in efforts of search and discovery, where massively parallel human computation can be used to identify anomalies of loosely defined characteristics within large volumes of ultra-high resolution multi-spectral satellite imagery. As human generated data is inherently noisy and subjective in nature, a statistical approach is taken towards consensus based data validation. We show that a spatial landscape can serve as the framework for collaborative computation through an overview of our initial efforts in archaeology, and the subsequent applications in disaster assessment, and search and rescue.

This chapter situates and considers several different facets of human computation in electronic literature and digital art. Electronic literature encompasses works in literary forms that are particular to the computer or the network context. Human computation is examined as an element of the development of collective narratives online, in which different roles are defined in architectures of participation. The form, structure, and common features of notable human-computation based artworks are identified. The human computation processes of collectively written and internet-harvested haiku generators are contrasted with each other to reveal their different models of situating the relationship between computational process and human authorship. Literary meta-critiques of human computation technologies such as Google’s machine reading of Gmail and reCAPTCHA’s use of human language recognition are discussed as electronic literature is positioned in a critical, if symbiotic, relationship to human computation.

Human computation techniques, such as crowdsourcing and games, have demonstrated their ability to accomplish portions of information retrieval (IR) tasks that machine-based techniques find challenging. Query refinement is one such IR task that may benefit from human involvement. We conduct an experiment that evaluates the contributions of participants from Amazon Mechanical Turk (N = 40). Each of our crowd participants is randomly assigned to use one of two query interfaces: a traditional web-based interface or a game-based interface. We ask each participant to manually construct queries to respond to a set of OHSUMED information needs and we calculate their resulting recall and precision. Those using a web interface are provided feedback on their initial queries and asked to use this information to reformulate their original queries. Game interface users are provided with instant scoring and asked to refine their queries based on their scores. In our experiment, crowdsourcing-based methods in general provide a significant improvement over machine algorithmic methods, and among crowdsourcing methods, games provide a better mean average precision (MAP) for query reformulations as compared to a non-game interface.

This chapter proposes a novel approach to credit risk rating based upon Network Analysis and enabled by Human Computation. Credit risk rating, which is essential on financial markets, has become difficult with the advent of financial instruments called derivatives and structured notes and of credit management techniques called securitization. The consequences have been dramatic: A wide-spread improper credit risk rating in the presence of these instruments and techniques has been recognized as a major cause of the financial crisis of 2007–2009 which sparked worldwide recessions. This chapter first proposes to collect risk estimates from debtors and derivatives’ parties and to aggregate these estimates into eigenvector centralities expressing a systemic rating of the credit risk faced by the market’s agents. This rating is shown to hold the promise of overcoming many deficiencies of current credit risk rating. Then, practical and theoretical implications of the proposed approach are discussed. Finally, observing that Human Computation systems and markets are related, it is argued that both Human Computation systems and markets are promising applications for approaches of the kind proposed here.

This chapter considers the means by which many people can work together to generate new ideas that have practical value. A familiar example of such a process is “brainstorming”, where people build off of each other’s ideas. Network technology and social collaboration have allowed us to improve traditional brainstorming so that more people can contribute ideas and work together more effectively irrespective of time asynchronicity or geographical distance. This chapter describes the techniques we have found to be instrumental for achieving innovation on an organizational scale.

The advent of human computation fostered by the massive diffusion of social media in personal life will change also the workplace. We are witnessing the emergence of Social Business Process Management, defined as the integration of business process management with social media, with the aim of enhancing the enterprise performance by means of a controlled participation of external stakeholders to process design and enactment. This Chapter discusses a model-driven approach to the design of participatory and socially enacted business processes. Our proposal comprises a methodology for identifying relevant social requirements in business processes, a notation for expressing social process aspects (formulated as a BPMN 2.0 extension), and a technical framework for implementing social processes as Web applications integrated with public or private Web social networks. The work is part of the ongoing BPM4People project, an initiative funded by the Seventh Framework Programme of the European Commission.

Wicked problems represent a class of problems so difficult, often people cannot determine if a proposed solution will solve the problem or not. Human computation offers a potential to solve these problems by incorporating a variety of worldviews, but first the solvers must agree on the problem they are solving.

Defined by Luis von Ahn as “systems that combine humans and computers to solve large-scale problems that neither can solve alone,” human computational systems have attracted enough attention by researchers and developers to form a loosely connected community. This handbook aims to form stronger relationships between these researchers, many of whom happened into the field through artificial intelligence (AI) or human computer interfaces (HCI). In this section, several of these pioneers share “words of wisdom” from their own experiences with human computation. This section will be of considerable interest to anyone who is curious about how HC systems could be implemented, and those who would like to enhance existing systems.

Techniques and Modalities

highlights reusable techniques and approaches that can be applied to address common problems in human computation. Some recommendations are borrowed from other fields, such as human computer interfaces and biology. Others address issues that are unique to human computation, such as motivating contributors, incorporating the contributor’s personal context and aggregating multiple perspectives.

Social content collection sites allow regular netizens to create communities of interest and share information at unprecedented scale. As a point of reference, MediaWiki (the wiki that powers Wikipedia) has millions of installations that allow non-programmers to contribute content. Because the content has very little structure, the information cannot be easily aggregated to answer simple questions. In recent years, several approaches have emerged for social knowledge collection, allowing a community of contributors to structure content so that information can be aggregated to answer reasonably interesting albeit simple factual queries. This chapter gives an overview of existing social knowledge collection research, ranging from intelligent interfaces for collection of semi-structured repositories of common knowledge, semantic wikis for organizing community resources, and collaborative ontology editors to create consensus taxonomies with classes and properties. The chapter ends with a reflection on open research problems in this area.

The spreading use of Internet-connected

smart phones and portable devices

is allowing an increasingly large number of people to bring wherever they go an intelligent tool with

sensing and acting capabilities

. It is no wonder that this led also to an explosion of location-based applications for mobile devices, including an ever developing market of

gaming applications

. Even within the boundaries of Human Computation research, applications are appearing to employ human workers on the move to solve simple computational challenges. Still, a full exploitation of mobile capabilities for

location-based Games with a Purpose

is not completely accomplished. In this chapter we analyse a number of location-based mobile applications, within and outside the Human Computation realm, to understand their distinctive features and unveil their potential to be adapted and employed to solve Human Computation tasks. We then present a

reference methodology

for the design and development of location-based mobile Games with a Purpose and we frame its adoption in the broader concept of

Citizen Computation

, our vision of a rising field at the crossroads of Human Computation, Crowdsourcing and Citizen Science.

Human Computation (HC) has traditionally used web services and distributed networks to enable remote people to perform micro-tasks without regard to their location. In contrast, Augmented Reality (AR) is concerned with providing seamless digital enhancements to a person’s physical environment, and location is a key element. This chapter discusses the opportunity for HC to enhance AR applications, especially through aiding with content creation and validation. It also reviews recent efforts to use AR technology as an interface to Human Computation and introduce more location awareness into HC applications. Finally it describes opportunities for future research using new AR display technology and support for remote collaboration.

Systems involving human computation often rely on computation being distributed spatially and temporally, enabling large-scale human-driven information processing. This distribution suggests that human computation systems may be effectively supported with pervasive computing technologies, which aim to invisibly embed networked computation in everyday life. In this chapter, I consider previous work at the intersection of human computing and pervasive computing, focusing on how human computation has been deployed through mobile platforms and how localized humans can act as computer-based sensors. I suggest a number of questions for guiding future research, framed around the question of: “in what ways does the situatedness enabled by pervasive systems influence human computation?” In addition, I discuss how the pervasive computing lens of “seamless” interaction highlights issues in human computation systems of rendering both computers and computation users invisible; this lens suggests further considerations in developing pervasive human computation systems.

This chapter discusses a computational model for collective problem solving that enables distributed individuals to describe a complex problem and to explore possible solutions. In this case, crowdsourcing is applied to the whole process of problem solving. Contributors will be involved in defining and decomposing problems in addition to finding and evaluating potential solutions. The goal of this chapter is to introduce tools for a particular kind of collective action—one in which contributors collaborate to solve problems in which they have a personal interest or stake.

DAGGRE is a research project that aims to improve the forecasting methods of world events using the first combinatorial prediction market in the world. The DAGGRE prediction market aggregates estimates from hundreds of participants to forecast the outcomes considering potential links between events. This market also aggregates estimates from a series of autotraders (algorithms) that trade live alongside of human users. The combinatorial prediction market allows Bayes Net models to be implemented and tested against the aggregate information extracted through crowdsourcing. On several of these Bayes Nets, we implemented a Bayes Net autotrader and this research shows the forecasting results of the Bayes Net autotrader in a combinatorial prediction market.

Crowdsourcing has become an increasingly popular means of flexibly deploying large amounts of human computational power. The present chapter investigates the role of microtask labor marketplaces in managing human and hybrid human machine computing. Labor marketplaces offer many advantages that in combination allow human intelligence to be allocated across projects rapidly and efficiently and information to be transmitted effectively between market participants. Human computation comes with a set of challenges that are distinct from machine computation, including increased unsystematic error (e.g. mistakes) and systematic error (e.g. cognitive biases), both of which can be exacerbated when motivation is low, incentives are misaligned, and task requirements are poorly communicated. We provide specific guidance about how to ameliorate these issues through task design, workforce selection, data cleaning and aggregation.

A recurrent challenge for human computation is motivation. Motivation is not only a prevailing topic for crowd based human computation it is also multifarious. Contributors support human computation projects for money, fun, and many other reasons. Probably the most appealing motivation from a requester’s perspective is an intrinsic interest in the task itself, although this is a rare situation. Therefore, when designing a human computation system a key challenge to accept and handle is to offer a valuable reward for contributors. One possible approach to this challenge is to design human computation systems in a way that makes their use an inherently pleasurable experience. A powerful concept to make tasks more pleasurable is to use game design to add playful elements to the task or merge the task completely into a digital game. This chapter describes concepts, methods, and pitfalls of this approach. It will give hints to identify suitable tasks, design an overall strategy, and deal with the evaluation of data in playful human computation systems.

This chapter explores the relationship between human computation and human-computer interaction (HCI). HCI is a field concerned with innovating, evaluating and abstracting principles for the design of usable interfaces. Significant work on human computation has taken place within HCI already (see Quinn and Bederson (Human computation: a survey and taxonomy of a growing field. In: Proceedings of the SIGCHI conference on human factors in computing systems (CHI ‘11), ACM, New York, pp 1403–1412, 2011) and, beyond HCI (Jamieson et al. Research directions for pushing harnessing human computation to mainstream video games. In: Meaningful play, East Lansing, 18–20 Oct 2012, 2012) for reviews of this work) and, as a result of the encounter between HCI and human computation, there are many results concerned with the relevance of interaction design for human computation systems. Rather than attempt to cover this wide range of issues comprehensively, this chapter focuses on providing a broad critique of the nature of the concepts, orientations and assumptions with which human computation systems design is considered within HCI. In particular it addresses two of the five foundational questions for human computation systems suggested by Law and von Ahn: (1) how to guarantee solutions are accurate, efficient and economical; and (2) how to motivate human components in their participation and expertise and interests (Law and von Ahn Human computation. Morgan & Claypool Synthesis Lectures on Artificial Intelligence and Machine Learning, 2011). These two key human-related issues lead us to address the ways in which designers conceive of, model and frame

the human element of interactive systems

and how this is relevant in informing our understanding of

the human element of human computation systems

. Building on empirical work in human computation games (e.g., Bell et al. (2008)), this critique seeks to reorient human computation’s perspective on human conduct as a fundamentally interpretive and socially organised accomplishment that is negotiated between humans in human computation systems, rather than an

algorithmic

process. Key elements of this reorientation argued in the chapter are: (1) that the human perspective should be considered a

foundational issue

in human computation; (2) that meaning within human computation systems is

situated

(i.e., within a particular context); and (3) that the ways in which human computation systems are experienced by human participants

fundamentally frames

their interaction with it and thus also the

products

of these interactions.

This chapter discusses how human computation can be conceptualized as a specific method within a broader context of collaborative systems. It considers how the theory of collective action and existing models of computer-supported collaboration can inform the design of new approaches to human computation. Accordingly, it proposes a conceptual design framework for collaborative human computation and illustrates its application to a prototypical design of an application integrating human computation with collective action.

The speed and transformative power of human cultural evolution is evident from the change it has wrought on our planet. This chapter proposes a human computation program aimed at (1) distinguishing algorithmic from non-algorithmic components of cultural evolution, (2) computationally modeling the algorithmic components, and amassing human solutions to the non-algorithmic (generally, creative) components, and (3) combining the two to develop human-machine hybrids with previously unforeseen computational power that can be used to solve real problems. Drawing on recent insights into the origins of evolutionary processes from biology and complexity theory, human minds are modeled as self-organizing, interacting, autopoietic networks that evolve through a Lamarckian (non-Darwinian) process of communal exchange. Existing computational models as well as directions for future research are discussed.

Many of the problems that are addressed by human computation systems can be framed as search problems: exploring values of input parameters and evaluating the resulting output of the system. For example, finding the correct labels for a set of objects can be framed as searching the sets of possible label-object pairings for the best matchings. By framing it in this way, research on collective search can be brought to bear on the understanding of human computation. In this chapter, I draw the comparison between problem solving and search, and then describe research on collective search and how the results pertain to human computation. I conclude by discussing future directions for research that integrates human computation and collective search.

Herein we entertain the prospect that engineered approaches to human computation can foster more effective collaborations than are possible today. It is commonly known that adding more people to a group effort eventually produces diminishing returns. Need this be the case? Recent evidence suggests that group efficacy is related less to the individuals in a group and more to the quality of their interactions. Furthermore, each person added to a larger group creates many more possible pairwise relationships than adding a person to a smaller group does. Taken together, this would seem to suggest the opposite of what is observed, that there should be increasing returns when adding people to a group. That there are not implies that the costs associated with adding people to a group accrue faster than the benefits. These considerations compel an amelioration strategy that involves both increasing the value and decreasing the burden of group interactions. Toward that end, a new human computation paradigm is proposed, inspired by the successes of natural systems. This “organismic computing” approach seeks to improve collaboration efficacy via the affordances of

shared sensing

,

collective reasoning

, and

coordinated action

. In addition, a technique involving

simulated augmented reality

is introduced to enable a pilot study that compares organismic computing to other collaboration methods within a virtual environment. Results from this study point to increasing rather than decreasing returns for larger groups under this new collaboration model.

The idea behind the traditional conception of human computation has been intensely reductionist—it seeks to identify aspects of human capability that are useful in isolation, to wrap those capabilities in abstractions that allow their utility and applicability to be assessed independent of the identity of the human involved (or, indeed, of the very fact that a human is involved), and to design systems that maintain these abstractions sufficiently well for the desired computation to be performed in a way that satisfies the purposes of its consumer.

Crowdsourcing using independent tasks provides an effective means of leveraging human intelligence to solve discretized problems. However, this model cannot handle acquiring input on an ongoing task from workers. In order to expand the power of crowd algorithms, we present an overview of approaches to continuous real-time crowdsourcing that engages workers for longer periods of time, allowing workers to receive feedback from the system as the task evolves due to their and other’s input. We describe how models of continuous crowdsourcing can be used to enable task completion using both synchronous and asynchronous groups of workers. We then explore a new model of continuous real-time crowdsourcing called a “crowd agent” that allows groups of workers to interact with users and their environment as if they were a single individual. This model provides a means of abstracting away the collective in crowdsourcing by making it appear as a single intelligence.

Semantic Web Technologies and Human Computation systems have great promise in working together. Human Computation has been used as a solution to help with curating Semantic Web data in the Linked Open Data cloud. In turn, the Semantic Web helps could be used to provide better user continuity and platform consistency across Human Computation systems, allowing for more complex, connected and global Human Computation system to be explored. In this chapter we explore previous work in this combination of the Semantic Web and Human Computation, and raise some challenges and research goals for future work.

Current Human Computation systems suffer from substantial limitations on the complexity of the operations they can perform; although these limitations can be mitigated by careful workflow design, those workflows can, in turn, limit task heterogeneity. Some limitations are due to the underlying software platform, but others are due to a lack of flexibility and communication ability on the part of the non-human component of the HC system. In short, the software component is often not intelligent enough to communicate its needs effectively to the human component, and it is not intelligent enough to understand the humans’ attempts to satisfy those needs. Curious Cat is an AI system, built on top of the Cyc platform (Lenat, Commun ACM 38:32, 1995; Panton et al., Common sense reasoning—from Cyc to intelligent assistant. In: Cai Y, Abascal J (eds) Ambient intelligence in everyday life. LNAI 3864. Springer, pp 1–31, 2006), whose conversational interactions crowdsource information about the world for the purpose of providing highly specific assistance and recommendations. In this chapter, we describe the general goal of specific assistance, outline the features of the Cyc platform that render it suitable for pursuing this goal, and illustrate by an example both how conversational knowledge capture from humans is achieved, and how the resulting knowledge is used to support assistance.

To better model human as computing resources, we identify six unique characteristics: (1) humans can solve computer hard problems; (2) humans are very good at exception handling, (3) humans have creativity, (4) humans have cognitive load limitation, (5) humans are vulnerable to psychological manipulation, (6) humans are prone to errors, especially for reflective tasks. We discuss how to design intuitive algorithms by taking all the unique characteristics into consideration, and the scalability issues once intuitive algorithms are developed. There are many open questions in this research direction; we are only scratching the surface.

Human computation and crowdsourcing are increasingly gaining momentum. Many platforms already exist providing basic features for crowdsourcing different types of tasks on the Web. Service Oriented Architectures (SOA) provide the ideal technical framework to support interactions with both Human-Provided Services (HPS) and Software-Based Services (SBS). A unified service-oriented computing approach allows combining the capabilities of humans and software services. Here we discuss the functional and non-functional requirements of service-oriented protocols for human computation. Human interactions in service-oriented systems need to be enabled in a different manner than interactions with software services. We describe the mapping of human interactions onto a service-oriented infrastructure.

This chapter presents a case study of a computing platform for cybernetic dataspaces. The core component of the platform is a language named CyLog that models humans as rational data sources to give an integrated abstraction of human/machine computation. Crowd4U is a non-commercial microtask-based platform being developed by universities. It has an engine for executing CyLog code, provides a pool of microtasks for crowdsourcing, and supports a variety of incentive and task-assignment structures. This chapter overviews CyLog and Crowd4U, and discusses the lessons learned from our experience of crowdsourcing projects with them.

The environment in which agents are situated has been recognized as an explicit and exploitable element in the design of Multi-Agent Systems (MAS). It can be assigned a number of responsibilities whose mechanisms for fulfillment would be more difficult to design solely using the notion of agents. To support the engineering of means to fulfill these responsibilities, we propose a novel nature-inspired approach developed by the EU project SAPERE. In particular, the intent of this chapter is to present a framework-based approach providing for context-awareness, dependability, openness, flexible and robust evolution. In such a framework all of these issues can be solved via a limited set of “laws” embedded in the framework to support and govern its self-organizing activities.

Data from ‘human sensors’ is increasingly easy to collect. Yet how may systems be designed that put it to use? This chapter discusses this question in three steps. First, we describe how the increasing ubiquity of digital systems is facilitating the creation of streams of human data. We characterise these data sources according to their purpose, obtrusiveness, structure, and hierarchy. Then, we address the kinds of systems that are already reaping the benefits of these data sources; they are broadly categorised as recommendation, retrieval, and behaviour-mediating systems. Finally, we describe a case study of potential systems that may be built to support urban travellers by leveraging the data that travellers themselves create while navigating their city. The chapter concludes with three open research challenges, related to understanding the context of data creation, the systems that are designed to use this data, and how to best architect a bridge between the two.

The idea of treating humans as computational units has challenged and redefined our understanding of what computing is. Since Luis von Ahn introduced CAPTCHA a decade ago, a fast-rising number of crowdsourcing games have used human computation to solve a wide range of problems. The rapid development of crowdsourcing games has outpaced our understanding of the theory and algorithms that are common to them. Indeed, in a 2008 article in Communications of the ACM, Jeannette Wing notes that one of the five unsolved problems in Computer Science is to define computing when it can be performed by both humans and machines.

Although the idea is an old one, there has been a recent boom in research into the Wisdom of Crowds, and this appears to be at least partly due to the now widespread availability of the Internet, and the advent of social media and Web 2.0 applications. In this paper, we start by laying out a simple conceptual framework for thinking about the Wisdom of the Crowds. We identify six core aspects that are part of any instance of the Wisdom of the Crowds. One of these aspects, called

aggregation

, is the main focus of this paper. An aggregation method is the method of bringing the many contributions of a crowd together into a collective output. We discuss three different types of aggregation methods: mathematical aggregation, group deliberation and prediction markets.

Many interesting and successful human computation systems leverage the complementary computational strengths of both humans and machines to solve these problems. In this chapter, we examine Human Computation as a type of Human-Computer Collaboration—

collaboration involving at least one human and at least one computational agent

. We discuss recent advances in the open area of function allocation, and explore how to balance the contributions of humans and machines in computational systems. We then explore how human-computer collaborative strategies can be used to solve problems that are difficult or computationally infeasible for computers or humans alone.

Building an informed crowdsourced workflow can help improve the quality of crowdsourcing results by allowing workers to collaborate and build on each others’ work. This topic has been widely adopted and studied.

Research into algorithms for coordinating computational agents that cooperatively solve problems can shine light on potential strategies for coordinating human computation. Here, we briefly summarize key concepts manifested in distributed intelligent agent algorithms, and highlight some opportunities for translating pertinent concepts to benefit human computation.

Crowds can generate creative ideas by working in parallel to modify and combine each other’s ideas. Specifically, crowd members can be organized by a human-based evolutionary algorithm. New ideas are created from scratch, they are ranked, then selected for modification or combination by other crowd members. The end result of this process is a population of ideas that will be better than starting ideas along all measured dimensions. This technique is particularly useful when problems are difficult to formalize and require human judgment at both the alternative generation and evaluation stages.

Recommender systems aim to filter information for the user, usually based on personalization techniques. As social media becomes more popular, the need for recommender systems to help each user reach the most attractive and relevant information becomes acute. Social recommender systems provide just that: filtering social content, activities, tags, people, and communities, and suggesting them for the user. The social web offers many new forms of data and metadata for social recommendation algorithms to take advantage of. In this chapter, we will review different social recommendation algorithms and their way to exploit different types of social data and metadata to enhance their effectiveness.

The key question addressed in this section is, why do people participate in human computation systems and how can high-quality participation be encouraged? This can be decomposed into more focused questions: What draws them to participate? What motivates them, drives them to keep working or contributing? How does this differ with respect to different types of human computation systems? How do you improve the quality of the participation? Can one design a system to guide the participants to provide highest quantity and best quality output? Is there an inherent tradeoff between quantity and quality?

One of the most significant challenges facing some Human Computation Systems is how to motivate participation on a scale required to produce high quality data. This chapter discusses methods that can be used to design the task interface, motivate users and evaluate the system, using as an example Phrase Detectives, a game-with-a-purpose to collect data on anaphoric co-reference in text.

Virtual citizen science (VCS) creates Internet-based projects that involve volunteers collaborating with scientists in authentic scientific research. Understanding what motivates volunteers to contribute to these projects is key to their growth and success. After reviewing the existing research on motivation to volunteer in VCS projects, we present our own research about the motivations of both experienced and first-time volunteers in Zooniverse, a collection of VCS projects. The volunteers’ responses to surveys and interviews help to provide a more complete sense of the possible motives for participating in VCS, how the motives are related to different VCS activities, and directions for future research.

With the increasing presence of online groups in so many sectors of our lives, it becomes more and more important to understand what influences the performance of these collectives. However, historically, research to measure and thereby improve performance for online groups has produced disparate conclusions. In this chapter, we propose enhancing the performance by cultivating collective intelligence in online groups—a new way of conceptualizing and measuring collective performance. We describe recent research on collective intelligence, and develop a framework for thinking about how processes critical to collective intelligence can be enhanced in online groups through the use of collaboration tools. In conclusion, we encourage researchers to explore and fine-tune these tools as a means of continuing to enhance the collective intelligence of online groups.

Continued technological advances have allowed human computation to span geographical, cultural, and temporal boundaries giving rise to technology-mediated collaborations. Yet, there is little consistency across disciplines about the definitions and roles of social processes in technology-mediated collaborations. This chapter highlights the variability in definitions for social processes across disciplines that might be responsible for inconsistent conclusions about technology-mediated collaboration effectiveness. We encourage researchers to critically compare definitions of social process variables across disciplines and to consider using an inductive approach to identify new social processes that might be uniquely adaptive in technology-mediated collaborations.

The success of a human computation system depends critically on the humans in the system actually behaving, or acting, as necessary for the system to function effectively. Since users have their own costs and benefits from participation, they will undertake desirable actions only if properly

incentivized

to do so: Indeed, while there are a vast number of human computation systems on the Web, the extent of participation and quality of contribution varies widely across systems. How can a game-theoretic approach help understand why, and provide guidance on designing systems that incentivize high participation and effort from contributors?

In many systems considered in this book, computation is an emergent property of a large population of interacting individuals. The role of analysis is to uncover and validate the microscopic mechanisms that govern an individual’s behavior. The products of analysis are descriptive models and theories of individual behavior, and a framework that explains the collective behavior that arises from interactions among many individuals. In addition to being descriptive, the models are often used to predict emergent collective behavior and motivate the design of future human computational algorithms and user interfaces that support them.

Online social media has emerged as a critical factor in information dissemination, search, marketing, expertise and influence discovery, and potentially an important tool for mobilizing people. It has also given researchers access to massive quantities of social data for empirical analysis. These data sets offer a rich source of evidence for studying dynamics of individual and group behavior, the structure of networks and global patterns of the flow of information on them. However, in most previous studies, the structure of the underlying networks was not directly visible but had to be inferred from the flow of information from one individual to another. As a result, we do not yet understand dynamics of information spread on networks or how the structure of the network affects it. We analyze data from two popular social news sites, Digg and Twitter, to understand the mechanisms of information diffusion in social networks, in the process, uncovering the primary role played by individual’s cognitive constraints in online social behavior.

Agent-based modeling (ABM) is a common computational analysis tool to study system dynamics. In the framework of ABM, the system consists of multiple autonomous and interacting agents. We can explore emergent collective patterns by simulating the individual operations and interactions between agents. As a case study, we present an experiment using an agent-based model to study how competition for limited user attention in a social network results in collective patterns of meme popularity. The model is inspired by the long tradition that represents information spreading as an epidemic process, where infection is passed along the edges of the underlying social network. The model also builds upon empirical observations on how individual humans behave online. The combination of social network structure and finite agent attention is sufficient for the emergence of broad diversity in meme popularity and lifetime. The case study illustrates how one can analyze the kind of emergent human computation that makes some memes very popular.

We describe a stochastic framework for modeling social behavior in social media, and apply this probabilistic method to one such site, the news aggregator Digg. This stochastic model identifies the interaction between how the web site displays content to users and how interesting the content is to those users. We show how this model predicts a story’s eventual popularity from users’ early reactions to it, and estimate the prediction reliability. This modeling framework can help evaluate alternative design choices for the effectiveness of social media web sites.

Most real-world networks have a modular structure, i.e., they are composed of clusters of well connected nodes, with relatively lower density of links across different clusters. Here we report on our studies of a simple cascading process in a structured heterogeneous population composed of two loosely coupled communities. We demonstrate that under certain conditions the cascading dynamics in such a network has a two-tiered structure that characterizes activity spreading at different rates in the communities. We also demonstrate that the structure has implication on problems such as influence maximization. In particular, it is shown that targeting heuristics that work provably well for homogenous networks can produce significantly sub-optimal results for heterogenous networks. We suggest a simple modification of the heuristics that accounts for the community structure, and observe improved performance.

In recent years, social synchrony has attracted much attention from different research areas including biology, physics, psychology, and engineering. It is widely believed that synchrony, as an outcome of evolutionary selection, can increase the cohesion of social groups and thus lead them to perform better when dealing with complex tasks. This chapter briefly reviews several quantitative aspects of social synchrony, including how to measure and how to model it, the impact on it of the social network structure underlying the group, and its benefits to cooperation and productivity. We provide a case study of social synchrony among software developers in Apache, a distributed Open Source Software (OSS) project. In it, we illustrate how one could quantitatively study aspects of social synchrony. The results suggest that Apache software developers synchronize their work with each other, and work together in larger groups in relatively short periods. Such working synchrony increases productivity, in terms of the number of lines of code produced, and improves the efficiency of coordination among developers, in terms of communication overhead.

“Gamification”, the application of gameplay to real-world problems, enables the development of human computation systems that support decision-making through the integration of social and machine intelligence. One of gamification’s major benefits includes the creation of a problem solving environment where the influence of cognitive and cultural biases on human judgment can be curtailed through collaborative and competitive reasoning. By reducing biases on human judgment, gamification allows human computation systems to exploit human creativity relatively unhindered by human error. Operationally, gamification uses simulation to harvest human behavioral data that provide valuable insights for the solution of real-world problems.

The rewards and rules of human collaboration systems shape the behavior of the human participants, often leading to behaviors the human computation system designers never envisioned. Most software designers make the mistake of assuming that people will follow the intent of the rules they set up in the program. But, as the rising wave of cyber-crime shows this people do what they can get away with. People will do anything they can to achieve rewards, and sometimes the reward means breaking the system for the joy of figuring out how to solve a puzzle.

Many forms of human computation require an investment of labor on the part of humans. Where does human computation intersect with traditional notions of employment? What labor standards ought to apply? This chapter explores how existing employment law regimes might operate in this relatively new context. First it attempts to distinguish between “work” and “non-work” forms of human computation, for purposes of determining when labor standards are even relevant. The next section examines various regulatory obstacles and unsettled legal questions likely to arise in online distributed work platforms, including jurisdiction, coverage, and compensation. The following section describes several voluntary measures employers could (but are not legally required to) take to establish a threshold level of fairness, transparency, and dignity in the work. The last section identifies vital opportunities created by human computational work, as a guide for stakeholders and regulatory authorities.

This chapter addresses the sinister counterpoint to the much celebrated positive opportunities of human computation systems. We characterize exploits that target workers, requesters, and a system as a whole. We identify opportunities for malicious users to leverage human computation systems to exploit external populations. Finally, we sketch some preliminary directions towards mitigating these concerns.

Privacy considerations are unavoidable in human computing endeavors, and both our decision making biases and biology may play a much larger role than commonly acknowledged in privacy related decision making. Decision making considerations such as temporal discounting and how our neurochemistry and neurobiology, including production of compounds such as dopamine, are impacted by social engagement need to be considered in human computing efforts if we are to protect participants privacy rights. Addressing privacy rights in human computing could be highly disruptive to existing approaches to online interaction design and commercial business models and will likely require adoption of privacy principles such as Privacy By Design to be successful.

With the expression

social collaboration

we refer to the processes of helping multiple people to interact and share information in order to achieve common goals. Nowadays, collaboration and social dissemination of information are facilitated by the Internet and

Social Network Services

(SNS). The reliance of social collaboration on SNS might seem surprising given the differences between their group-centric and individual-centric views. In particular, social collaboration services focus on group activities, identifying groups and collaboration spaces in which messages are explicitly directed at the group and the group activity feed is seen the same way by everyone. In contrast, social networking services generally focus on single personalized activities, sharing messages in a more-or-less undirected way and receiving messages from many sources into a single personalized activity feed.

This chapter discusses the lessons learned from computer security in the past decades and how it applies to human computation systems for solving unsolved problems. The distributed nature and inherent trust found in collaboration systems will raise particular challenges to the unique assets of problem solving systems. The chapter discusses the problem solving systems must address.

Surely you have heard of Pandora, who according to Greek mythology was the first woman on Earth. Perhaps even more famous is the container she was gifted by the gods and instructed to never open. Of course, ultimately, Pandora succumbed to her curiosity and let escape all of the evils of the universe before she managed to replace the lid. But maybe you were not aware of this important detail concerning the fate of her container:

hope remained sealed within

.

The present chapter wishes to investigate the wider context of human computation, viewing it as merely one approach within the broad domain of distributed human-computer symbiosis. The multifarious developments in the “social” Internet have shown the great potential of large-scale collaborative systems that involve both people and the various information and communication technologies (ICT) that process, store and distribute data. Here, I wish to explore this development in the broadest sense, as the self-organization of a distributed intelligence system at the planetary level—a phenomenon that has been called the “global brain”.

In a future world with pervasive Human Computation (HC), there may be profound effects on how humanity functions at multiple levels from individual behaviors to species/wide changes in evolutionary development. What would such an HC/shaped human society look like? This hypothetical society would be the result of successful adaptations that provide both increased benefit to the high/level facilitators of large-scale computations as well as sufficient incentives to individuals to participate in those computations.

The future of human computation (HC) benefits from examining tasks that agents already perform and designing environments to give those tasks computational significance. We call this

natural human computation

(NHC). We consider the possible future of NHC through the lens of Swarm!, an application under development for Google Glass. Swarm! motivates users to compute the solutions to a class of economic optimization problems by engaging the attention dynamics of crowds. We argue that anticipating and managing economies of attention provides one of the most tantalizing future applications for NHC.

Cumulative culture is the engine that drives the remarkable power of the global human computer. It enables societies to act as extremely powerful computers by ratcheting up technological and other cultural innovations. Once culture can accumulate, the ability of a society to maintain and spread complex technologies is directly related to the size of the population and its connectivity with other populations, finessing the strict limits on individual intelligence. Larger and more connected societies can maintain more complex technologies. This also means that sudden isolation or a drop in population size can lead to a loss of technology. In this chapter we first discuss how cumulative culture increases the evolutionary fitness of a population of social learners. We then focus on complex technology as a marker of cumulative cultural evolution, and discuss how technological complexity increases when cultures are both more populous and more connected. We discuss the fragility of our modern complex societies in response to disasters that may shrink the population or isolate groups. We end with a discussion of how our cultural norms and institutions shape the problems tackled by the global human computer.

As human computation tools and techniques increase in power and pervasiveness so too does their impact on factors affecting the likelihood of armed conflict, and on armed conflict itself. Some factors such as rapid economic decline, environmental stress, private motivation, a failure of the social contract, and social distance to potential enemies have been shown through empirical studies to increase the likelihood of armed conflict. The same studies have identified full democracy and high levels of education as decreasing the likelihood of conflict. Informed and inspired by this growing body of research, we discuss the current and future impact of human computation taking a socio-technical perspective. In particular we focus on ways human computation can facilitate novel forms of interaction between humans toward specific objectives. Topics include the potential of human computation to influence interactions that in turn support democracy, consumer awareness, human connections, education, poverty reduction, citizen journalism, crisis informatics, digital archives, reconciliation, and warfare.

Human computation includes the accumulation of social wisdom in the form of progressive social agendas. These agendas developed rapidly during the twentieth and twenty-first centuries through the distributed intelligence made possible by advanced information technologies. Progressive social agendas build toward equality among groups differentiated by race, class, gender, sexual orientation, and similar factors. It is no accident that these trends, distinctive in human history, occurred during an era in which national and international communication and the creation and widespread dispersal of information were enabled by advanced technologies. In this chapter I discuss future threats to social progress and the importance of maintaining the information and communication technologies and infrastructures that underpin such progress.

Today humans face many challenges as a species, including some that pose grave risks. Technology has been a significant contributor to these risks, but it may also lead to solutions. In the first part of this chapter, we consider how Human Computation (HC), the study of humans as computational elements in a purposeful system, has already been helpful in solving problems. We further consider why HC may be instrumental for mitigating future risks. In the second part of this chapter, we examine the maturity of human computation as both a practice and a discipline. This analysis informs a proposal for technical maturation as well as a formal definition of the field and its distinguishing qualities, all in service of accelerating research and ensuring responsible use of any resultant capabilities. Though the ideas in this chapter may be informed by engagement with the HC community, this manifesto represents a personal perspective.