Citizen Science: An Information Quality Research Frontier

In recent decades, the IS research community has begun to explore relevant topics outside of traditional organizational settings (Levy and Germonprez 2017; Winter et al. 2014). Calls are increasing to conduct research in information systems that is explicitly concerned for large social groups and society (Goes 2014; Melville 2010; Seidel et al. 2013). Citizen science is uniquely positioned to establish IS as a discipline concerned with social progress and social improvement.

Studying IQ in citizen science stands to benefit sciences broadly and has potential for major social improvements (Burgess et al. 2017; McKinley et al. 2016). Much hope is vested in citizen science, which is most strongly adopted in sciences tackling humanity’s “evil quintet” of climate change, overexploitation, invasive species, land use change, and pollution (Theobald et al. 2015). As Light and Miskelly (2014, p. 14) put it, “[t]he urgency of environmental issues draws our attention to the management of finite resources, and the potential of digital tools to help us work with them effectively.” These are conversations that IS can ill afford to ignore in the Twenty-First Century.

Citizen science is also a booming hobby for online participants worldwide and a growing trend in science. In biodiversity research alone, it is estimated that more than two million people are engaged in major citizen science projects, contributing up to $2.5 billion of in-kind value annually (Theobald et al. 2015). As of 2009, the public was already providing 86% of biological survey data in Europe (Schmeller et al. 2009). Constantly pushing the boundaries of what is possible, scientists are engaging ordinary people in an ever increasing range of tasks. Volunteers may be asked to observe plants and animals (Hochachka et al. 2012), transcribe historical texts (Eveleigh et al. 2013), map cultural and geographic objects (Haklay and Weber 2008), identify natural resources to preserve and protect from development and exploitation (Vitos et al. 2013), design new drugs (Khatib et al. 2011), search for extraterrestrial intelligence (Korpela 2012), catalog galaxies (Fortson et al. 2011), describe oceans floors and document marine life (Pattengill-Semmens and Semmens 2003), track butterflies (Davis and Howard 2005), and report on active wildfires (Goodchild and Glennon 2010), among many other tasks. Citizen science can take volunteers into wilderness areas, as in the case of some biological monitoring projects, but are just as likely to occur in kitchens or backyards (some projects require mailing specimens, such as sourdough starters or soil samples), can be conducted entirely online (such as peering at photographs taken by the Hubble telescope to classify galaxies or scanned specimens to transcribe museum collections), or require little direct involvement, like in volunteer computing projects such as SETI@Home’s search for extraterrestrial intelligence and a wide variety of other research projects supported by the BOINC distributed computing architecture (Hand 2010; Louv et al. 2012).

Ingenuous application of online technologies, coupled with enthusiasm and creativity of ordinary people, have already resulted in a number of high-profile scientific discoveries. Table 1 provides several examples of discoveries originating in citizen science projects.

Notably, citizen science information quality is already a well-explored subject in sciences (discussed further below). It builds on the long tradition of involving humans in data collection that invariably involved external research participants, field research settings, noisy and unreliable sources, and investments into innovative data collection infrastructure. Some date the origins of citizen science to Victorian-era Britain (or earlier) and online citizen science was in fact one of the early adopters of content producing technologies known as Web 2.0 (Bauer et al. 2000; Bonney et al. 2009; Brossard et al. 2005; Goodchild 2007; Louv et al. 2012; Osborn et al. 2005; Wiersma 2010). By bringing IS expertise to citizen science, IS researchers can contribute to advancing an established and socially valuable body of long-standing cumulative research. This work has potential to advance practice as a value multiplier.

Information quality is a core concern in science, and therefore also in citizen science. As Prestopnik and Crowston (2012b, p. 9) conclude from observing a real-world citizen science project development: “[scientists’] primary goal is that [citizen science projects] should produce large amounts of very high quality data. Virtually all other considerations are secondary” (emphasis added). Similarly, data quality is “[o]ne of the highest priorities” for the Atlas of Living Australia, which maps biodiversity of the continent based on citizen-supplied data, as well as many other sources, including government agencies (Belbin 2011). Of the challenges that face the practice of citizen science, information quality is consistently considered paramount (Cohn 2008; Kosmala et al. 2016; Lewandowski and Specht 2015; Sheppard and Terveen 2011; Wiggins et al. 2011).

The persistent and often lopsided focus on IQ in citizen science contrasts with many other types of UGC (e.g., social networking, micro-task markets, blogs), where information quality competes with other priorities for attention, such as profit, enjoyment of participants, ability to advertise products, disseminating content, attracting followers, co-creating products, and solving practical problems (Ipeirotis et al. 2010; Levina and Arriaga 2014; Sorokin and Forsyth 2008; Susarla et al. 2012; Wattal et al. 2010; Zwass 2010). In contrast, in the context of science, these considerations are present but typically draw less attention because they are less core to the success of scientific research and are generally considered to be operational issues. Empirical evidence lies at the heart of the scientific method, making information quality a critical and central citizen science theme. Yet, this persistent interest in IQ is what makes citizen science an exciting setting for better understanding and advancing the general information quality research tradition.

As with any complex social phenomena, citizen science is more than just data collection and information quality. It also includes considerations of the engagement of volunteers in doing research; increasing public trust in the scientific enterprise; co-creation of scientific endeavors; engagement of the public in policies and debates arising as a result of science; building communities; involving people in addressing climate change and environmental sustainability; fostering education, literacy, and awareness of science among the general population (Bonney et al. 2009; Irwin 1995; Irwin and Michael 2003; Levy and Germonprez 2017; Sieber 2006). While these are important to the phenomenon and practice of citizen science, conducting research in information quality in citizen science holds undisputed value among scientists, and IS researchers are particularly well positioned to conduct research affecting the core issue in this emerging field.

IS researchers are frequently challenged to embark on risky research that tackles difficult and ill-defined, wicked problems (Hevner et al. 2004). Due to its unique nature, citizen science has presented a number of information quality research problems that are either unreported in traditional organizational settings or familiar issues with notable contextual considerations.

One of the interesting aspects of citizen science is its typically democratic and open nature (Irwin 1995). Citizen science is fundamentally about empowering ordinary people in advancing research endeavors. Consequently, it is against the spirit of citizen science to discriminate against participants based on lack of scientific training or low literacy (ESCA 2015). Citizen science also puts an important premium on unique local knowledge. Yet it is precisely these characteristics that may endanger information quality (Gura 2013):

“You don't necessarily know who is on the other end of a data point … It could be a retired botany professor reporting on wildflowers or a pure amateur with an untrained eye. As a result, it is difficult to guarantee the quality of the data.”

In many emerging crowdsourcing environments, such as micro-task markets (Garcia-Molina et al. 2016; Lukyanenko and Parsons 2018; Ogunseye et al. 2017; Palacios et al. 2016), one of the ways to deal with information quality is to restrict participation to those members of the community deemed competent to produce high-quality information (Allahbakhsh et al. 2013; Chittilappilly et al. 2016; Ogunseye et al. 2017). While this is technically also possible to do in citizen science, it both runs against the spirit of openness and inclusion and can jeopardize the recruitment and retention of contributors that is key to success in projects that rely on voluntary labor. Yet, as noted earlier, high-quality consistent data is needed to make strong inferences and conclusions. Thus, scientists face a dilemma of optimizing information quality while keeping participation levels high enough to meet research needs and remaining sensitive to all relevant stakeholders. The challenge therefore becomes to preserve the openness of citizen science, allowing anybody interested to participate while delivering information of sufficient quality for scientific research.

Another consequence of open and often anonymous participation is that citizens often have diverse backgrounds and hold different views from each other. Sometimes the differences between scientists and contributors as well as the nature of the information are truly “extreme,” as in the case of a conservation project in the Congo rainforest basin where data collectors include non-literate seminomadic hunter-gatherers (Pygmies), local farmers, loggers, and foresters (Rowland 2012; Stevens et al. 2014; Vitos et al. 2013). Lukyanenko et al. (2014a, 2014b) show that when views of volunteer contributors differ from those of professional experts (scientists) and one another, quality of contributions may suffer.

For the reasons above, quality remains a wicked problem. Indeed, concerns about IQ are so ingrained that scientists continue to face challenges in sharing findings based on citizen data with the scientific community even when they can conclusively demonstrate that data quality is high (Law et al. 2017; Lewandowski and Specht 2015; Riesch and Potter 2013). It is this untamed nature that makes citizen science information quality research interesting and productive.

Information quality research in organizational contexts studies many factors to improve IQ, including total data production monitoring and control, adherence to best practices in IS development (e.g., requirements elicitation, participative design, conceptual, database and interface design), training of data creators (e.g., database clerks), clarity of data entry instructions, close monitoring of the data production process, providing timely quality feedback and continuous improvement of information production processes (Ballou et al. 2003; Batini and Scannapieca 2006; Lee et al. 2002; Levitin and Redman 1995; Redman 1996; Strong et al. 1997; Wang et al. 1995a, 1995b). Many of these strategies have been applied and evaluated in citizen science projects (Bonney et al. 2009; Kosmala et al. 2016; Lewandowski and Specht 2015; Newman et al. 2003; Wiggins et al. 2011). Surveys of citizen science projects show that multiple quality assurance and quality control strategies are usually implemented (Wiggins and Crowston 2014). Thus, citizen science is already a partial extension of the IQ tradition in traditional settings.

Much like data entry in organizations, some citizen science projects require testing or certification in a certain skill to participate (Crall et al. 2011; Lewandowski and Specht 2015). Often projects naturally attract or are specifically marketed to members of the general public with domain expertise, such as avid birders (e.g., Cohn 2008). Insisting on high domain expertise or prequalifying tests of expertise can severely curtail participation in many projects and runs contrary to the open spirit of citizen science (Lukyanenko et al. 2014b; Wiggins et al. 2011). Alternative approaches, such as embedded assessments that evaluate learning or skill through the process of participation as opposed to using a quiz or other explicit instrument, are currently a hot topic of research in evaluation related to citizen science (Becker-Klein et al. 2016; Nelson et al. 2017; Ogunseye and Parsons 2016). A typical embedded assessment task for an online content classification project injects a handful of images into the workflow that have been pre-classified by experts to evaluate contributors’ agreement with experts, producing individual-level performance measures.

Another traditional measure to improve IQ is training for data creators. Training has been widely applied and studied in the citizen science context. Anecdotal and direct evidence of the positive impact of training on accuracy is reported (Delaney et al. 2008; Newman et al. 2003; Wiggins et al. 2011). Citizen science studies explore theoretical boundaries on how training is designed and delivered, identifying limitations in projects with many users, globally distributed participants, ill-defined tasks, minimal resources for training, or large and complex domains (e.g., entirety of natural history). Furthermore, requiring extensive training may discourage participation, can lead to hypothesis guessing, and may result in other biases (Lukyanenko et al. 2014b; Ogunseye et al. 2017). For example, participants who know the objective of the project may overinflate or exaggerate information (Galloway et al. 2006; Miller et al. 2012). Some have explored the use of tasks which specifically do not require training (Eveleigh et al. 2014; Lukyanenko et al. 2019), while others have investigated the possibility of attracting diverse crowds so that training induced biases are mitigated due to the diversity of the participants (Ogunseye et al. 2017; Ogunseye and Parsons 2016). All these are novel ideas for traditional information quality research. As training is prevalent in corporate settings, findings from citizen science may suggest new approaches for this context as well.

In citizen science, as in broader IQ research, information quality is considered a multidimensional construct composed of such dimensions as accuracy, timeliness, completeness (Arazy and Kopak 2011; Ballou and Pazer 1995; Girres and Touya 2010; Haklay and Weber 2008; Lee et al. 2002; Nov et al. 2014; Rana et al. 2015; Sicari et al. 2016). One of the findings of traditional IQ research are the trade-offs between dimensions (Batini and Scannapieca 2006; Pipino et al. 2002; Scannapieco et al. 2005). For example, previous research in corporate settings proposed trade-offs between consistency and completeness (Ballou and Pazer 2003), and accuracy and timeliness (Ballou and Pazer 1995). These findings have been corroborated in the citizen science context as well.

For example, research established a common trade-off in citizen science between accuracy and data completeness (Parsons et al. 2011). Specifically, to maximize accuracy, projects may be tempted to create restrictive data collection protocols and limit participation to the most knowledgeable contributors. While this strategy leads to increased accuracy of classification (e.g., identification of species), it also limits the amount of data (or completeness) such projects may otherwise achieve. A pressing challenge in citizen science is finding ways to mitigate such trade-offs, allowing many people to participate and provide information in an unconstrained manner, while ensuring that the information provided is adequate for scientists to use in research.

A novel trade-off between consistency and discovery appears to be further inherent to the citizen science context. Consistency is an important dimension of citizen science quality as it promotes reliability and comparability of results, often requisite for scientific data analysis. For example, in the River Watch Project (www.​iwinst.​org), “it is critical that the data is collected in exactly ‘the same way, every time’ so that any differences that are found are real and not due to inconsistent procedures. The goal is to minimize ‘variables’ as much as possible, so that data can be meaningfully compared.” (Sheppard and Terveen 2011, p. 33). Researchers in citizen science have applied analysis techniques such as confusion matrices, finding that contributors who are consistently wrong can be just as valuable as those who are consistently right for certain tasks, because consistent responses can be corrected to make the data useful (Simpson et al. 2013).

Such standardization, of course, may preclude some projects from attaining another important scientific objective, discovery, where variability is a potential requisite (Lukyanenko, Parsons and Wiersma 2016, Nielsen 2011). Recent work in IS began to examine whether using innovative instance-based approaches to data collection and storage can eliminate trade-offs that appeared to be inevitable in citizen science (Parsons et al. 2011). An instance-based approach to storage and collection focuses on capturing unique attributes of individual objects of interest (e.g., birds, plants, geological and geographic forms, events), rather than their similarity. This information that is inherently variable and sparse can then be made consistent through the application of high performance data mining algorithms and inferential rules. Recent experiments demonstrate the advantages of this approach over traditional methods of data collection and storage in increasing simultaneously accuracy, completeness and discovery (Lukyanenko et al. 2014b, a), however the ability of this approach to promote consistency is unknown, suggesting a need for additional research in this direction (Lukyanenko, Parsons, Wiersma, et al. 2016). Other analytical techniques have also been applied, such as using effort information to standardize data (Bird et al. 2014), or using modeling (Sousa et al. 2018; Strien et al. 2013), applying different analytic approaches to solving some of the problems in participant variability, which also may have utility for IS research. Considering this work, there is a clear need for further research on dimensions and the nature of IQ tradeoffs in citizen science.

As illustrated by the examples above, a good deal of citizen science research is quite similar to the typical work information systems scholars have conducted in other contexts. This makes the findings from citizen science potentially transferable to broader areas of IQ and allows it to contribute by corroborating the existing theories and implementing proposed approaches. Yet another exciting aspect of citizen science is a growing number of ideas and strategies that are original and unique, promising novel insights and opportunities for the broader information quality research tradition. We focus the remainder of this section on select contributions that advance or challenge established theoretical notions and offer innovative practical solutions.

An important contribution of citizen science context is better understanding of individual human factors’ impact on IQ. In traditional organizational settings, common training, background, instructions, and organizational culture and norms resulted in a relatively homogenous views of people in the same roles. Similarly, data creators in corporate environments are compelled to create high quality data as part of the general disposition to perform work well, despite known individual differences, e.g., in virtual teams (Watson-Manheim et al. 2002). Consequently, research on individual factors in IQ has been scarce. In a recent “information quality research challenge paper”, Alonso (2015) laments: “[m]ost of the data quality frameworks … superficially [touch] the quality of work that is produced by workers. New labeling tasks that rely heavily on crowdsourcing need to pay attention to the different social aspects that can influence the overall quality of such labels. Payments, incentives, reputation, hiring, training, and work evaluation are some of those aspects” (p. 2).

By contrast to employees in corporate settings, the voluntary nature of content creation in citizen science means we must assume variation in backgrounds, motivation, knowledge, literacy skills, and other abilities, which motivates increased attention to the impact of individual factors on IQ. Many studies have demonstrated the importance and positive impact of motivation on quantity of contributions, noting the particular importance of intrinsic motivation in citizen science (Flanagin and Metzger 2008; Nov et al. 2011; Prestopnik et al. 2013). Nov et al. (2014) report positive impact of collective motives (contribution to collective goals) and rewards, such as increased reputation among project members and beyond or making new friends, on accuracy of contributions.

The nature of citizen science data collection also introduces biases in data that are generally assumed not to be present when information production occurs in controlled settings. These include self-selection biases, predisposition of volunteers to notice charismatic, unusual, easy to report events and objects, and spatial and temporal biases (Janssens and Kraft 2012), all of which are also known problems in professional science (Kosmala et al. 2016). Citizen science researchers have documented, investigated, and proposed solutions to detect and mitigate these biases when analyzing data and drawing conclusions (Crall et al. 2011; Delaney et al. 2008; Girres and Touya 2010; Wiersma 2010). These ideas could significantly enrich IS quality research, where individual variation in information production has attracted attention only recently (Burton-Jones and Volkoff 2017), and assumptions of bias, or lack thereof, may merit re-examination.

Data collection in traditional organizational settings was typically conducted in a well-controlled environment. Among other things it meant that in designing data collection process, IS specialists decided the hardware available in the process of design. In citizen science, controlling which devices volunteers use is much more problematic (He and Wiggins 2017). Citizen science research stands to enrich information quality literature by examining the impact of hardware and devices as well as the role of physical settings for doing data entry (Eveleigh et al. 2014; Stevens et al. 2014; Wiggins and He 2016), factors that have not received much attention in traditional IQ research. Among other things, research in citizen science has underscored the importance of designing for multiple devices and creating processes which are simple, flexible, easy to use, and work on devices with memory and Internet connectivity constraints (e.g., mobile apps or miniaturized systems) (Cohn 2008; Higgins et al. 2016; Kim et al. 2013; Newman et al. 2010; Prestopnik and Crowston 2012a). Another interesting idea, rarely investigated in corporate settings, is to allow users to explore, experiment, tinker, and even “fail” when working with the systems as a strategy for identifying ways to develop more robust and usable tools (Newman et al. 2010).

A general lesson emerging from this research is that the physical environment and hardware characteristics have a measurable impact on information quality – a novel consideration for traditional information systems research, which routinely abstracts away implementation details when dealing with representational issues (Jabbari et al. 2018; Jabbari Sabegh et al. 2017; Wand and Weber 1995). This insight may be increasingly relevant in “bring your own device” and app-driven organizational settings (French et al. 2014; Hopkins et al. 2017; Jordan 2017; Weeger et al. 2015).

Another original contribution of citizen science is the exploration of social factors in content creation. Social networking and interactivity between participants is a common design feature of citizen science IS (Haklay and Weber 2008; Silvertown 2010). Yet, unlike general social networking projects (such as Facebook, Twitter, Youtube), in citizen science these features are implemented and investigated with explicit information quality objectives, making this a particularly interesting context for understanding the implications of sociality on information production (Nov et al. 2014; Silvertown 2010). Silvertown (2010) suggested that social networking could aid novices in species identification through the process of peer identification, which Wiggins and He (2016) examined in a study of community-based data validation, where social practices and device types used to collect information impacted information quality. Social networking is generally expected to improve information accuracy, foster peer-based learning and co-creation where participants are able to both generate and evaluate content, develop new knowledge, and refine existing skills, all of which can lead to higher quality UGC (Lukyanenko 2014; Silvertown 2010; Tan et al. 2010). Contributors remain aware of the social context in which observations are being made: the study by Nov et al. (2011) found the impact of social motives on quantity of observations.

As the same time, overreliance on social media in the context of citizen science may impose scientific limitations and negative impacts on quality. High user interactivity may lead to group think or hypothesis guessing. As scientists frequently conduct online experiments in the context of citizen science (e.g., Lukyanenko et al. 2014a), high social interactivity may engender treatment spillover, when participants in one condition may communicate with the participants in another (e.g., treatment or control) condition. Social media also brings additional challenges of managing a less structured user engagement, which can be fraught with cyberbullying, trolling or other malignant user behavior prevalent on social media platforms (Kapoor et al. 2018), and may undermine citizen science projects.

As social networking has only recently been incorporated into corporate information production (Andriole 2010), there is scarce work on IQ implications of these platforms, and findings from citizen science may offer useful insights to social networking-related IQ research in corporate settings. Consider malfeasance, a persistent concern in open online settings (Law et al. 2017), where employing social networking has shown promise for data sabotage detection (Delort et al. 2011; Wiggins and He 2016). Research has also considered features of systems that make them more robust to malfeasance (Wiggins and He 2016), as anecdotal evidence has suggested extremely low levels of malicious behavior in citizen science. These lessons may be applicable in more traditional settings to detect data sabotage by disgruntled employees. Research on social factors in citizen science can also contribute to the emerging area of online digital experimentation conducted for marketing or product development purposes in commercial contexts (e.g., Crook et al. 2009; Davenport and Harris 2017; Kohavi et al. 2013).

Citizen science sheds new light on and invites reconsideration of several long-standing assumptions of IS research. Some researchers argue that novel challenges of understanding and improving quality in citizen science warrant extending prevailing definitions of IQ and suggest rethinking traditional methods of improvement (Lukyanenko and Parsons 2015b). The organizational IQ paradigm focused on the data consumers and has defined quality as fitness of data to their needs (Madnick et al. 2009; Wang and Strong 1996). This view remains central to much of citizen science, as scientists routinely approach citizen science the same way they approach scientific research: they formulate predefined hypotheses, set goals for the projects, and then use the data provided by the volunteers to corroborate their predictions and assumptions (Lukyanenko and Parsons 2018; Wiersma 2010). This strategy is also sound as it reduces noise and variation in the data and makes the resulting data much easier to interpret and analyze.

Notwithstanding the value of approaching citizen science from the traditional fitness for use perspective, it may have limitations for some projects. A new definition of IQ has been proposed defining it as “the extent to which stored information represents the phenomena of interest to data consumers and project sponsors, as perceived by information contributors” (Lukyanenko et al. 2014b, p. 671). This definition avoids the traditional “fitness for use” definition (Madnick et al. 2009) since contributors may struggle to provide data that satisfies data consumer needs and may not even be fully aware of them. Thus, holding data contributors to data consumer standards may curtail their ability to provide high quality content as defined by data consumers, or suggest a need to refine instructions, procedures, or expectations. Guided by this definition, a series of laboratory and fields experiments demonstrated that accuracy and completeness of citizen science data can indeed be improved by relaxing the requirements to comply with data consumer needs (Lukyanenko et al. 2014a, 2014b; Lukyanenko et al. 2019), which is a standard project design strategy for achieving data quality targets in citizen science. Further, for projects focusing on new, emerging phenomena, it may be challenging to anticipate the optimal structure of citizen science data due to the different needs of diverse data consumers, so traditional solutions for storage premised on a priori structures (e.g., relational databases) may be inadequate in this setting (Sheppard et al. 2014). The new definition, however, remains controversial: the original definition was clearly useful for project leaders, and the data resulting from putting the views of data contributors first may be sparse and difficult to work with or even intractable for scientific requirements. Nonetheless, the advantages for some contexts suggest that the traditional fitness for use view may not fully capture the diverse and changing IQ landscape. These findings further provide a novel connection between conceptual modeling grammars (i.e., rules for building conceptual models and constructs that can be used to do so, see Gemino and Wand 2004, Mylopoulos 1998, Wand and Weber 2002, Woo 2011), database design approaches, and IQ, which are traditionally considered quite different research domains. The citizen science context revealed an exciting new possibility for combining research strands within IS.

Another controversial idea born out of citizen science is the notion that expertise may be harmful to IQ (Ogunseye and Parsons 2016). Contrary to the prevailing common sense assumption, Ogunseye and Parsons argue that it is the essence of expertise to be focused, and potentially ignore what is deemed irrelevant based on prior knowledge. Citizen science is a setting where discoveries are possible, and thus, it is important to have an open mind (Lukyanenko et al. 2016a). Crowdsourcing strategies where added value is premised on diverse contributions, e.g., open innovation (Brabham 2013; Howe 2008), are a prime example of this principle.

An active stream of research has studied design antecedents of quality in citizen science: many novel IS design solutions are rapidly tested with information quality being the most common outcome variable evaluated. Due to the importance of participants’ motivation in creating large quantities of accurate information, design antecedents focused on motivation have received the most attention, including design features that support promotion of collective goals, ability to interact and make friends, rewards, and game-like features (Elevant 2013; Eveleigh et al. 2013; Nov et al. 2011; Prestopnik et al. 2013; Prestopnik and Crowston 2011). It can also offer a setting for research on participatory design (Bowser et al. 2017; Lukyanenko et al. 2016b; Preece et al. 2016), another major IS topic (Bodker 1996; Bratteteig and Wagner 2014; Garrity 2001; Klein and Hirschheim 2001; Kyng 1995).

Growing evidence demonstrates benefits of gamification for data quantity and discovery. For example, Foldit is a highly successful project that turns a complex and esoteric task of designing proteins into an interactive game resulting in important discoveries in healthcare and biology (Belden et al. 2015; Khatib et al. 2011; Khoury et al. 2014). Defying the notoriously mysterious nature of quantum theory, the Quantum Game Jam project (scienceathome.​org) effectively engages ordinary people in such tasks as solving the Schrödinger equation. As gamification is quite novel in IS (e.g., Amir and Ralph 2014; Richards et al. 2014) citizen science can pave way for wider adoption of gamification in other settings. At the same time, gamification has potential to subvert IQ by incentivizing “gaming the system” and irritating or demotivating some of the most prolific and reliable contributors (Bowser et al. 2014), suggesting both that the benefits of gamification are audience specific, and that additional work is needed on this topic.

The quality challenges in citizen science also fuel the search for innovative design solutions, such as applying artificial intelligence techniques like machine learning. Artificial intelligence appears fruitful where knowledge and abilities of volunteers are deemed limited (Bonney et al. 2014a, 2014b; Hochachka et al. 2012). Typically, however, machine learning is applied post hoc – after data is captured – to detect trends or label items based on data provided (Hochachka et al. 2012; Tremblay et al. 2010). Citizen science is one of the pioneers of artificial intelligence used to augment and improve quality at the point of data creation. For example, the Merlin App aids people with bird species identification based on uploaded photographs and other basic information, using data from eBird contributors to predict likely species matches. The decision is made in real time and allows people to confirm or reject the automatic recommendation; the machine learning results can also be used to dynamically generate more effective data collection forms. The same project uses post hoc data mining to sift through millions of bird sightings to identify outliers, automatically flag erroneous species identifications, and detect potential vandalism. The substantial progress in the application of artificial intelligence and high-performance computing as well as growing evidence of effective adaptation of successful project designs led to recent Nature (Show 2015) and Science (Bonney et al. 2014a, 2014b) articles making the cautious conclusion that the problem of data quality in citizen science is “becoming relatively simple to address” (Show 2015, p. 265) for some projects. Such a conclusion, of course, is based on limited indicators of quality based on accuracy of identification (e.g., of biological species) which ignores many other potentially relevant dimensions (Lukyanenko et al. 2016a). At the same time, such signals from the leaders in the citizen science community are quite telling of the significant progress the field made in a very short period of time, which also stands ready to advance the broader field of IS.

Finally, a notable contribution of the citizen science context is investigating IQ in real settings. As visitors of websites can be easily converted into research subjects by assigning them to different control and treatment conditions (Lukyanenko and Parsons 2014), many citizen science experiments can be conducted in field settings (Delaney et al. 2008; Lewandowski and Specht 2015). Despite challenges in controlling for confounds, field experiments offer several advantages over laboratory experiments or surveys. Using a real setting allows tracking real user behavior (as opposed to behavioral intentions). It also allows for research with the population of interest rather than surrogates (e.g., students). Conducting research in a real setting also increases external validity compared to similar studies conducted in laboratory environments. As field experimentation is generally scarce in IS (Siau and Rossi 2011; Zhang and Li 2005), citizen science field experimental evidence can be used to corroborate and compare findings from more prevalent surveys and laboratory data (e.g., Jackson et al. 2016).

Table 3 summarizes the exciting new contributions of information quality research and citizen science to the broader information quality discipline.

The examples of research in citizen science provided above are not meant to be comprehensive and exhaustive. Instead, they illustrate the potential of this research domain for advancing knowledge and practice in IS related to IQ. As discussed earlier, citizen science information quality research actively investigates traditional aspects of information quality, novel concepts, methods for conducting research, and approaches to IS development and IQ management. Currently rooted in specific scientific disciplines, citizen science studies have much to offer to the traditional information quality work in IS. At the same time, as information quality is one of the core subject areas for the information systems research, citizen science promises unique opportunities for cross-pollination of theories and methods, and for exporting foundational knowledge from IS to sciences where practitioners actively seek collaborators but may be unaware of IS research community. In the next section, we further consider the role citizen science information quality research may play in connecting IS scholars with the broader scientific community.