11. Developing Misinformation Immunity in a Post-Truth World: Human Computer Interaction for Data Literacy

People’s engagement with and understanding of media devices and digital systems have been intertwined with their levels of trust towards institutions and other people. These two main relations have influenced people’s media literacies and their data literacies, particularly during the Covid-19 pandemic. We developed our data literacies framework, which we call Data Citizenship (Carmi et al., 2020), across three main dimensions: (1) Data doing—Citizens’ everyday engagements with data (for example, using data in an ethical way); (2) Data thinking—Citizens’ critical understanding of data (for example, verifying information and sources online); and (3) Data participating—Citizens’ proactive engagement with data and their networks of literacy (for example, helping others with their data literacy through games/chatbots). Trust is a common thread that relates to all these three dimensions and consequently how people navigate the datafied ecosystem.

When it comes to trust in institutions, this relates to people’s critical thinking about how reliable they perceive a specific institution to be (e.g. a news outlet or health institute) and therefore whether they should read or believe their messages. Fletcher et al. (2020), for example, show how during the Covid-19 pandemic people’s attitudes in the United Kingdom towards the trustworthiness of news outlets have decreased from 57% to 46%, and their trust levels in the government have declined from 67% to 48%. In addition, as Cushion et al. (2021) have shown, UK citizens have broadly managed to identify “fake news”, but they felt confused by the statistics and the neglect of important facts, such as how the pandemic was being handled by the UK government. According to Cushion et al. (2021), this was mainly due to editorial decisions where sufficient details were not conveyed to citizens, with the result that people felt misinformed. Cushion et al. points out both how people’s trust in the main sources of information has deteriorated during the pandemic but also, importantly, that the arguments presented to them by mainstream media were confusing and were causing a proliferation of misinformation.

Such a situation cannot be simply solved by relying on fact-checking organisations as arbiters of trustworthy information. Not only is there a great abundance of non-fact checked information but the epistemology of fact-checking is far from standardised and affected by selection effects and other types of biases (Uscinski & Butler, 2013).

Drawing from our research (Yates et al., 2021), it seems that the answer lies in a collective effort: We found that people mainly rely on their personal networks of literacy to verify information and learn new data literacy skills. We see this as a modern digital version of the 2-step-flow model of influence, originally conceived by Katz (1957) taking place in citizens networks of literacy. In relation to the Fake News Immunity Chatbot, this means that people can develop their data thinking skills using the game. This teaches them how to critically use fallacy theory as they search for and identify reliable/trusted sources and how to locate reliable articles on social media. It also encourages them to practice data participating by either playing with others in their networks of literacy or through propagating these key ideas through these networks. This can help different people in their networks with lower data literacies.

The pandemic has created an epistemological situation in which what counts as true is continuously updated. A vaccine trial could, for instance, by default, offer reliable but constrained truths about potential side-effects, while reporters rely on second-hand evidence due to lockdown restrictions. In such a scenario, the distinction between mediation, the “material prerequisites for representation in media”, and representation, “the semiotic operation, that is, the creation of meaning in the mind” (Elleström, 2017, 663) is almost removed since the representation necessarily happens in a mediated environment. That is, our sense-making processes have to rely on truth claims which depend on the reasons provided by a media product supporting their trustworthiness (Elleström, 2014, 2017, 2021).

In the era of citizens’ journalism, the news making process is inherently transmedial, and it assumes the shape of a polylogue (Musi & Aakhus, 2018) where multiple users (often anonymous) negotiate different opinions across different venues (from social media to fora). In the absence of a gatekeeping process, the discourse through which a news claim is shaped becomes the main guarantor for its truth. Thus, its persuasiveness through rhetorical strategies impacts on our perception of truthfulness. In light of this, fallacies, i.e. arguments that seem valid but are not (Hamblin, 1970), are likely to support claims which, even if not false, might be misleading and trigger misperceptions of truthfulness.

We call these misinformation news “semi-fake” since “created/shared by the authors with the intention of circulating fabricated information and hard to be flagged by the public through common ground knowledge” (Musi & Reed, 2022, 17). Fallacies, thus, constitute useful means to identify fake news. This is especially true when misinformation—information which is misleading but not necessarily non-factual or created with the intention of causing harm—rather than disinformation—i.e. information which is blatantly false—is at stake.

Two caveats have, however, to be considered. Firstly, verifiable news can also be supported by fallacious arguments, and secondly, there is so far no agreed upon taxonomy for fallacies. As to the former, our goal is to make people aware of fallacious arguments as means for scrutiny rather than truth verdicts. Our guidelines for the analysis of fallacies are, in fact, based on critical questions which cast doubt on various aspects of the news. As to the latter, we have adopted an empirical approach in selecting a decalogue of fallacies relevant for the current misinformation ecosystem. We analysed a preliminary set of 40 fact-checking commentaries and their source articles randomly picked from the fact-checker Healthfeedback.org. We developed guidelines for 10 fallacy types found in the data, which include a definition, identification questions, and an intuitive example, as presented below.

Using these ten fallacy classifications, we undertook an annotation exercise involving two minimally trained students and an expert annotator to solve cases of disagreement to analyse a dataset of news web scraped by five English factcheckers (Snopes, The Ferret, FullFact, Politifact, and Healthfeedback.org) from January 2020 to end of June 2020 and from January 2021 to end of March 2021. The resulting dataset consisted of 1500 claims.

The results show that the 10 fallacies are associated with all cases of misinformation according to the distributions displayed in Fig. 11.1. We developed series of recommendations on how to identify such fallacies, which are publicly available in a simplified format.⁵

A pie chart plots the distribution of fallacy types. Some of the values are as follows. Cherry picking, 20.3%. Evading the burden of proof, 20.1%. Hasty generalization, 14.9%. Vagueness, 14.8%. Strawman, 7.5%. Red Herring, 6.7%. False authority, 6.1%.

The annotation experiment has shown that non-experts are able to successfully identify the majority fallacies when aided with a set of heuristic guidelines. However, the task is highly complex and cognitively taxing, as shown by the cases of disagreement in the annotation exercise. For this reason, an active learning environment grounded in educational technology promises to foster engagement while retaining a focus on the task at hand, as discussed below.

According to the report Testing and Refining Criteria to Assess Media Literacy Levels in Europe (Shapiro & Celot, 2011) commissioned by the European Union, one of the factors which hinders critically reflective skills is screen time since it affects the way we access information (in a fragmented rather than holistic vein). Such a negative correlation is particularly problematic in times such as the pandemic where educational settings are constrained to virtual environments. The situation is further complicated by the proliferation of fake news which is frequently hard to pin-point, especially as it proliferates through social media. In such a scenario digital fluency is hard to achieve. Teaching people how to identify reliable sources of information is, in fact, not enough in the current (mis)information ecosystem since official news venues can convey misleading information regardless of their intentions.

Relying on fact-checkers’ comments as material to develop their critical media literacy (Kellner & Share, 2007) also has its limitations. Besides struggling to keep up with the abundance of information spread online, fact checkers make use of ratings that do not inform the readers about the basis for the misinformation (e.g. Mostly False/Half True). Rather, these online factor checkers simply apply a veracity value with little pedagogical relevance. For example, the following news was analysed by the factchecker Snopes: “In January 2021, the World Health Organization warned that pregnant women should avoid the COVID-19 vaccine”. The rating assigned by the fact-checker is Mixture, which points to the presence of both elements of truth and of falsity without, however, instructing on how to identify misleading aspects. For example, is it true that pregnant women should avoid the vaccine or is it not accurate to state that the WHO expressed a view upon the issue?

Crisis situation such as the pandemic bring to the fore two challenges identified by Kline (2016) in relation to the critical media literacy framework. Firstly, while the pedagogical focus is on teaching how the content of media messages responds to political and economic agendas, the problem of media at the level of form is underestimated. Secondly, when it comes to misinformation, media affordances play a crucial role. Word limits imposed by a social media platform can, for instance, cause cherry-picking behaviours of information. Rendering a headline more clickable might bring to misrepresentations of various kinds, while the “sharing without caring” widespread attitude might make misleading content viral.

In relation to fallacies, two main interconnected educational challenges imposed by the virtual medium need to be considered to guarantee an effective learning environment: (1) learning how to recognize fallacies, as any other critical thinking skill, implies a thinking slow process (Kahneman, 2011) which clashes with the scrolling behaviours of today’s news readers; and (2) the digital medium tends to reduce the attention span if not involving interaction. To cope with these issues, we developed the Fake News Immunity Chatbot which is an online publicly accessible game. This game aligns with the pedagogical hallmark of boosting active learning methodologies. That is, through the game, users learn how to fact-check news using fallacies by talking to a set of characters which embody ancient philosophers. Gamification has, in fact, been proven to foster students’ commitment and motivation, which may lead to improvement of critical skills (Huang & Soman, 2013). For example, the GoViralGame⁶ was recently launched by Cambridge researchers to introduce players to four tactics (e.g. using charged language) used to spread fake news online. The concept behind the game is that exposing people to a mild dose of the ways used to disseminate false information will help to generate inoculation. Inoculation Theory (Compton, 2013; McGuire & Papageorgis, 1961) is based on the assumption that “just as vaccines generate antibodies to resist future viruses, inoculation messages equip people with counter arguments that potentially convey resistance to future misinformation, even if the misinformation is congruent with pre-existing attitudes” (Cook et al., 2017, 4).

In our case, the Fake News Immunity Chatbot leverage fallacies as a vaccine to inoculate against fake news. As opposed to GoViralGame, it is focused on misinformation rather than disinformation and on identifying triggers, which can make a news misleading even when the author did not mean to spread false content. To assess the efficacy of gamification to teach critical thinking, we tested the Fake News Immunity Chatbot with two cohorts of postgraduate students (36 participants overall). After having played with the chatbot, students are asked to complete a questionnaire to evaluate the chatbot at different levels. The choice of having students self-reflect upon various aspects of the chatbot instead of merely tracking their interaction times and behaviours is driven by two main reasons: (a) It increases the perception of their role and responsibility as beta testers in a research-led teaching environment, and (b) it serves the learning outcome of making students mull over the role that human computer interaction might have in facilitating rather than replacing human decision making and reasoning (Vinuesa et al., 2020). The gamification exercise is discussed below.

The Fake News Immunity Chatbot has been created with the overall goal of reverse-engineering the manipulation of information. It is designed to use Fallacy Theory to teach citizens how to act as fact-checkers by training them in critical thinking. As users, citizens are in essence signing up to be students of a fact-checking initiative. After having been fronted with a summary of the chatbot rationale, users are introduced to the other participants, the three avatars of the Ancient philosophers Aristotle, Gorgias, and Socrates and the avatars of the members of the research team, among whom they are asked to select an interlocutor. The conversation begins in media res, with the selected avatar asking the user to assess the reliability of a news item, then cross-checking it with the fact-checker’s answer. After this first prompt, if the user decides to be willing to learn how to fact-check through fallacies, they are asked to read a news item and answer questions by the philosophers, while challenged in their decision-making process. For example, Fig. 11.2 shows an example of how the user is guided through the fact-checking process by identifying fallacies.

A screenshot of fake news immunity chatbot. It has avatars of Aristotle, Gorgias, and Socrates at the left. At the right, the chat between the avatars and the user identifies that the post discussed is a word-of-mouth story, as it doesn't give any evidence or trustworthy sources.

As the chatbot is educational, its dynamics beyond the conversational outline have been designed to match those gamification principles that have turned out to be most pedagogically effective (Stott & Neustaedter, 2013):

Drawing from Huang and Soman (2013), we designed the gamification experience accounting for: (1) target audience and context, (2) learning objectives, (3) structure of the experience, and (4) gamification elements. As to the audience, the students that participated in the study were enrolled in the postgraduate modules “Artificial Intelligence and Communication” (MSc in Data Science and Artificial Intelligence) and “Discourse, Rhetoric and Society” (MSc in Strategic Communication) at the University of Liverpool in the United Kingdom. Both student cohorts were introduced during the modules to key concepts related to the (mis)information ecosystem, such as the distinction between misinformation, disinformation, and malinformation (Carmi et al., 2020), the blurred notion of fake news (Tandoc et al., 2018) and the fact-checking process. Both modules took the form of workshops and followed a blended learning approach during which students were presented with notions followed by active learning activities and discussions in small groups. The modules were hosted on Zoom due to the pandemic, which resulted in various challenges, including Zoom fatigue. Students were also provided with an overview of the scope, the goals, and the methodologies adopted within the research project. However, they were not presented with a thorough explanation of fallacy theory and its relevance for misinformation before playing with the Fake News Immunity Chatbot. They were, instead, instructed about the role played by their feedback as beta testers in improving the chatbot in view of its launch to foster their sense of self-ownership and motivation.

The learning objectives of the sessions hosting the gamification experience were to: (1) learn how to fact-check news through fallacies and (2) reflect upon the role played by human computer interaction in learning skills. From the analysts’ point of view, we wanted to understand (1) whether the heuristic implemented in the chatbot is effective in teaching how to recognize fallacies, and (2) what are best strategies to guarantee human-computer trust in the context of the misinformation ecosystem. The latter aspect is crucial in building effective pedagogical digital interventions for data literacy, but it has so far been under-investigated. The majority of studies have tackled the need for human-computer trust scales which cut across domains (Gulati et al., 2019).

However, the information ecosystem is peculiar since social bots are generally discussed as fake news spreaders. They are, therefore, potentially associated with dis-information by the larger public, rather than as gatekeepers of truth. Furthermore, while persuasive technology is usually associated with human-likeliness, this tendency might not apply to a context where radical uncertainty makes peers somewhat unreliable as experts.

The gamification experience was structured in the same way for both cohorts. That is, students had 15 minutes to freely play with the chatbot, after which they were asked to fill in a questionnaire on Qualtrics⁷ to evaluate the chatbot for an estimated time of 10 minutes. Even though students were then encouraged to discuss their experience with their peers, the training activity was mainly self-led to avoid face-threatening situations (e.g. a student might be faster in correctly completing a stage). The students could select which game level (i.e. credulous, skeptic, and agnostic) to start with to allow them to build their own strategies in training as factcheckers (Simões et al., 2013).

The questionnaire has been designed to assess four different facets of the chatbot in line with evaluation criteria set up by Jain et al. (2018). The aspects considered, the associated questions and their underlying rationale are the following:

Rationale: The questions are aimed at understanding what conversational features prompt students’ engagement in a virtual setting. Even though the ultimate goal of chatbot developers is that of achieving human-like conversation, the natural language patterns with the strongest educational value in a gamification environment have been understudied. For instance, the presence of options that allow to delay choices (e.g. “maybe later”, “I do not know”) seem to increase users’ engagement in commercial chatbots (Valério et al., 2020), but might have a different outcome in an educational setting.

Rationale: The questions are aimed at eliciting what personality traits are perceived by the students as positive in a virtual pedagogical interaction, thus facilitating learning. In designing the questions, we used findings from persuasive technologies studies. For example, as explained by Fogg (2002): a) We are more likely to be persuaded by computing technology that we perceive similar to us (principle of similarity), b) we tend to be persuaded by computing technology that offers us praise of some sort (principle of praise), and c) we tend to feel the need to reciprocate when computing technology has provided some benefits to us (principle of reciprocity).

Rationale: The questions are aimed at getting information as to the role played by multimodal input in creating a favourable learning setting. In particular, Q13 it is formulated on the basis of the hypothesis that computing technology that shows the role of authority is generally perceived as more trustworthy and, thus, persuasive.

The results of the questionnaire show that some facets of the chatbot design have been deemed as more controversial than others. To start with, the rhythm of the conversation has variously been perceived as “slow” (40.54%), “just right” (29.73%) or “fast” (27.03%), and “too slow” by a minority (2.70%). No one perceived the rhythm to be “too fast”. The turn-taking has been designed to mimic human behaviour, adding a 250 milliseconds delay per word in coming up with a new conversational move. As to the tone, that has been meant to be colloquial but lexically rigorous, it has been assessed by the majority as “just right” (64.86%). More varied has been the assessment of the presence of a multi-agent conversation, considered “interesting” (38.30%), “informative” (27.66%), but also “confusing” (29.79%) and rarely “boring” (4.26%). Overall, the participation in the conversation has been felt by the majority of the participants as sometimes “active” (37.84%) or “just right” (32.43%), but less frequently as “not active” (18.92%) or “active” (10.81%). As to Q5, 20% of respondents remarked that the philosophers did not remember their name, 16% recognized the usefulness of having a set of questions to help them in the fact-checking process, and 50% declared that the binary choice was sometimes difficult to make.

Turning to the chatbot personality, there is no clear preference for one philosopher over another. The adjectives providing reasons for the positive sentiment are variously distributed as displayed in Fig. 11.3, with the most common adjective being “knowledgeable” (19.59%.).

A pie chart of the frequency of adjectives describing reasons for liking a philosopher avatar. Knowledgeable, 19.59%. Helpful, 15.46%. Smart, 12.37%. Provocative, 10.31%. Open minded, 8.25%. Friendly, 7.22%. Expert, 6.19%. Reliable, 6.19%. Unpredictable, 4.12%. Nosy, 3.09%. Organized, 2.06%.

The majority of the respondents answered “no” to Q8 (64.86%), suggesting that liking a character does not increase the likelihood of reaching out to them to seek help, most likely to avoid face-threatening feelings. From the open question Q9, it emerges that five qualities (frequency > 5) are commonly perceived as characterizing a good teacher beyond the fallacy scenario: namely, knowledgeable, friendly, helpful, clarity, and passion, as displayed in Fig. 11.4. Therefore it seems that folk values associated to quality in an ideal pedagogical setting underpin positive attitudes towards different participants in the praxis of a digital game.

A circle chart plots the most frequent qualities defining a good teacher. The qualities in descending order are as follows. Knowledgeable. Friendly. Helpful. Clarity. Passion.

Moving to questions related to the interface, there is variation in terms of associated feelings (Fig. 11.5), with no single feeling emerging as significant. In fact, the most common response was “other”, closely followed by “relaxed”, “overwhelmed”, “bored”, and “amused.”

A bar graph plots feelings triggered by the chatbot interface. The values are as follows. Relaxed, 22%. Bored, 18%. Overwhelmed, 19.50%. Amused, 18%. Other, 24%. Values are estimated.

Browsing through the explanations provided for “other”, a feeling of confusion regarding actions to take is the one most frequently voiced. Similarly, the advocated changes in the interface cut across different facets, including stylistic features such as font, colours, examples, language, avatars, and others (Fig. 11.6).

A pie chart plots aspect that could be improved in the chatbot interface. Other, 27.59%. Colors, 22.41%. Language, 20.69%. Font, 12.07%. Examples, 12.07%. Avatars, 5.17%.

Focusing specifically on the avatars, the large majority of respondents would have not changed any of them (Fig. 11.6). As to trustworthiness (Q11), only 21 students provided an answer. One third of the responses pointed to a lack of preference, while among the chosen avatars, the ones most frequently selected are Aristotle, the Principal Investigator of the project, and Socrates. Unfortunately, few participants provided a justification for their choice. When reasons were given, these related to familiarity for the Principal Investigator and helpfulness for Socrates (e.g. “Socrates, even if he didn’t provide straight answers, he pointed me in the direction that I had to look in”).

In terms of functionality, students encountered a wide range of fallacies, with cherry- picking (25.30%), hasty generalization (13.25%), and red herring (10.84%) on top of the frequency scale. Apart from one student, they all declared they were able to describe the fallacy they encountered. Manually checking the answers, they turned out to be all correct. Furthermore, differences in phrasing used by different students to describe the same fallacy show that they did come up with personal elaborations without stemming from dictionary-like definitions (e.g. cherry-picking: “Selectively picking supports to provide a predecided argument”; “The information might be chosen for a specific purpose, but may not tell the whole picture or may misguide you”). Interestingly, the students tend to think they might be able to recognize the fallacy they learnt in different contexts, such as different news (Fig. 11.7). Even though the responses were overall positive, there is uncertainty as to whether they will be able to fully transfer the learnt knowledge.

A horizontal bar graph plots the perceived ability of recognizing fallacies across different context. The values are as follows. Yes, 54%. Maybe, 46%. No, 0%.