Promoting and countering misinformation during Australia’s 2019–2020 bushfires: a case study of polarisation

The 2020 Australian ‘Black Summer’ bushfires (a.k.a., wildfires) burnt over 16 million hectares, destroyed over 3500 homes, and caused at least 33 human and a billion animal fatalities,¹ and attracted global media attention. During the bushfires, as in other crises, social media provided a mechanism for people in the fire zones to provide on-the-ground reports of what was happening around them, a way for those outside to get insight into the events as they occurred (including authorities and media), but also a way for the broader community to connect and process the imagery and experiences through discussion. The lack of the traditional information mediator or gatekeeper role played by the mainstream media on social media permits factual errors, mis-interpretation and outright bias to proliferate without check in a way it could not in decades past. Our previous analysis of online discussion at this time (Weber et al. 2020) showed:In Weber et al. (2020), we collected 18 days of Twitter data to examine the discussion on #ArsonEmergency and replicate findings reported in ZDNet (Stilgherrian 2020). In the process, analyses revealed two clearly polarised groups in the retweet network, distinctly different in behaviour and content. Supporters promoted the narrative that Australia’s ‘Black Summer’ bushfires were primarily caused by arson, relying on misinformation and biased reporting, while the Opposers countered with official announcements and fact-check articles. We determined that the Opposers responded to the growing Supporter and Unaffiliated account activity on #ArsonEmergency, when it was exposed in a ZDNet article (Stilgherrian 2020). The publication of the ZDNet article was the trigger that drew the attention of the mainstream media, which promulgated the exposé further, drawing in many more Unaffiliated accounts and noticeably changed the nature of the discussion. Using a different bot detection system, many fewer bots were found than in the analysis reported in the ZDNet article.

The analyses did not explore whether the polarisation in the retweet network was also reflected in networks made from other interactions, nor how the groups responded to each others’ behaviour beyond the initial Opposer response, nor any exploration of growth patterns of the groups, in case they artificially grew their numbers with external help (e.g. from beyond national borders). Questions remain about the communication strategies used, including whether there is any evidence of coordinated amplification (Weber and Neumann 2021), which may suggest the presence of an organised misinformation or disinformation campaign. Although evidence of bots was found to be limited, later work by Graham and Keller (2020) and Keller et al. (2020) clarified the significant role of trolls in the discussion, arguing that it amounted to participation in a broader disinformation campaign, thus exploration of trolling and similar inauthentic behaviours (Gleicher 2018) remain to be explored. Although we do not consider the intention of those promulgating false narratives in the data, whether the actions constitute an (unintentional) misinformation or a (deliberate) disinformation campaign, our analyses can shed light on the presence and activity of trolls and other inauthentic accounts.

We present a mixed-method analysis of the Twitter activity using the term ‘ArsonEmergency’ approximately a week before and after the publication of the ZDNet article (Stilgherrian 2020), which we treat as an ‘intervention’ to counter misinformation in the discussion. We make use of a combination of complementary social network analyses (SNA), behavioural and content analyses pivoted around the intervention, the polarised groups and unaffiliated discussion participants. Analysis of the networks of different interactions in the data reveal how central these groups became and to what degree they connected to each other and the broader discussion. We also consider patterns of inauthentic behaviour by presenting and demonstrating a simple method for exposing inauthentic tweets by their text patterns.

The study of Twitter during crises and times of political significance is well-established (Bruns and Liang 2012; Bruns and Burgess 2012; Flew et al. 2014; Marozzo and Bessi 2017; Graham et al. 2020), and has provided recommendations to governments and social media platforms alike regarding its exploitation for timely community outreach. The social media response of the Australian Queensland State Government was praised for its use of social media to manage communication during devastating floods (Bruns and Burgess 2012), and analyses of coordinated behaviour have revealed significant organised anti-lockdown behaviour during the COVID pandemic (Graham et al. 2020; Magelinski and Carley 2020; Loucaides et al. 2021) and in the lead up to the January 6 Capitol Riots in America (Scott 2021; Ng et al 2021). The continual presence of trolling and bot behaviour diverts attention and can confuse the public at times of political significance, whether it is to generate artificial support for policies and their proponents (Keller et al. 2017; Rizoiu et al. 2018; Woolley and Guilbeault 2018), harass opponents (Keller et al. 2017; CREST 2017) or just pollute existing communication channels (Woolley 2016; Nasim et al. 2018; Kušen and Strembeck 2020). Malign actors can also foster online community-based conflict (Kumar et al. 2018; Datta and Adar 2019; Mariconti et al. 2019) and polarisation (Conover et al. 2011; Garimella et al. 2018; Morstatter et al. 2018; Villa et al. 2021).

Studies of the spread of rumours and counter-efforts on Twitter during crises, and people’s response to them, have also confirmed the importance of authorities injecting reliable clarifications into the information space. Wang and Zhuang (2018) studied how Twitter users responded when they encountered misinformation in a number of different discussions. They developed a decision tree that modelled a variety of user behaviours including initially attempting to confirm a rumour, simply doubt it or disseminate it, and then also behaviour after their disseminated post had been debunked in a reply. Populating the decision tree with real-world data, they found that questions regarding rumours were frequently answered, but of the vast majority of people who disseminated the rumours in question (via retweet), fewer than 20% of users would take some remedial action if their retweet was debunked. The decision tree concept was extended by Agarwal et al. (2022) using a game-theoretic approach, creating decision models that examined how authorities can select which rumours to debunk or clarify, and how important timeliness is compared with the quality of the clarification. Using several datasets relating to true and false rumours, they showed the value of authorities providing clarifying information, which dampened the dissemination of false rumours, as well as that, up to a point, it is better for authorities to delay debunking a false rumour until high quality information is available. Hunt et al (2020) examined the cross-platform information sharing behaviour of Twitter users during Hurricanes Harvey and Irma, which both affected America in September 2017. Although most retweets were by non-verified accounts, 90% of verified account retweets were of factual reports from government agencies. The most frequently mentioned URLs and URL domains belonged to government agencies and news media, and most referred to easily accessible public websites, rather than posts on other social media sites, similar to previous findings regarding EXTERNAL URLs (Weber et al. 2020). They also found that posts debunking rumours can have a continued effect: they observed that a given rumour lasted only a third as long during the second hurricane, as the debunking information was still present in the online discussion. Thus, there is significant value in government agencies having an active social media presence during crises, as a source for credible and correct information.

Misinformation can be regarded as the sharing of unintentionally false information, while disinformation is shared knowing that it is false or biased in a deliberate attempt to deceive others (Wardle and Derakhshan 2017). Information campaigns rely on groups of accounts disseminating a particular narrative as propaganda, using truthful information, true information with a biased presentation, or outright falsehoods (Kavanagh and Rich 2018). The blending of true and false information makes it harder to identify (Starbird 2019) and easier to convince others to share, thereby making modern information operations ‘participatory’ in nature (Starbird et al. 2019). Due to the degree to which social media is enmeshed in modern life, misinformation on social media is a topic of much study (Kumar and Shah 2018; Starbird 2019; Singer and Brooking 2019; Graham et al. 2020), with growing attention to its overall effect on society (Starbird 2019; Carley 2020), but many relevant current events are yet to be explored in the peer-reviewed literature. Instead, researchers have turned to other methods to quickly warn of the dangers of misinformation via other channels; examples include Graham and Keller ’s interview with the technology magazine ZDNet (Stilgherrian 2020) and their follow-up article on The Conversation (Graham and Keller 2020), a publisher of ‘research-based news and analysis’,⁴ while commissioned reports provide an opportunity to present more comprehensive yet still not peer-reviewed analyses (e.g. Wardle and Derakhshan 2017; Graham et al. 2020; Smith et al. 2020). Because social media has become such a mainstay of modern communication, misinformation on social media is often amplified on the mainstream media (MSM), or by prominent individuals, often when it aligns with their ideological outlook, which then feeds back into social media as people discuss it further; such cycles have been known to be deliberately fostered (Benkler et al 2018; Starbird and Wilson 2020; Badham 2021). Patterns of fire-related misinformation similar to those observed on #ArsonEmergency were repeated in the US during Californian wildfires in mid-2020, even causing armed vigilante gangs to form to counter non-existent Antifa activists who were blamed for the fires on social media.⁵ Arson was again blamed for the 2021 fires around the Mediterranean, throughout southern Europe and in northern Africa,⁶ even as the United Nations’ Intergovernmental Panel on Climate Change released its sixth Assessment Report stating that humans’ effect on climate is now ‘unequivocal’ (IPCC In Press). Furthermore, when the misinformation involved relates to conspiracy theories involving public health measures during a global pandemic, the risk is that adherents will turn away from other evidence-based policies, as we see with vaccine hesitancy (Ball and Maxmen 2020), adoption of flat earth beliefs (Brazil 2020) and other conspiratorial anti-government sentiments (The Soufan Center 2021).

Online conversation dynamics can produce information environments vulnerable to misinformation, such as echo chambers and filter bubbles (Pariser 2012; Bruns 2019). While studying COVID-19 vaccine-related narratives in mid-2020, Smith et al. (2020) highlight two primary ‘market failures of the information industry: data deficits and data oversupply’ (p.20, Smith et al. 2020). These both relate to the amount of information, particularly credible information, in a discussion relative to the demand. A data oversupply results in a crowded information space, where people are easily confused and overloaded by (sometimes contradictory) information, which causes them to disengage. A data deficit, in contrast, occurs when there is a lack of credible information about an issue but significant demand for it. Although data deficits are not deliberately created (experts may not realise what information people require or which require it), it can be vulnerable to the introduction of misinformation and exploited with disinformation. In our study, it is arguable that the ’data deficit’ of #ArsonEmergency was, in fact, created deliberately ( Graham and Keller 2020, argue this), thereby establishing a data void into which disinformation based on the arson narrative could be allowed to flourish before linking it to broader discussions.

Studies of virality of social media content also examine similar natural experiments. The question of predictability of retweet cascades has been explored. Cheng et al. (2014) examined how to predict the magnitude of a cascade of retweets or Facebook shares, finding that its breadth (the maximum frequency of shares within a certain period after the original post) is a strong indicator of the final overall number of shares. Rizoiu et al. (2018) proposed a point process model of retweet cascades to measure how influential accounts are in their exploration of social bot manipulation of Twitter during the 1st 2016 US presidential debate. Other studies have examined how the ‘buzz’ of Facebook posts can predict whether they can go viral (Deusser et al. 2018), and then also the nature of these ‘buzzes’ as ‘firestorms’ of anger, ‘lovestorms’ filled with messages of affection and support or ‘hot topics’ of enthusiasm (Jansen 2019). Strathern et al. (2020) explored the triggers behind firestorms, particularly those based on moral infractions. Conclusions from these studies can inform communication strategies to respond to bad press or other events of unwanted attention. In this study, it can be argued that the intervention was aimed at exposing a previously obscured discussion to generate a response to it, rather than to quell a response.

Finally, the natural experiment model has been used in the study of online discussion dynamics. Wang et al. (2018) studied the effect of user verification on the spread of fake news on Weibo, finding that fake news is dampened when people know they are discoverable (via verification with the platform), but this effect is outweighed by greater engagement if the verification is made public (e.g. with a badge icon) due to the credibility it implies. A study of which EU politicians benefit most from bot followers on Twitter examined the drop in follower numbers after Twitter’s July 2018 purge of bot accounts (Silva and Proksch 2020). Results showed the radical right lost the most. Given political parties rely on being ‘popular’ and the media’s use of Twitter to judge this, it suggests they received the most benefit, as it drew unwarranted attention to their causes. Here, our natural experiment model allows us to examine the change in dynamics of an online discussion before and after the ZDNet article’s publication.

The aim of these research questions is to provide guidelines to exploring a communication space in which it is known there is some kind of misinformation campaign using a natural experiment approach. They encourage identification of the communities promoting and potentially countering the misinformation, differences in their behaviour and communication strategies, and to what extent inauthentic behaviour (such as use of automation and trolling) is exploited. Furthermore, based on this context, they direct the examination to consider any changes before and after the intervention and how they relate to the intervention.

In the remainder of this paper, we describe our mixed-method analysis and the datasets used. We briefly recap the timeline analysis (Weber et al. 2020), before examining the growth of the discussion, comparing it to other contemporary related and unrelated contentious hashtag-based discussions. We address our research questions and discuss how our findings can be generalised to analyse and counter misinformation elsewhere before concluding with thoughts for future work.

This study focuses on about a week of Twitter activity before and after the publication of the ZDNet article (Stilgherrian 2020), which effectively acted as an intervention to counter the misinformation in the discussion. The ZDnet article was published at 6:03am GMT (5:03pm AEST⁸) on 7 January 2020, and was then reported more widely in the MSM morning news, starting around 13 h later. We use the same temporal markers as Weber et al. (2020) to define three phases:Our primary interest, however, lies in the differences before and after Phase 1 ends.

Figure 1 shows the number of tweets posted each hour in the ‘ArsonEmergency’ dataset, and highlights the phases and notable events including: the publication of the ZDNet article; when the story hit the MSM; the time at which the Rural Fire Service (RFS) and Victorian Police countered the narratives promoted on the #ArsonEmergency hashtag; the publication of a Conversation article clarifying the ZDNet findings (Graham and Keller 2020); and the clear subsequent diurnal cycle.

Since late September 2020, Australian and international media had reported on the bushfires around Australia, including stories and photos drawn directly from social media, as those caught in the fires shared their experiences. No one hashtag had emerged to dominate the online conversation and many were in use, including #AustraliaFires, #ClimateEmergency, #bushfires and #AustraliaIsBurning.

The ZDNet article publication generated an immediate response, with over 1, 200 tweets in Phase 2 by 927 accounts, 824 of which had not tweeted in Phase 1. Nearly 85% of the Phase 2 tweets were retweets, and 60% of those promoted the ZDNet article, including 357 retweets of a single tweet mentioning it. Most Phase 2 activity occurred in the Australian evening period, and the diurnal cycle visible in the time series indicates the majority of activity was local to Australia (or at least to its timezones). The biggest reaction came immediately after the story reached the morning news the next day, while the publication of the Conversation article appears not to have generated a significant response.

To consider if the pattern of discussion growth in ‘ArsonEmergency’ is typical, we compared the discussion with two other contemporary discussions in terms of user growth (i.e. number of new accounts joining the discussion) and tweet growth (Fig. 2). The similarity in the user and tweet growth lines indicates that as new accounts joined each discussion, they usually only posted a single tweet. The #Brexit discussion lacks a clear intervention event and so its growth is smooth and consistent.⁹ In contrast, ‘AustraliaFire’ discussion appears to be a hashtag campaign instigated by people in Pakistan and Germany resulting in 45k retweets. Many of the retweeting accounts were suspended, so it is possible they were driven by botnets, and the campaign stops growing suddenly after a few days. The ‘ArsonEmergency’ dataset’s growth pattern clearly shows the point of the intervention, but it continues to grow for several more days after the initial response.

The relative behaviour of the communities over the collection period, shown in Fig. 4, informs several key observations. The first is the impact of the story reaching the MSM: the peaks of both Opposer and Unaffiliated contributions are on the morning of Phase 3, immediately after the story appeared on the morning bulletins. Despite the much greater number of Unaffiliated accounts (11,782), their peak is only a little more than twice that of the 593 Opposer accounts. Unaffiliated and Supporter accounts are active during the entire collection, but Supporters’ activity is prominent each day in Phase 3, and peaks on the second day of Phase 3. That peak might have occurred as a response to the previous peak, as by that time the news would have had a full day to disseminate around the world. By reaching a broader audience via the MSM, more Supporter accounts may have been drawn into the online discussion. Supporter activity increased dramatically in Phase 3, potentially in response to the Opposer pushback in Phase 2.

Analysis confirms that the composition of the Unaffiliated participants did change across the phases. Few of the 1680 Unaffiliated accounts active in Phase 1 appeared in the later phases (30 in Phase 2 and 427 in Phase 3), and their contributions were not significantly higher than the new Unaffiliated accounts (Fig. 5). It is clear that Supporters and Opposers’ contributions changed dramatically, however. Supporters were the most active in the larger phases, particularly in the last. Opposers were most active in the final phase also, but contributed at a rate of less than half that of Supporters, though they did contribute slightly more than Supporters per account in Phase 2, immediately after the release of the ZDNet article.

Examining the accumulation of new accounts (Fig. 6a) and new tweets (Fig. 6b) shows that #ArsonEmergency was steadily accruing Supporters until the ZDNet article (Stilgherrian 2020), at which point the community was established and remained active for several days into Phase 3. The Opposer community joined almost entirely in Phase 2, and its activity was mostly confined to that phase, while the Unaffiliated continued to join the discussion well into Phase 3. The publication of the ZDNet article appears to have drawn in large numbers of Opposers and Unaffiliated, while the Supporter growth immediately plateaued.

Finally, a clear diurnal effect can be in Fig. 4 with daily peaks of activity occurring during Australian daytime hours, implying that the majority of the activity is domestic. Analysis of the ‘lang’ field in the tweets¹⁰ confirmed that over 99% of tweets used ‘en’ (English, 90.5%) or ‘und’ (undefined, 8.7%). An analysis of self-reported locations indicated that most accounts were Australian (88% of Opposers, 77% of Supporters and 72% of Unaffiliated), while foreign accounts were most often from the US or UK.

User behaviour on Twitter can be examined through the features used to connect with others and through content. Here, we consider how active the different groups were across the phases of the collection, and then how that activity manifested itself in the use of mentions, hashtags, URLs, replies, quotes and retweets. The statistics in Table 2 are a superset of those discussed in our previous work (Weber et al. 2020), but serve to inform the later analyses in this paper.

Overall, as shown in the bottom section of Table 2, Supporter accounts tweeted much more often than other accounts, and used more hashtags, mentions, quotes, replies and URLs, but retweeted less often than both Opposers and Unaffiliated accounts. This suggests that Supporters were generating their own content (not just retweeting it), and attempting to engage with others through the use of platform features, implying a high degree of motivation on their part. These mere frequencies do not provide an understanding on who was being interacted with, the nature of those interactions or their content.

When contrasting the content of the two affiliated groups, we considered the hashtags and external URLs used. A hashtag can provide a proxy for a tweet’s topic, and an external URL can refer a tweet’s reader to further information relevant to the tweet, and therefore tweets that use the same URLs and hashtags can be considered related.

To investigate whether coordinated dissemination of content was occurring, we performed co-retweet, co-hashtag and co-URL analyses (Weber and Neumann 2021), searching for sub-communities of accounts that retweeted the same tweets, and shared the same hashtags, URLs and URL domains within the same timeframes (denoted by \(\gamma\)). The analyses result in weighted networks consisting of the sub-communities as disconnected components of accounts, the edge weights of which indicate the frequency of co-linking or co-mentioning of a hashtag. Further, to examine how the sub-communities relate to one another, we can then re-introduce the URLs and domains as explicit ‘reason’ nodes into these networks, making them bigraphs in which communities are joined according to these ‘reason’ nodes (Weber and Neumann 2021).

There is good reason to believe that #ArsonEmergency was deliberately created (Graham and Keller 2020), forming a ‘data deficit’ (Smith et al. 2020) for the sharing of misinformation regarding the arson narrative. This could form an isolated echo chamber for recruiting a new user base and radicalising it. Then, once established, it could link into the broader discussions by using a variety of hashtags in their tweets, which is what we observed. Radicalisation may not have been the ultimate goal of this particular community, but the technique could equally be used to garner support. Large isolated communities of accounts have been discovered by researchers before,¹⁷ and moderate levels of activity could remain undetected, particularly if participants avoided using other hashtags in their #ArsonEmergency tweets (which would link to other hashtag communities). #ArsonEmergency was discovered because participating accounts were known to Graham and Keller . This study provides confirmation of the presence of trolling, but no direct evidence of disinformation (cf., Graham and Keller 2020; Keller et al. 2020).

At a high level, this situation involved a number of events: The bot analyses were preliminary and did not stand further scrutiny (Graham and Keller 2020; Weber et al. 2020),¹⁸ but only days later the researchers clarified that much of the behaviour may have been due to (human) trolls (Graham and Keller 2020), and later presented evidence to suggest that the activity may have been coordinated with a broader disinformation campaign (Keller et al. 2020). The initial article, however, was enough to draw public attention to it, initially through ZDNet’s audience, spurring the Opposer community to form. By mid-morning after the news reached the breakfast MSM, it had had time to spread around the world as well as become known amongst the broader Australian online community. By the end of Phase 2, official announcements refuting the information were reported, and these became the focal points of the Opposer and Unaffiliated URL sharing.

In this way, a counter-misinformation strategy could involve: There are limitations with this strategy and reasons why its success in this case study may be tightly associated with the specific context. By the time the broader Unaffiliated community entered the discussion, not only were there official statements and fact-checks to share, but they had also seen months of coverage of the bushfires on televisions around the world, so in a sense they were already primed to reject claims that the fires were not unusual. The ‘AustraliaFire’ discussion provides a strong example of this: the hashtag campaign was underpinned by mass retweeting of a few tweets that focused on the suffering of people and animals, thus establishing a moral imperative to respond to the crisis, strengthened with emotion. To be successful then, counter-efforts should either prime the Unaffiliated somehow (this is probably typically uncontrollable) or rely on some aspect of the Unaffiliated community’s mindset that acts as the priming mechanism, whether it be appealing to widespread ethical standards, rational thinking and/or the use of reputable sources.¹⁹

The Supporter and Opposer communities were relatively easy to label, based on manual inspection of their most retweeted accounts. In studies of larger datasets, other methods may need to be considered, relying on automated analyses or other cues. Textual analyses, such as topic modelling of profile descriptions, could reveal a community’s major interests, but such descriptions are often very terse and sometimes do not align with account behaviour. In their study of the 2017 German election, Morstatter et al. (2018) identified several very large clusters of accounts using Louvain (Blondel et al 2008), but then determined the content of each major cluster using a hierarchical topic modelling technique applied to the hashtags they used. As discussed in Sect. 3.3, hashtags can be considered proxies for discussion themes. Furthermore, the predominant language used in the clusters also helped reveal distinct German-speaking alt-right and English-speaking alt-right clusters.

Our previous analysis (Weber et al. 2020) indicated that Supporters used original content and personal engagement more than Opposers and were active throughout the discussion. The analyses of the discussion’s interaction networks in this work suggest Supporters were deeply and widely embedded. Yet the analysis of inauthentic behaviour revealed that significant portions of these interactions with the broader community were not constructive, and some could be considered clear harassment.

These findings emphasise the value of mixed-method analyses, including using complementary methods, which take advantage of their relative strengths.

Methodologically, the approach taken in this paper has taken advantage of recent advances in network science, bolstered them with established network, statistical and bot analyses, and proposed text patterns as a simple approach to identifying inauthentic behaviour. This final element helps illuminate the tone of interactions between the Supporter community and those outside it, on which network analyses shed little light. We observed that Supporters’ inauthentic behaviour seemed to mostly increase after the intervention, particularly the targeting of tweets with @mentions. It remains unclear whether this contributed to the Unaffiliated accounts’ shift away from the arson narrative.

The co-activity analyses used in this study further validate the utility of the approach, which has gained attention recently (Weber and Neumann 2021; Pacheco et al. 2021; Magelinski et al 2021). Similar recent work has applied co-URL and co-domain analysis to expose information polluters on the basis of the news they disseminate (and the sources from which it comes) (Truong et al. 2022). The inclusion of a temporal constraint aids in identifying concerted coordination over grassroots coordination, but improvements could be introduced to account for the scale of the discussion and co-activity coincidence rates.

Further research is required to examine the dynamic aspects of the social and interaction structures formed by groups involved in spreading misinformation to learn more about how to better address the challenge they pose to society. Future work will draw more on social network analysis based on interaction patterns and content (Bagrow et al. 2019) as well as developing a richer, more nuanced understanding of the Supporter community itself, including revisiting the polarised accounts over a longer time period and consideration of linguistic differences. A particular challenge is determining a social media user’s intent when they post or repost content, which could help distinguish between disinformation intended to deceive, and merely biased presentation of data or misinformation that aligns with the user’s worldview.