The five tests: designing and evaluating AI according to indigenous Māori principles

Artificial intelligence technologies (AI) are being rapidly deployed across an array of high-stakes areas, from welfare to law enforcement, healthcare, and recruitment. For technology pundits, this transformation is a positive one, accelerating innovation and ushering in progress and prosperity (Brynjolfsson and McAfee 2011, 2014). But more critical research has highlighted the social and environmental fallout of AI-driven shifts, its ability to extract capital in novel ways while increasing precarity and inequality. Digital platforms allow homework under a piecework model, a highly exploitative form of labor (Dubal 2020). AI systems can perpetuate gendered stereotypes and contribute to racial injustice (Buolamwini and Gebru 2018; Benjamin 2019). AI systems meticulously track workers, rewarding and punishing individuals based on their performance (Munn 2017). And if workers have suffered, so too has the environment, as high-carbon, high-energy technologies consume natural resources and enact a heavy toll on a warming planet (Munn 2022b).

So while AI is novel, it often seems to continue long-standing paradigms of technology in the service of capital, reducing agency and autonomy (Marx 1977), increasing the precarity of labor (Berardi 2009), undermining the well-being of workers (Huws 2014), and amplifying forms of racialized and gendered inequality (Noble 2018; Beller 2018). This scholarship suggests that the human harms documented in recent AI-driven initiatives are not merely “teething problems,” but part of a broader paradigm of capitalist and colonialist values at the core of our current economic and technological systems.

Where do these values come from? AI technologies, like any technology, are embedded with certain values, norms, and priorities drawn from a particular (colonial) history and a particular development environment (white, male, patriarchal, heteronormative). Beller (2018) has shown how the historical development of computation, broadly understood, was intimately connected with capital and its drive to instrumentalize racialized and gendered difference. This means that the values at the heart of contemporary AI systems are not neutral or disinterested, but rather particular and purposeful. Far from being universal, artificial intelligence can be better understood as “artificial Western ethno-intelligence” (Williams and Shipley 2020). For this reason, MacQuillan (2019) suggests that whenever AI is adopted without constraints, it will amplify the injustice of the status quo.

Because current AI technologies and their values compound inequality and injustice, an alternate set of values, paradigms, and priorities are urgently needed. AI developers cannot carry out the deep transformations needed to support inclusivity and sustainability while continuing to draw on the same hegemonic epistemological and ethical systems. As AI systems grow in power and permeate further into high-stakes domains of political and social life, the question we are faced with becomes more stark. Will AI technologies continue to extract personal data, to exacerbate inequalities of wealth and power, and to render life more precarious for some of the most marginal and vulnerable (Mejias and Couldry 2019; Ciston 2019; Checketts 2022)? Or can we welcome new knowledge paradigms, establish alternate priorities, and progress towards technologies that underpin care for each other and for the earth in crisis?

Where can we draw an alternate set of AI priorities and principles from? Indigenous cosmologies, epistemologies, and ways of being and doing provide one promising approach. Williams and Shipley (2020) suggest that AI applications might benefit from indigenous wisdom, augmenting the often narrow Western focus on utility and efficiency with concepts such as harmony with others, deeper ecological understanding, and close kinship networks. Similarly, Irwin and White (2019, 1) assert that “indigenous philosophy has a lot to offer the world, as we face the necessary shift from an exploitative, extractive economy, to a more sustainable one.”

The aim here is not to “diversify” (in a superficial sense) AI technologies, nor to “solve” or streamline existing AI processes, but to instead radically challenge the foundational assumptions of these technologies. At the same time, this article seeks to do more than critique or debunk. Indeed one of the motivations for this research is that the 5 tests are actionable or operationable, suggesting concrete ways they might be employed in design and evaluation.

This article builds on very recent work exploring indigenous AI alternative approaches to AI technologies. Lewis et al. (2020) carried out workshops and interviews with a number of indigenous groups across Aotearoa, Australia, North America, and the Pacific to develop a rich position paper concerning indigenous protocols and artificial intelligence. While promising, the authors acknowledge that this research is very much in progress. The paper’s diverse mixture of technology descriptions, design guidelines, artworks, and poetry reflect this nascent quality.

Some of this research has started to consider how AI technologies might be conceived and constructed. After working closely with Aboriginal technologists, Abdilla et al. (2021) have shared their insights about how an indigenous-centered AI might be developed. The authors argue that indigenous AI should be regional in its conception and development, be guided by local indigenous laws, and be designed with future cultural and technical interrelationships in mind.

Similarly, but in a Māori context, Shedlock and Hudson (2022) have offered a kaupapa Māori model for the creation of IT artifacts, including AI applications. The duo argue that current approaches to AI reproduce colonial paradigms and historical inequalities. What is needed is a solution developed by Māori and for Māori. The model has three core components: adequately framing the purpose and aim of the artifact according to Māori knowledge systems; meaningfully engaging with end-users during the design process; and maintaining accountability and rapport with communities over the lifetime of the project. These interventions begin to move from theory to practice, exploring how indigenous values might be designed into products and services.

How might we begin to design and evaluate AI from an indigenous perspective? This article offers one approach by taking up the work of Māori scholar Sir Hirini Moko Mead. Mead is a highly regarded anthropologist, historian, and prominent Māori leader who has founded an indigenous tertiary institution and also represented several iwi in disputes. Mead (2016) unpacked key Māori concepts, practices, and paradigms in his groundbreaking Tikanga Māori book, which seeks to provide guidance about tikanga Māori or the correct way of doing something.

After stepping through each of these concepts, Mead closes the book with what he calls the five tests. Mead recognises that there are new global issues that will constantly emerge, from surrogate motherhood to same-sex marriage and genetic engineering. These novel issues have not been encountered before and are not explicitly dealt with by Tikanga Māori or Mātauranga Māori. An existing Māori position cannot simply be plucked from history or tradition. It must be discovered.

To aid in this discovery, Mead (2016, 336) offers the five tests as a framework of assessment, “a method or methods for assessing a situation or event that challenges our thinking and our values.” Mead (337) stresses that this process results in a position, not the definitive position. Different people and communities will make the assessment in different ways. Despite this disclaimer, a tikanga Māori framework can be immensely helpful in considering a controversial new issue from a Māori (vs a Western or non-indigenous) perspective, working through the benefits and risks, and arriving at a viewpoint. The next sections step through each test, explain key concepts, and discuss how they could be applied to AI technologies.

As the problems with current AI paradigms become increasingly clear, the need for alternative visions, values, and frameworks grows urgent. Indigenous concepts from Aotearoa provide a distinctly different set of principles and priorities—a way of knowing and being at once ancient and fresh that productively challenges Western technocratic norms. Importantly, these are not simply high-minded ideals but pragmatic principles that can be applied to novel circumstances and new technical conditions. The Five Tests provide a means for carrying out this evaluation, allowing us to weigh up the potentials and problems of AI according to a rubric that centers on human dignity, communal integrity, and ecological sustainability.

Designing focuses on the form and function of technology, its appearance and its operations. This is not just the user interface, but more fundamentally the set of decisions that lead to this particular product, this particular technology. In the context of AI, this enfolds aspects like the production of training data, the definition of “ground truth,” the architectures used (i.e. transformer vs convolutional), the optimization function, the deployment of the model, and the feedback mechanisms it surfaces to users. These aspects, from classification to explainability and outputs, can be understood as the design space of a model (Morris et al. 2022a, b). The Five Tests can be applied to these decisions-in-progress, guiding the development of an AI-based intervention. How might this design respect the sacred (tapu)? How might it preserve or enhance the life force of the people and environments it impacts (mauri)? If this design has negative impacts, how might those be reconciled in an acceptable way (take-utu-ea)? These kinds of questions can be asked again and again throughout the design process, from conception through to development and launch. They provide a very different set of criteria from typical business maxims. Engaging genuinely with these questions and “resolving” them through code, architectures, interfaces, and affordances would result in a distinctive technology, something more considered, more inclusive, and more ecologically attuned.

Design also implies redesign, the ability to assess how something is performing and alter particular aspects to improve it. This would apply to AI products and services that have already been developed and deployed. How exactly might these tests be taken up by companies or organizations in this case? This process might begin with a workshop that provided a foundational understanding of indigenous values. This workshop would also enable different stakeholders to come together around the evaluation process. This kind of cross-organizational collaboration is crucial considering the diverse forms of data and feedback that are required. Holistic evaluation could certainly include an engineering team and their quantitative metrics—but might also encompass an environmental report and a series of interviews with community leaders. For some of the tests, evaluative instruments would need to be designed; for others, like Mauri, existing measures might be used. Each of the Five Tests could be weighted according to their importance for the AI developer and for the end-users involved. Scores for each test would highlight areas that need improvement and further iteration.

Critics might object that these measures are unsatisfactory—somewhat improvised and incomplete—and they would be correct. But this is what product development and policy development look like in practice, a process of attempts and iterations. By muddling through (Lindblom 2018) and comparing against prior versions, a material artifact such as code, infrastructures, platforms, or legislation directed at them is gradually improved. Version 0.2 is slightly more diverse, or equitable, or sustainable than Version 0.1. Perfect is the enemy of good.

The more serious criticism in this context would actually be co-option. Indigenous principles are not a smorgasbord where principles can be chosen as desired. In the same vein, splicing one or two concepts into a broader framework of Western values too often leaves them watered down or tokenistic. It would be easy for governments and corporations to gain social and cultural prestige by superficially parroting some of these values without any significant commitment behind them—indeed, we see such a pattern repeatedly in the past. For this reason, the pragmatic, present-focused response discussed above needs to be accompanied by a slower reflection on the nature of power and one’s role within it.

The second pathway might thus be termed decolonizing, a deeper and more sustained confrontation with current AI regimes. Band-aid fixes and nods to “diversity” or more “humane” AI have proven to be woefully inadequate (Munn 2022a). The failure of such superficial interventions only serve to demonstrate that current technologies are built on a far deeper strata of capitalism and coloniality, perpetuating systems of inequality and oppression. For all their purported rationality and neutrality, technologies often reinforce this violence, causing collateral damage to the marginal and the vulnerable who are least equipped to deal with it (McQuillan 2022).

The exact nature of the Five Test’s provocation to current AI regimes is a matter of interpretation, but a few challenges might briefly be sketched. First, it rejects the generic, universalizing frame often imposed by technology companies and even by AI ethicists. Instead, it is grounded in particular worldviews, it highlights communal needs, and it tends to frame impacts according to local norms. Second, it refuses to neatly compartmentalize human well-being and ecological well-being. Instead, it stresses their connection and interdependence moving into the future: social and environmental justice are inseparable (Rixecker and Tipene-Matua 2003). And third, it does not shirk responsibility for its activities, bracketing off downstream effects as a problem for the state or individuals, as is typical of many technology companies. Instead, it carefully considers potential impacts and develops ways to mitigate them or redress them to satisfy involved parties. Even in this brief sketch, we can recognize a way of intervening that is slower, more considered, and more considerate of life in its various forms—the antithesis of our current AI regimes and their production culture of “moving fast and breaking stuff.”

The Five Tests, then, are not just about instrumentalization but about confrontation: they pose a more fundamental challenge to current AI paradigms and practices. They raise a series of key questions: what should this data-driven technology be doing? How might we design these technologies in ways that are more inclusive, communal, and sustainable? And what values and norms are we using to judge the success of a particular technology? These are epistemological questions, concerning the knowledge systems that we use to understand the world. They are cultural and historical questions, concerning the violent domination of some peoples by others. And they are social questions, concerning a particular understanding of a healthy society and the good life. These questions resonate with recent calls to decolonize AI (Mohamed et al 2020; Hanna 2022). Understanding and undoing systems of inequality that have been formalized and fossilized over time is a massive undertaking. Such a programme is daunting when one considers the full range of institutions, relationships, and practices that would need to be rethought (Adams 2021). And yet—as digital technologies and technical systems become increasingly rolled out in high-stakes areas and permeate into our life world—this seems to be the long-term project that social justice demands.

These two pathways are complementary, each augmenting the other. Technical design and codesign initiatives from developers and end-users must be coupled with a radical rethinking of dominant philosophies and paradigms: the bottom-up must be combined with the top-down (Cruz 2021). A genuine engagement with this work might see a company initiate a rapid redesign and evaluation, something material, messy, and ad-hoc that nevertheless generates insights and iterative versions of software—but it would also see that company put in place long-term measures to reflect on their priorities as an organization, to understand the power asymmetries at work in the world, and to restructure their ways of being-and-doing as this journey unfolds. The radical and the actionable must be intertwined.