Varieties of transparency: exploring agency within AI systems

See Ferreira et al. (2021).

Wang (2008).

See Bernard Marr, The Revolutionary Way of Using Artificial Intelligence in Hedge Funds: https://​www.​forbes.​com/​sites/​bernardmarr/​2019/​02/​15/​the-revolutionary-way-of-using-artificial-intelligence-in-hedge-funds-the-case-of-aidyia/​#17eb640157ca (last Accessed: 13 June 2021). Also, in extreme cases, such systems may be used for the algorithmic regulation of society (Cristianini and Scantamburlo 2020).

See: https://​www.​science.​org/​news/​2020/​08/​ai-invents-new-recipes-potential-covid-19-drugs

See: https://​replika.​ai/​

See Lupton (2016).

See Floridi et al. (2018).

For important discussions of these themes, however, see Coeckelbergh (2020), de Fine Licht and de Fine Licht (2020) and Walmsley (2020).

For an overview, see Müller (2020). See also de Fine Licht and de Fine Licht (2020).

In their influential work, Turilli and Floridi write: “In the disciplines of computer science and IT studies, however, ‘transparency’ is more likely to refer to a condition of information invisibility, such as when an application or computational process is said to be transparent to the user” (Turilli and Floridi 2009, p. 105).

More on this in Sect. 3.

Wheeler (2019) identifies the problem in the following quote: “Sometimes, technology is described as being transparent when a specified class of users is able to understand precisely how it functions. This is a perfectly reasonable notion of transparency, but note that a device which is transparent in this sense may be broken or malfunctioning, and so will not be transparent in the phenomenological sense, and that a device which is phenomenologically transparent-in-use may be impenetrable in its inner workings, and so will not be transparent in the ‘open to understanding’ sense. Therefore, there is a double dissociation between the two concepts” (Wheeler 2019, p. 859).

Walmsley (2020) classifies different notions of transparency and distinguishes between “outward” transparency that targets various epistemic and ethical features of AI systems and functional transparency. We do not have sufficient space to address the difference between Walmsley’s taxonomy and ours. However, we wish to highlight that these different varieties of transparency correspond to different forms of what we call reflective transparency. We thank an anonymous reviewer for bringing this to our attention.

See de Fine Licht and de Fine Licht (2020).

This is sometimes referred to as “opening the black box”. See Zednik (2021).

See Karen Hao’s work on the carbon footprint of deep learning: https://​www.​technologyreview​.​com/​2019/​06/​06/​239031/​training-a-single-ai-model-can-emit-as-much-carbon-as-five-cars-in-their-lifetimes/​ (last accessed: 3 December, 2020).

See, for instance, the report that Amnesty International and Afrewatch have presented on child labour and cobalt mines: https://​www.​amnesty.​org/​en/​latest/​news/​2016/​01/​child-labour-behind-smart-phone-and-electric-car-batteries/​ (last accessed: 2 June, 2021).

See Smart et al. (2017), pp. 77–78.

Clowes (2020).

Classic examples of this sort of transparency in the literature include Heidegger (1927) and Maurice Merleau-Ponty (1945). For more contemporary approaches, see Clark (2008), Heersmink (2015), Wheeler (2019) and Andrada (2020).

It has usually been argued that transparency entails a lack of conscious thought or reflection for the artefact’s proficient use. See Heersmink (2015), Andrada (2020).

This is empirically supported by experiments, such as those performed by Maravita and Iriki (2004).

Transparency has played an important role in the hypothesis of the extended mind, where some form of transparency-in-use is an indicator of mental or cognitive extension (Clark and Chalmers 1998; Clark 2008; Andrada 2020). Here we do not want to enter into the debate concerning the plausibility or otherwise of the extended mind thesis. Nevertheless, as will become clear, we do consider a certain degree of transparency-in-use to be central to a successful human–technology interaction.

Heersmink (2015) refers to representational transparency as “informational transparency”. In order to avoid confusion with our notion of information transparency, we have chosen to speak of “representational transparency”.

Note that this distinction holds for informational devices, but not for all cases of transparency-in-use.

See for instance https://​calmtech.​com/​ (last accessed: 2 August, 2021).

See Wheeler (2019), for an account of the risks that transparency entails in some of our cognitive processes. See also Andrada (2021), for more on the connection between transparency and an agent’s epistemic standing.

See Clowes (2020).

See Carter (2020).

Thanks to an anonymous reviewer for bringing this to our attention.

This relates to the so-called control problem (Bostrom 2014; Russell 2019), which can be viewed as a problem of (collective) human agency gaining control over (in this case) a super-intelligent AI, in order to avoid an existential threat.

Thanks to an anonymous reviewer for encouraging us to develop this point further.

The link between transparency-in-use and agency also highlights the importance of accessible and inclusive technologies. See Andrada (2020), for more on the relationship between phenomenological transparency, technologies and diverse embodiments.

We wish to warn the reader that we are not saying that this distinction is correct. In fact, there might be good reasons to think that, even if we can make such a distinction for certain theoretical purposes, the relationship between such forms of agency would be much more dynamic and intertwined. Nevertheless, this distinction may help to clarify our proposed analysis.

It is important to point out that the situation would be different if we wanted to determine the correct degree and type of transparency required for enhancing trust and trustworthiness in AI systems. From a practical perspective, theorists such as O’Neill (2020) have questioned the extent to which transparency is effective in supporting assessments of trustworthiness, and there are additional reasons to doubt the assumed relationship between transparency and trust; that is, the idea that (reflective) transparency always plays a positive role in cultivating trust or supporting assessments of system trustworthiness (see also Nguyen 2021). On the other hand, users trust AI systems for engaging in various actions. They do not want to constantly check on well-functioning equipment, because that impedes their ability to act with it. That is why some degree of transparency-in-use seems to be necessary for trustworthiness. The crucial thing to bear in mind here is that the adequate type and degree of transparency required for promoting trust and trustworthiness in AI systems might turn out to be different from the level of transparency required for promoting agency. We hope to come back to this issue in future work, but this is already enough to show how applying our taxonomy to different normative frameworks can help to illuminate different dimensions of human–technology interactions and AI ethics.

See Clowes (2019a, b), for examples of how the use of Fitbit and personal tracking systems is often a way of practising agency.