Cognitive architectures for artificial intelligence ethics

Over time, as we recognize more consequences of our actions, our societies tend to give us both responsibility and accountability for these consequences—credit and blame depending on whether the consequences are positive or negative. Artificial intelligence only changes our responsibility as a special case of changing every other part of our social behaviour [… However, r]esponsibility is not a fact of nature. Rather, the problem of governance is as always to design our artifacts—including the law itself—in a way that helps us maintain enough social order so that we can sustain human dignity and flourishing.

(Bryson 2020, p. 5)

To my mind, progress on giving computers moral intelligence cannot be separated from progress on other kinds of intelligence; the true challenge is to create machines that can actually understand the situation that they confront. As Isaac Asimov’s stories demonstrate, a robot can’t reliably follow an order to avoid harming a human unless it can understand the concept of harm in different situations. Reasoning about morality requires one to recognize cause-and-effect relationships, to imagine different possible futures, to have a sense of the beliefs and goals of others, and to predict the likely outcomes of one’s actions in whatever situation one finds oneself. In other words, a prerequisite to trustworthy moral reasoning is general common sense, which, as we’ve seen, is missing in even the best of today’s AI systems.

(Mitchell 2019, p. 156-157).

As artificial intelligence (AI) thrives and propagates through modern life, changing our society as electricity changed our society a century ago,¹ a key question to ask is how to include humans in future AI? In the age of (vividly) personalised web search results and social media feeds, AI clearly informs to a non-insignificant degree what information we receive about the world around us in a sort of ‘filter bubble’ (Pariser 2011); automatically choosing what is most relevant and interesting to us on our behalf. While useful in some contexts (e.g., automated movie and song suggestions), this gives AI the power to choose what we see and hear about the world or more appropriately—our world—the one that has been created for us and communicated to us by AI, often largely without any input on our own part. Furthermore, as “we are exposed to certain kinds of stimuli” AI can “learn how we respond to them and how these stimuli can be used to trigger certain behavioural responses” (Helbing 2019, p. 28), tiptoeing eerily close to an infringement of free will and self-determination as if we would each be part of a gigantic Skinner box. While AI already brings and will bring a lot more social and societal benefits, AI systems can be designed to manipulate our decisions and behaviour towards a specific agenda, using methods linked to messages and values we are most susceptible and sensitive to. This of course can be used for good, but in the wrong hands it obviously causes concerns. In general, technological developments often start with good intentions behind them. However, the line between ‘good’ and ‘bad’ are something early adopters define for themselves and can lead them to walk the grey areas in between. As do the notions behind ‘big nudging’ and ‘citizen scores’ developed and advanced by data-driven cybernetic societies, such as Singapore and China (Helbing 2019). As opposed to the idea of libertarian paternalism that emphasizes the importance of freedom of choice and the goal to influence people’s behaviour to make their lives better, healthier, and, therefore, longer (Thaler and Sunstein 2009). It is still early days for AI and so, we need to be prepared for the directions which future AI may take, and the implications of these in a social, health and well-being, and moral context. In other words, to take a longer term perspective and in doing so, to entertain the many possible paths a future AI-fuelled human life could take.

Despite increasing regulations, many critics cite the ‘black box’ characteristics of AI systems, models, and behaviours as problematic for existing and future AI (Carabantes 2020). This is despite human-involvement at every stage of the production process, from conception and design through to implementation. Sometimes, we do not know what really goes on inside or how and why certain conclusions are met. If we do not understand them, should we let them reign so free and pervasive through human life and society? Human life is complex, messy, and unpredictable. Something humans do have is intentionality; reasons for what they do and why they do it (Searle and Willis 1983). We share this expectation of intentionality with others (Malle and Knobe 1997) and we will come to expect this of future AI also. Explainable AI (X-AI) and transparent reasoning aims to aid in our ability to understand and communicate how an AI system or model makes it decisions in clear and coherent ways (Gunning et al. 2019). For example, X-AI requires that “[w]hen asked a question about its activities, the agent must be able to retrieve the ways in which its choices relate to norms and then communicate them in accessible terms” (Langley 2019, p. 9778). X-AI also prompts the human designer and user to reflect on their own knowledge, biases, and possible (mis)conceptions as they make sense of the AI’s reasoning and naturally, compare it to their own via introspection (Richards 2019) and counterfactuals (Costello and McCarthy 1999) (i.e., imagined what-if scenarios, situations, and non-experiences). This allows also in some sense a forecast for (un)intended consequences, particularly useful in restoration/conservation settings or where payoffs are realised in the distant future (Mozelewski and Scheller 2021). For example, in the case of more advanced future AI a.k.a. strong AI. Furthermore, this allows an opportunity to implement the necessary controls before systems become live; useful in situations where the human consequences could be significant. Reflection is a powerful tool which gives us opportunity to adapt to changing situations by thinking about outcomes achieved and how we got there. In the process, we change the way we think (i.e., goals, intentions, motivations) and act, and build on our tacit, learned, and experience-based knowledge base. It also allows us to explain and make sense of our reasoning and further, to be transparent when communicating to others the rules and procedures we apply to get there. This is an important factor in trust development (Chen and Barnes 2014), and an indicator of human (mis)use of automation at the human–machine interface (Chen and Barnes 2014; Visser et al. 2018), i.e., when being operated directly by the end-user. The lack of transparency makes AI systems more vulnerable and potentially subject to sabotage and misuse. Mitchell (2019), for example, stresses:

Machine learning is being deployed to make decisions affecting the lives of humans in many domains. What assurances do you have that the machine creating your news feed, diagnosing your diseases, evaluating your loan applications, or – God forbid – recommending your prison sentence have learned enough to be trustworthy decision makers? (p. 142).

A focus on cognitive architectures can help increase procedural transparency, reducing sabotage and misuse (e.g., via introducing commonsense) and help to move towards a better understanding of how to model aspects, such as emotions, well-being, or empathy. Cognitive architectures allow to answer why questions and put weight on the ability to envision alternative options and realities (counterfactuals) and compare them. To interpret data also means to formulate a model of the data generating process and reflect on actions taken or not taken. Cognitive architectures can help navigate in a world rich in causal and unpredictable forces which is a challenge when applying purely a machine learning approach:

Like the prisoners in Plato’s famous cave, deep-learning systems explore the shadows on the cave wall and learn to accurately predict their movements. They lack the understanding that the observed shadows are mere projections of three-dimensional objects moving in a three-dimensional space. Strong AI requires this understanding” (Pearl and Mackenzie 2018, p. 362).

Pearl and Mackenzie (2018) argue that a strong AI system would require a causal model of the world and a causal model of its own software (reflect on its own actions). In addition, a memory that records how intents in its mind are connected to events in the outside world (p. 367). This echoes calls made more recently by brain scientist, Jeff Hawkins, in his book A Thousand Brains: A New Theory of Intelligence. Hawkins (2021) argues the mind is constantly creating and revising its models of the world and the objects in it based on how we interact with them. As new problems call for new solutions, additional cognitive functions emerge to offer a path forward due their flexibility. This then requires the ability to think about our thinking (i.e., meta-cognition), revise what we currently known to be true based on new evidence (i.e., learning and extension (not replacement and sometimes revision) of existing knowledge), and to imagine and choose between the potential consequences of various actions which offer solutions to the problem and situation at hand.

A focus on cognitive architectures is, therefore, useful also from a societal perspective as a better procedural understanding can help increase trust in AI systems via a better understanding and interpretation and improved assignment of accountability. Understanding cognitive architectures are also important as many AI systems are still quite limited relative to what human intelligence can achieve and certainly fall short of the expectations of artificial general intelligence (AGI) or strong AI of the future—taking a longer term perspective. We argue that to find answers to the core AI questions that Mitchell (2019) classifies the “Great AI Trade-Off’ requires a better understanding of cognitive architectures:

Should we embrace the abilities of AI systems, which can improve our lives, and allow these systems to be employed ever more extensively? Or should we be more cautious, given current AI’s unpredictable errors, susceptibility to bias, vulnerability to hacking and lack of transparency in decision-making? To what extent should humans be required to remain in the loop in different AI applications? What should we require of an AI system in order to trust it enough to let it work autonomously?

Cognitive architectures are conceptual models of intelligent minds—be it human, animal, or artificial—as they learn, process, store, and reuse knowledge and information, and make and carry out decisions to problems they face. Cognitive architectures can help implement transparency and explainability in AI with different levels or subsystems each performing distinct yet interrelated cognitive functions including value and goal setting, planning, deliberation, and action to name a few—particularly in development of artificial moral agents (Cervantes et al. 2020). Cognitive architectures also improve the ability to communicate to a wider audience beyond experts and specialists in academia and government and interact with them socially (Samsonovich 2020). It provides clear frameworks that are flexible to the needs of the user, architect, and environment of application. Furthermore, it helps clarify the assignment of responsibility across different levels of a multi-level cognitive framework, lending itself to division of labour and division of credit and blame (i.e., accountability). However, as Griffiths and Lucas (2016) contend, everyday “[e]conomic transactions, such as legal transactions, do not take place in a vacuum, but in a social and moral context” (p. 30). This speaks to the importance of social and moral factors in everyday transactions that strong AI, as humans have, will likely face on a regular day-to-day basis. Furthermore, these transactions are by “human beings with a ‘mindset’ of motivations and aspirations which determine how they react to the particularity of the time and circumstances in which they find themselves” (Griffiths and Lucas 2016, p. 30). This shows the (p)relevance of goals, aspirations, and motivations in human life and society and also requires AI reasoning about others’ mental models and states—another support for cognitive architectures in ethical AI. Doing so requires opening up the “black box” of AI; especially as they act, interact, and adapt in a human world and how they interact with other AI in this world.

In this contribution, we argue for the application of cognitive architectures for AI ethics. In particular, for their potential contributions to AI transparency, explainability, and accountability. Future AI will face many dilemmas and ethical issues unforeseen by their creators beyond those commonly discussed (e.g., trolley problems and variants of it) and to which solutions cannot be hard-coded and are often still up for debate. Given the sensitivity of such social and ethical dilemmas and the implications of these for human society at large, when and if our AI make the “wrong” choice we need to understand how they got there to make corrections and prevent recurrences, and if necessary, punish or reward. This is particularly true in situations, where human livelihoods are at stake (e.g., health, well-being, finance, law) or when major individual or household decisions are taken. In the next section, we introduce AI ethics as they apply to society, namely: ethics in design, ethics by design, and ethics for design (Dignum 2019). Following this, we discuss cognitive architectures as they apply to intelligent minds in individuals and collectives. We then discuss their relevance to AI ethics and their potential contributions to transparency, explainability, and accountability. Finally, we summarise with implications, shortcomings and challenges, and future perspectives for cognitive architectures in AI ethics and future AI more generally.

Human life is full of moral and ethical dilemmas and as such, plenty of opportunity to practice our moral reasoning and ethical decision-making. These transactions “do not take place in a vacuum, but in a social and moral context” (Griffiths and Lucas 2016, p. 30) and hence, an abstraction or idealisation that is void of historical time, and space makes little sense. Should Odysseus risk encountering Charybdis and lose his entire ship and companions, or should he sail closer to the evil monster Scylla which would lead in losing six of his crew members:

We then sailed on up the narrow strait with wailing. For on one side lay Scylla and on the other divine Charybdis terribly sucked down the salt water of the sea. Verily whenever she belched it forth, like a cauldron on a great fire she would seethe and bubble in utter turmoil, and high over head the spray would fall on the tops of both the cliffs. However, as often as she sucked down the salt water of the sea, within she could all be seen in utter turmoil, and round about the rock roared terribly, while beneath the earth appeared black with sand; and pale fear seized my men. So we looked toward her and feared destruction; but meanwhile Scylla seized from out the hollow ship six of my comrades who were the best in strength and in might (Homer 1945, The Odyssey, p. 449).

When choosing what to do we rely on our morals. We also subscribe to shared ethical principles with those we interact with frequently and have relationships with. Morals (a.k.a. moral values) are values (i.e., codes, standards of practice, guiding principles) that protect and enhance life for all, self and others. Being moral means knowing the difference between right and wrong and wanting/choosing to do what is right. Of course, what is right and wrong is subjective; a matter of opinion and preference and is often driven by culture, religion, and environment (Awad et al. 2018) among other contextual factors. Morality also develops and evolves over a lifetime (Brady and Hart 2007); from focusing on personal interests through to maintaining social norms and then more independent critical thinking about morals and ethics. Being ethical means carrying out morals, typically those shared with other individuals, groups, and institutions. In other words, ethics are those morals which emerge from shared understanding and agreement, and hence, are often tied to a specific socio-political environment, time and place. They are often used to foster safety, security, and cooperation in communities of individuals who interact frequently. However, being ethical does not always equate to being moral. For example, adherence to the criminal’s code of silence—designed to protect criminals from police conviction—is ethical behaviour from the standpoint of other criminals. It is viewed favourably upon by other criminals and rejection of this code often leads to serious consequences (e.g., “snitches get stitches”). Morally speaking, lying and misleading are morally inadmissible acts as is criminal activity in the first place. Hence, sometimes (often) ethics and morals collide.

As AI proliferates further through human life and takes on increasingly difficult tasks in more complex environments, the likelihood that AI will face problems and events with moral and ethical implications will increase dramatically. This is particularly true, where human life and/or livelihoods are at stake and when big government, organisational, household, or individual decisions are made. This is where AI ethics has a solid role to play. Broadly speaking, the AI ethics literature can be broadly clustered by the ethics of AI (i.e., ethical issues related to or caused by AI) and ethical AI (i.e., machine ethics / the ethical and moral behaviour of AI) (Siau and Wang 2020). Essentially, this boils down to the What and How of AI. In other words, what are the effects of AI on society (and vice versa), and how can we ensure AI act ethically by design, monitoring, and regulation. While this is a fairly intuitive (dualistic) distinction, we prefer the responsibility approach (i.e., implementation-focus) of Dignum (2019) who clusters AI ethics into three areas of focus:

This provides a mean to discuss what ethical AI should or ought to look like, the roles and functions they should perform in society, and how this can be practically realised. Furthermore, what the role of regulators should be and how should we balance ethical, legal, economic, and social considerations to achieve something which benefits society as a whole. Clearly, the AI ethics literature is broad and varied. For example, Jobin et al. (2019) in their review of 84 AI ethics soft-law and grey literature find eleven overarching ethical values and principles (by frequency of appearance): transparency, justice and fairness, non-maleficence, responsibility, privacy, beneficence, freedom and autonomy, trust, dignity, sustainability, and solidarity. The core ethical principles which featured in more than half of the sources include transparency, justice and fairness, non-maleficence, responsibility, and privacy. AI4People² also include beneficence and autonomy and highlight the salience of explicability in enabling the application of ethics principles to AI (Floridi et al. 2018). For example, they highlight “… for AI to be beneficent and non-maleficent, we must be able to understand the good or harm it is actually doing to society” (p. 700). In this contribution, we focus on explicability (i.e., able to be explained, interpreted and/or accounted for), which itself is backed by transparency and explainability, and lends support to greater accountability. This stems from “… the need to understand and hold to account the decision-making processes of AI” (Floridi et al. 2018, p. 700) and to ensure the people and companies responsible for developing and deploying the AI are also held accountable in the case of negative outcomes for individuals and society. We discuss the relevance of cognitive architectures to this cause later in Sect. 4.

In the following subsections (2.1 to 2.3), we introduce and unpack each of Dignum’s (2019) AI Ethics clusters in turn. In particular, we focus on the AI ethics principles of transparency, explainability, and accountability whose presence is felt right across these three domains of AI ethics and for the reasons laid out above. In other words, we see accountability to be critical to the sustainability of AI (longer term implications in fostering safety, security, and accountability across every stage of the AI production process or lifecycle). To this end, transparency and explainability are required for policymaking, community consultation, layman communication and understanding, interdisciplinary knowledge sharing, and monitoring adherence to AI rules and regulations.

In AI, the idea of the mind as a collection of agents (i.e., many cognitive agents or processes / sets of processes that are nearly incomprehensible from one another) is not new, but the diversity in ways to think, represent, and act is still lacking in today’s narrow-minded AI. A narrow brute force approach that relies on machine learning instead of stronger AI can backfire from an ethical perspective. For example, biased training data can lead to ethical issues, such as discrimination. In many instances in life there is no or little training data available (Marcus and Davis 2019) and this is true also for commonsense knowledge. On the other hand, focusing and implementing cognitive architectures can help in dealing with uncertainty, and incomplete and/or inconsistent data and information.

While the influence of dual process theory¹⁶ in AI and the social sciences cannot be not understated (Milli et al. 2019),¹⁷ such black and white “dumbbell” thinking tends to constrain thought and theory if taken too literally (Minsky 1988). There is no reason that we need to stop with just two categories of human cognition; as if all deliberative, reflective, and meta-cognitive (thinking about thinking) cognition is of the same sort and should be classed as such. Stanovich (2004) and Evans (2006) argue that while most researchers refer to type 1 as a single system, it is actually a set of (autonomous) systems and processes which each fulfil distinct but related cognitive functions. Glӧckner and Witteman (2016) for example categorise intuition (one form of type 1 thinking) into associative, analogous, accumulative, and constructive intuition, discussing how this differentiation helps clarify and provide deeper insight into the relationship between intuition and decision-making. We can also split type 2 thinking into at least algorithmic and reflective cognition as Stanovich (2009) does; however, we may need to dig deeper if we are to truly understand the full breadth and depth of human behaviour we see out in the real world.

Table 1 below provides one such example of a general cognitive architecture developed by BICA Society (Biological Inspired Cognitive Architectures) with 5 levels of cognition. From ‘low’ to ‘high’ level cognitive functions this includes reflexive, reactive/adaptive, proactive/deliberative, reflective, and meta-cognitive/self-aware. This clearly goes beyond the type 1, type 2 dichotomies by separating out cognitive functions which may more or less sit between or across the typical dual-process distinction, allowing for a richer account of complex behaviours, such as motivations, emotions, empathy, and more.

Cognitive architectures are a subset of agent architectures¹⁸ and come in symbolic (i.e., top-down approach), connectionist (or emergent, i.e., bottom-up approach), or some hybrid combination of these (e.g., serial, or parallel processing, modularised designs, layered hierarchical systems). Over the years, there have been some estimated three hundred cognitive architectures proposed and developed to varying degrees—ranging from those which are solely conceptual through to those which are practically realisable and those actually realised. Of those, Kotserube and Tsotso (2020) survey 84 cognitive architectures developed over the last four decades and clustered them by their perception modality, attentional mechanisms, memory organization, types of learning, and practical applications. Some of the most popular and cited cognitive architectures include ACT-R, Soar, CLARION, ICARUS, EPIC, and LIDA. Others in Duch et al. (2008), Samsonovich (2010), Thórisson and Helgasson (2012), Goertzel et al. (2014), and Ganesha and Venkatamuni (2017) provide additional surveys of the literature on cognitive architectures—a wide, interdisciplinary, and varied body of work. What becomes clear is the lack of general consensus on what cognitive premises and assumptions to work from and further, on any unified theory of cognition in the first place. Future AI—the general sort of AGI envisioned by technology theorists, futurists, and science-fiction writers—requires a broad set of competencies and ways to think, and the ability to regulate behaviour and choose between alternative sets of action possibilities. However, the exact criteria to ascertain AGI is debated. For example, Newell’s (1980, 1992) criteria for AGI includes adaptive behaviour, real-time operability, (normative) rationality, a deep and broad knowledge base, learning, development, linguistic capabilities, self-awareness, and brain realisation. Sun’s (2004) desiderata for AGI cognitive architectures in addition includes ecological realism, bio-evolutionary realism, cognitive realism, routinisation, and diversity of methodologies and techniques (and synergistic interactions between them). Adams et al. (2012) suggests perception, memory, attention, reasoning, mobility, planning, motivation, learning, emotion, communication, social interaction, modelling self and others, creativity, and arithmetic criteria. Furthermore, Minsky et al. (2004) propose nine types of reasoning required by future strong AI: spatial, physical, bodily, visual, psychological, social, reflective, conversational, and educational. The most common (core) set of competencies, however, includes perception, learning, reasoning, decision-making, planning, and acting (Metzler and Shea 2011). Others in Vernon et al. (2007), Langley et al. (2009), and Asselman et al. (2015) suggest yet even more AI characteristics to focus on.

Worth acknowledging is that cognitive architectures are complex systems (Menzel and Giurfa 2001; Schmid et al. 2011) which themselves are generally hierarchical in structure (Simon 2001) and nearly decomposable (Simon 1962). In other words, we are able to make distinctions between sub-systems (levels) which themselves are nearly independent, but still intertwined and hence evolve together in time (and space). The higher (slower and coarser) levels in the system conserve stability, while the lower (faster and finer) levels allow novelty and testing of innovations, mutations, and adaptations to challenges and opportunities in the agents’ world. Newell (1990) observed that human activity unfolds on different levels of cognitive processing and can be grouped by timescales at 12 different orders of magnitude (starting at 100 µs and up to months/years), providing support to hierarchical thinking and cognition.¹⁹ In the same way that many human systems are complex hierarchical structures, the human mind comprises a set of nested cognitive functions which have evolved over time to help us solve new problems as we face them (Hawkins 2021). The economy is always discovering, creating, and in process (Arthur 2006), new problems constantly arise and are solved (or linger). These new problems demand attention and novel thinking to solve them in a never-ending act–react–adapt cycle. Of course, we also leverage our knowledge and past experiences in doing so, highlighting the adaptive and path-dependent nature of intelligence.

Cognitive architectures have been deployed in a variety of domains and operational contexts (Kotseruba et al. 2016); human performance modelling, games and puzzles, robotics, psychological experiments, and natural language processing, to name a few. What is lacking from existing cognitive architectures (or at least the current AI implementations of them) is much of what makes us human (Langley 2017): the ability to understand and interpret imprecise and complex concepts (e.g., based on commonsense, analogous thinking, or abductive reasoning), dynamic memory and continual (online) learning (Diaz-Rodrigues et al. 2018), creativity, emotions and metacognition (and the interactions between them), personality and goal reasoning, and motivation (Dörner and Güss 2013; Güss and Dörner 2017). Furthermore, commonsense is a rare element for today’s AI but is crucial to achieving human-level (general) intelligence (McCarthy 2007). Commonsense requires the sort of (implicit) knowledge that children seem to grasp readily but of which machines struggle greatly. For example, consider the following: Jack walked into the loungeroom and picked up the TV remote. Jack then walked into the kitchen. Where is the TV remote now? Despite it not being explicitly stated, one generally assumes the answer is of course the kitchen. This example may seem trivial but the mechanisms underlying our understanding and comprehension of the situation are not. Generally speaking, commonsense requires many ways to think, and cognitive architectures can offer this diversity in ways to reason about the world around us (Lieto et al. 2018; McCarthy et al. 2002; Singh et al. 2004; Shylaja et al. 2017).

Minsky (2000) stresses the importance of (many) knowledge representations, (relevant) knowledge retrieval, negative expertise (i.e., learning from failures), and self-reflection for commonsense AI. Without these we would learn constricted models of the world, not learn from our mistakes, not reflect on what went right or wrong, and even if we did learn, there is no guarantee we could call upon these lessons learned in future analogous problems and situations. Commonsense is conditional on a commonsense knowledge base; knowledge about the world and things around us that we as humans usually assume are obvious (e.g., grass is green, sky is blue, fire is hot). Such commonsense is amassed through experiencing the world and interacting with it.²⁰ Pearl and Mackenzie (2018) propose a “ladder of causation” (seeing, doing, and imagining²¹) that intelligent agents could use to model how the world works. To be practical, this also requires linking knowledge to uses, goals, or functions (Minsky 2000). In other words, defining knowledge by its usefulness to us in getting what we want. For example, augmenting “… knowledge with additional kinds of procedural and heuristic knowledge, such as descriptions of (1) problems that this knowledge item could help solve; (2) ways of thinking that it could participate in; (3) known arguments for and against using it; and (4) ways to adapt it to new contexts” (Minsky et al. 2004). Furthermore, to understand truly ‘human’ concepts of emotions, love, envy we require much more sophisticated architectures accounting for different sorts of emotions. For example, a reactive layer for primary emotions (e.g., being frightened), a deliberative layer for secondary emotions (e.g., relief), and a meta-management (reflective) layer for tertiary emotions that are typically associated with humans (e.g., love, excitement, anticipation) (Sloman, 2000, 2001).

Hierarchic structures occur frequently in physical, biological, and social systems alike (Simon 1962). This provides space for cross-fertilisation of knowledge and insights across disciplinary boundaries. For social systems, interactions between elements and the intensity of them are a defining feature. How social infrastructure (e.g., libraries, parks, community centres) interacts with physical (e.g., roads, bridges, telecommunication networks) and environmental (e.g., natural resources, lakes, rivers and wetlands, rainforests) infrastructures is still a nascent area of research (Latham and Layton 2019) and could be explored by AI. Simon (1962) again provides an important insight: “if the process absorbs free energy the complex system will have a smaller entropy than the elements; if it releases free energy, the opposite will be true” (p. 471). In other words, as long as there is an external source of energy to draw upon, the system will remain relatively stable. However, when energy becomes scarce things may start to unhinge. The social element of human–machine interaction should not be understated; many people anthropomorphise AI agents (Salles et al. 2020) and hence interact with them in ways which require social skills that emulate at least some level of ‘human’ competency. Acknowledging this then requires a representation of the social processes of behaviour and decision-making that are implementable by algorithms.²² This also requires understanding the various drivers of human social behaviour, such as social norms and status, political viewpoints, personality, and culture, among others. Furthermore, to then be able to identify and correct/adjust for these differences in real-time. Even in the social sense, the way we interact with other people in society resembles a rather well-defined hierarchic structure (Simon 1962). Operationally, “[t]he groupings in this structure may be defined … by some measure of frequency of interaction in this sociometric matrix” (Simon 1962, p. 469), at least theoretically speaking we should be able to computationally interpret social hierarchy and structure.

Event calculus (EC) (Shanahan 1999; Brandano 2001) and variants²³ of it (Sadri and Kowalski 1995; Miller and Shanahan 2002) provide an intuitive way to reason (deductive-, inductive-, and abductively) about the world around and how our actions and the actions of others influence it. EC is a many-sorted first-order logic (Mueller 2014) that allows to reason about agents, objects, and events in dynamic ecological systems over time and space, as well as to reason about agent knowledge, goals, and beliefs (e.g., using EFEC) and hypothetical events and situations with a branching time approach (e.g., using NEC). This also provides a base from which commonsense reasoning can emerge (Mueller 2014), as will be expected in future strong AI. Commonsense reasoning can be used to improve AI understanding in a human world and enable greater flexibility to adapt to unexpected real-world problems by reasoning about actions and events and to make plans for the future to satisfy needs and achieve goals. EC provides the means to represent knowledge about events (e.g., a person crossing the road), fluents (i.e., a time-varying property of the world, such as the location of a car driving down the road or a propositional truth/false statement), and timepoints (i.e., representing an instant of time like 8.00am on a Monday morning) declaratively as logical formulas by employing some or all of the common EC predicates shown in Table 2 below as well as domain-specific axioms for EC extensions and new practical applications. For each new domain, we can add domain-specific axioms to handle new scenarios and phenomena, such as actions with non-deterministic effects, concurrent actions, continuous change, and hierarchical/compound actions. For example, using the examples provided by Mueller (2008, p. 33) in Table 2.1 (adapted and provided in Supplementary Information), we may describe any commonsense domain by new event effects and occurrences via adding axioms to of the existing corpus of axioms in whichever version of the event calculus you’re employing. Shanahan (1999) provides a practical, tutorial-style introduction to the development of domain descriptions using event calculus.

EC offers a very generalisable toolkit to reason about the world, others, and yourself. It can be used in individual and collective accounts as ‘narratives’—the horsepower—are a flexible conceptual tool. Tools like EC provide a method of reasoning about action and change on a timeline which actual events occur (Mueller 2008). Situational calculus on the other hand explores hypotheticals and requires more complete specification of hypothetical actions on outcomes (Kowalski and Sadri 1997). This could allow counterfactual reasoning about the world in imagined scenarios which may or may not ever transpire. EC appears more organic in the sense that allows an incomplete ‘narrative’ to be specified over perfectly specified situations and demonstrates the clear path-dependency that pervades human life. However, both event and situation calculus support “context-sensitive effects of events, indirect effects, action preconditions, and the common-sense law of inertia” (Mueller 2008, p. 671), demonstrating their usefulness in AI for human society. For example, EC has been used to represent and monitor of social commitments (Chesani et al., 2013), model agents’ emotional reactions to real-world events (Sarlej and Ryan 2011), model agents’ moral reasoning and ethical responsibility (Berreby et al. 2015), monitor compliance to obligations in the context of business process checking (Hashmi et al. 2014), monitor business constraints in maritime safety and the behaviour of vessels (Montali et al. 2014), formalise business and production workflows (Cicekli and Cicekli 2006) and policy specification and analysis (Bandara et al. 2003), track infectious disease spread (Chaudet 2006), and is used in event recognition (Katzouris et al. 2019; Skarlatidis et al. 2015), activity recognition (Artikis et al. 2019; Kafalı et al. 2017; Kim et al. 2019), natural language, and story understanding tasks (Abdel Salam et al. 2013; Mueller 2003, 2004b, 2006), among other practical applications.²⁴ Furthermore, we can use EC to represent the goals, values, motivations, and intent of our AI and link them through symbolic nets to events, outcomes, percepts, and actions (e.g., using EFEC—see footnote 23). These can then be represented to humans in interpretable forms by leveraging the intuitive and descriptive nature of EC. This allows better understanding and interpretation by a wider audience. Strangely enough, EC is largely absent from mainstream literature on cognitive architectures despite holding promise in multiple areas that plague today’s cognitive architectures.

Cognitive architectures are closely linked to ethics as internal representation deals with aspects such as goals, preferences, desires, and beliefs. It is important to understand how humans and AI may (complementarily or not) integrate with each other and other cognitive artifacts in their social, cultural, and material environment. After all, ethical and moral reasoning is also heavily influenced by such mechanisms (Awad et al. 2018). Large variations across different social and cultural groups suggests that reasoning about problems with moral and ethical implications is influenced by the presence (or not) of social institutions and cultural norms (e.g., norms of fairness, and collectivism or individualism). Experimental evidence from behavioural economics shows us that cooperation, sharing, and punishment behaviours closely correspond to membership and engagement with cultural groups (Henrich et al. 2001). In other words, we act in similar ways to those we identify with. Despite this, systems thinking for ethical issues rarely goes beyond a single issue or initiative. This is likely because it is very difficult to engage in recursive reasoning about such complex systems, feedbacks, feedforwards, and interactions. For many, it is difficult to conceive how individual actions can relate to ecosystem outcomes and in turn, how ecosystems influence our behaviours. Furthermore, how networks of individuals develop and sustain emergent properties which are not attributed merely to the sum of individual actions (Holling 2001). Understanding the degree and varieties of cognitive integration between cognitive agents and artifacts requires focusing on dimensions, such as information flows, reliability, durability, trust, procedural transparency, informational transparency, individualisation, and transformation (Heersmink 2015). This can be directly applied to research on distributed morality²⁵ (Floridi 2013) and may also advance the study of global collective behaviour.²⁶ Moral rules and ethical guidelines are distributed throughout groups of like and frequently interacting people (e.g., countries, cultures, families, friend groups, organisations) and some rules propagate widely across groups (e.g., fairness, empathy, cooperation) due to their ability to provide conditions which support peace and prosperity for all. How humans and strong AI that are engaging in moral and ethical reasoning interact with cognitive artifacts (e.g., social and cultural norms), and the degree by which their decisions are informed by them is crucial to shining light on the ‘black box’ problem. Marcus and Davis (2019), for example, stress that.

the decisions that the program is making, being computed ‘algorithmically,’ have an aura of objectivity that impresses bureaucrats and company executives and cows the general public. The workings of the programs are mysterious – the training data is confidential, the program is proprietary, the decision-making process is a ‘black box’ that even the program designers cannot explain – so it becomes almost impossible for individuals to challenge decisions that they feel are unjust” (p. 34).

Procedural transparency (i.e., transparency in methods, processes, and procedures used to make decisions) is also beneficial in (at least some) public governance and policy settings (Cucciniello et al. 2017) and it is thought to underpin much of democratic society. An intelligent future would benefit from this also and cognitive architectures offer a path forward.

Altman (2014) shows us that mental models shape the choices we make relating to social, economic, and moral problems. Kahneman and Tversky (1984) show us the way problems and choices are framed (i.e., choice architecture) also influences the ordering of preference between alternatives. This begs caution and consideration in how human designers frame ethical goals and values to cognitive architectures and also how they should frame moral problems, situations, action, and events in the first place. The key here is that differences in how we perceive and reason about these problems arise from the mental models we begin with. They act as filters for what things we can perceive, they simplify reasoning and decision making, and they are sensitive to manipulation, positive or otherwise. In society, we see how subscribers of certain narratives act in collectively coherent ways that are consistent with whatever cause they subscribe to. There is evidence such narratives play a crucial role in many (socio)economic fluctuations (Shiller 2017, 2019), in ethics, and in moral reasoning (Roberts 2012; Brody and Clark 2014). The problem is when maladaptive ones rise and take hold; these beasts can wreak havoc on peoples’ well-being and livelihood and bring destructive force to human social and societal systems, well-intentioned or not. Avoiding negative outcomes (and promoting positive ones) of future AI requires keeping a watching eye over potential maladaptive ethical and moral traits forming in a society of humans and AI, devising interventions to rectify issues (if any), and if desired, carrying these plans out. This could be supported by a “new[er] kind of AI program—oversight programs—that will monitor, audit, and hold operational AI programs accountable” (Etzioni and Etzioni 2016). Equally, to hold humans involved in the process accountable also.

As mentioned earlier, cognitive architectures allow to answer why questions and put weight on the ability to envision alternative options and realities and compare them. In the next three sections, we explore why cognitive architectures are relevant for AI ethics principles: transparency, explainability, and accountability. For each we highlight elements of AI ethics in design, by design, and for design. Furthermore, we discuss how cognitive architectures offer unique advantage and insight compared to other AI technologies.

In this paper we have emphasized that an AI that is ethical and reliable will require strong and general (future) or strong AI that considers cognitive architecture aspects to function in our complex and constantly changing world with so many unpredictable elements that are hard to program and anticipate. In recent years, we have obtained wonderful progress with an AI that is narrow which distracts for the importance of a cognitive architecture as a way of providing AI with a deeper understanding of the world and those in it. The great success in well-defined areas (e.g., board games such as chess or Go or game shows such as Jeopardy!) cannot be scaled up to a complex real-world environment. We also need to get better (theoretical) understanding of why Deep Learning (DL) works so effectively in those specialized circumstances. For example, more progress will be achieved in understanding the power and characterization of multi-layer neural networks and how to reason about meta-level reasoning (for a discussion, see, e.g., Perez 2018). We, therefore, need to be open to a large set of tools and methods to deal with such complexity. Marcus and Davis (2019) point out that “what we have for now are basically digital idiots savants” (p. 13) criticizing that “we ceding more and more authority to machines that are unreliable and, worse, lack any comprehension of human values. The bitter truth is that for now the vast majority of dollars invested in AI are going towards solutions that are brittle, cryptic, and too unreliable to be used in high-stakes problems” (p. 15). They argue we are experiencing a short-term obsession with narrow AI that goes for the low-hanging fruits rather than the more challenging and long-standing problems. A focus on cognitive architectures can counteract such tendencies. It may also provide ways of thinking how cognitive architectures can be improved by insights from approaches, such as DL. For example, cognitive architectural approaches have struggled historically in incorporating learning elements. However, cognitive architectures are essential as the real world is an open system that requires constant adjustments to what is changing in its surroundings. Future research could explore in more detail what we can learn in the area of open and emergent systems and complexity research as, for example, done by scholars at the Santa Fe Institute (see, e.g., Krakauer 2019) to transfer those insights into AI. This could help AI to better tackle the world in all its complexity and richness.

We are still in the process of understanding how to analyse evolving or ever-unfolding systems, systems in which we do not know what can and will happen next, which puts a toll on our current tools of thought and exploration that strongly emphasize the virtue of rationality and reason. Strong AI capable of reasoning about situations with moral and ethical elements require commonsense to allow adaptation to context-specific issues and considerations and hence, need to remain responsive to the environment and those in it. For example, for AI to abide by even Asimov’s first law³¹ requires it first understand the meaning of ‘causing harm to humans’. This requires may ways of thinking and also representing knowledge on many layers and abstractions. Cognitive architectures which go beyond shallower models of cognition (e.g., system 1, system 2) allow this flexibility and in so, robustness and resilience in facing new problems and situations, including those we may not have ever experienced before (i.e., learning of the fly, ‘winging’ it). Adding commonsense reasoning into AI systems will help to reduce system vulnerabilities or sabotage attempts that have substantial ethical implications. In addition, commonsense reasoning is a step towards thinking how social or emotional intelligence can be included. However, for that, we also need to improve our computational theories around psychological processes. The Moral Competence in Computational Architectures for Robots initiative financed by the Department of Defence including scholars from Tufts University, Brown University, RPI, Georgetown University, and Yale University, for example, tries to develop computational architectures that are capable of moral reasoning via identifying the “logical, cognitive, and social underpinnings of human moral competence”.³²

No doubt there will be shortcomings and pitfalls of AI. This is to be expected as it has occurred time again with other foundational technologies in history. Especially those in the earlier stages of mass technological adoption. Crucial for wider adoption and integration in human life and society is transparency, explainability, and accountability for AI in design right through to implementation and upkeep (considering the full lifecycle). These require opening the black box of AI, something we have argued that cognitive architectures show promise. This requires to also go beyond the power of data and focus on aspects, such as the ethical values that designers and programmers use to make a fair, transparent, and safe world. In addition, the means by which we communicate and deliberate these to a wider audience of technical and non-technical stakeholders alike.