Key Take-Aways
-
Management is essential. Drawing substantive lines between ethical, and unethical, uses of AI is only the first step. The organization must also manage its operations to ensure that it stays within these boundaries. This includes making a particular person and/or committee responsible and accountable for the data ethics management function.
-
New management functions and structures are needed. The data ethics management function goes beyond the privacy one. It addresses bias, manipulation, opacity and other risks that go well beyond privacy violations. And it aims towards a beyond compliance goal rather than compliance with privacy regulations. Given this, data ethics management requires organizations either to expand the privacy role, or to establish new positions and entities capable of managing the data ethics function.
-
Organizational location varies. Organizations made different choices as to where to house the data ethics officer or committee. Some companies chose to locate the data ethics function in their privacy unit since it has traditionally handled externalities associated with use of personal data. Others locate it in the legal unit, or elsewhere in the organization.
-
Data ethics officers and committees play important roles. Some organizations localize the data ethics management function in a data ethics officer—a position that was only just emerging at the time of our research—or expand the privacy officer position to encompass it. Some create an internal, cross-functional data ethics committee. In a company that has both a data ethics officer and an AI ethics committee, the two may share responsibility for establishing policies and procedures and for making data ethics judgment calls, with the committee usually deciding the highest stakes issues.
-
The data ethics function tends to be a strategic, rather than a compliance-oriented, one. Privacy management focuses primarily on compliance with privacy laws. By contrast, data ethics management goes beyond compliance with existing laws in order to build and sustain trust and prepare for future regulation. This is a strategic, rather than a compliance-oriented, function. In some companies, the strategy unit was the one that pushed for data ethics management.
It is not enough to draw substantive lines. A company must also manage its operations to ensure that it abides by the lines that it has drawn. The interviewees spent the bulk of their time describing the management practices that their companies use to try to achieve this. These practices break down into three main areas: organizational structure, processes for spotting data ethics issues, and processes for deciding those issues. This chapter describes the emerging structures for data ethics management. Chapter 8 describes the processes for spotting and deciding data ethics issues.
Anzeige
7.1 Organizational Structures
An organization that wants to accomplish something must generally localize responsibility for achieving the objective in a person or group that can then be held accountable. In setting up a data ethics management operation, one of the first things that the companies we spoke to needed to decide was who within the organization should “own” this area, and where should they sit within the company as a whole. Who should be responsible for data ethics?
7.1.1 Privacy Office
The majority of the companies that we spoke to assigned this function to a Chief Privacy Officer or some other privacy manager. The interviews suggest the thinking behind this. For some time now, the main risks associated with personal information have been privacy harms. When companies that use advanced analytics and AI confront new threats from their uses of personal data—bias, manipulation, etc.—they take them to the privacy office. As one interviewee explained about the privacy team that they lead: “We’ve become the de facto ethics team. We’re the people that people come to with far more than just privacy questions, so we end up being a conduit for that.... they say ‘alright well, these are the sorts of questions this team does, we’ll take it to them.’” (Interviewee #10). Statements like this suggest that the companies allocated this role to the Chief Privacy Officer and privacy team more by default than by design.
7.1.2 Legal Department
Another common choice was the legal or compliance office, units that may, or may not, encompass the privacy function.1 One interviewee explained that the Legal Department is generally responsible for doing due diligence on uses of data throughout the company. This gives it representation throughout the company and so enables it to spot and process data ethics issues wherever they arise (Interviewee #19). A second interviewee drew a distinction between the Legal and Compliance Departments and explained that Legal was preferable for the data ethics function because it is accustomed to making risk-based judgments under conditions of uncertainty, whereas Compliance is more used to bright-line rules.
My area reports up through the law department, which is interesting, because when I originally assumed this role, it was part of compliance . . . It made sense to move under legal, we also wanted to get out of the checkbox kind of compliance thinking. When you think of compliance, you think, "I check the box and I take care of what I need to do." . . . [T]hat's really . . . not the appropriate way we want our folks to think about it. (Interviewee #16)
The survey data suggests that, in most companies, either the legal department or the privacy office (which may, in some companies, be part of Legal), has primary responsibility for managing the ethical issues that the company’s use of advanced analytics may create. We asked respondents: “Who in your company has primary responsibility for managing ethical risks associated with big data analytics?” Table 7.1 displays these results and indicates that the Chief Privacy Officer or a Legal executive have primary ownership for the ethical risks. We asked a follow-up about this person’s background and learned that over 50% of the specific individuals charged with managing ethical risks have a legal or compliance background.
Table 7.1
Who in your company has primary responsibility for managing ethical risks associated with big data analytics?
Percent | |
|---|---|
No one in particular | 10.7 |
Privacy officer or similar | 32.1 |
Legal or compliance executive or manager | 32.1 |
Other high-level officer (e.g., Chief Data Officer) | 3.6 |
Data ethics officer or similar | 14.3 |
Other | 7.1 |
Anzeige
7.1.3 The Chief Data Ethics Officer
An interesting development was the emergence during the interview period (2017–2019) of a new executive position related to advanced analytics and customer trust, the Chief Data Ethics Officer,2 and, in some organizations, the creation of an Office of Data Ethics. In some companies, this function was combined with the privacy one. In others, it was distinct. As the time that we performed this research, the Data Ethics Officer role was still quite rare. Companies that had made a significant commitment to data ethics management made up our entire interview sample and, due to selection bias, were likely over-represented in our survey sample as well. Yet only one in five companies in the interview sample had recently created a data ethics officer or similar position, and only 17% of those in the survey sample had done so (Table 7.2). By contrast, almost ninety percent of survey respondents indicated that their company had a Chief Privacy Officer (Table 7.3).
Table 7.2
Does your company have a Chief Data Ethics Officer?
Percent | |
|---|---|
No | 82.8 |
Yes | 17.2 |
Table 7.3
Does your company have a Chief Privacy Officer?
Percent | |
|---|---|
No | 10.3 |
Yes | 89.7 |
The Chief Data Ethics Officer role goes well beyond that of the typical Chief Privacy Officer. To begin with, the Data Ethics Officer is responsible for all data about humans that could harm people, not just personally identifiable information (PII). A Chief Privacy Officer, by contrast, generally focuses on PII. As one former chief privacy officer explained:
I've just changed the name of the global program and my title has officially changed. My official title is now [title that includes “Data Ethics”] and I've changed the name of the global program to [name that includes “Data Ethics.”] And it is because the way that we've done it at [company] is full accountability of all the data that we process and that we steward. That's a very different thing than ensuring you of just privacy requirements like notice and choice. [The idea that the company] should be comprehensively accountable for the data collection, the data activation, the data transformation, the data distribution, is a very next-generation program. It's always been built on ethics. We've been talking about the program as ethical data use for about five years. Then I, as I say, a few weeks ago, I made the official change. That's our journey. (Interviewee #6)
The data ethics function also goes beyond privacy to encompass responsibility for other advanced analytics and AI-related risks such as bias, manipulation, labor displacement and many of the other threats described above.
While privacy officers tend to focus on compliance with privacy laws, the data ethics function must focus on beyond compliance solutions since the law generally does not yet address the threats that advanced analytics can pose. One such professional explained that at the beginning of their tenure the CEO said to her: “I want compliance out of your title. This is not about compliance. This is about customer trust. Let's figure out a new title. So that's the birth of the title.” (Interviewee #20). Another expressed a similar evolution:
We actually added data ethics last year, so my title and my department changed. . . . if we are to do what we need to do for our customers . . . [w]e need to get folks to think of what privacy means a little differently, that it isn't simply complying with the law or policies, it is looking at things through an ethical lens. Because much of what we're doing with data is . . . in a space that is not occupied by law. . . . [D]ata ethics is getting a primary spot. That's the name of our department now. (Interviewee #16)
7.1.4 The Data Ethics Committee
Another important management innovation was the creation of a new entity, the data ethics committee. These bodies could craft the organization’s data ethics strategy, set data ethics policy, and decide or make recommendations on the highest stakes and most difficult data ethics issues.
These functions required multiple types of expertise and perspectives, and companies generally designed the committees as cross-functional entities that bridged a number of departments. The precise make-up varied from company to company but generally included representatives from the legal, privacy, security, communications, data analytics and engineering departments, as well as the affected business unit (Interviewees #6, 14). Some included individuals from government affairs (Interviewee #19), or from corporate social responsibility, (Interviewee #2). The committee might also seek input from C-Suite executives, including the CEO.
Data ethics committees, while growing in popularity, remained a minority approach at the time of our 2019 survey. Even in the survey sample, which likely over-represented companies that took data ethics seriously, only 33% of the respondents used such a committee for formal review of data ethics concerns. Over 40% of respondents indicated that their company had only an informal review process, or no process at all (see Table 7.4 for full distribution of companies’ processes). We expect the use of data ethics committees to increase as a growing number of companies confront the risks that their use of advanced analytics can create.
Table 7.4
What is your company’s process for identifying ethical risks?
Percent | |
|---|---|
We do not have a process set up currently | 18.5 |
Informal screening or review–by a person or office (such as a data ethics executive or team) | 22.2 |
Formal screening or review–by a person or office (such as a data ethics executive or team) | 11.1 |
Formal screening or review by an internal committee, advisory board, or specialized body (e.g., ethics committee, IRB, etc.) | 33.3 |
Screening or review of another sort | 11.1 |
I do not know | 3.7 |
7.1.5 Philosophers in the Corporate Ranks
Another personnel-related innovation was the hiring of PhD philosophers onto the privacy and data ethics team. One interviewee, explaining the role that the philosopher plays in their groups, discussed the debate that the company had as to whether to create encrypted communications that the government could not access:
[A]t the heart of that is the question, what is the consequences of that, and even that, why do we have government? What is the purpose of government and what happens if we change the fundamental way the world operates by creating this extra-governmental space and is that good or bad . . . . And so being able to think through those questions and recognize those questions is a big part of what we do. Lawyers . . . our job is to look at the legal implications; engineers’ tunnel vision is: “I want something that works fast and effectively,” and so philosophers are helpful in dragging us out of those mindsets and thinking about, looking at the broader implications. It’s an incredibly valuable insight. And we’re employing philosophers, which has got to be valuable. (Interviewee #10)
This comment suggests that the data ethics team’s need to consider broadly the social implications of advanced information technologies has led to the integration of philosophers trained to think rigorously about such matters.
7.1.6 From Compliance to Strategy
The growth of data ethics management, as personified by the Chief Data Ethics Officer and the Data Ethics Committee, may signify a fundamental change in the way that companies manage data-related risks. The interviewees explained that, traditionally, the Privacy Officer’s role was to make sure that the company complied with governing privacy laws. This made the Privacy Officer a type of internal cop, and the privacy function a drag on the business operation, even if a very necessary one.
Data ethics, by contrast, is not about compliance. It is about going beyond compliance with existing legal requirements to mitigate risk and maintain the company’s reputation as a good steward of people’s data. It seeks to build the trust that stakeholders (customers, users, business partners, regulators, and the general public) have in the company. This makes data ethics management similar to other business units—those focused on quality and reliability, communications, or customer relations—whose ultimate goal is to build and preserve the company’s trusted relationships with customers, regulators and other important stakeholders. While corporate staff have tended to view the privacy function as a box that the business units need to check, they are increasingly coming to appreciate the data ethics as contributing to the core business mission of building trust and goodwill. If privacy was a compliance function, then, increasingly, data ethics is a strategic activity. One interviewee who had made the change from privacy officer to data ethics officer spoke about the transition in just this way:
[The shift from privacy officer to ethics officer] is reflective of a really different way of approaching the subject . . . [R]eframing the whole discussion around customer trust has transformed the way I'm able to talk to the business. Before . . . the goal was to simply to get it by me, to check the compliance function. . . . [Then] I went in and I said, hey, this is about whether our customers trust us. . . . So that was the lens that the business understood. They understood how important it is to keep customer trust. They want more customers. So when I talked to them about the customer experience and customer trust, it completely turned it around. . . . The reality is we're ending up going so much farther and building things that are far superior in terms of the customers’ experience around privacy. Just because I started with how the business wants to design products and services. (Interviewee #20)
Another interviewee whose position had grown from privacy to data ethics explained the distinction in strikingly similar terms: “Privacy became more of an operational function for the organization.... we became an enterprise solution.” (Interviewee #16). One interviewee told us that it was neither the compliance, privacy nor legal offices that pushed for the establishment of a data ethics function; it was the strategy office. “They were the ones that saw the need for it and created it.” (Interviewee #2). The fact that, in this case at least, the impetus for data ethics management sprung from the strategy group further suggests the changing role of data risk governance from a legal or compliance function to one linked much more closely to enterprise strategy. We anticipate that, in the years to come, more companies will create structures for data ethics management and that they will do so largely for strategic reasons such as those outlined above in Chap. 5.
Open Access This chapter is licensed under the terms of the Creative Commons Attribution 4.0 International License (http://creativecommons.org/licenses/by/4.0/), which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license and indicate if changes were made.
The images or other third party material in this chapter are included in the chapter's Creative Commons license, unless indicated otherwise in a credit line to the material. If material is not included in the chapter's Creative Commons license and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder.