Should Artificial Intelligence Be More Regulated?

Many discussions are taking place at the moment about Artificial Intelligence (AI), about ways AI may benefit mankind and about risks of AI. Autonomous cars, automated trust assignment to individuals, and autonomous weapons are only a few examples how AI can change our life. Some people warn us that AI can be even more dangerous than nuclear power. On the other hand it seems impossible and undesirable to stop development of AI and its applications. Thus, the question arises what should be the role of governments. Should AI be more regulated with respect to research and/or its usage? This question was addressed at WCC 2018 in a panel discussion with four panel members:

An audience of 40 participants actively engaged in the discussion. The session was recorded on video [1].

To start the debate, two panel members presented arguments in favor of more regulation and two members presented arguments against more regulation.

Our starting point is ethics. The implementation of AI systems, including of AI driven robotics, poses a number of ethical challenges. A non-exhaustive list includes:

The question is: where can we rely on the ethical actions of developers and users of AI and where is this clearly not adequate and therefore regulation is necessary. Some of the applications of AI are in domains that are already regulated and have a long history of regulation, medicine and finance are two obvious examples. But the existing regulations may not cover the application of AI. These regulations may need to be enhanced to prevent harm to patients or unfair financial practices. Autonomous cars should not be allowed to go on the road until there is an agreed allocation of responsibility and therefore liability for harm. You need a third party insurance that covers the driver. But in an autonomous car there is no driver. So who is responsible/liable: the manufacturer of the car, the manufacturer of failing components, the car salesperson, the owner of the car? And what if the car is hacked? Or if software updates have not been installed? There is no doubt that existing regulations do not cover autonomous cars and that this is needed.

Many people have already asked for a ban on autonomous lethal weapons, comparable to nuclear and chemical and biological weapons. While controlling adherence to such a ban may be difficult and sometimes such weapons are used nevertheless, laws and regulations have a powerful effect on the public opinion.

Economists predict a growth in jobs due to AI, but only in the long term. In the short term jobs will be lost. Regulation may be needed to provide funding for retraining employees for a new job.

AI can be used to de-anonymize personal data that has been anonymized. GDPR may already be a good step in the right direction restoring the control over personal data to the owner instead of the company. However the protection in the GDPR against AI based use of personal data is weak and needs strengthening.

Machine learning and AI systems are complex systems. In a number of application areas we should think about regulation. Consider machine learning and AI systems as a product and therefore regulation is focused on the application of AI, on the product. In the medical domain we see systems that can make prognoses and by doing that impact people’s health. It is important that the system can not make errors. It is the task of the producer/vendor to take care of that. Compare it with the process to get a new drug (medicine) accepted. Strict procedures and tests take place before the new drug is allowed to be put on the market. Producers of AI should provide assurance that their product is working correctly and this should be enforced through regulation. Regulation does not always have to be laws, it also can be community agreed rules and processes or evaluation and certification. Another element concerns the question whether an AI system should be able to explain the decision it took. For some domains and applications this may not be necessary, for others it is, think about legal decisions (e.g. AI supported court cases). It should be mandatory for such systems to be able to explain. That may not be easy, when is an explanation clear enough, what is the context.

Another issue is intellectual property rights. Advanced systems can write poems and stories or compose music. Who owns this and benefits from the profits this might generate? Regulation may be needed to clarify such rights.

If regulation is needed, it could be considered to not only look at legal regulations but also at initiatives about principles. An example of these are the Asilomar AI Principles. Industry giants and experts such as Elon Musk and Stephen Hawking have advocated for humane and “safe” robotics. Along with hundreds of researchers and experts in the fields, they have proposed 23 “guiding principles” that will ensure the development of AI for the benefit of mankind [2, 3].

Although one might in principle be pro regulation of certain aspects of AI system, it simply seems to be impossible. The problem is we don’t know what it is, an AI system. AI is not a monolithic system, it is in other systems, it is in our cars, in our search engines, in our shopping cart. AI is a functionality of existing systems. It is completely impossible to control the development of these techniques and also it is impossible to control its use. Other areas that have been regulated also show this. Two examples from the weapons industry. Nuclear weapons, we all know how difficult it is in certain parts of the world to control the development of nuclear weapons. It is a highly political issue. The other example is chemical weapons. They are banned and nevertheless used. It is impossible to control the use of technology and it is impossible to control the development of technology. We also should not want this because technology is a driving force of our society. We want to learn more. We shouldn’t stop science or regulate science. There may be some exceptions with respect to ethical issues. We don’t know exactly what to regulate and how to regulate it. The United Nations does not succeed in getting a letter signed by all countries about the goal of a ban on lethal autonomous weapons. Some countries have major interests in such an industry or other arguments for not signing it.

Another example concerns autonomous cars. Where liability and insurance issues might be regulated there are also ethical issues. The German government drafted a report saying that an autonomous car should never be able to face an ethical dilemma situation. That is impossible, it would be similar to saying that human drivers should never face an ethical dilemma. Furthermore the report argues that algorithms should be checked and that self-adapting systems should not be applied in autonomous cars. That is also strange, an autonomous car should learn by driving and adapt. It is also unimaginable that a human driver would be forbidden to learn from mistakes.

In a distant past when cars were just introduced, there was a rule in the UK that in front of a car there should be someone walking with a red flag to warn people. Perhaps we should use a red flag to warn (or better: make people aware) that you are dealing with an AI system. A Blade Runner situation where it is difficult to distinguish humans from machines should be avoided. Regulation might be helpful for that.

During the debate the arguments pro and con were both challenged and supported and some new issues were raised. This chapter provides a selection of the main topics discussed. Sometimes in a Q&A format, sometimes just as additional remarks.

AI tools, systems, technology could/should be regulated as human beings are also regulated. What about AI as a scientific discipline, should that be regulated as well? In a sense this is already regulated in the same way as other scientific disciplines like medicine and genetics. When applying for research funding for instance the request has to be judged on many aspects including ethical issues. What should not be regulated are the goals to pursue with research.

A comparison with the regulation of the Internet can be made. We now realize that we were too late thinking about regulation of the Internet when the Internet was created and that makes it difficult to repair it now. Maybe it is also due to the way scientists think. The benefits prevail, especially in areas where AI can assist professionals such as medical doctors who are already overburdened to take over part of the routine work. And focusing on the benefits, the risk of abuse of technology that is created with the best of intentions might be overlooked.

Regulations should be in place in certain areas but an additional question is who will be responsible for those regulations. If it is the lawmakers do they know enough about the topic to draft good regulations. A lot of poor law is written because of insufficient knowledge of the subject matter. Society/lawmakers lag behind with respect to technological developments. We as professionals at the forefront of these developments are better placed to judge where these developments might lead and what might be an appropriate societal safeguard long before the lawmakers can make those decisions. This means that we as an IT community have an obligation to engage with legislators to support them in drafting decent legislation. We should at least make an effort to be involved.

The issue was raised that a request for funding of scientific research usually has to pass via ethical committees, because funds are tax payers money. That is not the case for research and product development done by industry. AI as technology may perhaps not face ethical questions but the applications do. Is the current ethical oversight (for academic funding) sufficient? During many years of AI research ethical questions never popped up, we researched nice technology. Because there were no real-life applications. Now this is changing for instance with autonomous cars. And that introduces also the question of impact of the application. The impact of an application is often not or rarely assessed before selling or using it.

An interesting perspective was mentioned from a small and medium sized enterprise point of view. When you develop a new product, regulation in the beginning is an obstacle and difficult. However, it can also mean a benefit if you can advertise that your product meets certain regulations. And the competition has to keep up with that. Also good for consumers who can see that a product meets certain requirements as laid down in regulations.

Regulation in the globalized world is difficult. It is not enough to have regulations in a country or a region. However, so far regulation on a global scale, for instance via UN, is not successful. If we want to regulate a borderless development such as AI, we need to do that on a global scale otherwise it is meaningless. The statement that regulation will only work if it is on a global scale was challenged. Take for example the argument that companies will go to a country that is not regulated. That will not help them because they can maybe produce the product in such a country but if they want to sell it in a country that has regulation, it will not be able to do so unless it complies with the regulation. GDPR is a good example. US companies have to comply with GDPR if they want to do business in the EU. It can work well even if it is jurisdiction based.

Another talk at WCC about shifting identities triggered an issue of importance to AI. Identities touch a multidisciplinary field. Regulating this part of science also means that we need to be very clear where we want to go, what we want to be in the future. It was mentioned that perhaps a link can be made to the work on consciousness. Psychologists and philosophers are trying to find out what does it mean for humans to have consciousness. As developers of AI we are working on AI systems that have a kind of consciousness. A German philosopher Metzinger argues that we should never try to bring consciousness into an artificial system because then we would be able to bring harm to them and we are not allowed to do harm to other human beings.

It is our duty as IT professionals to explain AI to people. We should be able to understand how algorithms work. And to explain the choices that have been made in designing the algorithm and the effects the algorithm may have.

While there was not a complete agreement on everything and the outcome of the discussion was not fully conclusive, in general a broad consensus could be noted on a number of issues.

Artificial Intelligence is a broad term that includes science, technology, applications and products. AI can bring benefits but it also can introduce (new) risks. The answer to the question “should it be (more) regulated” depends on a variety of aspects, it can’t be a yes or no. Consider things case by case. The term regulation is not precise, it can mean a law but it can also mean mutually agreed rules and procedures.

We do not have clear easy answers but we should make efforts and increase awareness. We should debate and work on documents to indicate critical points. We cannot control everything. You should know who you are talking to. Difficult, challenging but not a reason for not trying.

It is important as professionals to engage in discussions like this. We as an IT community have an obligation to engage with legislators to support them in drafting decent legislation.

We should develop methodologies to certify AI products. There is a role for IFIP and other professional societies to think about ways about how to define workflows for approving AI based products.

April 25^th 2018 the European Commission issued a Communication COM (218) on Artificial Intelligence for Europe [4]. This Communication sets out a European initiative on AI, which aims to ensure an appropriate ethical and legal framework. The Commission will (selective quotes):

This Communication addresses precisely the issues raised in the discussion. It also invites stakeholders to participate in the efforts. Let’s contribute to these and other efforts in the world. There is a need and an opportunity for us as IT professionals to pick up the challenge and to continue the discussion. There is momentum now, let’s not waste the opportunity. We don’t want to observe in ten years’ time that we again missed the boat (after the Internet). We also have to research some fundamental questions, where do we want to go with regulation, where do we want to go with applications, who do we want to be. This is an appeal to all participants and readers who are interested in continuing this debate in a search for guidance. If you want to get involved, let me know. Send an e-mail to the address at the start of the paper.