Skip to main content
Erschienen in:

01.08.2015 | Open Forum

Racing to the precipice: a model of artificial intelligence development

verfasst von: Stuart Armstrong, Nick Bostrom, Carl Shulman

Erschienen in: AI & SOCIETY | Ausgabe 2/2016

Einloggen

Aktivieren Sie unsere intelligente Suche, um passende Fachinhalte oder Patente zu finden.

search-config
loading …

Abstract

This paper presents a simple model of an AI (artificial intelligence) arms race, where several development teams race to build the first AI. Under the assumption that the first AI will be very powerful and transformative, each team is incentivised to finish first—by skimping on safety precautions if need be. This paper presents the Nash equilibrium of this process, where each team takes the correct amount of safety precautions in the arms race. Having extra development teams and extra enmity between teams can increase the danger of an AI disaster, especially if risk-taking is more important than skill in developing the AI. Surprisingly, information also increases the risks: the more teams know about each others’ capabilities (and about their own), the more the danger increases. Should these results persist in more realistic models and analysis, it points the way to methods of increasing the chance of the safe development of AI.

Sie haben noch keine Lizenz? Dann Informieren Sie sich jetzt über unsere Produkte:

Springer Professional "Wirtschaft+Technik"

Online-Abonnement

Mit Springer Professional "Wirtschaft+Technik" erhalten Sie Zugriff auf:

  • über 102.000 Bücher
  • über 537 Zeitschriften

aus folgenden Fachgebieten:

  • Automobil + Motoren
  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Elektrotechnik + Elektronik
  • Energie + Nachhaltigkeit
  • Finance + Banking
  • Management + Führung
  • Marketing + Vertrieb
  • Maschinenbau + Werkstoffe
  • Versicherung + Risiko

Jetzt Wissensvorsprung sichern!

Springer Professional "Wirtschaft"

Online-Abonnement

Mit Springer Professional "Wirtschaft" erhalten Sie Zugriff auf:

  • über 67.000 Bücher
  • über 340 Zeitschriften

aus folgenden Fachgebieten:

  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Finance + Banking
  • Management + Führung
  • Marketing + Vertrieb
  • Versicherung + Risiko




Jetzt Wissensvorsprung sichern!

Springer Professional "Technik"

Online-Abonnement

Mit Springer Professional "Technik" erhalten Sie Zugriff auf:

  • über 67.000 Bücher
  • über 390 Zeitschriften

aus folgenden Fachgebieten:

  • Automobil + Motoren
  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Elektrotechnik + Elektronik
  • Energie + Nachhaltigkeit
  • Maschinenbau + Werkstoffe




 

Jetzt Wissensvorsprung sichern!

Fußnoten
1
Though high uncertainties do not imply safety.
 
2
Some examples of bad values could be ‘prevent human suffering’ via killing people off and ‘make people happy’ via wireheading(Yudkowsky 2008; Bostrom 2014).
 
3
And only the winning team—if another team gets a disastrous AI first by taking lower precautions, they will ‘won’ the race to build the first AI.
 
4
Of course, the model can be refined in various ways. One could make capacity information uncertain and fuzzy, one could have different levels of enmity between different teams, one could incorporate uncertainty about the safety levels and the ultimate outcomes, and so on. Or one could have a dynamic process to determine the outcome, rather than rushing straight to the Nash equilibrium. But the simple model is enough to gain useful insights.
 
5
It may seem unusual for teams to not know their own capabilities in the real world. However, this is close to the situation we find ourselves with current AI research: people and organisations have a pretty clear idea of what resources and knowledge they have, but do not know how hard AI is or what routes are most likely to lead there. They are thus effectively ignorant of their own AI-building capabilities.
 
6
If makes no sense that a team with higher capability would have a lower chance of winning (if so, they would voluntarily destroy part of their capability).
 
7
Such secrecy can interfere with trust building, though, making it hard to reach agreements between teams if such agreement is needed.
 
8
This is because only the teams with low capability take risks in cases of private information, and the more teams there are, the less likely it is that the winner will be low capability.
 
Literatur
Zurück zum Zitat Armstrong S (2013) General purpose intelligence: arguing the orthogonality thesis. Anal Metaphys 12:68–84 Armstrong S (2013) General purpose intelligence: arguing the orthogonality thesis. Anal Metaphys 12:68–84
Zurück zum Zitat Armstrong S, Sotala K (2012) How we’re predicting ai-or failing to. In: Romportl J, Ircing P, Zackova E, Polak M, Schuster R (eds) Beyond AI: artificial dreams. University of West Bohemia, Pilsen, pp 52–75 Armstrong S, Sotala K (2012) How we’re predicting ai-or failing to. In: Romportl J, Ircing P, Zackova E, Polak M, Schuster R (eds) Beyond AI: artificial dreams. University of West Bohemia, Pilsen, pp 52–75
Zurück zum Zitat Armstrong S, Sandberg A, Bostrom N (2012) Thinking inside the box: controlling and using an oracle ai. Minds Mach 22:299–324CrossRef Armstrong S, Sandberg A, Bostrom N (2012) Thinking inside the box: controlling and using an oracle ai. Minds Mach 22:299–324CrossRef
Zurück zum Zitat Armstrong S, Sotala K, hÉigeartaigh SO (2014) The errors, insights and lessons of famous ai predictions—and what they mean for the future. In: Proceedings of the 2012 Oxford Winter intelligence conference Armstrong S, Sotala K, hÉigeartaigh SO (2014) The errors, insights and lessons of famous ai predictions—and what they mean for the future. In: Proceedings of the 2012 Oxford Winter intelligence conference
Zurück zum Zitat Bostrom N (2011) Information hazards: a typology of potential harms from knowledge. Rev Contemp Philos 10:44–79 Bostrom N (2011) Information hazards: a typology of potential harms from knowledge. Rev Contemp Philos 10:44–79
Zurück zum Zitat Bostrom N (2012) The superintelligent will: motivation and instrumental rationality in advanced artificial agents. Minds Mach 22:71–85CrossRef Bostrom N (2012) The superintelligent will: motivation and instrumental rationality in advanced artificial agents. Minds Mach 22:71–85CrossRef
Zurück zum Zitat Bostrom N (2013) Existential risk prevention as global priority. Glob Policy 4:15–31CrossRef Bostrom N (2013) Existential risk prevention as global priority. Glob Policy 4:15–31CrossRef
Zurück zum Zitat Bostrom N (2014) Superintelligence: paths, dangers, strategies. Oxford University Press, Oxford Bostrom N (2014) Superintelligence: paths, dangers, strategies. Oxford University Press, Oxford
Zurück zum Zitat Chalmers D (2010) The singularity: a philosophical analysis. J Conscious Stud 17:7–65 Chalmers D (2010) The singularity: a philosophical analysis. J Conscious Stud 17:7–65
Zurück zum Zitat Good I (1965) Speculations concerning the first ultraintelligent machine. Adv Comput 6:31–83CrossRef Good I (1965) Speculations concerning the first ultraintelligent machine. Adv Comput 6:31–83CrossRef
Zurück zum Zitat Henkel RD, Miller T, Weyant RS (2012) Monitoring select agent theft, loss and release reports in the United States–2004–2010. Appl Biosaf 17:171–180CrossRef Henkel RD, Miller T, Weyant RS (2012) Monitoring select agent theft, loss and release reports in the United States–2004–2010. Appl Biosaf 17:171–180CrossRef
Zurück zum Zitat Heymann DL, Aylward RB, Wolff C (2004) Dangerous pathogens in the laboratory: from smallpox to today’s sars setbacks and tomorrow’s polio-free world. Lancet 363:1566–1568CrossRef Heymann DL, Aylward RB, Wolff C (2004) Dangerous pathogens in the laboratory: from smallpox to today’s sars setbacks and tomorrow’s polio-free world. Lancet 363:1566–1568CrossRef
Zurück zum Zitat Kahn H (1960) On thermonuclear war. Transaction publishers, Piscataway Kahn H (1960) On thermonuclear war. Transaction publishers, Piscataway
Zurück zum Zitat Muehlhauser L, Salamon A (2012) Intelligence explosion: evidence and import. In: Eden A, Søraker J, Morr J, Steinhart E (eds) The singularity hypothesis: a scientific and philosophical assessment. Springer, Berlin Muehlhauser L, Salamon A (2012) Intelligence explosion: evidence and import. In: Eden A, Søraker J, Morr J, Steinhart E (eds) The singularity hypothesis: a scientific and philosophical assessment. Springer, Berlin
Zurück zum Zitat Omohundro SM (2008) The basic AI drives. Front Artif Intell Appl 171:483–492 Omohundro SM (2008) The basic AI drives. Front Artif Intell Appl 171:483–492
Zurück zum Zitat Sandberg A, Bostrom N (2011) Machine intelligence survey. Technical report, future of humanity institute, Oxford University #2011-1:1–12 Sandberg A, Bostrom N (2011) Machine intelligence survey. Technical report, future of humanity institute, Oxford University #2011-1:1–12
Zurück zum Zitat Yudkowsky E (2008) Artificial intelligence as a positive and negative factor in global risk. In: Bostrom N, Ćirković MM (eds) Global catastrophic risks. Oxford University Press, New York, pp 308–345 Yudkowsky E (2008) Artificial intelligence as a positive and negative factor in global risk. In: Bostrom N, Ćirković MM (eds) Global catastrophic risks. Oxford University Press, New York, pp 308–345
Metadaten
Titel
Racing to the precipice: a model of artificial intelligence development
verfasst von
Stuart Armstrong
Nick Bostrom
Carl Shulman
Publikationsdatum
01.08.2015
Verlag
Springer London
Erschienen in
AI & SOCIETY / Ausgabe 2/2016
Print ISSN: 0951-5666
Elektronische ISSN: 1435-5655
DOI
https://doi.org/10.1007/s00146-015-0590-y