Skip to main content
Fachgebiete
Automobil + Motoren Bauwesen + Immobilien Business IT + Informatik Elektrotechnik + Elektronik Energie + Nachhaltigkeit Finance + Banking Management + Führung Marketing + Vertrieb Maschinenbau + Werkstoffe Versicherung + Risiko
DE
EN
Bücher
Zeitschriften
Themenseiten
Organisationspsychologie Projektmanagement Marketing Smart Manufacturing
Weitere Formate
Podcasts Webinare Technik Webinare Wirtschaft Kongresse Veranstaltungskalender Awards
Jetzt Einzelzugang starten
Zugang für Unternehmen
Referenzkunden
SLX-Digitalkonferenz: Zukunftswerkstatt 2024

Mehr Umsatz mit Top-Kontakten

Am 7. Mai erfahren Sie, wie Vertriebsteams Leadgenerierung, Leadnurturing und Leadstrategien effizient steuern können.
Erweiterte Suche
Anmelden
nach oben
AI & SOCIETY
Erschienen in: AI & SOCIETY 2/2016

01.08.2015 | Open Forum

Racing to the precipice: a model of artificial intelligence development

verfasst von: Stuart Armstrong, Nick Bostrom, Carl Shulman

Erschienen in: AI & SOCIETY | Ausgabe 2/2016

Einloggen

Aktivieren Sie unsere intelligente Suche, um passende Fachinhalte oder Patente zu finden.

search-config
loading …

Abstract

This paper presents a simple model of an AI (artificial intelligence) arms race, where several development teams race to build the first AI. Under the assumption that the first AI will be very powerful and transformative, each team is incentivised to finish first—by skimping on safety precautions if need be. This paper presents the Nash equilibrium of this process, where each team takes the correct amount of safety precautions in the arms race. Having extra development teams and extra enmity between teams can increase the danger of an AI disaster, especially if risk-taking is more important than skill in developing the AI. Surprisingly, information also increases the risks: the more teams know about each others’ capabilities (and about their own), the more the danger increases. Should these results persist in more realistic models and analysis, it points the way to methods of increasing the chance of the safe development of AI.
Vorheriger Artikel Auto-Catastrophic Theory: the necessity of self-destruction for the formation, survival, and termination of systems
Nächster Artikel The importance of a human viewpoint on computer natural language capabilities: a Turing test perspective

Sie haben noch keine Lizenz? Dann Informieren Sie sich jetzt über unsere Produkte:

Springer Professional "Wirtschaft+Technik"

Online-Abonnement

Mit Springer Professional "Wirtschaft+Technik" erhalten Sie Zugriff auf:

  • über 102.000 Bücher
  • über 537 Zeitschriften

aus folgenden Fachgebieten:

  • Automobil + Motoren
  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Elektrotechnik + Elektronik
  • Energie + Nachhaltigkeit
  • Finance + Banking
  • Management + Führung
  • Marketing + Vertrieb
  • Maschinenbau + Werkstoffe
  • Versicherung + Risiko

Jetzt Wissensvorsprung sichern!

Jetzt informieren

Springer Professional "Wirtschaft"

Online-Abonnement

Mit Springer Professional "Wirtschaft" erhalten Sie Zugriff auf:

  • über 67.000 Bücher
  • über 340 Zeitschriften

aus folgenden Fachgebieten:

  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Finance + Banking
  • Management + Führung
  • Marketing + Vertrieb
  • Versicherung + Risiko




Jetzt Wissensvorsprung sichern!

Jetzt informieren

Springer Professional "Technik"

Online-Abonnement

Mit Springer Professional "Technik" erhalten Sie Zugriff auf:

  • über 67.000 Bücher
  • über 390 Zeitschriften

aus folgenden Fachgebieten:

  • Automobil + Motoren
  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Elektrotechnik + Elektronik
  • Energie + Nachhaltigkeit
  • Maschinenbau + Werkstoffe




 

Jetzt Wissensvorsprung sichern!

Jetzt informieren
Fußnoten
1
Though high uncertainties do not imply safety.
 
2
Some examples of bad values could be ‘prevent human suffering’ via killing people off and ‘make people happy’ via wireheading(Yudkowsky 2008; Bostrom 2014).
 
3
And only the winning team—if another team gets a disastrous AI first by taking lower precautions, they will ‘won’ the race to build the first AI.
 
4
Of course, the model can be refined in various ways. One could make capacity information uncertain and fuzzy, one could have different levels of enmity between different teams, one could incorporate uncertainty about the safety levels and the ultimate outcomes, and so on. Or one could have a dynamic process to determine the outcome, rather than rushing straight to the Nash equilibrium. But the simple model is enough to gain useful insights.
 
5
It may seem unusual for teams to not know their own capabilities in the real world. However, this is close to the situation we find ourselves with current AI research: people and organisations have a pretty clear idea of what resources and knowledge they have, but do not know how hard AI is or what routes are most likely to lead there. They are thus effectively ignorant of their own AI-building capabilities.
 
6
If makes no sense that a team with higher capability would have a lower chance of winning (if so, they would voluntarily destroy part of their capability).
 
7
Such secrecy can interfere with trust building, though, making it hard to reach agreements between teams if such agreement is needed.
 
8
This is because only the teams with low capability take risks in cases of private information, and the more teams there are, the less likely it is that the winner will be low capability.
 
Literatur
Armstrong S (2013) General purpose intelligence: arguing the orthogonality thesis. Anal Metaphys 12:68–84
Armstrong S, Sotala K (2012) How we’re predicting ai-or failing to. In: Romportl J, Ircing P, Zackova E, Polak M, Schuster R (eds) Beyond AI: artificial dreams. University of West Bohemia, Pilsen, pp 52–75
Armstrong S, Sandberg A, Bostrom N (2012) Thinking inside the box: controlling and using an oracle ai. Minds Mach 22:299–324CrossRef
Armstrong S, Sotala K, hÉigeartaigh SO (2014) The errors, insights and lessons of famous ai predictions—and what they mean for the future. In: Proceedings of the 2012 Oxford Winter intelligence conference
Bostrom N (2011) Information hazards: a typology of potential harms from knowledge. Rev Contemp Philos 10:44–79
Bostrom N (2012) The superintelligent will: motivation and instrumental rationality in advanced artificial agents. Minds Mach 22:71–85CrossRef
Bostrom N (2013) Existential risk prevention as global priority. Glob Policy 4:15–31CrossRef
Bostrom N (2014) Superintelligence: paths, dangers, strategies. Oxford University Press, Oxford
Chalmers D (2010) The singularity: a philosophical analysis. J Conscious Stud 17:7–65
Good I (1965) Speculations concerning the first ultraintelligent machine. Adv Comput 6:31–83CrossRef
Henkel RD, Miller T, Weyant RS (2012) Monitoring select agent theft, loss and release reports in the United States–2004–2010. Appl Biosaf 17:171–180CrossRef
Heymann DL, Aylward RB, Wolff C (2004) Dangerous pathogens in the laboratory: from smallpox to today’s sars setbacks and tomorrow’s polio-free world. Lancet 363:1566–1568CrossRef
Kahn H (1960) On thermonuclear war. Transaction publishers, Piscataway
Muehlhauser L, Salamon A (2012) Intelligence explosion: evidence and import. In: Eden A, Søraker J, Morr J, Steinhart E (eds) The singularity hypothesis: a scientific and philosophical assessment. Springer, Berlin
Omohundro SM (2008) The basic AI drives. Front Artif Intell Appl 171:483–492
Sandberg A, Bostrom N (2011) Machine intelligence survey. Technical report, future of humanity institute, Oxford University #2011-1:1–12
Yudkowsky E (2008) Artificial intelligence as a positive and negative factor in global risk. In: Bostrom N, Ćirković MM (eds) Global catastrophic risks. Oxford University Press, New York, pp 308–345
Metadaten
Titel
Racing to the precipice: a model of artificial intelligence development
verfasst von
Stuart Armstrong
Nick Bostrom
Carl Shulman
Publikationsdatum
01.08.2015
Verlag
Springer London
Erschienen in
AI & SOCIETY / Ausgabe 2/2016
Print ISSN: 0951-5666
Elektronische ISSN: 1435-5655
DOI
https://doi.org/10.1007/s00146-015-0590-y

Weitere Artikel der Ausgabe 2/2016

AI & SOCIETY 2/2016 Zur Ausgabe

Book Review

Bernhard Irrgang: Critics of technological lifeworld, collection of philosophical essays

Original Article

Presence: Is it just pretending?

Curmudgeon Corner

Thirty years on

Open Forum

The importance of a human viewpoint on computer natural language capabilities: a Turing test perspective

Original Article

Simulation, self-extinction, and philosophy in the service of human civilization

Open Forum

Growth strategy for Finnish science parks under external economic crises