skip to main content
survey

Security Modeling of Autonomous Systems: A Survey

Published:13 September 2019Publication History
Skip Abstract Section

Abstract

Autonomous systems will soon be integrating into our lives as home assistants, delivery drones, and driverless cars. The implementation of the level of automation in these systems from being manually controlled to fully autonomous would depend upon the autonomy approach chosen to design these systems. This article reviews the historical evolution of autonomy, its approaches, and the current trends in related fields to build robust autonomous systems. Toward such a goal and with the increased number of cyberattacks, the security of these systems needs special attention from the research community. To gauge the extent to which research has been done in this area, we discuss the cybersecurity of these systems. It is essential to model the system from a security perspective, identify the threats and vulnerabilities, and then model the attacks. A survey in this direction explores the theoretical/analytical system and attack models that have been proposed over the years and identifies the research gap that needs to be addressed by the research community.

References

  1. Hezam Akram Abdul-Ghani, Dimitri Konstantas, and Mohammed Mahyoub. 2018. A comprehensive IoT attacks survey based on a building-blocked reference model. Int. J. Adv. Comput. Sci. Appl. (IJACSA) 9, 3 (2018).Google ScholarGoogle Scholar
  2. Manuj Aggarwal. 2017. Blockchain in Robotics—A Sneak Peek into the Future. Retrieved July 18, 2018 from https://tinyurl.com/y5flry5n.Google ScholarGoogle Scholar
  3. Raja Naeem Akram, Pierre-François Bonnefoi, Serge Chaumette, Konstantinos Markantonakis, and Damien Sauveron. 2016. Secure autonomous UAVs fleets by using new specific embedded secure elements. In Proceedings of the 2016 IEEE Trustcom/BigDataSE/ISPA. IEEE, 606--614.Google ScholarGoogle ScholarCross RefCross Ref
  4. Mohammed Saeed Al-Kahtani. 2012. Survey on security attacks in vehicular ad hoc networks (VANETs). In Proceedings of the 6th International Conference on Signal Processing and Communication Systems (ICSPCS’12). IEEE, 1--9.Google ScholarGoogle ScholarCross RefCross Ref
  5. Hamad Al-Mohannadi, Qublai Mirza, Anitta Namanya, Irfan Awan, Andrea Cullen, and Jules Disso. 2016. Cyber-attack modeling analysis techniques: An overview. In Proceedings of the 2016 IEEE 4th International Conference on Future Internet of Things and Cloud Workshops (FiCloudW). IEEE, 69--76.Google ScholarGoogle ScholarCross RefCross Ref
  6. Dario Albani, Joris I. Jsselmuiden, Ramon Haken, and Vito Trianni. 2017. Monitoring and mapping with robot swarms for agricultural applications. In Proceedings of the 14th IEEE International Conference on Advanced Video and Signal Based Surveillance (AVSS). IEEE, 1--6.Google ScholarGoogle ScholarCross RefCross Ref
  7. Homa Alemzadeh, Daniel Chen, Xiao Li, Thenkurussi Kesavadas, Zbigniew T. Kalbarczyk, and Ravishankar K. Iyer. 2016. Targeted attacks on teleoperated surgical robots: Dynamic model-based detection and mitigation. In Proceedings of the 46th Annual IEEE/IFIP International Conference on Dependable Systems and Networks (DSN’16). IEEE, 395--406.Google ScholarGoogle Scholar
  8. Rasim Alguliyev, Yadigar Imamverdiyev, and Lyudmila Sukhostat. 2018. Cyber-physical systems and their security issues. Comput. Ind. 100 (2018), 212--223.Google ScholarGoogle ScholarCross RefCross Ref
  9. Edward Amoroso. 2019. Security Advantages of Software Defined Networking (SDN). Retrieved February 26, 2019 from http://tinyurl.com/yynq6gpg.Google ScholarGoogle Scholar
  10. Maria Azees, Pandi Vijayakumar, and Lazarus Jegatha Deborah. 2016. Comprehensive survey on security services in vehicular ad-hoc networks. IET Intel. Transport Syst. 10, 6 (2016), 379--388.Google ScholarGoogle ScholarCross RefCross Ref
  11. Lisanne Bainbridge. 1983. Ironies of automation. In Analysis, Design and Evaluation of Man--Machine Systems 1982. Elsevier, 129--135.Google ScholarGoogle ScholarCross RefCross Ref
  12. Lina Bariah, Dina Shehada, Ehab Salahat, and Chan Yeob Yeun. 2015. Recent advances in VANET security: A survey. In Proceedings of the 2015 IEEE 82nd Vehicular Technology Conference (VTC Fall). IEEE, 1--7.Google ScholarGoogle ScholarCross RefCross Ref
  13. Ramon Bauza, Javier Gozalvez, and Joaquin Sanchez-Soriano. 2010. Road traffic congestion detection through cooperative vehicle-to-vehicle communications. In Proceedings of the 2010 IEEE 35th Conference Local Computer Networks (LCN). IEEE, Denver, CO. https://ieeexplore.ieee.org/abstract/document/5735780/. Google ScholarGoogle ScholarDigital LibraryDigital Library
  14. Gordon D. Baxter, John Rooksby, Yuanzhi Wang, and Ali Khajeh-Hosseini. 2012. The ironies of automation: still going strong at 30? In ECCE. 65--71. Google ScholarGoogle ScholarDigital LibraryDigital Library
  15. Jenay M. Beer, Arthur D. Fisk, and Wendy A. Rogers. 2014. Toward a framework for levels of robot autonomy in human-robot interaction. J. Human-Robot Interact. 3, 2 (2014), 74--99. Google ScholarGoogle ScholarDigital LibraryDigital Library
  16. Vahid Behzadan. 2017. Cyber-physical attacks on UAS networks—Challenges and open research problems. arxiv:1702.01251 http://arxiv.org/abs/1702.01251.Google ScholarGoogle Scholar
  17. Brian Beyst. 2016. Comparing ThreatModeler to Microsoft Threat Modeling Tool (TMT). Retrieved May 26, 2018 from https://tinyurl.com/y379hnrb.Google ScholarGoogle Scholar
  18. Sourabh Bhattacharya and Tamer. Basar. 2010. Game-theoretic analysis of an aerial jamming attack on a UAV communication network. In Proceedings of the American Control Conference (ACC). IEEE, 818--823. DOI:https://doi.org/978-1-4244-7427-1Google ScholarGoogle ScholarCross RefCross Ref
  19. Sourabh Bhattacharya and Tamer Basar. 2012. Multi-layer hierarchical approach to double sided jamming games among teams of mobile agents. In Proceedings of the IEEE Conference on Decision and Control (2012). 5774--5779.Google ScholarGoogle ScholarCross RefCross Ref
  20. Trevor J. Bihl, Chad Cox, and Todd Jenkins. 2018. Finding common ground by unifying autonomy indices to understand needed capabilities. In Sensors and Systems for Space Applications XI, Vol. 10641. International Society for Optics and Photonics, 106410G.Google ScholarGoogle Scholar
  21. Tamara Bonaci, Jeffrey Herron, Tariq Yusuf, Junjie Yan, Tadayoshi Kohno, and Howard Jay Chizeck. 2015. To make a robot secure: An experimental analysis of cyber security threats against teleoperated surgical robots. arXiv preprint arXiv:1504.04339 (2015).Google ScholarGoogle Scholar
  22. Jeffery Bradshaw, Andrzej Uszok, Renia Jeffers, Niranjan Suri, Patric Hayes, Mark Burstein, Alessandro Acquisti, Brett Benyo, M. Breedy, Marco Carvalho, et al. 2003. Representation and reasoning for DAML-based policy and domain services in KAoS and Nomads. In Proceedings of the 2nd International Joint Conference on Autonomous Agents and Multiagent Systems. ACM, 835--842. Google ScholarGoogle ScholarDigital LibraryDigital Library
  23. Jeffrey M. Bradshaw, Alessandro Acquisti, James Allen, Maggie R. Breedy, Larry Bunch, Nate Chambers, Paul Feltovich, Lucian Galescu, Michael A. Goodrich, Renia Jeffers, et al. 2004a. Teamwork-centered autonomy for extended human-agent interaction in space applications. In Proceedings of the AAAI 2004 Spring Symposium. 22--24.Google ScholarGoogle Scholar
  24. Jeffrey M. Bradshaw, Patrick Beautement, Maggie R. Breedy, Larry Bunch, Sergey V. Drakunov, Paul J. Feltovich, Robert R. Hoffman, Renia Jeffers, Matthew Johnson, Shriniwas Kulkarni, et al. 2004b. Making agents acceptable to people. In Intelligent Technologies for Information Analysis. Springer, 361--406.Google ScholarGoogle Scholar
  25. Jeffrey M. Bradshaw, Hyuckchul Jung, Shri Kulkarni, Matthew Johnson, Paul Feltovich, James Allen, Larry Bunch, Nathanael Chambers, Lucian Galescu, Renia Jeffers, et al. 2005. Kaa: Policy-based explorations of a richer model for adjustable autonomy. In Proceedings of the 4th International Joint Conference on Autonomous Agents and Multiagent Systems. ACM, 214--221. Google ScholarGoogle ScholarDigital LibraryDigital Library
  26. David J. Bruemmer, Donald D. Dudenhoeffer, and Julie L. Marble. 2002. Dynamic-autonomy for urban search and rescue. In AAAI Mobile Robot Competition. 33--37.Google ScholarGoogle Scholar
  27. Patrick Caughill. 2017. An Artificial Intelligence Has Officially Been Granted Residency. Retrieved May 26, 2018 from http://tinyurl.com/y9q3zpz4.Google ScholarGoogle Scholar
  28. Dino Causevic. 2018. How Machine Learning Can Enhance Cybersecurity for Autonomous Cars. Retrieved October 1, 2018 from http://tinyurl.com/yyrhhsg7.Google ScholarGoogle Scholar
  29. CCW. 2018. Five Years of Campaigning, CCW Continues. Retrieved June 09, 2018 from https://www.stopkillerrobots.org/2018/03/fiveyears/.Google ScholarGoogle Scholar
  30. Cesar Cerrudo and Lucas Apa. 2017. Hacking robots before skynet. IOActive Website 2017 (2017), 1--17.Google ScholarGoogle Scholar
  31. George W. Clark, Michael V. Doran, and Todd R. Andel. 2017. Cybersecurity issues in robotics. In IEEE Conference on Cognitive and Computational Aspects of Situation Management (CogSIMA’17). IEEE, 1--5.Google ScholarGoogle Scholar
  32. Bruce T. Clough. 2002. Metrics, Schmetrics! How the Heck Do You Determine a UAV’s Autonomy Anyway? Technical Report 990. Air Force Research Lab, Wright-Patterson AFB OH. 313--319 pages. http://oai.dtic.mil/oai/oai?verb=getRecord&metadataPrefix===html&identifier===ADA515926.Google ScholarGoogle Scholar
  33. Cognilytica. 2018. Will There Be Another AI Winter? Retrieved June 26, 2018 from https://www.cognilytica.com/2018/02/22/will-another-ai-winter/.Google ScholarGoogle Scholar
  34. Ciprian Adrian Corneanu, Marc Oliu Simón, Jeffrey F. Cohn, and Sergio Escalera Guerrero. 2016. Survey on rgb, 3D, thermal, and multimodal approaches for facial expression recognition: History, trends, and affect-related applications. IEEE Trans. Pattern Anal. Mach. Intell. 38, 8 (2016), 1548--1568.Google ScholarGoogle ScholarDigital LibraryDigital Library
  35. Gary Cornelius, Patrice Caire, Nico Hochgeschwender, Miguel A. Olivares-Mendez, Paulo Esteves-Verissimo, Marcus Völp, and Holger Voos. 2017. A perspective of security for mobile service robots. In Proceedings of ROBOT 2017: 3rd Iberian Robotics Conference. 88--100.Google ScholarGoogle Scholar
  36. Nikolaus Correll and Alcherio Martinoli. 2009. Multirobot inspection of industrial machinery. IEEE Rob. Autom. Mag. 16, 1 (2009), 103--112. https://ieeexplore.ieee.org/abstract/document/4799452/.Google ScholarGoogle ScholarCross RefCross Ref
  37. Liam Critchley. 2018. How Machine-Based Learning Will Protect Automobiles from Cyber Attacks. Retrieved October 1, 2018 from https://www.azom.com/article.aspx?ArticleID=15659.Google ScholarGoogle Scholar
  38. Mary L. Cummings, Sylvain Bruni, S. Mercier, and P. J. Mitchell. 2007. Automation Architecture for Single Operator, Multiple UAV Command and Control. Technical Report. Massachusetts Inst. of Tech Cambridge.Google ScholarGoogle Scholar
  39. Mary Missy Cummings. 2014. Man versus machine or man+ machine?IEEE Intell. Syst. 29, 5 (2014), 62--69.Google ScholarGoogle ScholarCross RefCross Ref
  40. Praveen Damacharla, Ahmad Y. Javaid, Jennie J. Gallimore, and Vijay K. Devabhaktuni. 2018. Common metrics to benchmark human-machine teams (HMT): A review. IEEE Access 6 (2018), 38637--38655.Google ScholarGoogle ScholarCross RefCross Ref
  41. Drew Davidson, Hao Wu, Robert Jellinek, Thomas Ristenpart, and Vikas Singh. 2016. Controlling UAVs with sensor input spoofing attacks. In Proceedings of the 10th USENIX Conference on Offensive Technologies (WOOT’16). 221--231. Google ScholarGoogle ScholarDigital LibraryDigital Library
  42. Alex Davies. 2018. The Self-Driving Startup Teaching Cars to Talk. Retrieved February 26, 2019 from https://www.wired.com/story/driveai-self-driving-design-frisco-texas/.Google ScholarGoogle Scholar
  43. Office of the Secretary of Defense, Washington D.C. 2001. Unmanned Aerial Vehicles Roadmap 2000-2025. Technical Report. Defense Pentagon, Washington, D.C.Google ScholarGoogle Scholar
  44. Meghann Lomas De Brun, Vera Zaychik Moffitt, Jerry L. Franke, Dimitri Yiantsios, Trevor Housten, Adria Hughes, Shannon Fouse, and Drew Housten. 2008. Mixed-initiative adjustable autonomy for human/unmanned system teaming. In AUVSI Unmanned Systems North America Conference.Google ScholarGoogle Scholar
  45. Matthew DeBord. 2018. Waymo Has Launched Its Commercial Self-driving Service in Phoenix and It’s Called “Waymo One”. Retrieved February 26, 2019 from http://tinyurl.com/y25d4g9x.Google ScholarGoogle Scholar
  46. Tamara Denning, Cynthia Matuszek, Karl Koscher, Joshua R. Smith, and Tadayoshi Kohno. 2009. A spotlight on security and privacy risks with future household robots. In Proceedings of the 11th International Conference on Ubiquitous Computing - Ubicomp’09. ACM, 105. Google ScholarGoogle ScholarDigital LibraryDigital Library
  47. Munjal Desai. 2007. Sliding Scale Autonomy and Trust in Human-robot Interaction. Master’s thesis. 1442064.Google ScholarGoogle Scholar
  48. Munjal Desai and Holly A. Yanco. 2005. Blending human and robot inputs for sliding scale autonomy. In Proceedings of the IEEE International Workshop on Robot and Human Interactive Communication (ROMAN’05). IEEE, 537--542.Google ScholarGoogle Scholar
  49. A. Dhamgaye and N. Chavhan. 2013. Survey on security challenges in VANET. Int. J. Comput. Sci. Network 2, 1 (2013), 1.Google ScholarGoogle Scholar
  50. M. Bernardine Dias, Balajee Kannan, Brett Browning, E. Gil Jones, Brenna Argall, M. Freddie Dias, Marc Zinck, Manuela M. Veloso, and Anthony J. Stentz. 2008. Sliding autonomy for peer-to-peer human-robot teams. Intell. Auton. Syst. 10, IAS 2008 (2008), 332--341.Google ScholarGoogle Scholar
  51. Derui Ding, Qing-Long Han, Yang Xiang, Xiaohua Ge, and Xian-Ming Zhang. 2018. A survey on security control and attack detection for industrial cyber-physical systems. Neurocomput. 275 (2018), 1674--1683. Google ScholarGoogle ScholarDigital LibraryDigital Library
  52. Gregory A. Dorais, R. Peter Bonasso, David Kortenkamp, Barney Pell, and Debra Schreckenghost. 1999. Adjustable autonomy for human-centered autonomous systems. In Proceedings of the 16th International Joint Conference on Artificial Intelligence Workshop on Adjustable Autonomy Systems. 16--35.Google ScholarGoogle Scholar
  53. Luke Dormehl. 2017. MiRo Is the Robot Dog that Promises to Be a Geek’s Best Friend. Retrieved May 20, 2019 from https://tinyurl.com/yxz74w93.Google ScholarGoogle Scholar
  54. Warren R. Dufrene. 2005. An approach for autonomy: A collaborative communication framework for multi-agent systems. In Workshop on Radical Agent Concepts. Springer, 147--159. Google ScholarGoogle ScholarDigital LibraryDigital Library
  55. Muawia Abdelmagid Elsadig and Yahia A. Fadlalla. 2016. VANETs security issues and challenges: A survey. Indian J. Sci. Technol. 9, 28 (2016).Google ScholarGoogle Scholar
  56. Mica R. Endsley. 1987. The application of human factors to the development of expert systems for advanced cockpits. In Proceedings of the Human Factors and Ergonomics Society Annual Meeting, Vol. 31. SAGE Publications Sage CA, Los Angeles, CA, 1388--1392.Google ScholarGoogle ScholarCross RefCross Ref
  57. Mica R. Endsley. 1988. Design and evaluation for situation awareness enhancement. In Proceedings of the Human Factors Society Annual Meeting, Vol. 32. SAGE Publications Sage CA: Los Angeles, CA, 97--101.Google ScholarGoogle ScholarCross RefCross Ref
  58. Mica R. Endsley. 1995. Towards a new paradigm for automation: Designing for situation awareness. IFAC Proceedings Volumes 28, 15 (1995), 365--370.Google ScholarGoogle ScholarCross RefCross Ref
  59. Mica R. Endsley and David B. Kaber. 1999. Level of automation effects on performance, situation awareness and workload in a dynamic control task. Ergon. 42, 3 (1999), 462--492.Google ScholarGoogle ScholarCross RefCross Ref
  60. Control Engineering. 2016. IoT to IoAT: Internet of Autonomous Things Devices Provides Solutions. Retrieved June 18, 2018 from https://tinyurl.com/y5nlonp8.Google ScholarGoogle Scholar
  61. Aisha D. Farooqui and Muaz A. Niazi. 2016. Game Theory Models for Communication between Agents: A Review. Vol. 4. 13 pages.Google ScholarGoogle Scholar
  62. Nicholas Fearn. 2018. The Cutting-Edge Tech Set to Define 2018. Retrieved from http://www.techx365.com/author.asp?section_id=686&doc_id===739321.Google ScholarGoogle Scholar
  63. David Feil-Seifer, Kristine Skinner, and Maja J. Matarić. 2007. Benchmarks for evaluating socially assistive robotics. Interact. Stud. 8, 3 (2007), 423--439.Google ScholarGoogle ScholarCross RefCross Ref
  64. Eduardo Castelló Ferrer. 2016. The blockchain: A new framework for robotic swarm systems. arXiv preprint arXiv:1608.00695 (2016).Google ScholarGoogle Scholar
  65. Tomas Foltyn. 2018. Cybersecurity Trends 2019: Privacy and Intrusion in the Global Village. Check Point Research. Retrieved from http://tinyurl.com/yxeph2cb.Google ScholarGoogle Scholar
  66. T. Fong, N. Cabrol, C. Thorpe, and C. Baur. 2001. A personal user interface for collaborative human-robot exploration. In 6th International Symposium on Artificial Intelligence, Robotics, and Automation in Space (iSAIRAS'01).Google ScholarGoogle Scholar
  67. Stefano Fontanelli, Enrico Bini, and Paolo Santi. 2010. Dynamic route planning in vehicular networks based on future travel estimation. In Proceedings of the 2010 Vehicular Networking Conference (VNC). IEEE, Jersey City, NJ. https://ieeexplore.ieee.org/abstract/document/5698247/.Google ScholarGoogle ScholarCross RefCross Ref
  68. Douglas M. Gage. 1985. Security considerations for autonomous robots. In 1985 IEEE Symposium on Security and Privacy. IEEE, 224--224.Google ScholarGoogle ScholarCross RefCross Ref
  69. Dom Galeon. 2017. World’s First AI Citizen in Saudi Arabia is Now Calling for Women’s Rights. Retrieved May 26, 2018 from http://tinyurl.com/yyqcksxq.Google ScholarGoogle Scholar
  70. Mevlut Turker Garip, Mehmet Emre Gursoy, Peter Reiher, and Mario Gerla. 2015. Congestion attacks to autonomous cars using vehicular botnets. In NDSS Workshop on Security of Emerging Networking Technologies (SENT), San Diego, CA.Google ScholarGoogle ScholarCross RefCross Ref
  71. Michael Giering, Vivek Venugopalan, and Kishore Reddy. 2015. Multi-modal sensor registration for vehicle perception via deep neural networks. In Proceedings of the 2015 High Performance Extreme Computing Conference (HPEC). IEEE, 1--6.Google ScholarGoogle ScholarCross RefCross Ref
  72. Saira Gillani, Farrukh Shahzad, Amir Qayyum, and Rashid Mehmood. 2013. A survey on security in vehicular ad hoc networks. In International Workshop on Communication Technologies for Vehicles. Springer, 59--74.Google ScholarGoogle ScholarCross RefCross Ref
  73. Michael A. Goodrich, Alan C. Schultz, et al. 2008. Human—robot interaction: a survey. Foundations and TrendsR in Human—Computer Interaction 1, 3 (2008), 203--275. Google ScholarGoogle ScholarDigital LibraryDigital Library
  74. James Goppert, Weiyi Liu, Andrew Shull, Vincent Sciandra, Inseok Hwang, and Hal Aldridge. 2012. Numerical analysis of cyberattacks on unmanned aerial systems. AIAA Infotech at Aerospace Conference and Exhibit 2012.Google ScholarGoogle Scholar
  75. Andy Greenberg. 2015. Hackers Remotely Kill a Jeep on the Highway—With Me In It. Retrieved May 26, 2018 from http://tinyurl.com/o9coyn4.Google ScholarGoogle Scholar
  76. Pinyao Guo, Hunmin Kim, Nurali Virani, Jun Xu, Minghui Zhu, and Peng Liu. 2017. Exploiting physical dynamics to detect actuator and sensor attacks in mobile robots. CoRR abs/1708.01834 (2017).Google ScholarGoogle Scholar
  77. David Hambling. 2017. Ships Fooled in GPS Spoofing Attack Suggest Russian Cyberweapon. Retrieved July 18, 2018 from http://tinyurl.com/ycuzl3pz.Google ScholarGoogle Scholar
  78. Elyes Ben Hamida, Hassan Noura, and Wassim Znaidi. 2015. Security of cooperative intelligent transport systems: Standards, threats analysis, and cryptographic countermeasures. Electron. 4, 3 (2015), 380--423.Google ScholarGoogle ScholarCross RefCross Ref
  79. Ali Hamieh, Jalel Ben-Othman, and Lynda Mokdad. 2009. Detection of radio interference attacks in VANET. In Proceedings of the Global Telecommunications Conference, 2009 (GLOBECOM). IEEE, Honolulu, HI. Google ScholarGoogle ScholarDigital LibraryDigital Library
  80. M. Hans, B. Graf, and R. D. Schraft. 2002. Robotic home assistant care-o-bot: Past-present-future. In Proceedings of the 11th IEEE International Workshop on Robot and Human Interactive Communication, 2002. 380--385.Google ScholarGoogle ScholarCross RefCross Ref
  81. Musad Haque, Electa Baker, Christopher Ren, Douglas Kirkpatrick, and Julie A. Adams. 2018. Analysis of biologically inspired swarm communication models. In Advances in Hybridization of Intelligent Methods. Springer, 17--38.Google ScholarGoogle Scholar
  82. Benjamin Hardin and Michael A. Goodrich. 2009. On using mixed-initiative control: A perspective for managing large-scale robotic teams. In Proceedings of the 4th ACM/IEEE International Conference on Human Robot Interaction. ACM, 165--172. Google ScholarGoogle ScholarDigital LibraryDigital Library
  83. F. W. Heger and S. Singh. 2006. Sliding autonomy for complex coordinated multi-robot tasks: Analysis 8 experiments. In Proceedings, Robotics: Systems and Science, Philadelphia. http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.148.98928rep=rep18type=pdf.Google ScholarGoogle Scholar
  84. Frederik W. Heger, Laura M. Hiatt, Brennan Sellner, Reid Simmons, and Sanjiv Singh. 2005. Results in sliding autonomy for multi-robot spatial assembly. European Space Agency, (Special Publication) ESA SP603 (2005), 489--496.Google ScholarGoogle Scholar
  85. Fiona Higgins, Allan Tomlinson, and Keith M. Martin. 2009. Threats to the swarm: Security considerations for swarm robotics. Int. J. Adv. Secur. 2, 283 (2009), 288--297.Google ScholarGoogle Scholar
  86. Kashmir Hill. 2016. Security robot accidentally attacks child. Retrieved April 03, 2018 from https://tinyurl.com/yynv5se3.Google ScholarGoogle Scholar
  87. Homer. {n.d.}. The Iliad. In Vol. Book XVIII. circa.Google ScholarGoogle Scholar
  88. Christopher-Eyk Hrabia, Nils Masuch, and Sahin Albayrak. 2015. A metrics framework for quantifying autonomy in complex systems. In German Conference on Multiagent System Technologies. Springer, 22--41. Google ScholarGoogle ScholarDigital LibraryDigital Library
  89. H. M. Huang. 2004. Autonomy levels for unmanned systems (ALFUS) framework Volume I : Terminology version 2.0. Technical Report.Google ScholarGoogle Scholar
  90. Hui-Min Huang. 2007. Autonomy levels for unmanned systems (ALFUS) framework: Safety and application issues. In Proceedings of 2007 Workshop on Performance Metrics for Intelligent Systems (PerMIS’07). 48--53. Google ScholarGoogle ScholarDigital LibraryDigital Library
  91. Hui-Min Huang, James S. Albus, Elena R. Messina, Robert L. Wade, and R. W. English. 2004a. Specifying autonomy levels for unmanned systems: Interim report. In Proceedings of SPIE Defense and Security Symposium 2004. 386--397.Google ScholarGoogle Scholar
  92. Hui-Min Huang, Elena Messina, and James Albus. 2003b. Autonomy level specification for intelligent autonomous vehicles: Interim progress report. In 2003 Performance Metrics for Intelligent Systems (PerMIS) September (2003). 1--7.Google ScholarGoogle Scholar
  93. Hui-Min Huang, Elena Messina, and James Albus. 2003a. Toward a Generic Model for Autonomy Levels for Unmanned Systems (ALFUS). Technical Report. National Institute of Standards and Technology, Gaithersburg, MD.Google ScholarGoogle Scholar
  94. Hui-Min Huang, Elena Messina, Robert Wade, Ralph English, Brian Novak, and James Albus. 2004b. Autonomy measures for robots. In ASME 2004 International Mechanical Engineering Congress and Exposition. American Society of Mechanical Engineers, 1241--1247.Google ScholarGoogle ScholarCross RefCross Ref
  95. Hui-Min Huang, Kerry Pavek, James Albus, and Elena Messina. 2005. Autonomy levels for unmanned systems (ALFUS) framework: An update. Proc. SPIE 5804, June (2005), 439--448.Google ScholarGoogle ScholarCross RefCross Ref
  96. Tom Huddleston Jr. 2018. Move Over Tesla, This Self-driving Car Will Let You Sleep or Watch a Movie during Your Highway Commute. Retrieved February 26, 2019 from http://tinyurl.com/yad58cgj.Google ScholarGoogle Scholar
  97. Todd Humphrey. 2012. Todd Humphreys’ Research Team Demonstrates First Successful GPS Spoofing of UAV. Retrieved July 18, 2018 from http://tinyurl.com/yydz75x5.Google ScholarGoogle Scholar
  98. -->Google ScholarGoogle Scholar
  99. Rosteslaw M. Husar and Jerrell Stracener. 2014. System autonomy modeling during early concept definition. Editorial Preface 5, 6 (2014). -->Google ScholarGoogle Scholar
  100. -->Google ScholarGoogle Scholar
  101. Mustafa Hussain, James Dewey, and Nadir Weibel. 2017. EAI endorsed transactions. -->Google ScholarGoogle Scholar
  102. Ians. 2015. Surgical Robots to Become Ubiquitous in Indian Hospitals. Retrieved September 13, 2018 from http://tinyurl.com/y2rfp2es.Google ScholarGoogle Scholar
  103. Smart Industry. 2017. Blockchain Makes IoT Devices Autonomous. Retrieved 2018-06-19 from https://tinyurl.com/y5w784re.Google ScholarGoogle Scholar
  104. Farha Jahan, Ahmad Y. Javaid, Weiqing Sun, and Mansoor Alam. 2015. GNSSim: An open source GNSS/GPS framework for unmanned aerial vehicular network simulation. EAI Endorsed Trans. Mobile Commun. Appl. 2, 6 (2015), 1--13.Google ScholarGoogle Scholar
  105. Ahmad Y. Javaid, Farha Jahan, and Weiqing Sun. 2017. Analysis of global positioning system-based attacks and a novel global positioning system spoofing detection/mitigation algorithm for unmanned aerial vehicle simulation. Simul. 93, 5 (2017), 427--441. Google ScholarGoogle ScholarDigital LibraryDigital Library
  106. Ahmad Y. Javaid, Weiqing Sun, and Mansoor Alam. 2015. Single and multiple UAV cyber-attack simulation and performance evaluation. EAI Endorsed Trans. Scalable Inf. Syst. 2, 4 (2015), 1--11.Google ScholarGoogle Scholar
  107. Ahmad Y. Javaid, Weiqing Sun, Vijay K. Devabhaktuni, and Mansoor Alam. 2012. Cyber security threat analysis and modeling of an unmanned aerial vehicle system. In 2012 IEEE Conference on Technologies for Homeland Security (HST). IEEE, 585--590.Google ScholarGoogle ScholarCross RefCross Ref
  108. Kanwaldeep Kaur and Giselle Rampersad. 2018. Trust in driverless cars: Investigating key factors influencing the adoption of driverless cars. J. Eng. Tech. Manage. 48 (2018), 87--96.Google ScholarGoogle ScholarDigital LibraryDigital Library
  109. Alan Kim, Brandon Wampler, James Goppert, Inseok Hwang, and Hal Aldridge. 2012. Cyber attack vulnerabilities analysis for unmanned aerial vehicles. Infotech@Aerospace 2012 (2012), 1--30.Google ScholarGoogle Scholar
  110. Matthew Klenk, Matt Molineaux, and David W. Aha. 2013. Goal-driven autonomy for responding to unexpected events in strategy simulations. Comput. Intell. 29, 2 (2013), 187--206.Google ScholarGoogle ScholarCross RefCross Ref
  111. Timo Korthals, Mikkel Kragh, Peter Christiansen, Henrik Karstoft, Rasmus N. Jørgensen, and Ulrich Rückert. 2018. Multi-modal detection and mapping of static and dynamic obstacles in agriculture for process evaluation. Front. Rob. AI 5 (2018), 28.Google ScholarGoogle ScholarCross RefCross Ref
  112. Karl Koscher, Alexei Czeskis, Franziska Roesner, Shwetak Patel, Tadayoshi Kohno, Stephen Checkoway, Damon McCoy, Brian Kantor, Danny Anderson, Hovav Shacham, et al. 2010. Experimental security analysis of a modern automobile. In Proceedings of the IEEE Symposium on Security and Privacy (SP’10). IEEE, 447--462. Google ScholarGoogle ScholarDigital LibraryDigital Library
  113. C. G. Leela Krishna and Robin R. Murphy. 2017. A review on cybersecurity vulnerabilities for unmanned aerial vehicles. In Proceedings of the IEEE International Symposium on Safety, Security and Rescue Robotics (SSRR’17). IEEE, 194--199.Google ScholarGoogle Scholar
  114. Rajesh Kumar, Mohd Abuzar Sayeed, Vishal Sharma, and Ilsun You. 2017. An SDN-based secure mobility model for UAV-ground communications. In Proceedings of the International Symposium on Mobile Internet Security. Springer, 169--179.Google ScholarGoogle Scholar
  115. Cheolhyeon Kwon, Weiyi Liu, and Inseok Hwang. 2013. Security analysis for cyber-physical systems against stealthy deception attacks. In Proceedings of the 2013 American Control Conference (ACC). IEEE, 3344--3349.Google ScholarGoogle Scholar
  116. Cheolhyeon Kwon, Scott Yantek, and Inseok Hwang. 2016. Real-time safety assessment of unmanned aircraft systems against stealthy cyber attacks. J. Aerosp. Inf. Syst. 13, 1 (2016), 27--45.Google ScholarGoogle ScholarCross RefCross Ref
  117. Dimosthenis Kyriazis and Theodora Varvarigou. 2013. Smart, autonomous and reliable Internet of Things. Procedia Comput. Sci. 21 (2013), 442--448.Google ScholarGoogle ScholarCross RefCross Ref
  118. Hoa La Vinh and Ana Rosa Cavalli. 2014. Security attacks and solutions in vehicular ad hoc networks: A survey. Int. J. Ad Hoc Networking Syst. (IJANS) 4, 2 (2014), 1--20.Google ScholarGoogle ScholarCross RefCross Ref
  119. Michel Lacerda, Dongjin Park, Srujal Patel, and Daniel Schrage. 2018. A Mars exploration concept systems design with an innovative unmanned autonomous vehicle and “carrier” ground rover configuration. Part I: System design. In Proceedings of the 2018 Aviation Technology, Integration, and Operations Conference. 3262.Google ScholarGoogle ScholarCross RefCross Ref
  120. Fred Lambert. 2018. Watch What Tesla Autopilot Can See in Incredible 360º Video. Retrieved 2018-11-26 from https://electrek.co/2018/11/26/tesla-autopilot-360-video/.Google ScholarGoogle Scholar
  121. John Leonard, Jonathan How, Seth Teller, Mitch Berger, Stefan Campbell, Gaston Fiore, Luke Fletcher, Emilio Frazzoli, Albert Huang, Sertac Karaman, et al. 2008. A perception-driven autonomous urban vehicle. J. Field Rob. 25, 10 (2008), 727--774. Google ScholarGoogle ScholarDigital LibraryDigital Library
  122. Francisco J. Rodríguez Lera, Camino Fernández Llamas, Ángel Manuel Guerrero, and Vicente Matellán Olivera. 2017. Cybersecurity of robotics and autonomous systems: Privacy and safety. In Robotics-Legal, Ethical and Socioeconomic Impacts. InTech.Google ScholarGoogle Scholar
  123. Hui Lin, Homa Alemzadeh, Daniel Chen, Zbigniew Kalbarczyk, and Ravishankar K. Iyer. 2016. Safety-critical cyber-physical attacks: Analysis, detection, and mitigation. In Proceedings of the Symposium and Bootcamp on the Science of Security. ACM, 82--89. Google ScholarGoogle ScholarDigital LibraryDigital Library
  124. Lanny Lin, Michael A. Goodrich, and Spencer Clark. 2012. Sliding autonomy for UAV path planning: Adding new dimensions to autonomy management. J. Hum.-Rob. Interact. 1, 1 (2012), 78--95.Google ScholarGoogle Scholar
  125. Nikita Lyamin, Alexey Vinel, Magnus Jonsson, and Jonathan Loo. 2014. Real-time detection of denial-of-service attacks in IEEE 802.11 p vehicular networks. IEEE Commun. Lett. 18, 1 (2014), 110--113.Google ScholarGoogle ScholarCross RefCross Ref
  126. Bharat B. Madan, Manoj Banik, and Doina Bein. 2019. Securing unmanned autonomous systems from cyber threats. J. Defense Model. Simul. 16, 2 (2019), 119--136.Google ScholarGoogle ScholarCross RefCross Ref
  127. Federico Maggi, Davide Quarta, Marcello Pogliani, Mario Polino, Andrea M. Zanchettin, and Stefano Zanero. 2017. Rogue Robots: Testing the Limits of an Industrial Robot’s Security. Technical Report. Trend Micro, Politecnico di Milano.Google ScholarGoogle Scholar
  128. Katrina Mansfield, Timothy Eveleigh, Thomas H. Holzer, and Shahryar Sarkani. 2013. Unmanned aerial vehicle smart device ground control station cyber security threat model. In IEEE International Conference on Technologies for Homeland Security (HST’13). IEEE, 722--728.Google ScholarGoogle ScholarCross RefCross Ref
  129. Stephen McBride. 2018. The Driverless Car Revolution Has Begun—Here’s How To Profit. Retrieved from http://tinyurl.com/ydzby4yr.Google ScholarGoogle Scholar
  130. Joe W. McDaniel. 1988. Rules for fighter cockpit automation. In Proceedings of the IEEE 1988 National Aerospace and Electronics Conference (NAECON’88). IEEE, 831--838.Google ScholarGoogle ScholarCross RefCross Ref
  131. Dhwani Mehta, Mohammad Faridul Haque Siddiqui, and Ahmad Y. Javaid. 2018. Facial emotion recognition: A survey and real-world user experiences in mixed reality. Sens. 18, 2 (2018), 416.Google ScholarGoogle ScholarCross RefCross Ref
  132. Mohamed Nidhal Mejri and Jalel Ben-Othman. 2014b. Detecting greedy behavior by linear regression and watchdog in vehicular ad hoc networks. In Proceedings of the 2014 IEEE Global Communications Conference (GLOBECOM). IEEE, Austin, TX.Google ScholarGoogle ScholarCross RefCross Ref
  133. Mohamed Nidhal Mejri and Jalel Ben-Othman. 2014a. Entropy as a new metric for denial of service attack detection in vehicular ad-hoc networks. In Proceedings of the 17th ACM International Conference on Modeling, Analysis and Simulation of Wireless and Mobile Systems. ACM, 73--79. Google ScholarGoogle ScholarDigital LibraryDigital Library
  134. Mohamed Nidhal Mejri, Jalel Ben-Othman, and Mohamed Hamdi. 2014. Survey on VANET security challenges and possible cryptographic solutions. Veh. Commun. 1, 2 (2014), 53--66. Google ScholarGoogle ScholarDigital LibraryDigital Library
  135. Mohamed Nidhal Mejri and Mohamed Hamdi. 2015. Recent advances in cryptographic solutions for vehicular networks. In International Symposium Networks, Computers and Communications (ISNCC’15). IEEE.Google ScholarGoogle ScholarCross RefCross Ref
  136. Kathryn Merrick, Medria Hardhienata, Kamran Shafi, and Jiankun Hu. 2016. A survey of game theoretic approaches to modelling decision-making in information warfare scenarios. Future Internet 8, 3 (2016), 34.Google ScholarGoogle ScholarCross RefCross Ref
  137. Vera Zaychik Moffitt, Jerry L. Franke, and Meghann Lomas. 2006. Mixed-initiative adjustable autonomy in multi-vehicle operations. In Proceedings of AUVSI, Orlando, Florida (2006).Google ScholarGoogle Scholar
  138. Matthew Molineaux, Matthew Klenk, and David W. Aha. 2010. Goal-driven autonomy in a navy strategy simulation. In AAAI. 1548--1554. Google ScholarGoogle ScholarDigital LibraryDigital Library
  139. Salama A. Mostafa, Mohd Sharifuddin Ahmad, and Aida Mustapha. 2017. Adjustable autonomy: A systematic literature review. Artif. Intell. Rev. (2017), 1--38. Google ScholarGoogle ScholarDigital LibraryDigital Library
  140. Salama A. Mostafa, Mohd Sharifuddin Ahmad, and Aida Mustapha. 2019. Adjustable autonomy: A systematic literature review. Artif. Intell. Rev. 51, 2 (2019), 149--186. Google ScholarGoogle ScholarDigital LibraryDigital Library
  141. Jordan Navarro. 2018. A state of science on highly automated driving. Theoretical Issues in Ergonomics Science 20, 3 (2019), 366--396.Google ScholarGoogle ScholarCross RefCross Ref
  142. Navya. 2018. Navya History. Retrieved March 2, 2019 from https://www.navya-corp.com/index.php/fr/navya/histoire.Google ScholarGoogle Scholar
  143. Annalee Newitz. 2013. 15 Books That Will Change the Way You Look at Robots. Retrieved March 03, 2018 from http://tinyurl.com/yy2mkkwg.Google ScholarGoogle Scholar
  144. Quamar Niyaz, Weiqing Sun, and Ahmad Y. Javaid. 2016. A deep learning based DDoS detection system in software-defined networking (SDN). arXiv preprint arXiv:1611.07400 (2016).Google ScholarGoogle Scholar
  145. Donald A. Norman. 1990. The problem with automation: Inappropriate feedback and interaction, not over-automation. Phil. Trans. R. Soc. Lond. B 327, 1241 (1990), 585--593.Google ScholarGoogle ScholarCross RefCross Ref
  146. University of Nottingham--Mixed Reality Laboratory. 2016. Future Everyday Interaction with the Autonomous Internet of Things (A-IoT). Retrieved June 19, 2018 from https://tinyurl.com/yylnuarh.Google ScholarGoogle Scholar
  147. Ebenezer A. Oladimeji, Sam Supakkul, and Lawrence Chung. 2006. Security threat modeling and analysis: A goal-oriented approach. In Proceedings of the 10th IASTED International Conference on Software Engineering and Applications (SEA’06). Citeseer, 13--15.Google ScholarGoogle Scholar
  148. Ancient Origins. 2013. Talos Crete. Retrieved March 05, 2018 from http://www.ancient-origins.net/myths-legends/talos-crete-00157.Google ScholarGoogle Scholar
  149. Joe Pappalardo. 2018. The Dream of Drone Delivery Just Became Much More Real. Retrieved February 26, 2019 from http://tinyurl.com/yylzkvqs.Google ScholarGoogle Scholar
  150. Raja Parasuraman, Toufik Bahri, John E. Deaton, Jeffrey G. Morrison, and Michael Barnes. 1992. Theory and Design of Adaptive Automation in Aviation Systems. Technical Report. Catholic Univ of America, Washington D.C., Cognitive Science Lab. http://books.google.com.my/books?id=DEtSOwAACAAJ.Google ScholarGoogle Scholar
  151. R. Parasuraman, T. B. Sheridan, and C. D. Wickens. 2000. A model for types and levels of human interaction with automation. IEEE Trans. Syst. Man Cybern. Part A Syst. Humans 30, 3 (May 2000), 286--297. Google ScholarGoogle ScholarDigital LibraryDigital Library
  152. Simon Parkinson, Paul Ward, Kyle Wilson, and Jonathan Miller. 2017. Cyber threats facing autonomous and connected vehicles: Future challenges. IEEE Trans. Intell. Transp. Syst. 18, 11 (2017), 2898--2915.Google ScholarGoogle ScholarDigital LibraryDigital Library
  153. Scott Drew Pendleton, Hans Andersen, Xinxin Du, Xiaotong Shen, Malika Meghjani, You Hong Eng, Daniela Rus, and Marcelo H. Ang. 2017. Perception, planning, control, and coordination for autonomous vehicles. Machines 5, 1 (2017), 6.Google ScholarGoogle ScholarCross RefCross Ref
  154. Jonathan Petit and Steven E. Shladover. 2015. Potential cyberattacks on automated vehicles. IEEE Trans. Intell. Transp. Syst. 16, 2 (2015), 546--556.Google ScholarGoogle ScholarDigital LibraryDigital Library
  155. Jonathan Petit, Bas Stottelaar, Michael Feiri, and Frank Kargl. 2015. Remote attacks on automated vehicles sensors: Experiments on camera and lidar. Black Hat Europe 11 (2015), 2015.Google ScholarGoogle Scholar
  156. T. Proscevicius, A. Bukis, V. Raudonis, and M. Eidukeviciute. 2011. Hierarchical control approach for autonomous mobile robots. Elektronika ir Elektrotechnika 110, 4 (2011), 101--104.Google ScholarGoogle Scholar
  157. Oscar Puñal, Ana Aguiar, and James Gross. 2012. In VANETs we trust?: Characterizing RF jamming in vehicular networks. In Proceedings of the 9th ACM International Workshop on Vehicular Inter-Networking, Systems, and Applications. ACM, 83--92. Google ScholarGoogle ScholarDigital LibraryDigital Library
  158. Davide Quarta, Marcello Pogliani, Mario Polino, Federico Maggi, Andrea Maria Zanchettin, and Stefano Zanero. 2017. An experimental security analysis of an industrial robot controller. In 2017 38th IEEE Symposium on Security and Privacy (SP). IEEE, 268--286.Google ScholarGoogle ScholarCross RefCross Ref
  159. Nancy E. Reed. 2005. A user controlled approach to adjustable autonomy. In Proceedings of the 38th Annual Hawaii International Conference on System Sciences. 295b--295b. Google ScholarGoogle ScholarDigital LibraryDigital Library
  160. Felix Richter. 2018. Fatal Accidents Damage Trust in Autonomous Driving. Retrieved May 26, 2018 from https://tinyurl.com/y3lbkx8a.Google ScholarGoogle Scholar
  161. M. E. Rosheim. 2006. Leonardo’s Lost Robot. Springer Verlag, Berlin.Google ScholarGoogle Scholar
  162. Joey Roulette. 2019. Self-driving Buses Roll into Orlando’s Lake Nona, a Growing Testbed for “Smart City” Technology. Retrieved February 26, 2019 from http://tinyurl.com/yy3kc8kh.Google ScholarGoogle Scholar
  163. William B. Rouse. 1976. Adaptive allocation of decision making responsibility between supervisor and computer. In Monitoring Behavior and Supervisory Control. Springer, 295--306.Google ScholarGoogle Scholar
  164. Erol Sahin. 2005. Swarm robotics: From sources of inspiration to domains of application. In Swarm Robotics, Erol cSahin and William M. Spears (Eds.). Springer, Berlin, 10--20. Google ScholarGoogle ScholarDigital LibraryDigital Library
  165. Mukesh Saini, Abdulhameed Alelaiwi, and Abdulmotaleb El Saddik. 2015. How close are we to realizing a pragmatic VANET solution? A meta-survey. ACM Comput. Surv. (CSUR) 48, 2 (2015), 29. Google ScholarGoogle ScholarDigital LibraryDigital Library
  166. Anibal Sanjab, Walid Saad, and Tamer Başar. 2017. Prospect theory for enhanced cyber-physical security of drone delivery systems: A network interdiction game. In 2017 IEEE International Conference on Communications (ICC'17). IEEE, 1--6.Google ScholarGoogle ScholarCross RefCross Ref
  167. Sarah Sloat. 2016. BMW, Intel, Mobileye Link Up in Self-Driving Tech Alliance. Retrieved February 26, 2019 from http://tinyurl.com/y56bwcd2.Google ScholarGoogle Scholar
  168. Paul Scerri, David V. Pynadath, and Milind Tambe. 2003. Adjustable autonomy for the real world. In Agent Autonomy. Springer, 211--241.Google ScholarGoogle Scholar
  169. Brennan P. Sellner, Laura M. Hiatt, Reid Simmons, and Sanjiv Singh. 2006. Attaining situational awareness for sliding autonomy. In Proceedings of the 1st ACM SIGCHI/SIGART Conference on Human-Robot Interaction. ACM, 80--87. Google ScholarGoogle ScholarDigital LibraryDigital Library
  170. Shervin Shahrdar, Luiza Menezes, and Mehrdad Nojoumian. 2017. A survey on trust in autonomous systems. In Intelligent Computing.Google ScholarGoogle Scholar
  171. Hazim Shakhatreh, Ahmad Sawalmeh, Ala Al-Fuqaha, Zuochao Dou, Eyad Almaita, Issa Khalil, Noor Shamsiah Othman, Abdallah Khreishah, and Mohsen Guizani. 2018. Unmanned aerial vehicles: A survey on civil applications and key research challenges. arXiv preprint arXiv:1805.00881 (2018).Google ScholarGoogle Scholar
  172. Yogesh Kumar Sharma and Ashish Bagla. 2009. Security challenges for swarm robotics. SECURITY CHALLENGES 2, 1 (2009), 45--48.Google ScholarGoogle Scholar
  173. Thomas B. Sheridan. 1992. Telerobotics, Automation, and Human Supervisory Control. MIT Press, Cambridge, MA. Google ScholarGoogle ScholarDigital LibraryDigital Library
  174. Thomas B. Sheridan and William L. Verplank. 1978. Human and computer control of undersea teleoperators. Technical Report. Massachusetts Inst. of Tech. Cambridge Man-Machine Systems Lab.Google ScholarGoogle Scholar
  175. Forrest Shull. 2016. Cyber Threat Modeling: An Evaluation of Three Methods. Retrieved May 26, 2018 from https://tinyurl.com/y3j56guz.Google ScholarGoogle Scholar
  176. Elizabeth Snell. 2015. Phishing Attack Affects 3,300 Partners HealthCare Patients. Retrieved June 08, 2018 from http://tinyurl.com/y3v8on9f.Google ScholarGoogle Scholar
  177. Statista. 2018. Internet of Things (IoT) Connected Devices Installed Base Worldwide from 2015 to 2025 (in Billions). Retrieved June 18, 2018 from http://tinyurl.com/j3t9t2w.Google ScholarGoogle Scholar
  178. Aaron Steinfeld, Terrence Fong, David Kaber, Michael Lewis, Jean Scholtz, Alan Schultz, and Michael Goodrich. 2006. Common metrics for human-robot interaction. In Proceedings of the 1st ACM SIGCHI/SIGART Conference on Human-Robot Interaction. ACM, 33--40. Google ScholarGoogle ScholarDigital LibraryDigital Library
  179. Jie Su, Jianping He, Peng Cheng, and Jiming Chen. 2016. A stealthy GPS spoofing strategy for manipulating the trajectory of an unmanned aerial vehicle. IFAC-PapersOnLine 49, 22 (2016), 291--296.Google ScholarGoogle ScholarCross RefCross Ref
  180. Irshad Ahmed Sumra, Iftikhar Ahmad, Halabi Hasbullah, and Jamalul-lail bin Ab Manan. 2011. Classes of attacks in VANET. In Proceedings of the Saudi International Electronics, Communications and Photonics Conference (SIECPC’11). IEEE, Riyadh, Saudi Arabia.Google ScholarGoogle ScholarCross RefCross Ref
  181. Zhong-yang Tan, Ying, and Zheng. 2013. Research advance in swarm robotics. Defence Technol. 9, 1 (2013), 18--39. https://www.sciencedirect.com/science/article/pii/S221491471300024X.Google ScholarGoogle ScholarCross RefCross Ref
  182. Vrizlynn L. L. Thing and Jiaxi Wu. 2016. Autonomous vehicle security: A taxonomy of attacks and defences. In Proceedings of the 2016 IEEE International Conference on Internet of Things (iThings) and IEEE Green Computing and Communications (GreenCom) and IEEE Cyber, Physical and Social Computing (CPSCom) and IEEE Smart Data (SmartData). IEEE, 164--170.Google ScholarGoogle Scholar
  183. Jon Turi. 2014a. GE’s Bringing Good Things, and Massive Robots, to Life. Retrieved March 03, 2018 from https://www.engadget.com/2014/01/26/ge-man-amplifying-robots/.Google ScholarGoogle Scholar
  184. Jon Turi. 2014b. Tesla’s Toy Boat: A Drone before Its Time. Retrieved March 03, 2018 from https://www.engadget.com/2014/01/19/nikola-teslas-remote-control-boat/.Google ScholarGoogle Scholar
  185. Chris Urmson, Joshua Anhalt, Drew Bagnell, Christopher Baker, Robert Bittner, John Dolan, Dave Duggins, Dave Ferguson, Tugrul Galatali, Chris Geyer, Michele Gittleman, Sam Harbaugh, Martial Hebert, Tom Howard, Alonzo Kelly, David Kohanbash, Maxim Likhachev, Nick Miller, Kevin Peterson, Raj Rajkumar, Paul Rybski, Bryan Salesky, Sebastian Scherer, Young Woo-seo, Reid Simmons, Sanjiv Singh, Jarrod Snider, Anthony Stentz, William Red Whittaker, Jason Ziglar, Joshua Struble, and Michael Taylor. 2007. Tartan racing: A multi-modal approach to the DARPA urban challenge. Defense 94, 4 (2007), 386--387.Google ScholarGoogle Scholar
  186. George Vachtsevanos and Johan Reimann. 2004. An intelligent approach to coordinated control of multiple unmanned aerial vehicles. In Proceedings of the American Helicopter Society 60th Annual Forum, Baltimore, MD.Google ScholarGoogle Scholar
  187. Rick van der Kleij, Tom Hueting, and Jan Maarten Schraagen. 2018. Change detection support for supervisory controllers of highly automated systems: Effects on performance, mental workload, and recovery of situation awareness following interruptions. Int. J. Ind. Ergon. 66 (2018), 75--84.Google ScholarGoogle ScholarCross RefCross Ref
  188. Gabriel Vasconcelos, Gabriel Carrijo, Rodrigo Miani, Jefferson Souza, and Vitor Guizilini. 2016. The impact of DoS attacks on the AR. Drone 2.0. In Proceedings of the 2016 13th Latin American Robotics Symposium and 4th Brazilian Robotics Symposium (LARS/SBR). IEEE, 127--132.Google ScholarGoogle ScholarCross RefCross Ref
  189. Chad Vander Veen. 2015. 10 Years, 10 Milestones for Driverless Cars. Retrieved February 26, 2019 from https://tinyurl.com/y4wb3qwt.Google ScholarGoogle Scholar
  190. Pandi Vijayakumar, Maria Azees, and Arputharaj Kannan. 2015. Dual authentication and key management techniques for secure data transmission in vehicular ad hoc networks. IEEE Trans. Intell. Transp. Syst. 17, 4 (2015), 1015--1028.Google ScholarGoogle ScholarDigital LibraryDigital Library
  191. Daisuke Wakabayashi. 2018. Uber’s Self-Driving Cars Were Struggling Before Arizona Crash. Retrieved September 13, 2018 from http://tinyurl.com/y83sc39o.Google ScholarGoogle Scholar
  192. Alicja Wakulicz-Deja and Małgorzata Przybyła-Kasperek. 2007. Hierarchical multi-agent system. Stud. Inf. 28, 4 (2007), 63--80.Google ScholarGoogle Scholar
  193. Richard Waters and Tim Bradshaw. 2016. Rise of the Robots Is Sparking an Investment Boom. Retrieved May 26, 2018 from http://tinyurl.com/yymnbxrt.Google ScholarGoogle Scholar
  194. Ben George Weber, Michael Mateas, and Arnav Jhala. 2012. Learning from demonstration for goal-driven autonomy. In AAAI.Google ScholarGoogle Scholar
  195. David Welch and Keith Naughton. 2019. GM Falls Millions of Miles Short on Cruise Driving Projection. Retrieved February 26, 2019 from http://tinyurl.com/y5w4zpm7.Google ScholarGoogle Scholar
  196. Darrell M. West. 2017. Securing the Future of Driverless Cars. Retrieved February 26, 2019 from https://www.brookings.edu/research/securing-the-future-of-driverless-cars/.Google ScholarGoogle Scholar
  197. Earl L. Wiener and Renwick E. Curry. 1980. Flight-deck automation: Promises and problems. Ergon. 23, 10 (1980), 995--1011.Google ScholarGoogle ScholarCross RefCross Ref
  198. Norbert Wiener and Others. 1964. God and Golem, Inc.Google ScholarGoogle Scholar
  199. Joe Williams. 2018. 2019 May Be Year of the Driverless Car: Here’s Where Top Automakers Stand. Retrieved February 26, 2019 from http://tinyurl.com/yxgjhhrh.Google ScholarGoogle Scholar
  200. Sadie Williamson. 2018. Blockchain May Be the Answer to Making Self Driving Cars Safer. Retrieved October 1, 2018 from http://tinyurl.com/y3xv467u.Google ScholarGoogle Scholar
  201. Mark A. Wilson, James McMahon, Artur Wolek, David W. Aha, and Brian H. Houston. 2016. Toward goal reasoning for autonomous underwater vehicles: Responding to unexpected agents. In Proceedings of Goal Reasoning: Papers from the IJCAI Workshop.Google ScholarGoogle Scholar
  202. Wenguo Liu Winfield and Alan F. T. 2010. Modeling and optimization of adaptive foraging in swarm robotic systems. Int. J. Rob. Res. 29, 14 (2010), 1743--1760. Google ScholarGoogle ScholarDigital LibraryDigital Library
  203. Scott Wold and Paste Staff. 2015. The 100 Greatest Movie Robots of All Time. Retrieved March 04, 2018 from http://tinyurl.com/y3zbsxyk.Google ScholarGoogle Scholar
  204. Kyle Hollins Wray, Luis Pineda, and Sholo Zilberstein. 2016. Hierarchical approach to transfer control in semi-autonomous systems. In Proceedings of the 25th International Joint Conference on Artificial Intelligence. 517--523. http://www.ijcai.org/Proceedings/16/Papers/080.pdf. Google ScholarGoogle ScholarDigital LibraryDigital Library
  205. Guangyu Wu, Jian Sun, and Jie Chen. 2016. A survey on the security of cyber-physical systems. Control Theory Technol. 14, 1 (2016), 2--10.Google ScholarGoogle ScholarCross RefCross Ref
  206. Eray Yağdereli, Cemal Gemci, and A. Ziya Aktaş. 2015. A study on cyber-security of autonomous and unmanned vehicles. J. Defense Model. Simul. 12, 4 (2015), 369--381.Google ScholarGoogle ScholarCross RefCross Ref
  207. Andrew Ydenberg, Navtej Heir, and Bob Gill. 2018. Security, SDN, and VANET technology of driver-less cars. In Proceedings of the 2018 IEEE 8th Annual Computing and Communication Workshop and Conference (CCWC). IEEE, 313--316.Google ScholarGoogle ScholarCross RefCross Ref
  208. W. B. Yeats. 1933. The winding stairs and other poems. Reprinted by Kessinger Publishing.Google ScholarGoogle Scholar
  209. Siew Yong, Dale Lindskog, Ron Ruhl, and Pavol Zavarsky. 2011. Risk mitigation strategies for mobile Wi-Fi robot toys from online pedophiles. In 2011 IEEE 3rd International Conference on Privacy, Security, Risk and Trust (PASSAT) and 2011 IEEE 3rd International Conference on Social Computing (SocialCom). IEEE, 1220--1223.Google ScholarGoogle ScholarCross RefCross Ref
  210. ZDNet. 2018. 15 of the Best Movies About AI, Ranked. Retrieved March 03, 2018 from https://tinyurl.com/yculsjxp.Google ScholarGoogle Scholar
  211. Kim Zetter. 2014. Hospital Networks Are Leaking Data, Leaving Critical Devices Vulnerable. WIRED. Retrieved from http://tinyurl.com/y5kxsfol.Google ScholarGoogle Scholar
  212. Ning Zhang, Shan Zhang, Peng Yang, Omar Alhussein, Weihua Zhuang, and Xuemin Sherman Shen. 2017. Software defined space-air-ground integrated vehicular networks: Challenges and solutions. IEEE Commun. Mag. 55, 7 (2017), 101--109.Google ScholarGoogle ScholarDigital LibraryDigital Library
  213. Shlomo Zilberstein. 2010. Building strong semi-autonomous systems. Chaffins 2008 (2010), 1--20. Google ScholarGoogle ScholarDigital LibraryDigital Library

Index Terms

  1. Security Modeling of Autonomous Systems: A Survey

              Recommendations

              Comments

              Login options

              Check if you have access through your login credentials or your institution to get full access on this article.

              Sign in

              Full Access

              • Published in

                cover image ACM Computing Surveys
                ACM Computing Surveys  Volume 52, Issue 5
                September 2020
                791 pages
                ISSN:0360-0300
                EISSN:1557-7341
                DOI:10.1145/3362097
                • Editor:
                • Sartaj Sahni
                Issue’s Table of Contents

                Copyright © 2019 ACM

                Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

                Publisher

                Association for Computing Machinery

                New York, NY, United States

                Publication History

                • Published: 13 September 2019
                • Accepted: 1 May 2019
                • Revised: 1 April 2019
                • Received: 1 November 2018
                Published in csur Volume 52, Issue 5

                Permissions

                Request permissions about this article.

                Request Permissions

                Check for updates

                Qualifiers

                • survey
                • Research
                • Refereed

              PDF Format

              View or Download as a PDF file.

              PDF

              eReader

              View online with eReader.

              eReader

              HTML Format

              View this article in HTML Format .

              View HTML Format