Top

Autonomous Robots

Published in:

01-01-2015

Provably correct reactive control from natural language

Authors: Constantine Lignos, Vasumathi Raman, Cameron Finucane, Mitchell Marcus, Hadas Kress-Gazit

Published in: Autonomous Robots | Issue 1/2015

Activate our intelligent search to find suitable subject content or patents.

search-config

AI-assisted search

Off

Abstract

This paper presents an integrated system for generating, troubleshooting, and executing correct-by-construction controllers for autonomous robots using natural language input, allowing non-expert users to command robots to perform high-level tasks. This system unites the power of formal methods with the accessibility of natural language, providing controllers for implementable high-level task specifications, easy-to-understand feedback on those that cannot be achieved, and natural language explanation of the reason for the robot’s actions during execution. The natural language system uses domain-general components that can easily be adapted to cover the vocabulary of new applications. Generation of a linear temporal logic specification from the user’s natural language input uses a novel data structure that allows for subsequent mapping of logical propositions back to natural language, enabling natural language feedback about problems with the specification that are only identifiable in the logical form. We demonstrate the robustness of the natural language understanding system through a user study where participants interacted with a simulated robot in a search and rescue scenario. Automated analysis and user feedback on unimplementable specifications is demonstrated using an example involving a robot assistant in a hospital.

previous article Grasp quality measures: review and performance

Dont have a licence yet? Then find out more about our products and how to get one now:

Springer Professional "Wirtschaft+Technik"

Online-Abonnement

Mit Springer Professional "Wirtschaft+Technik" erhalten Sie Zugriff auf:

über 102.000 Bücher
über 537 Zeitschriften

aus folgenden Fachgebieten:

Automobil + Motoren
Bauwesen + Immobilien
Business IT + Informatik
Elektrotechnik + Elektronik
Energie + Nachhaltigkeit
Finance + Banking
Management + Führung
Marketing + Vertrieb
Maschinenbau + Werkstoffe
Versicherung + Risiko

Jetzt Wissensvorsprung sichern!

inform now

Springer Professional "Technik"

Online-Abonnement

Mit Springer Professional "Technik" erhalten Sie Zugriff auf:

über 67.000 Bücher
über 390 Zeitschriften

aus folgenden Fachgebieten:

Automobil + Motoren
Bauwesen + Immobilien
Business IT + Informatik
Elektrotechnik + Elektronik
Energie + Nachhaltigkeit
Maschinenbau + Werkstoffe

Jetzt Wissensvorsprung sichern!

inform now

This arguably unintuitive translation is due to specifications in LTLMoP being restricted to the GR(1) fragment of LTL.

Even though it was the role of the operator, not the robot, to rescue hostages, we label these examples as tagging errors because a command was given to the system and it was not properly understood. The desired response in this situation is to understand the requested action but report that the robot cannot perform it.

Berant, J., & Liang, P. (2014). Semantic parsing via paraphrasing. In Proceedings of the 52nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) (pp. 1415–1425).

Bhatia, A., Kavraki, L. E., & Vardi, M.Y. (2010). Sampling-based motion planning with temporal goals. In IEEE International Conference on Robotics and Automation (ICRA), IEEE (pp. 2689–2696).

Biere, A. (2008). PicoSAT essentials. Journal on Satisfiability Boolean Modeling and Computation (JSAT), 4, 75–97.MATH

Bikel, D. M. (2004). Intricacies of Collins’ parsing model. Computational Linguistics, 30(4), 479–511.CrossRefMATH

Bobadilla, L., Sanchez, O., Czarnowski, J., Gossman, K., & LaValle, S. (2011). Controlling wild bodies using linear temporal logic. In Robotics: Science and Systems (RSS).

Brooks, D., Lignos, C., Finucane, C., Medvedev, M., Perera, I., Raman, V., Kress-Gazit, H., Marcus, M., & Yanco, H. (2012). Make it so: Continuous, flexible natural language interaction with an autonomous robot. In Proceedings of the Grounding Language for Physical Systems Workshop at the 76th AAAI Conference on Artificial Intelligence.

Chen, D. L., & Mooney, R.J. (2011). Learning to interpret natural language navigation instructions from observations. In Proceedings of the 25th AAAI Conference on Artifical Intelligence (pp. 859–865).

Cizelj, I., & Belta, C. (2013). Negotiating the probabilistic satisfaction of temporal logic motion specifications. In IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) (pp. 4320–4325).

Clarke, E. M., Grumberg, O., & Peled, D. A. (1999). Model checking. Cambridge, MA: MIT Press.

Dzifcak, J., Scheutz, M., Baral, C., & Schermerhorn, P. (2009). What to do and how to do it: Translating natural language directives into temporal and dynamic logic representation for goal management and action execution. In IEEE International Conference on Robotics and Automation (ICRA) (pp. 4163–4168).

Fainekos, G. E. (2011). Revising temporal logic specifications for motion planning. In IEEE International Conference on Robotics and Automation (ICRA) (pp. 40–45).

Finucane, C., Jing, G., & Kress-Gazit, H. (2010). LTLMoP: Experimenting with language, temporal logic and robot control. In IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) (pp. 1988–1993).

Gabbard, R., Marcus, M., & Kulick, S. (2006). Fully parsing the Penn Treebank. In Human Language Technology Conference of the North American Chapter of the Association of Computational Linguistics (NAACL HLT) (pp. 184–191).

Karaman, Frazzoli. (2009). Sampling-based motion planning with deterministic \(\mu \)-calculus specifications. In IEEE Conference on Decision and Control (CDC) (pp. 2222–2229).

Kim, K., Fainekos, G. E., & Sankaranarayanan, S. (2012). On the revision problem of specification automata. In IEEE International Conference on Robotics and Automation (ICRA) (pp. 5171–5176).

Kloetzer, M., & Belta, C. (2008). A fully automated framework for control of linear systems from temporal logic specifications. IEEE Transactions on Automatic Control, 53(1), 287–297.CrossRefMathSciNet

Kress-Gazit, H., Fainekos, G. E., & Pappas, G. J. (2008). Translating structured english to robot controllers. Advanced Robotics, 22(12), 1343–1359.CrossRef

Kress-Gazit, H., Fainekos, G. E., & Pappas, G. J. (2009). Temporal-logic-based reactive mission and motion planning. IEEE Transactions on Robotics, 25(6), 1370–1381.CrossRef

Matuszek, C., Fox, D., & Koscher, K. (2010). Following directions using statistical machine translation. In Human-Robot Interaction (HRI) (pp. 251–258).

Matuszek, C., FitzGerald, N., Zettlemoyer, L., Bo, L., & Fox, D. (2012). A joint model of language and perception for grounded attribute learning. In Proceedings of the 29th International Conference on Machine Learning (ICML) (pp. 1671–1678).

Matuszek, C., Herbst, E., Zettlemoyer, L., & Fox, D. (2013). Learning to parse natural language commands to a robot control system. Experimental Robotics, 88, 403–415.CrossRef

Piterman, N., Pnueli, A., & Sa’ar, Y. (2006). Synthesis of reactive(1) designs. In Verification, Model Checking, and Abstract Interpretation (VMCAI) (pp. 364–380).

Poon, H., & Domingos, P. (2009). Unsupervised semantic parsing. In Proceedings of the 2009 Conference on Empirical Methods in Natural Language Processing (EMNLP) (pp. 1–10).

Raman, V., & Kress-Gazit, H. (2011). Analyzing unsynthesizable specifications for high-level robot behavior using LTLMoP. In Computer Aided Verification (CAV) (pp. 663–668).

Raman, V., & Kress-Gazit, H. (2013a). Explaining impossible high-level robot behaviors. IEEE Transactions on Robotics, 29, 94–104.CrossRef

Raman, V., & Kress-Gazit, H. (2013b). Towards minimal explanations of unsynthesizability for high-level robot behaviors. In IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) (pp. 757–762).

Raman, V., & Kress-Gazit, H. (2014). Unsynthesizable cores: Minimal explanations for unsynthesizable high-level robot behaviors. arXiv:1409.1455.

Raman, V., Lignos, C., Finucane, C., Lee, KCT., Marcus, M., & Kress-Gazit, H. (2013). Sorry Dave, I’m afraid I can’t do that: Explaining unachievable robot tasks using natural language. In Robotics: Science and Systems (RSS).

Schuler, K. (2005). Verbnet: A broad-coverage, comprehensive verb lexicon. PhD thesis, University of Pennsylvania.

Tellex, S., Kollar, T., Dickerson, S., Walter, M. R., Banerjee, A. G., Teller, S. J., & Roy, N. (2011). Understanding natural language commands for robotic navigation and mobile manipulation. In Proceedings of the 25th AAAI Conference on Artifical Intelligence (pp. 1507–1514).

Toutanova, K., Klein, D., Manning, C. D., & Singer, Y. (2003). Feature-rich part-of-speech tagging with a cyclic dependency network. In Proceedings of the 2003 Conference of the North American Chapter of the Association for Computational Linguistics on Human Language Technology (NAACL HLT) - Volume 1 (pp. 173–180).

Wongpiromsarn, T., Topcu, U., & Murray, R. M. (2010). Receding horizon control for temporal logic specifications. In Hybrid Systems: Computation and Control (HSCC) (pp. 101–110).

Title: Provably correct reactive control from natural language
Authors: Constantine Lignos
Vasumathi Raman
Cameron Finucane
Mitchell Marcus
Hadas Kress-Gazit
Publication date: 01-01-2015
Publisher: Springer US
Published in: Autonomous Robots / Issue 1/2015
Print ISSN: 0929-5593
Electronic ISSN: 1573-7527
DOI: https://doi.org/10.1007/s10514-014-9418-8

Springer Professional

Abstract

Please log in to get access to your license.

Dont have a licence yet? Then find out more about our products and how to get one now:

Springer Professional "Wirtschaft+Technik"

Springer Professional "Technik"

Other articles of this Issue 1/2015

Representing the robot’s workspace through constrained manipulability analysis

Comparing search algorithms for robotic underground chemical source location

Decentralized dynamic task planning for heterogeneous robotic networks

Semi-task-dependent and uncertainty-driven world model maintenance

Grasp quality measures: review and performance