Skip to main content
Erschienen in: Autonomous Robots 1/2015

01.01.2015

Provably correct reactive control from natural language

verfasst von: Constantine Lignos, Vasumathi Raman, Cameron Finucane, Mitchell Marcus, Hadas Kress-Gazit

Erschienen in: Autonomous Robots | Ausgabe 1/2015

Einloggen

Aktivieren Sie unsere intelligente Suche, um passende Fachinhalte oder Patente zu finden.

search-config
loading …

Abstract

This paper presents an integrated system for generating, troubleshooting, and executing correct-by-construction controllers for autonomous robots using natural language input, allowing non-expert users to command robots to perform high-level tasks. This system unites the power of formal methods with the accessibility of natural language, providing controllers for implementable high-level task specifications, easy-to-understand feedback on those that cannot be achieved, and natural language explanation of the reason for the robot’s actions during execution. The natural language system uses domain-general components that can easily be adapted to cover the vocabulary of new applications. Generation of a linear temporal logic specification from the user’s natural language input uses a novel data structure that allows for subsequent mapping of logical propositions back to natural language, enabling natural language feedback about problems with the specification that are only identifiable in the logical form. We demonstrate the robustness of the natural language understanding system through a user study where participants interacted with a simulated robot in a search and rescue scenario. Automated analysis and user feedback on unimplementable specifications is demonstrated using an example involving a robot assistant in a hospital.

Sie haben noch keine Lizenz? Dann Informieren Sie sich jetzt über unsere Produkte:

Springer Professional "Wirtschaft+Technik"

Online-Abonnement

Mit Springer Professional "Wirtschaft+Technik" erhalten Sie Zugriff auf:

  • über 102.000 Bücher
  • über 537 Zeitschriften

aus folgenden Fachgebieten:

  • Automobil + Motoren
  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Elektrotechnik + Elektronik
  • Energie + Nachhaltigkeit
  • Finance + Banking
  • Management + Führung
  • Marketing + Vertrieb
  • Maschinenbau + Werkstoffe
  • Versicherung + Risiko

Jetzt Wissensvorsprung sichern!

Springer Professional "Technik"

Online-Abonnement

Mit Springer Professional "Technik" erhalten Sie Zugriff auf:

  • über 67.000 Bücher
  • über 390 Zeitschriften

aus folgenden Fachgebieten:

  • Automobil + Motoren
  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Elektrotechnik + Elektronik
  • Energie + Nachhaltigkeit
  • Maschinenbau + Werkstoffe




 

Jetzt Wissensvorsprung sichern!

Fußnoten
1
This arguably unintuitive translation is due to specifications in LTLMoP being restricted to the GR(1) fragment of LTL.
 
2
Even though it was the role of the operator, not the robot, to rescue hostages, we label these examples as tagging errors because a command was given to the system and it was not properly understood. The desired response in this situation is to understand the requested action but report that the robot cannot perform it.
 
Literatur
Zurück zum Zitat Berant, J., & Liang, P. (2014). Semantic parsing via paraphrasing. In Proceedings of the 52nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) (pp. 1415–1425). Berant, J., & Liang, P. (2014). Semantic parsing via paraphrasing. In Proceedings of the 52nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) (pp. 1415–1425).
Zurück zum Zitat Bhatia, A., Kavraki, L. E., & Vardi, M.Y. (2010). Sampling-based motion planning with temporal goals. In IEEE International Conference on Robotics and Automation (ICRA), IEEE (pp. 2689–2696). Bhatia, A., Kavraki, L. E., & Vardi, M.Y. (2010). Sampling-based motion planning with temporal goals. In IEEE International Conference on Robotics and Automation (ICRA), IEEE (pp. 2689–2696).
Zurück zum Zitat Biere, A. (2008). PicoSAT essentials. Journal on Satisfiability Boolean Modeling and Computation (JSAT), 4, 75–97.MATH Biere, A. (2008). PicoSAT essentials. Journal on Satisfiability Boolean Modeling and Computation (JSAT), 4, 75–97.MATH
Zurück zum Zitat Bikel, D. M. (2004). Intricacies of Collins’ parsing model. Computational Linguistics, 30(4), 479–511.CrossRefMATH Bikel, D. M. (2004). Intricacies of Collins’ parsing model. Computational Linguistics, 30(4), 479–511.CrossRefMATH
Zurück zum Zitat Bobadilla, L., Sanchez, O., Czarnowski, J., Gossman, K., & LaValle, S. (2011). Controlling wild bodies using linear temporal logic. In Robotics: Science and Systems (RSS). Bobadilla, L., Sanchez, O., Czarnowski, J., Gossman, K., & LaValle, S. (2011). Controlling wild bodies using linear temporal logic. In Robotics: Science and Systems (RSS).
Zurück zum Zitat Brooks, D., Lignos, C., Finucane, C., Medvedev, M., Perera, I., Raman, V., Kress-Gazit, H., Marcus, M., & Yanco, H. (2012). Make it so: Continuous, flexible natural language interaction with an autonomous robot. In Proceedings of the Grounding Language for Physical Systems Workshop at the 76th AAAI Conference on Artificial Intelligence. Brooks, D., Lignos, C., Finucane, C., Medvedev, M., Perera, I., Raman, V., Kress-Gazit, H., Marcus, M., & Yanco, H. (2012). Make it so: Continuous, flexible natural language interaction with an autonomous robot. In Proceedings of the Grounding Language for Physical Systems Workshop at the 76th AAAI Conference on Artificial Intelligence.
Zurück zum Zitat Chen, D. L., & Mooney, R.J. (2011). Learning to interpret natural language navigation instructions from observations. In Proceedings of the 25th AAAI Conference on Artifical Intelligence (pp. 859–865). Chen, D. L., & Mooney, R.J. (2011). Learning to interpret natural language navigation instructions from observations. In Proceedings of the 25th AAAI Conference on Artifical Intelligence (pp. 859–865).
Zurück zum Zitat Cizelj, I., & Belta, C. (2013). Negotiating the probabilistic satisfaction of temporal logic motion specifications. In IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) (pp. 4320–4325). Cizelj, I., & Belta, C. (2013). Negotiating the probabilistic satisfaction of temporal logic motion specifications. In IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) (pp. 4320–4325).
Zurück zum Zitat Clarke, E. M., Grumberg, O., & Peled, D. A. (1999). Model checking. Cambridge, MA: MIT Press. Clarke, E. M., Grumberg, O., & Peled, D. A. (1999). Model checking. Cambridge, MA: MIT Press.
Zurück zum Zitat Dzifcak, J., Scheutz, M., Baral, C., & Schermerhorn, P. (2009). What to do and how to do it: Translating natural language directives into temporal and dynamic logic representation for goal management and action execution. In IEEE International Conference on Robotics and Automation (ICRA) (pp. 4163–4168). Dzifcak, J., Scheutz, M., Baral, C., & Schermerhorn, P. (2009). What to do and how to do it: Translating natural language directives into temporal and dynamic logic representation for goal management and action execution. In IEEE International Conference on Robotics and Automation (ICRA) (pp. 4163–4168).
Zurück zum Zitat Fainekos, G. E. (2011). Revising temporal logic specifications for motion planning. In IEEE International Conference on Robotics and Automation (ICRA) (pp. 40–45). Fainekos, G. E. (2011). Revising temporal logic specifications for motion planning. In IEEE International Conference on Robotics and Automation (ICRA) (pp. 40–45).
Zurück zum Zitat Finucane, C., Jing, G., & Kress-Gazit, H. (2010). LTLMoP: Experimenting with language, temporal logic and robot control. In IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) (pp. 1988–1993). Finucane, C., Jing, G., & Kress-Gazit, H. (2010). LTLMoP: Experimenting with language, temporal logic and robot control. In IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) (pp. 1988–1993).
Zurück zum Zitat Gabbard, R., Marcus, M., & Kulick, S. (2006). Fully parsing the Penn Treebank. In Human Language Technology Conference of the North American Chapter of the Association of Computational Linguistics (NAACL HLT) (pp. 184–191). Gabbard, R., Marcus, M., & Kulick, S. (2006). Fully parsing the Penn Treebank. In Human Language Technology Conference of the North American Chapter of the Association of Computational Linguistics (NAACL HLT) (pp. 184–191).
Zurück zum Zitat Karaman, Frazzoli. (2009). Sampling-based motion planning with deterministic \(\mu \)-calculus specifications. In IEEE Conference on Decision and Control (CDC) (pp. 2222–2229). Karaman, Frazzoli. (2009). Sampling-based motion planning with deterministic \(\mu \)-calculus specifications. In IEEE Conference on Decision and Control (CDC) (pp. 2222–2229).
Zurück zum Zitat Kim, K., Fainekos, G. E., & Sankaranarayanan, S. (2012). On the revision problem of specification automata. In IEEE International Conference on Robotics and Automation (ICRA) (pp. 5171–5176). Kim, K., Fainekos, G. E., & Sankaranarayanan, S. (2012). On the revision problem of specification automata. In IEEE International Conference on Robotics and Automation (ICRA) (pp. 5171–5176).
Zurück zum Zitat Kloetzer, M., & Belta, C. (2008). A fully automated framework for control of linear systems from temporal logic specifications. IEEE Transactions on Automatic Control, 53(1), 287–297.CrossRefMathSciNet Kloetzer, M., & Belta, C. (2008). A fully automated framework for control of linear systems from temporal logic specifications. IEEE Transactions on Automatic Control, 53(1), 287–297.CrossRefMathSciNet
Zurück zum Zitat Kress-Gazit, H., Fainekos, G. E., & Pappas, G. J. (2008). Translating structured english to robot controllers. Advanced Robotics, 22(12), 1343–1359.CrossRef Kress-Gazit, H., Fainekos, G. E., & Pappas, G. J. (2008). Translating structured english to robot controllers. Advanced Robotics, 22(12), 1343–1359.CrossRef
Zurück zum Zitat Kress-Gazit, H., Fainekos, G. E., & Pappas, G. J. (2009). Temporal-logic-based reactive mission and motion planning. IEEE Transactions on Robotics, 25(6), 1370–1381.CrossRef Kress-Gazit, H., Fainekos, G. E., & Pappas, G. J. (2009). Temporal-logic-based reactive mission and motion planning. IEEE Transactions on Robotics, 25(6), 1370–1381.CrossRef
Zurück zum Zitat Matuszek, C., Fox, D., & Koscher, K. (2010). Following directions using statistical machine translation. In Human-Robot Interaction (HRI) (pp. 251–258). Matuszek, C., Fox, D., & Koscher, K. (2010). Following directions using statistical machine translation. In Human-Robot Interaction (HRI) (pp. 251–258).
Zurück zum Zitat Matuszek, C., FitzGerald, N., Zettlemoyer, L., Bo, L., & Fox, D. (2012). A joint model of language and perception for grounded attribute learning. In Proceedings of the 29th International Conference on Machine Learning (ICML) (pp. 1671–1678). Matuszek, C., FitzGerald, N., Zettlemoyer, L., Bo, L., & Fox, D. (2012). A joint model of language and perception for grounded attribute learning. In Proceedings of the 29th International Conference on Machine Learning (ICML) (pp. 1671–1678).
Zurück zum Zitat Matuszek, C., Herbst, E., Zettlemoyer, L., & Fox, D. (2013). Learning to parse natural language commands to a robot control system. Experimental Robotics, 88, 403–415.CrossRef Matuszek, C., Herbst, E., Zettlemoyer, L., & Fox, D. (2013). Learning to parse natural language commands to a robot control system. Experimental Robotics, 88, 403–415.CrossRef
Zurück zum Zitat Piterman, N., Pnueli, A., & Sa’ar, Y. (2006). Synthesis of reactive(1) designs. In Verification, Model Checking, and Abstract Interpretation (VMCAI) (pp. 364–380). Piterman, N., Pnueli, A., & Sa’ar, Y. (2006). Synthesis of reactive(1) designs. In Verification, Model Checking, and Abstract Interpretation (VMCAI) (pp. 364–380).
Zurück zum Zitat Poon, H., & Domingos, P. (2009). Unsupervised semantic parsing. In Proceedings of the 2009 Conference on Empirical Methods in Natural Language Processing (EMNLP) (pp. 1–10). Poon, H., & Domingos, P. (2009). Unsupervised semantic parsing. In Proceedings of the 2009 Conference on Empirical Methods in Natural Language Processing (EMNLP) (pp. 1–10).
Zurück zum Zitat Raman, V., & Kress-Gazit, H. (2011). Analyzing unsynthesizable specifications for high-level robot behavior using LTLMoP. In Computer Aided Verification (CAV) (pp. 663–668). Raman, V., & Kress-Gazit, H. (2011). Analyzing unsynthesizable specifications for high-level robot behavior using LTLMoP. In Computer Aided Verification (CAV) (pp. 663–668).
Zurück zum Zitat Raman, V., & Kress-Gazit, H. (2013a). Explaining impossible high-level robot behaviors. IEEE Transactions on Robotics, 29, 94–104.CrossRef Raman, V., & Kress-Gazit, H. (2013a). Explaining impossible high-level robot behaviors. IEEE Transactions on Robotics, 29, 94–104.CrossRef
Zurück zum Zitat Raman, V., & Kress-Gazit, H. (2013b). Towards minimal explanations of unsynthesizability for high-level robot behaviors. In IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) (pp. 757–762). Raman, V., & Kress-Gazit, H. (2013b). Towards minimal explanations of unsynthesizability for high-level robot behaviors. In IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) (pp. 757–762).
Zurück zum Zitat Raman, V., & Kress-Gazit, H. (2014). Unsynthesizable cores: Minimal explanations for unsynthesizable high-level robot behaviors. arXiv:1409.1455. Raman, V., & Kress-Gazit, H. (2014). Unsynthesizable cores: Minimal explanations for unsynthesizable high-level robot behaviors. arXiv:​1409.​1455.
Zurück zum Zitat Raman, V., Lignos, C., Finucane, C., Lee, KCT., Marcus, M., & Kress-Gazit, H. (2013). Sorry Dave, I’m afraid I can’t do that: Explaining unachievable robot tasks using natural language. In Robotics: Science and Systems (RSS). Raman, V., Lignos, C., Finucane, C., Lee, KCT., Marcus, M., & Kress-Gazit, H. (2013). Sorry Dave, I’m afraid I can’t do that: Explaining unachievable robot tasks using natural language. In Robotics: Science and Systems (RSS).
Zurück zum Zitat Schuler, K. (2005). Verbnet: A broad-coverage, comprehensive verb lexicon. PhD thesis, University of Pennsylvania. Schuler, K. (2005). Verbnet: A broad-coverage, comprehensive verb lexicon. PhD thesis, University of Pennsylvania.
Zurück zum Zitat Tellex, S., Kollar, T., Dickerson, S., Walter, M. R., Banerjee, A. G., Teller, S. J., & Roy, N. (2011). Understanding natural language commands for robotic navigation and mobile manipulation. In Proceedings of the 25th AAAI Conference on Artifical Intelligence (pp. 1507–1514). Tellex, S., Kollar, T., Dickerson, S., Walter, M. R., Banerjee, A. G., Teller, S. J., & Roy, N. (2011). Understanding natural language commands for robotic navigation and mobile manipulation. In Proceedings of the 25th AAAI Conference on Artifical Intelligence (pp. 1507–1514).
Zurück zum Zitat Toutanova, K., Klein, D., Manning, C. D., & Singer, Y. (2003). Feature-rich part-of-speech tagging with a cyclic dependency network. In Proceedings of the 2003 Conference of the North American Chapter of the Association for Computational Linguistics on Human Language Technology (NAACL HLT) - Volume 1 (pp. 173–180). Toutanova, K., Klein, D., Manning, C. D., & Singer, Y. (2003). Feature-rich part-of-speech tagging with a cyclic dependency network. In Proceedings of the 2003 Conference of the North American Chapter of the Association for Computational Linguistics on Human Language Technology (NAACL HLT) - Volume 1 (pp. 173–180).
Zurück zum Zitat Wongpiromsarn, T., Topcu, U., & Murray, R. M. (2010). Receding horizon control for temporal logic specifications. In Hybrid Systems: Computation and Control (HSCC) (pp. 101–110). Wongpiromsarn, T., Topcu, U., & Murray, R. M. (2010). Receding horizon control for temporal logic specifications. In Hybrid Systems: Computation and Control (HSCC) (pp. 101–110).
Metadaten
Titel
Provably correct reactive control from natural language
verfasst von
Constantine Lignos
Vasumathi Raman
Cameron Finucane
Mitchell Marcus
Hadas Kress-Gazit
Publikationsdatum
01.01.2015
Verlag
Springer US
Erschienen in
Autonomous Robots / Ausgabe 1/2015
Print ISSN: 0929-5593
Elektronische ISSN: 1573-7527
DOI
https://doi.org/10.1007/s10514-014-9418-8

Weitere Artikel der Ausgabe 1/2015

Autonomous Robots 1/2015 Zur Ausgabe

Neuer Inhalt