2013 LearningtoParseNaturalLanguageC
- (Matuszek et al., 2013) ⇒ Cynthia Matuszek, Evan Herbst, Luke Zettlemoyer, and Dieter Fox. (2013). “Learning to Parse Natural Language Commands to a Robot Control System.” In: Experimental Robotics. doi:10.1007/978-3-319-00065-7_28
Subject Headings: Natural-Language Interaction, Robot Control, Action Sequence, Person's Intent.
Notes
- This paper was first published in an ISER conference (ISER-2012).
Cited By
Quotes
Abstract
As robots become more ubiquitous and capable of performing complex tasks, the importance of enabling untrained users to interact with them has increased. In response, unconstrained natural-language interaction with robots has emerged as a significant research area. We discuss the problem of parsing natural language commands to actions and control structures that can be readily implemented in a robot execution system. Our approach learns a parser based on example pairs of English commands and corresponding control language expressions. We evaluate this approach in the context of following route instructions through an indoor environment, and demonstrate that our system can learn to translate English commands into sequences of desired actions, while correctly capturing the semantic intent of statements involving complex control structures. The procedural nature of our formal representation allows a robot to interpret route instructions online while moving through a previously unknown environment.
1 Motivation and Problem Statement
In this paper, we discuss our work on grounding natural language–interpreting hu- man language into semantically informed structures in the context of robotic per- ception and actuation. To this end, we explore the question of interpreting natural language commands so they can be executed by a robot, specifically in the context of following route instructions through a map.
Natural language (NL) is a rich, intuitive mechanism by which humans can inter- act with systems around them, offering sufficient signal to support robot task plan- ning. Human route instructions include complex language constructs, which robots must be able to execute without being given a fully specified world model such as a map. Our goal is to investigate whether it is possible to learn a parser that produces correct, robot-executable commands for such instructions. We treat grounding as a problem of parsing from a natural language to a formal control language capable of representing a robot’s operation in an environment. Specifically, we train a se- mantic parsing model that defines, for any natural language sentence, a distribution over possible robot control sequences in a LISP-like control language called Robot Control Language, or RCL.
- Fig. 1: The task: Going from NL to robot control. First, the natural language command is parsed into a formal, procedural description representing the intent of the person. The robot control commands are then used by the executor, along with the local state of the world, to control the robot, thereby grounding the NL commands into actions while exploring the environment.
The key contributions of this work are to learn grounding relations from data (rather than predefining a mapping of natural language to actions), and to execute them against a previously unseen world model (in this case, a map of an environ- ment), as illustrated in Fig. 1. Training is performed on English commands anno- tated with the corresponding robot commands. This parser can then be used to trans- form new route instructions to execution system inputs in an unfamiliar map. The resulting system can represent control structures and higher-order concepts. We test our approach using a simulator executing the commands produced.
The remainder of this paper is organized as follows. In the next section, we dis- cuss related work in human-robot interaction, natural language understanding, and robot navigation and instruction-following. Sec. 3 describes the technical approach, the formal execution language we define for this work, and our parser learning sys- tem. Sec. 4 and Sec. 5 describe the experimental evaluation performed and the re- sults obtained, and we close with a discussion of insights gained from this work.
2 Related Work
Robot navigation is a critical and widely-studied task in mobile robotics, and fol- lowing natural-language instructions is a key component of natural, multi-modal human robot interaction. Previous efforts have treated the language grounding task as a problem of parsing commands into formal meaning representations. Several efforts [14, 23] parse natural route instructions to sequences of atomic actions that must be grounded into fully specified world models. Other systems learn to parse navigation instructions, but limit their formal language to a set of predefined parses [25].
Our work falls also into the broader class of grounded language acquisition [24], in which language is learned from situated context, usually by learning over a cor- pus of parallel language and context data. Other work shows how parsed natural language can be grounded in a robot’s world and action models, taking perceptual and grounding uncertainty into account, thereby enabling instruction for robot nav- igation, GUI interaction, and forklift operation [30, 4, 28].
Parsing natural language to expressive formal representations such as λ -calculus has been demonstrated [31, 1, 18]. λ -calculus is able to represent complex robot control systems [8]; however, to the best of our knowledge, such parser learning approaches have not yet been applied in the context of robotics. Logic-based control systems have been used successfully in robotics [11, 3, 5, 9, 16], providing a powerful framework that can be readily mapped to robot actions, and combina- tory categorial grammars have been used for semantic mapping [21]; in contrast to our framework, however, most approaches rely on a manually constructed parser to map from NL commands to λ -calculus, rather than learning grounding relations from data.
Our work is most similar to that of Chen & Mooney [7, 6], who perform parser learning over a body of route instructions through a complex indoor environment containing objects and landmarks with no prior linguistic knowledge. However, their work assumes initial knowledge of a map, and does not represent complex control structures. Compared to our previous approach to parser learning for route instruction following, the system presented here can represent control structures such as ‘while,’ higher-order concepts such as ‘nth,’ and set operations, and is able to follow such directions through unknown maps.
3 Technical Approach
…
References
;
| Author | volume | Date Value | title | type | journal | titleUrl | doi | note | year | |
|---|---|---|---|---|---|---|---|---|---|---|
| 2013 LearningtoParseNaturalLanguageC | Dieter Fox Luke Zettlemoyer Cynthia Matuszek Evan Herbst | Learning to Parse Natural Language Commands to a Robot Control System | 10.1007/978-3-319-00065-7_28 | 2013 |