Robot Learning from Human Teachers. Sonia Chernova

These studies provide strong evidence that gaze cues from a robot support conversational functions and result in a more natural interaction with a human. As an example of applying this to context of learning, [183] showed how using human-like visual saliency detection may help a robot learner segment a teaching demonstration into steps, and determine the right aspects of the state to pay attention to during the demonstration.

      Another way of directing attention is to emphasize or exaggerate parts of the desired movement. This form of instruction is challenging to adapt to LfD because the goal is not to reproduce the exaggeration itself, but instead to direct the focus of attention during learning.

2.2.2 DYNAMIC SCAFFOLDING

      Dynamic scaffolding is the notion that adults create a learning situation that is the right level of complexity for the learner. The adult adjusts dynamically to make sure the child is working within the Zone of Proximal Development, defined as the gap between what a learner has already mastered and what he or she can achieve with the aid of a teacher. In a way, the teacher creates “microworlds” for the learner to master parts of the task in isolation before moving on, providing safety and intermediate attainable goals [42]. For example, with language parents first treat anything as conversational speech, but eventually they raise their expectations, scaffolding the child’s conversational abilities [257]. In book reading, the parent will at first ask and answer their own questions, and later they will expect the child to participate in the question/answer game.

      Closely related to this idea is Lave and Wenger’s theory of legitimate peripheral participation, which states that the best way to learn is by starting on the sidelines and gradually gaining responsibility. This limits the opportunity for failure while still letting the newcomer play a legitimate part in the community. The level of scaffolding provided is an important factor in learning, instructors that always intervene to prevent problems may actually inhibit learning and the development of abilities to detect and prevent errors [219].

      The idea of scaffolding has been adapted into machine learning, and LfD specifically. Several LfD techniques have leveraged the human teacher in spacial scaffolding, in which the teacher restructures the learning environment to direct or focus the attention of the learner on the most relevant aspects of the task being learned [26, 227, 228]. Within other techniques, scaffolding is used as a means to build complex behaviors by combining or adapting simpler previously taught skills [13, 14, 129].

2.2.3 EXTERNALIZING AND MODELING METACOGNITION

      When working with children, adults often externalize the thinking process [23, 57]. In problem solving, a common simplification is to switch from an open-ended “wh” question (where, who, why, etc.), to yes/no questions when the child is having trouble. For example when asking “do you know where X is?” and the child says “no” or has trouble, the adult will switch to yes/no questions like “is it …?” to frame the search space. Often the yes/no questions are absurd to define the extremes of the space, instead exemplifying the process that the child should be using to come up with the answer for the question.

      Greenfield also observes that if a child turns to an adult during a task, the adult may ask a question or give a gesture hint. The questions asked are meant to elicit the thinking process. Additionally, an important role that the adult plays in a child’s learning process is linking new information to old, showing or suggesting to the child similarities between new problems and old ones [219]. A good teacher makes the information in a new problem compatible with what is known, guiding the generalization process, helping the child apply skills across various contexts.

      Importantly, in humans, the key element that enables the above techniques to be successful is meta-learning. Children can go from being directed in a task through leading questions and hints to internalizing that process and being able to achieve the task on their own. Thus, in robots, it is important to not only follow instructions and model the specific activity, but to learn task strategies (e.g., questions to ask, what to pay attention to, etc.), from these interactions.

2.3 ROLE OF COMMUNICATION IN SOCIAL LEARNING

2.3.1 EXPRESSION PROVIDES FEEDBACK TO GUIDE A TEACHER

      To be a good instructor, one must maintain a mental model of the learner’s state (e.g., what is understood so far, what remains confusing or unknown) in order to appropriately structure the learning task with timely feedback and guidance. The learner helps the instructor by expressing their internal state via communicative acts (e.g., expressions, gestures, or vocalizations that reveal understanding, confusion, attention, etc.). Through reciprocal and tightly coupled interaction, the learner and instructor cooperate to aid both the instructor’s ability to maintain a good mental model of the learner, and the learner’s ability to leverage from instruction to build the appropriate models, representations, and associations.

      With this view of learning as a tightly coupled collaboration, theories of human cooperative and collaborative activity help inform the design of robot learners. Cohen et al. analyzed task dialogs in which an expert instructed a novice assembling a physical device, and found that much of task dialog can be viewed in terms of joint intentions [72]. Their study identified key discourse functions including: organizational markers that synchronize the start of new joint actions (“now,” “next,” etc.), elaborations and clarifications for when the expert believes the apprentice does not understand, and confirmations establishing the mutual belief that a step was accomplished. Another important work is that of Bratman, in which he defines prerequisites for an activity to be considered shared and cooperative, stressing the importance of mutual responsiveness, commitment to the joint activity and commitment to mutual support [34]. Cohen et al. support these guidelines and also predict that an efficient and robust collaboration scheme in a changing environment needs an open channel of communication.

      These theories argue for the importance of sharing information through communication in order to maintain a successful collaborative activity. Thus, a robot learner that people will find collaborative and cooperative, must take into account nonverbal communication, such as gestures and gaze, to facilitate the interaction and maintain an understandable transparent interface between the human and the machine.

2.3.2 ASKING QUESTIONS

      In developmental psychology, the role of curiosity and inquiry is highlighted time and again as a crucial component to the learning process. Early in development this is characterized in self-learning where there is an active process of effectively asking questions of the environment. Piagetian self-regulatory reflexes (e.g., sucking, grasping, circular reactions) are crucial to early learning, helping infants/children obtain developmentally appropriate experiences for learning [207]. The work of Gopnik has additionally shown that children (and adults) are highly efficient in this process. In one study, Gopnik and colleagues demonstrated to children a “blicket machine” that made a sound when certain objects were put near it but not others. When asked to figure out how to make it go, they observed that 2, 3, and 4-year olds would efficiently explore the environment with actions (interventions) to uncover the pattern of conditional dependence between objects and the sound, inferring the causal structure of the machine [97].

      Later, children become experts in actively seeking knowledge from their social environment, first becoming proficient at deciding to whom to pay attention. Movellan showed that children are highly efficient in their behavior, and in the face of deciding whether or not someone or something is reacting contingently to themselves, optimize their actions to gain the most information [178]. Thus, even pre-verbal children that cannot “ask questions” in the traditional sense of the term, are not passive observers but active learners in their world.

      Educational psychology gives another view, looking at questions in a pedagogical context. Grasser and Person studied tutoring sessions in both grade school and college students, classifying a variety of question categories, under two main groups, those requiring short answers vs. long answers. They then studied the frequency and intent of various questions in real tutorial settings. They found the frequency of different types of questions was similar across two different settings, and that students primarily ask questions because of a knowledge deficit and to maintain common ground (e.g., confirming knowledge) [98]. In other research they have shown that the