Robot Learning from Human Teachers. Sonia Chernova

on the unique technical challenges associated with designing robots that learn from naive human teachers. We begin, in the introduction, with a unification of the various terminology seen in the literature as well as an outline of the design choices one has in designing an LfD system. Chapter 2 gives a brief survey of the psychology literature that provides insights from human social learning that are relevant to designing robotic social learners. Chapter 3 walks through an LfD interaction, surveying the design choices one makes and state of the art approaches in prior work. First, is the choice of input, how the human teacher interacts with the robot to provide demonstrations. Next, is the choice of modeling technique. Currently, there is a dichotomy in the field between approaches that model low-level motor skills and those that model high-level tasks composed of primitive actions. We devote a chapter to each of these. Chapter 7 is devoted to interactive and active learning approaches that allow the robot to refine an existing task model. And finally, Chapter 8 provides best practices for evaluation of LfD systems, with a focus on how to approach experiments with human subjects in this domain.

       KEYWORDS

      Learning from Demonstration, imitation learning, Human-robot Interaction

      Contents

       1 Introduction

       1.1 Machine Learning for End-Users

       1.2 The Learning from Demonstration Pipeline

       1.3 A Note on Terminology

       2 Human Social Learning

       2.1 Learning is a Part of All Activity

       2.2 Teachers Scaffold the Learning Process

       2.2.1 Attention Direction

       2.2.2 Dynamic Scaffolding

       2.2.3 Externalizing and Modeling Metacognition

       2.3 Role of Communication in Social Learning

       2.3.1 Expression Provides Feedback to Guide a Teacher

       2.3.2 Asking Questions

       2.4 Implications for the Design of Robot Learners

       3 Modes of Interaction with a Teacher

       3.1 The Correspondence Problem

       3.2 Learning by Doing

       3.3 Learning from Observation

       3.4 Learning from Critique

       3.5 Design Implications

       4 Learning Low-Level Motion Trajectories

       4.1 State Spaces for Motion Learning

       4.2 Modeling an action with Dynamic Movement Primitives

       4.3 Modeling Action with Probabilistic Models

       4.4 Techniques for Handling Suboptimal Demonstrations

       5 Learning High-Level Tasks

       5.1 State Spaces for High-Level Learning

       5.2 Learning a Mapping Function

       5.3 Learning a Task Plan

       5.4 Learning Task Objectives

       5.5 Learning Task Features

       5.6 Learning Frame of Reference

       5.7 Learning Object Affordances

       5.8 Techniques for Handling Suboptimal Demonstrations

       5.9 Discussion and Open Challenges

       6 Refining a Learned Task

       6.1 Batch vs. Incremental Learning

       6.2 Reinforcement Learning Based Methods

       6.3 Corrective Refinement from the Teacher

       6.4 Active Learning

       6.4.1 Label Queries

       6.4.2 Demonstration Queries

       6.4.3 Feature Queries

       6.5 Summary

       7 Designing and Evaluating an LfD Study

       7.1 Experimental Design

       7.2 Evaluating the Algorithmic Performance

       7.3 Evaluating the Interaction

       7.3.1 Subjective Measures

       7.3.2 Objective Measures

       7.4 Experimental Controls

       7.5 Experimental Protocol

       7.6 Data Analysis

       7.6.1 Choosing the Right Statistical Tool

       7.6.2 Drawing Conclusions

       7.7 Additional Resources

       8 Future Challenges and Opportunities

       8.1 Real Users, Real Tasks

       8.2 HRI Considerations

       8.3 Advancing Learning through Benchmarking and Integration

       8.4 Opportunities

       8.5 Additional Resources