Designing for Gesture and Tangible Interaction. Mary Lou Maher

any other except for brief quotations in printed reviews, without the prior permission of the publisher.

      Designing for Gesture and Tangible Interaction

      Mary Lou Maher and Lina Lee

       www.morganclaypool.com

      ISBN: 9781627056847 print

      ISBN: 9781627058865 ebook

      DOI 10.2200/S00758ED1V01Y201702HCI036

      A Publication in the Morgan & Claypool Publishers series

       SYNTHESIS LECTURES ON HUMAN-CENTERED INFORMATICS, #36

      Series Editors: John M. Carroll, Penn State University

      Series ISSN: 1946-7680 Print 1946-7699 Electronic

       Designing for Gesture and Tangible Interaction

       Mary Lou Maher and Lina Lee

      The University of North Carolina at Charlotte

       SYNTHESIS LECTURES ON HUMAN-CENTERED INFORMATICS #36

       ABSTRACT

      Interactive technology is increasingly integrated with physical objects that do not have a traditional keyboard and mouse style of interaction, and many do not even have a display. These objects require new approaches to interaction design, referred to as post-WIMP (Windows, Icons, Menus, and Pointer) or as embodied interaction design.

      This book provides an overview of the design opportunities and issues associated with two embodied interaction modalities that allow us to leave the traditional keyboard behind: tangible and gesture interaction. We explore the issues in designing for this new age of interaction by highlighting the significance and contexts for these modalities. We explore the design of tangible interaction with a reconceptualization of the traditional keyboard as a Tangible Keyboard, and the design of interactive three-dimensional (3D) models as Tangible Models. We explore the design of gesture interaction through the design of gesture-base commands for a walk-up-and-use information display, and through the design of a gesture-based dialogue for the willful marionette. We conclude with design principles for tangible and gesture interaction and a call for research on the cognitive effects of these modalities.

       KEYWORDS

      embodied interaction, gesture interaction, tangible interaction, interaction design, cognitive effects

       Contents

       Preface

       Acknowledgments

       1 Introduction

       1.1 Embodiment

       1.2 Affordance

       1.3 Metaphor

       1.4 Epistemic Actions

       1.5 This Book

       2 Tangible Interaction Design

       2.1 What is a Tangible Interaction?

       2.1.1 Tangible Keyboard

       2.1.2 Tangible Models

       2.2 Why is Tangible Interaction Interesting?

       2.2.1 Space-Multiplexed Input and Output

       2.2.2 Concurrency

       2.2.3 Strong Specific Devices

       2.2.4 Spatially Aware Computational Devices

       2.2.5 Spatial Reconfigurability of Devices

       2.3 What Are Key Design Issues for Tangible User Interfaces?

       2.3.1 Designing Interplay of Virtual and Physical

       2.3.2 Selecting from Multiple Gestures and Actions

       2.3.3 Crossing Disciplinary Boundaries

       2.3.4 The Lack of Standard Input and Output Interaction Models

       3 Designing for Tangible Interaction

       3.1 Designing the Tangible Keyboard

       3.1.1 Context of Use

       3.1.2 Design Goals

       3.1.3 Technology Selection

       3.1.4 Design Methods

       3.1.5 Interaction Model

       3.1.6 Design Heuristics

       3.2 Designing Tangible Models

       3.2.1 Context of Use

       3.2.2 Design Goals

       3.2.3 Technology Selection

       3.2.4 Design Methods

       3.2.5 Interaction Model

       3.2.6 Design Guidelines

       4 Gesture-based Interaction

       4.1 What Is a Gesture-based Interaction?

       4.1.1 Gesture Commands: Walk-Up-and-Use Information Display

       4.1.2 Gesture dialogue: The Willfull Marionette

       4.2 Why is Gesture-based Interaction Interesting?

       4.2.1 Public Walk-Up-and-Use Displays or Objects

       4.2.2 Remote Actuators, Sensors, and Displays

       4.2.3 Sociable Multiple-User Interaction

       4.2.4 Gesture Communication

       4.3 What Are Key Design Issues for Gesture-based Interaction?