The Handbook of Multimodal-Multisensor Interfaces, Volume 1. Sharon Oviatt

3

      10.1145/3015783.3015789 Chapter 4

      10.1145/3015783.3015790 Chapter 5

      10.1145/3015783.3015791 Chapter 6

      10.1145/3015783.3015792 Chapter 7

      10.1145/3015783.3015793 Chapter 8

      10.1145/3015783.3015794 Chapter 9

      10.1145/3015783.3015795 Chapter 10

      10.1145/3015783.3015796 Chapter 11

      10.1145/3015783.3015797 Chapter 12

      10.1145/3015783.3015798 Chapter 13

      10.1145/3015783.3015799 Index, Bios, Glossary

      A publication in the ACM Books series, #14

      Editor in Chief: M. Tamer Özsu, University of Waterloo

      First Edition

      10 9 8 7 6 5 4 3 2 1

      This book is dedicated to our families, whose patience and support sustained the year-long effort required to organize, write, and manage different stages of this multi-volume project.

       Contents

       Preface

       Figure Credits

       Introduction: Scope, Trends, and Paradigm Shift in the Field of Computer Interfaces

       Why Multimodal-Multisensor Interfaces Have Become Dominant

       Flexible Multiple-Component Tools as a Catalyst for Performance

       More Expressively Powerful Tools Are Capable of Stimulating Cognition

       One Example of How Multimodal-Multisensor Interfaces Are Changing Today

       Insights in the Chapters Ahead

       Expert Exchange on Multidisciplinary Challenge Topic

       References

       PART I THEORY AND NEUROSCIENCE FOUNDATIONS

       Chapter 1 Theoretical Foundations of Multimodal Interfaces and Systems

       Sharon Oviatt

       1.1 Gestalt Theory: Understanding Multimodal Coherence, Stability, and Robustness

       1.2 Working Memory Theory: Performance Advantages of Distributing Multimodal Processing

       1.3 Activity Theory, Embodied Cognition, and Multisensory-Multimodal Facilitation of Cognition

       Focus Questions

       References

       Chapter 2 The Impact of Multimodal-Multisensory Learning on Human Performance and Brain Activation Patterns

       Karin H. James, Sophia Vinci-Booher, Felipe Munoz-Rubke

       2.1 Introduction

       2.2 The Multimodal-Multisensory Body

       2.3 Facilitatory Effects on Learning as a Function of Active Interactions

       2.4 Behavioral Results in Children

       2.5 Neuroimaging Studies in Adults

       2.6 Neuroimaging Studies in Developing Populations

       2.7 Theoretical Implications–Embodied Cognition

       2.8 Implications for Multimodal-Multisensor Interface Design

       Focus Questions

       References

       PART II APPROACHES TO DESIGN AND USER MODELING

       Chapter 3 Multisensory Haptic Interactions: Understanding the Sense and Designing for It

       Karon E. MacLean, Oliver S. Schneider, Hasti Seifi

       3.1 Introduction

       3.2 Interaction Models for Multimodal Applications

       3.3 Physical Design Space of Haptic Media

       3.4 Making Haptic Media

       3.5 Frontiers for Haptic Design

       Focus Questions

       References