The Handbook of Multimodal-Multisensor Interfaces, Volume 1. Sharon Oviatt

the use of temperature to encode informaion.

      Vibrotactile feedback specifically refers to the use of vibration to encode information.

       3.1.1 Chapter Scope and Coverage

      This chapter outlines what makes haptic design different. As an aid to comprehension, readers are referred to this chapter’s Focus Questions and to the Glossary for a definition of terminology.

      We start with how our haptic sense is different (Section 3.1). Then, in four stages we distill insights from 20 years of designing haptic experiences ourselves and from studying skilled and novice designers as they work, often while using tools we have crafted for them. First, we establish why and when we should bother, by going through potential haptic contributions within a multimodal interaction (Section 3.2). Section 3.3 is about morphology: What can it consist of? At a high level, designing in a haptic medium is similar to any other; it’s the details that differ. Therefore, we will examine how by traveling through a conventional user experience (UX) design process (Section 3.4). We conclude by overviewing a few frontiers where we believe that accelerating innovation will soon pay off in solving many of the design obstacles we have identified (Section 3.5).

       3.1.2 Nature of the Haptic Sense

      A number of attributes together give a specific suitability profile to haptic media: simple messages graded in salience and nature, with availability corresponding to the user’s ability to physically access them.

      The haptic sense is distributed and multi-parametered. A complex diversity of skin and muscle mechanoreceptors permit the broad range of what we can physically feel: temperature, texture, forces, motion; a brush of fur, a breeze, a droplet of cold water, a swat or bump, road vibrations, a subtle weight shift of a heavy object we’re carrying. Sensory density and distribution changes across the body, and different receptors command differential response speed and specificity [Choi and Kuchenbecker 2013, IJsselsteijn 2003, Lederman and Klatzky 2009, Klatzky et al. 2013]. Imagine a machine that could sense—and make sense of—so many different things. It would require a lot of different sensors, plus compute power and sophisticated neural learning to integrate their diverse input. It is the same for living organisms.

      Haptics can be involved in bidirectional active sensorimotor exploration or query, or passive sensory reception of touches applied to one’s body by another person or thing. Much of our touching is in support of manipulation, and it is through manipulation that we can physically explore and sense environments. A designer must consider which information directly pertains to a manipulation, and whether this can be displayed to the body during a manipulation in a manner consistent with expectations drawn from real-world experience.

      Active and passive touch have different relationships to attention [Sarter 2013], providing different affordances and requirements for design. Passively experienced sensations may be an ambient interface, background source of information [MacLean 2009], which makes it to conscious attention only if there’s room; if salient, they’ll capture attention. Active exploration is usually in the attentional foreground; when a toucher is seeking something, he will probably notice it if it’s there.

      Perceived on the body, haptic perceptions are personal, private, and challenging to share [MacLean 2008a]. They involve social norms for interpersonal touching, as well as the appropriateness and safety or hygiene of touching other people and their belongings. Constant availability requires constant contact; otherwise, the user must know when to reach for a display. Haptics-suitable applications will have a built-in contact opportunity (a car seat, an object the user is already holding, or a wearable device); or can be designed holistically into a larger scene.

      Compared to visual and auditory channels, people tend to use touch for low-density information transfer. That said, the degree to which visually impaired individuals are able to extract greater density suggests this may indicate more about learning and communication norms than fundamental potential. With today’s technology, haptic media is usually displayed at lower information density than vision and even audition, but, conversely, it can be more convenient, immediate, and appropriately intrusive. Well-situated and timed signals can be extremely helpful to users as notifications, progress monitors, and manipulation-relevant details presented directly to the hand.

      Meanwhile, the ability of haptic display to ambiently convey more qualitative information is relatively untapped.

       3.1.3 Novelty of Haptic Media to Humans

      Our haptic vocabulary for physical sensations is relatively impoverished, impacting users’ ability to describe, communicate, and possibly even to perceive distinctions. While there have been and will be many efforts to create haptic lexicons, both in terms of abstract properties and their perceptibility [Maclean and Enriquez 2003, Ternes and MacLean 2008, Guest et al. 2011, Seifi et al. 2015] and for specific applications [Chan et al. 2008, Tam et al. 2013, Cauchard et al. 2016], it may be equally important to develop users’ ability to describe what they can feel and thus develop their appreciation of nuance—similarly to how novice wine lovers learning of olfactory and gustatory discernment is scaffolded by sommelier vocabulary [Obrist et al. 2013, Hwang et al. 2011, Lawless 1984].

      Beyond the question of vocabulary, most users are not accustomed to processing synthetically encoded haptic meaning. It is not a skill learned slowly since early childhood, like visual reading. Even for relatively simple communications, other modalities employ a sensory design language whose cultural foundations have developed and been imparted over years: westerners have learned to associate a graphical recycle bin icon with file deletion. For now, haptic applications may thus be limited to very easily acquired vocabularies, but the skill shown with longer training [Swerdfeger 2009] promises greater sophistication as the medium becomes more widespread.

3.1.4 How People Differ in Their Experience of Haptic Media

      Variations among individuals in their experience of haptic sensations mean that specific design elements may not work for everyone. There are at least three levels at which such individual differences appear, each with its own design significance.

      In haptic perception, individual mechanoreceptors register signals with varying resolutions (analogously to visual color-blindness), evident in nonuniform tactile threshold and difference detection abilities [Lo et al. 1984], and typically investigated with psychophysical studies which exclude subjective components. For subtle sensations such as programmable friction, differences among people become more prominent [Levesque et al. 2012]. Tactile acuity also declines with age, suggesting this channel is not ideally targeted for seniors [Stevens 1992, Stevens and Choo 1996]. There is empirical evidence that the perceptual space of sensations is impacted by these differences; for example, people varied in categorizing natural textures according to a 2D vs. a 3D perceptual space [Hollins et al. 2000].

      At the level of haptic processing and memory, numerous studies on human ability to identify and parse tactile patterns exemplify differences in ability to process and learn haptic stimuli, with tactile the most frequently studied, e.g., [Epstein et al. 1989]. In particular, an early study by Craig [1977] suggests two groups—learners and non-learners—in a spatio-temporal pattern matching task with the Optacon. A more recent study on a variable friction display reports notable differences in users’ recognition of friction patterns and their spatial density [Levesque et al. 2012]. People also differ in the degree to which they rely on touch for hedonic or information gathering purposes, suggesting modality-specific processing needs and abilities [Peck and Childers 2003]. Haptic processing abilities can be improved with practice: visually impaired individuals often develop exceptional tactile processing abilities independently of their degree of childhood vision, demonstrating substantial brain plasticity [Goldreich and Kanics 2003].

      Because synthetic