The Handbook of Multimodal-Multisensor Interfaces, Volume 1. Sharon Oviatt

vertical process, dependent on specifics of hardware, firmware, software, application, and multimodal context (Section 3.1.6). With the complexity of these many components, there can be a significant initial cost to setup a first haptic experience; then, adding this complexity to the time needed to program, recompile, or download to a microcontroller means iteration cycles have the potential to be slow and painful. Thus, increasing refinement fluidity is ripe for innovation. For example:

      Pipelines now connect initial design seamlessly through to final refinement [Schneider et al. 2015b, Schneider and MacLean 2016]. Continuity in future tools will provide fluid, transparent (rather than cumbersome, many-staged) connection between hardware and software tools at different design stages.

      Evaluation is as crucial as for any human-centered refinement cycle. While it will often require some form of sharing (coming up next), here we simply point out that the full spectrum of evaluative mechanisms and supports found in user experience development can be gainfully applied to haptic design, from lab-based comparative performance studies to qualitative examination of how usage strategies change when a physical dimension is deployed (e.g., [Minaker et al. 2016]).

      Customization tools are appearing at least at the level of prototyping and requirements generation [Schneider et al. 2015a, Seifi et al. 2014]. Force-feedback virtual environments support iteration and refinement through code, once the initial environment is setup. Software platforms like Unity [Unity Game Engine 2016] offer immediate control of variables in the UI itself.

      Tool context—calibration, customization, and sensing—in tools will help final haptic designs remain consistent depending on user activity (e.g., running impairs vibration sensitivity), individual differences, or other contextual concerns.

       Share

      Sharing designs is valuable at different stages of the design process [Kulkarni et al. 2012], whether for informal feedback from friends and colleagues, formal evaluation when refining designs, or distributing to the target audience for use and community for re-use [Shneiderman 2007].

      As haptic experiences must be felt, this process works best when collocated with only a few collaborators, whether by having collaborators work in the same lab, or by showing final experience in physical demos. During ideation, ideas can be generated when collaborating remotely, but physical devices need to be shipped back and forth and it is difficult to troubleshoot and confirm that configuration and physical setup are the exact same. Feedback also typically needs to be collocated, using in-lab studies or feedback, or shipping devices between collaborators. Furthermore, visual and audio design support very easy capture of ideas to share later, through smartphone cameras and microphones, that could later be browsed.

      So far, haptic broadcasting, analogous to broadcasting radio or television (e.g., Touch TV [Modhrain and Oakley 2001]) has been envisioned and explored. Follow-up work has added haptics to YouTube [Abdur Rahman et al. 2010] and movies [Kim et al. 2009]. Low-cost devices like the HapKit [Orta Martinez et al. 2016, Hapkit 2016] and Haply [Gallacher et al. 2016, Haply 2016] make haptics more ubiquitous, but remain troublesome to calibrate. To share ideas remotely on phones, proxies like visualizations or other types of haptics (phone vibrations) could be used [Schneider et al. 2016, HapTurk 2016]. Features like automatic calibration and proxies for use in online evaluation, and online communities more generally, are still in development.

       3.4.2 Schemas for Design

      Because haptic design is such a young field, there are many ways to approach it. One is to consider analogies to other fields, for example to draw on existing expertise in making sounds and multimedia. Another is to focus on the language of haptics, affect, and descriptive aspects of sensations, as laid out in Section 3.3.3. These approaches can productively be combined. In the following, we start with some general perspectives and techniques useful for haptic design, then delve into several specific schemas that haptic designers have made use of: sources of inspiration and conceptual scaffolding of what the finished design may be.

       General Methodological Perspectives

      Some higher-level perspectives offer useful outcome targets, collections of methods, and design attitudes to guide haptic practitioners in their process. DIY (do-it-yourself) haptics categorize feedback styles and design principles [Hayward and MacLean 2007, MacLean and Hayward 2008]. Ambience is proposed as one target for a haptic experience, where information moves calmly from a person’s periphery to their focused attention [MacLean 2009]. Haptic illusions can serve as concise ways to explore the sense of touch, explain concepts to novices and inspire interfaces [Hayward 2008]. “Simple Haptics” [Simple Haptics 2016], epitomized by haptic sketching, emphasizes rapid, hands-on exploration of a creative space [Moussette 2010, Moussette and Banks 2011] and has been enabled by recent and radical advances in mechatronic rapid prototyping technology. The notion of distributed cognition [Hutchins 1995] has particular relevance for haptic design, suggesting that people situate their thinking both in their bodies and in the environment. Finally, haptics courses are extremely helpful collections of skills and techniques, with foci including perception, control, and design [Okamura et al. 2012, Jones 2014]. Each of these different perspectives can help haptic designers think about how to design haptics more generally, and can augment schemas inspired from other fields.

       Design Schemas Inspired by Audio, Video and Multimedia

      Haptic designers have often appropriated design elements used in other fields. Haptic Icons [Maclean and Enriquez 2003], tactons [Brewster and Brown 2004], and haptic phonemes [Enriquez et al. 2006] are small, compositional, iconic representations of haptic ideas, inspired by comparable elements from graphical and sound design [Gaver 1986]. Touch TV [Modhrain and Oakley 2001], tactile movies [Kim et al. 2009], haptic broadcasting [Cha et al. 2009], and Feel Effects [Israr et al. 2014] aim to add haptics to existing media types, especially video.

      Music analogies and metaphors have frequently inspired haptic design tools, especially VT sensations. The Vibrotactile Score, a graphical editing tool representing vibration patterns as musical notes, is a major example [Lee and Choi 2012, Lee et al. 2009]. Other musical metaphors include the use of rhythm, often represented by musical notes and rests [Ternes and MacLean 2008, Brown et al. 2005, Chan et al. 2008, Brown et al. 2006b]. Earcons and tactons are represented with musical notes [Brewster et al. 1993, Brewster and Brown 2004], complete with tactile analoges of crescendos and sforzandos [Brown et al. 2006a]. The concept of a VT concert found relevant tactile analogues to musical pitch, rhythm, and timbre for artistic purposes [Gunther et al. 2002]. In the reverse direction, tactile dimensions have also been used to describe musical ideas [Eitan and Rothschild 2010].

       Language of Touch

      The language of tactile perception, especially its affective (emotional) terms, is an obvious possibility for framing haptic design. Language is a promising way to capture user experience, both more generally and for haptics in particular [Obrist et al. 2013], and can reveal useful parameters, e.g., how pressure influences affect [Zheng and Morrell 2012]. In Section 3.1.4, we noted how individuals differ in their experience of haptic stimuli, and this certainly has implications for the generation of stable, broadly understandable design languages in this modality. Reiterating those points: relatively (although not perfectly) consistent sensory dimensions have been established with psychophysical studies for both synthetic haptics and real-world materials, but for meaning-mapping, agreement becomes highly variable. Touch clearly communicates strongly to individuals, but it is difficult to describe, and there is less evidence for existence of a general tactile language that all individuals would agree with [Jansson-Boyd 2011]. The importance of learning and familiarity to cultural agreement on meaning has been barely looked at [Swerdfeger 2009].

      More research is clearly needed. Our own view is that some tactile elements can be consistently understood, but far more will be personally interpreted. The beauty and power of active haptic interfaces is that individualized approaches are possible, and solutions