The Handbook of Multimodal-Multisensor Interfaces, Volume 1. Sharon Oviatt

      Figure 3.3 Intuitive vs. abstract encoding of a direction vocabulary for a vibroactile seat. (a) User easily interprets intuitive encoding of direction with spatial parameters. (b) User learns abstract encoding of direction through temporal parameters.

3.4 Making Haptic Media

      How do we translate knowledge of the physical and semantic haptic design space into compelling, coherent, and learnable haptic media, given the many and particular challenges it presents? The answer is a robust and flexible process. We draw upon a design thinking approach, often described as a funnel of idea candidates wherein the designer iteratively generates, refines and narrows down multiple ideas in parallel until a final, well-developed, and trusted design concept remains (Figure 3.4.

      We look now at how generic forms of design thinking must be adapted when applied to haptics, and offer several different schemas for approaching haptic design (including those introduced earlier for the user’s view of haptic sensations see Section 3.3.3). We close with an inventory of current haptic design tools and techniques.

      Figure 3.4 Incorporating haptics into the design process. We adapt the classic design funnel, where multiple initial ideas are iteratively developed, then add four design activities we have found useful when supporting design: browsing, sketching, refining and sharing. (Based on Buxton [2007])

       3.4.1 Design Process

      Understanding how best to support design and creativity has long been an important research topic. There is increasing evidence that designers’ environment and tools shape their output, especially their exposure to previous designs, flexible and precise tools, and collaborators [Herring et al. 2009, Schneider and MacLean 2014, Kulkarni et al. 2012, Dow et al. 2011]. We will look at how four design activities—browsing, sketching, refining, and sharing—look in the context of a principled haptic media design process; and where these activities differ from designing in other modalities.

      Alongside these activities, designers are constantly engaged in other tasks such as devising effective haptic-meaning mappings (encoding, Section 3.3.3), and evaluating designs, often with rating scales or qualitative feedback—against criteria described in Sections 3.1.5 and 3.2.4). These tasks sequence and bind design activities in specific ways that help accomplish a design goal.

       Browse

      No idea is born in isolation. Individual designers have a repertoire of previous experiences they have encountered while learning or through practice [Schön 1982]. In addition, design often starts with a “gather” step [Warr and O’Neill 2005]: viewing examples for inspiration and problem definition. Gathering often occurs explicitly at the start of a design process, and can reoccur during iteration. Tangible examples are corkboards and mood boards, which allow ideas to “bake in” to the background [Buxton 2007]. Software tools like d.tour [Ritchie et al. 2011] and Bricolage [Kumar et al. 2011] recommend websites for inspiration and can automatically generate new ideas by combining sites. Haptic designers, however, encounter modality-specific barriers when gathering, managing, and searching for examples.

      First, we require a way to represent sensations, singly and in collections. How do we store, view, and organize haptic experiences? Haptic technologies are often inherently interactive, part of a multimodal experience with visual and audio feedback, and can take a variety of physical forms depending on the output (and input) device. This last point is particularly bothersome should the user not have access to the original device type—imagine trying to browse force-feedback sensations on your phone!

      Then we need means of classifying and organizing collections. Haptic language and cultures of meaning are still in active development. Without a commonly shared lexicon, organization dimensions, or even adjectives, it is difficult to curate collections. Compare this to sound: most musical terms have a long tradition with a clearly defined lexicon (e.g., crescendo, staccato); non-musical sound effects generally “sound like” something, and are often literal. With vision, one does not have to be a graphic designer or artist to instinctively understand “warm” and “cool” colors; the color wheel is introduced to us in grade school.

      Overviews allow us to skim collections. Visual or physical collections of examples are often displayed spatially for ambient reference or to enable quick scanning. When you cannot feel multiple things, it can be hard to get the big picture or swiftly peruse a collection. Both designer and end-users have needs for finding similar/different vibrations in a collection, requiring a low barrier-to-entry on any overview technique.

      Given the importance of browsing, it is no surprise that the haptics community has made some progress. Libraries such as the Haptic Touch toolkit [Ledo et al. 2012, HapticTouch Toolkit 2016] or Penn Haptic Texture toolkit [Culbertson et al. 2014, Penn Haptic Texture Toolkit 2016] are available to the community. The Haptic Camera allowed for easy capturing of door knob dynamics that can be stored and recreated later [MacLean 1996], inspiring similar camera-like devices like a portable texture recording device [Burka et al. 2016]. VibViz [Seifi et al. 2015, VibViz 2016] is an online, visualized collection of vibrotactile icons that explicitly tackles these issues, providing multiple classifications schemes (facets) and visualizations to rapidly skim and find vibrations. Visualization techniques are still early, but they help [Seifi et al. 2015], and careful design can help improve representation of perceptual qualities [Schneider et al. 2016].

       Sketch

      Sketching allows people to form abstracted, partial views of a problem or design, iterate very rapidly and explore concepts. This is mostly heavily used early in design, and plays a role in collaboration (discussed more under “Share” below). Of course, such a central technique is used as a key way of thinking about experience design [Buxton 2007]; some even consider sketching to be the primary language of design, equivalent to mathematics as a language for natural sciences [Cross 2006]. With haptic technology, there is no immediate way to handle two essential features: abstraction and ambiguity, and rapid iteration (addressed more fully in Section 3.4.3).

      With respect to abstractability, we note that haptics suffers from a dearth of notation. Sketching of physical devices or interfaces is well supported, with paper and pencil and innumerable software assists. Sketching motion, and in particular showing what is or might be felt in, say, a vibrotactile experience, is trickier. While we can sketch a visual interface and look at it, it is much harder to sketch a haptic sensation and imagine it without feeling it.

      Creative approaches are emerging. Most directly, Moussette and Banks [2011] teach Haptic Sketching [DesignThroughMaking 2016] with physical scraps and materials, combined with manual actuator and tools like Arduino, to build effective interactive haptic prototypes physically and programmatically in minutes or hours. Simple display-only sensations can be sketched (e.g., VT icons) using interactive design tools [Schneider and MacLean 2014, Hong et al. 2013].

       Refine

      Clearly apparent in Figure 3.4, design requires iteration to refine an initial set of ideas into a single well-developed one through concept generation followed by iterative revision, problem-solving and evaluation, until only small tweaks are necessary. This long view of the design process is necessary to see designs through to the end; furthermore, tweaking final designs is a valuable way to accommodate individual differences.

      Incorporating haptic technology into a design is