Designing for Gesture and Tangible Interaction. Mary Lou Maher

content on the subset being manipulated by referring to the composition on a larger display. With space-multiplexed input, each function to be controlled has a dedicated transducer, each occupying its own space (Fitzmaurice and Buxton, 1997). For example, in a Pattern Maker application on the Tangible Keyboard, each cube can be used to manipulate a shape, a color, and a scaling function. While the input devices are used to manipulate and input the composition, they also provide a visualization of subsets of the composition that can be repeated or rearranged as input.

       2) Tangible Models

      Tangible Models also has space-multiplexed input/output devices. The individual input/output blocks are each associated with a 3D digital model that is visible on the vertical display. The 3D models are rearranged on the tabletop in reference to a plan view of the composition, with visual feedback of the 3D scene on the vertical display. These multiple 3D blocks allow direct control of digital objects as space-multiplexed input devices, each specific to a function and independently accessible. The application of Tangible Models to the configuration design of rooms on a floor plan layout allows the user to assign 3D models such as walls and furniture from a library to each block. The user can rearrange the blocks to explore various design configurations by reaching for and moving the block as a handle for the 3D model.

2.2.2 CONCURRENCY

      A core property of TUIs is space-multiplexed input and output. This allows for simultaneous, but independent and potentially persistent selection of objects. TUIs have multiple devices available, and interactions that allow for concurrent access and manipulation of interface components (Fitzmaurice, 1996). In a traditional graphical user interface (GUI), one active selection is possible at a time and a new selection should be done in order to undo a prior one. Time-multiplexed input devices have no physical contextual awareness and lack the efficiency of specialized tools. The ability to use a single device for several tasks is a major benefit of the GUI, but given the nature of interaction, where only one person can edit the model at a time, the GUI environment may change interactivity in collaborative design (Magerkurth and Peter, 2002). However, TUIs can possibly eliminate many of the redundant selection actions and make selections easier. In terms of collaborative interactions, the TUI environment enables designers to collaborate on handling the physical objects more interactively by allowing concurrent access with multiple points of control (Maher and Kim, 2005).

       1) Tangible Keyboard

      Tangible Keyboard focuses on the user experience during a creative task in which the user has multiple tangible objects that are manipulated to compose and synthesize elements of new design. Multiple tangible objects offer greater flexibility, allowing each input device to display different types of function. In the Pattern Maker application, each input device displays a single shape, color, or scale. A shape can also be modified on a cube by rearranging these different shapes, colors, and scales. Shape cubes can be manipulated independently but also modified with color or scale cubes to create new design patterns. Concurrency is achieved through simultaneous access to multiple physical devices where each one displays its own shape, color, or scale.

       2) Tangible Models

      Tangible Models provides a similar experience, but with each tangible object assuming the geometric properties of a 3D object. The spatial rearrangement of the 3D models is directly correlated with the 3D composition. Concurrency is achieved through simultaneous access to multiple 3D models, each on a separate physical object. A protocol study of designers using Tangible Models, described in Kim and Maher (2008), showed that users were more focused on the spatial and functional relationships among the individual 3D objects than on the properties of each object when compared to an interaction design that was time-multiplexed (keyboard and mouse). With the direct, naïve manipulability of physical objects and rapid visualization, designers in the TUI environment produced more cognitive actions and completed the design tasks faster.

2.2.3 STRONG SPECIFIC DEVICES

      TUIs provide strong specific devices for interacting with the system. This offers more efficiency because the tangible objects are designed to be more specialized and tailored for working on a given task in order to increase the directness and intuitiveness of interactions (Le Goc et al., 2015; Hornecker, 2005). The design of appropriate physical representations is a very important aspect of tangible interface design (Ullmer and Ishii, 2000). To create strong specific devices, the most common approach is to utilize existing objects into which position sensors or ID tags are inserted. Alternatively, strong specific devices are achieved with Augmented Reality (AR), where each physical device is associated with a virtual object. The user interacts with a virtual object by manipulating the corresponding physical object (Waldner et al., 2006). While seeing virtual imagery superimposed on physical objects, the user perceives interaction with the digital object. These specialized interactive objects may lack generality and therefore may not be effective for some tasks. This loss of generality may be overcome by the advantages provided by task-specific physical tools (Fitzmaurice, 1996). Tangible user interaction with physical objects that have a specialized form and appearance offer affordances normally associated only with the physical object.

       1) Tangible Keyboard

      In the case of the Tangible Keyboard, the form is constant (cube-like objects with a display) and the appearance (display) is variable. The image on the display is designed to fit the context of the tasks supported by the application. The affordances of these specific devices are those associated with the shape of the object and the content on the display. In the Pattern Maker application, shape cubes are rearranged to form patterns and color cubes are tilted to pour a new color on a shape. These strong specific devices do not have the generality of the mouse for selecting any function, but provide strong feedback on the functions enabled by the application.

       2) Tangible Models

      With Tangible Models, simple 3D blocks as tangible devices are rearranged on a tabletop system with a vertical display of the 3D scene. Each block is associated with a single 3D model, providing a strong specific device for creating a composition of a scene or spatial design. With static mappings and multiple input objects, 3D blocks as tangible input elements can be expressive and provide affordances specific to the object they represent. The visualization of each 3D block directly indicates its meaning or function while the user is moving the pieces and making a composition.

2.2.4 SPATIALLY AWARE COMPUTATIONAL DEVICES

      Spatially aware computational devices and spatial configurations are important concepts in embodied interaction. A physical object in TUIs is typically aware of its spatial position and is registered with a central processing unit. Both position and orientation are critical pieces of information to be sensed (Fitzmaurice, 1996; Valdes et al., 2014). Proximity information is possible through communication to a central processing unit or independent sensors on each device. Applications that are more graphic than alphanumeric benefit from having spatially aware input devices, as graphical tasks are inherently spatial in nature (Fitzmaurice, 1996). Spatially aware computational devices allow users to interact with complex information spaces in a physical way by changing the spatial position and orientation of tangible devices.

       1) Tangible Keyboard

      Tangible Keyboard is built on the hardware/software platform of Sifteo cubes™ and the sensors include adjacency awareness and accelerometers. These sensors allow the cubes to be aware of other cubes and the movements of each cube, such as shaking, tilting, and turning over. By communicating with a central processor, the rearrangement and movements of the cubes can be mapped onto input events related to the composition task. For the Pattern Maker application, spatial awareness of the individual devices allows the user to form compositions and to modify the shape, color, and scale of elements of the composition.

       2) Tangible Models

      Tangible Models is built on the software platform of the ARToolkit, in which spatial awareness is achieved by a camera that senses the location of predefined markers (Kato et al., 2001). The assignment of each marker