The Handbook of Multimodal-Multisensor Interfaces, Volume 1. Sharon Oviatt

to intersensory redundancy?

      2.3. What types of behaviors are enhanced by multimodal interactions with the environment across development? In what way are these behaviors changed by this experience?

      2.4. How does learning through action change the brain? Include examples of learning about objects and symbols. What would be a practical implication of learning about three-dimensional objects through action vs. through observation?

      2.5. Why would handwriting facilitate learning and recruit brain systems used for reading more than typing?

      2.6. How can we use our knowledge of brain systems to aid in the construction of multimodal interfaces?

      2.7. Given the information provided in this chapter, what are some other multimodal interfaces that would facilitate learning?

      2.8. Would the understanding of word meaning be facilitated by multimodal learning? Would some words benefit more from this type of learning than others?

      2.9. Are there concepts that we learn independently of our multimodal interactions with the world? What are some examples? How would such concepts be represented in brain systems?

References

      W. J. Adams, E. W. Graf, and M. O. Ernst. 2004. Experience can change the “light-from-above” prior. Nature Neuroscience, 7(10):1057–1058. DOI: 10.1038/nn1312. 58

      D. M. Adams, A. T. Stull, I. Roewer, and M. Hegarty. 2011. Effects of using mental and manual rotation training on mental and manual rotation performance. In L. A. Carlson, C. Hölscher, and T. F. Shipley, editors. Expanding the Space of Cognitive Science: Proceedings of the 33rd Annual Meeting of the Cognitive Science Society. pp. 1268–1273. Cognitive Science Society, Austin, TX. 59

      K. Adolph. 1997. Learning in the development of infant locomotion. Monographs of the Society for Research in Child Development, 62(3):1–162. 64

      K. Adolph, L. B. Karasik, and C. Tamis-LeMonda. 2010. Motor skill. In M. H. Bornstein, editor. Handbook of Cultural Developmental Science. pp. 61–88. Psychology Press, New York. 64

      K. E. Adolph, C. S. Tamis-LeMonda, S. Ishak, L. B. Karasik, and S. A. Lobo. 2008. Locomotor experience and use of social information are posture specific. Developmental Psychology, 44(6):1705–1714. DOI: 10.1037/a0013852. 64

      S. H. Ambrose. 2001. Paleolithic technology and human evolution. Science, 291(5509):1748–1753. 79

      M. K. Babcock and J. J. Freyd. 1988. Perception of dynamic information in static handwritten forms. American Journal Psychology, 101(1):111–130. 60

      L. E. Bahrick and R. Lickliter. 2002. Intersensory redundancy guides early perceptual and cognitive development. In R. Kail, editor. Advances in Child Development and Behavior. vol. 30, pp. 153–187. Academic Press, New York. 52

      M. Banks and P. Salapatek. 1983. Infant visual perception. In M. M. Haith and J. J. Campos, editors. P. H. Mussen (Series Ed.). Handbook of Child Ppsychology: Vol. 2. Infancy and Developmental Psychobiology. pp. 43–572. Wiley, New York. 62

      L. W. Barsalou. 2008. Grounded cognition. Annual Review of Psychology, 59:617–645. DOI: 10.1146/annurev.psych.59.103006.093639. 79

      L. W. Barsalou, W. K. Simmons, A. Barbey, and C. Wilson. 2003. Grounding conceptual knowledge in modality-specific systems. Trends in Cognitive Science, 7(2):84–91. 70

      P. Bartolomeo, A. Bachoud-Lévi, S. Chokron, and J. Degos. 2002. Visually- and motor-based knowledge of letters: evidence from a pure alexic patient. Neuropsychologia, 40:1363–1371. 61

      V. W. Berninger, K. Vaughan, R. D. Abbott, A. Brooks, S. P. Abbott, L. Rogan, E. Reed, and S. Graham. 1998. Early intervention for spelling problems: Teaching functional spelling units of varying size with a multiple-connections framework. Journal of Educational Psychologu, 90(4):587–605. 60

      B. I. Bertenthal, D. R. Proffitt, and J. E. Cutting. 1984. Infant sensitivity to figural coherence in biomechanical motions. Journal of Experimental Child Psychology, 37:213–230. 63

      E. G. Boring. 1964. Size-constancy in a picture. American Journal of Psychology, 77(3):494–498. 85

      C. B. Boronat, L. J. Buxbaum, H. B. Coslett, K. Tang, E. M. Saffran, D. Y. Kimberg, and J. A. Detre. 2005. Distinctions between manipulation and function knowledge of objects: evidence from functional magnetic resonance imaging. Cognitive Brain Research, 23(2–3):361–373. DOI: 10.1016/j.cogbrainres.2004.11.001. 70

      S. Broaders, S. W, Cook, Z. Mitchell, and S. Goldin-Meadow. 2007. Making children gesture reveals implicit knowledge and leads to learning. Journal of Experimental Psychology General, 136(4):539–550. DOI: 10.1037/0096-3445.136.4.539.

      P. Broca. 1861. Perte de la parole, ramollissement chronique et destruction partielle du lobe antérieur gauche du cerveau. Bull Soc Anthropol, 2(1), pp. 235–238. 70

      A. J. Butler and K. H. James. 2013. Active learning of novel sound-producing objects: motor reactivism and enhancement of visuo-motor connectivity. Journal of Cognitive Neuroscience, 25(2):203–218. DOI: 10.1162/jocn_a_00284. 71, 72

      L. J. Buxbaum and E. M. Saffran. 2002. Knowledge of object manipulation and object function: dissociations in apraxic and nonapraxic subjects. Brain Language, 82:179–199. 70

      C. G. Christou and H. H. Bülthoff. 1999. View dependence in scene recognition after active learning. Memory & Cognition, 27(6):996–1007. 58

      J. Campos, D. Anderson, M. Barbu-Roth, E. Hubbard, M. Hertenstein, and D. Witherington. 2000. Travel broadens the mind. Infancy, 1(2):149–219. 69

      J. Campos, B. I. Bertenthal, and R. Kermoian. 1992. Early experience and emotional development: The emergence of wariness of height. Psychological Science, 3(1):61–64. 64

      P. Cisek. 1999. Beyond the computer metaphor: behaviour as interaction. Journal of Consciousness Studies, 6(11–12):125–142. 60

      L. Cohen and S. Dehaene. 2004. Specialization within the ventral stream: the case for the visual word form area. Neuroimage, 22(1):466–476. DOI: 10.1016/j.neuroimage.2003.12.049. 69

      E. Combe and M. Wexler. 2010. Observer movement and size constancy. Psychological Science, 21(5):667–675. DOI: 10.1177/0956797610367753. 57

      L. Craighero, L. Fadiga, C. Umilta, and G. Rizzolatti. 1996. Evidence for visuomotor priming effect. Neuroreport, 8:347–349. 69

      J. Dewey. 1896. The reflex arc concept in psychology. Psychological Review 3:357–370. 69

      S. B. Ehrlich, S. C. Levine, and S. Goldin-Meadow. 2006. The importance of gesture in children’s spatial reasoning. Developmental Psychology, 42(6):1259–1268. DOI: 10.1037/0012-1649.42.6.1259.

      R. Ellis and M. Tucker. 2000. Micro-affordance: The potentiation of components of action by seen objects. British Journal of Psychology, 91:451–471. 69

      M. O. Ernst and M. S. Banks. 2002. Humans integrate visual and haptic information in a statistically optimal fashion. Nature, 415:430–433. 58

      S. Exner. 1881. Untersuchungen über die Lokalisation der Functionen in der Grosshirnrinde des Menschen. Wilhelm Braunmüller, Vienna, Austria. 70

      L. M. Flevares and M. Perry. 2001. How many do you see? The use of nonspoken representations in first-grade mathematics lessons. Journal of Educational Psychology, 93(2):330–345. DOI: 10.1037/0022-0663.93.2.330

      J.J. Freyd. 1983. Representing the dynamics of a static form. Memory & Cognition, 11:342–346. 60

      V. Fuentes-Santamaria, J. C. Alvarado, J. G. McHaffie, and B. E. Stein. 2009. Axon morphologies and convergence patterns of projections from different sensory-specific cortices of the anterior ectosylvian sulcus onto multisensory neurons