My thesis focuses on sensorimotor simulation – the mental recreation of previously experienced perceptions and actions. My primary argument is that sensorimotor simulation drives multimodal communication, particularly the production of iconic and metaphoric speech and gesture.
In the case of iconicity, sensorimotor simulation drives the coinage of words and gestures that resemble their referents. Even as iconicity diminishes over time in individual words due to historical processes of conventionalization and grammaticalization, iconicity in the lexicon as a whole is replenished by novel iconic coinages, expressive modifications of existing iconic terms, and iconic gestures. The impetus for iconicity in multimodal communication, I argue, is driven by sensorimotor simulation.
According to conceptual metaphor theory, abstract domains that cannot be perceived directly (target domains) are understood by analogy to concrete domains that can be perceived directly (source domains). Metaphoric thought processes can thus be understood as sensorimotor simulation of a concrete source domain when thinking about an abstract target domain. This process drives the production of metaphoric language and gestures, and also the design of cultural artifacts such as data visualizations.