Narrative Review
Effects of visual, auditory, and combined cues on human movement and brain regions involved in perception action

Kedar K. V. Mate¹
Published online: July 5, 2022

¹Director of Outcomes Research, Centre for Neurological Restoration, Neurological Institute, Cleveland Clinic, Cleveland, Ohio, USA
Corresponding Author: Kedar K. V. Mate, email: kedar.mate@mail.mcgill.ca
DOI: 10.26443/mjm.v20i2.928

Introduction

Humans function within a multisensory environment, where sound often accompanies action (clapping, crushing a can, or hammering). The most common sensory stimuli are visual and auditory signals, which, when combined, amplify the perception of surrounding objects and movement.(1) It is well established that the response (or reaction) is optimal with multisensory stimulation instead of individual stimulus(2). Multisensory integration involves complex sensory interactions that help to perceive the environment more accurately.(3) The perceptual system integrates the audiovisual information to provide a comprehensive picture for optimal functioning within multisensory surroundings, evident by a decrease in reaction time to multisensory stimulation.(4, 5) Additionally, multisensory signals provide information about the environment that would be insufficient when obtained from anyone sensory signal in isolation.(6)

The perceptual-motor system has been studied for over half a century. This narrative review will present the interaction of sensory cues and movement as well as brain areas involved in perception-action. The review will also provide an overview of the literature on unintentional synchronization. This comprehensive review is divided into three sections: Section 1 will present the effects of visual, auditory, and combined cues on human movement; Section 2 will highlight brain region activation in response to sensory cues and the mirror neuron network, and Section 3 will discuss unintentional synchronization.

Methods

Only the English language literature related to sensory cues and human movement was searched in PubMed and Google Scholar. A pragmatic search strategy approach using a saturation matrix typically used in qualitative research was deployed, whereby the search was terminated once the same articles appeared. The aim of this work is not to present a reproducible search but to present the breadth of information on a topic. Literature in the field of biofeedback was excluded.

Section 1: Effect of individual sensory cues on movement

Visual cue:

Visual stimulus is the most dominant and ecologically salient source of information during walking.(7-10) Johansson et al. (1973) conducted a pioneer study to determine the ability of the visual system to recognize biological motion. In his study, the visual stimulus was constructed using a point-light attached to human joints the observers were asked to identify.(11) The study demonstrated the ability to interpret visual signals related to human movements in order to make meaningful inferences using past knowledge. We are constantly exposed to real multisensory stimuli complex scenes , so the perception of envirnonment complex scenes would differ from the relatively simple point-light design in the study. Moreover, it is shown that complex visual scenes result in distributed activations of different brain regions. The ventral temporal cortex (lateral and medial fusiform) is activated in response to video displays as opposed to point-light images.(12) The ventral temporal cortex is also responsive to static and moving human stimuli and objects (such as a saw or hammer) in motion. In contrast, the lateral temporal cortex is specifically responsive to moving complex stimuli such as articulated human motion.(12) This differential neural activation arose from the additional information in video display, such as form, color and texture that was absent in point-light display. Furthermore, the perception of motion is also affected based on the observer’s position. It is shown that the perception of motion of the point-light images differs between a static observer posture (standing on a treadmill or sitting on a static bicycle) and dynamic self-motion, such as healthy individuals walking on a treadmill.(13) The perception of point-light images is least accurate when the observer walks than when sitting.(13) In research and clinical practice, visual cues such as laser light and stride length markings on the floor are often used as a strategy to improve walking performance in people with Parkinson’s.(14) Recently, a trial studied the use of visual cues such as steppingstones for the rehabilitation of post-stroke walking.(15, 16) The use of steppingstones displays as a visual cue in healthy elderly subjects was reported to positively affect gait parameters.(17) The steppingstone visual display was compared to metronome beats among adults during a treadmill walking task and showed that the visual information resulted in quick gait recovery in response to perturbations compared to auditory beats.(17) Visual cue displayed over the treadmill determines step length (and consequently step frequency within a range of walking speeds) and directs spatial position for foot placement. The disadvantage of visual cues such as steppingstones, flashlights, or stripes on the floor, is that the cues have no ecological meaning.

Auditory cue:

Auditory system is a fast-processing sensory system that rapidly captures and extracts meaning from the received signals.(18, 19) Reaction time to auditory cues is faster by 20-50 milliseconds compared to visual or tactile cues.(18, 19) External auditory cues, such as metronome beats and music, are shown to have beneficial effects on walking in various neurological populations such as stroke, Parkinson’s, Huntington’s conditions.(14, 20-28) Unpredictable changes in auditory cue frequency provided by the metronome are shown to induce perturbation during walking in patients with stroke.(29, 30) This suggests that changes in the frequency of metronome sequences produces direct frequency entrainment capable of triggering instantaneous gait adjustments.

The critical aspect of rhythmic auditory cueing is the underlying rhythmicity or periodicity that enables auditory-motor interaction and also determines the strength of those interactions.(31) The auditory system is sensitive to time information and builds precise, stable time traces that acts as a motor template and helps individuals sync their movements.(31) The rhythmicity of the cue serves as an anticipatory and continuous external frame of reference. If the beats occur at regular intervals, there is a strong tendency to anticipate the next stimulus. This ‘anticipation tendency’ guides subsequent movement in advance. A stable auditory-motor synchronization is indicated by minimal variability in timing between the movement and the external cue. Steady-state auditory-motor synchronization is shown to occur within 2 to 3 repetitions of exposure to rhythmic metronome beats during finger tapping.(32) This ability of the sensory information from auditory cues helps to establish a stable synchronization pattern rapidly.

Monotonous metronome beats provide periodicity; this is in contrast to musical cues that are rich in other information such as melodicity. Auditory entertainment cues such as musical rhythms during a walking task in healthy young adults(33) and elderly individuals(34) resulted in reduced variability in synchronization compared to a metronome. Acoustic stimuli delivered as rhythmic music resulted in increased stride length and gait velocity compared to no cue stimuli or metronome conditions.(34) There is evidence that music-supported therapy has a beneficial effect in people post-stroke, as it provides an opportunity for the repetitive movement practice.(35)

Auditory-motor synchronization is well reported in the field of musical performance. Musical performance is a complex process that requires fast feedforward and feedback loops to rapidly process auditory, visual, and motor signals while referencing the learned sequence of musical output.(36, 37) Professional musicians are known to display precise auditory-motor synchronization.(36) Imaging studies in skilled musicians show brain activation in response to either auditory stimuli or passive finger movements, suggesting a co-activation phenomenon.(36) Auditory-motor coupling is defined as co-activation of cortical auditory and sensorimotor hand regions in either pure auditory or silent motor tasks.(38) Auditory-motor coupling is suggested to activate similar cortical networks both in trained and naïve musicians within 20 minutes of practice.(39-41) This suggests that music is able to establish a relatively fast coupling effect within a short span of exposure. In addition, professional musicians display involuntary auditory-to-motor (listening to music – purely acoustic) and motor-to-auditory co-activation (silent finger movements – purely motor).(37, 38, 42-46) Similar brain activation is also seen with imagery of sound or music playing motor action (non-acoustic).(47-49) A well-trained pianist showed involuntary, without actual finger movements, activation of the motor cortex (M1)(42), bilateral supplementary motor area (SMA), the primary motor cortex (PMC), anterior cingulate gyrus and parietal cortex(50-52), on listening to piano music, compared to a non-trained musician; this suggests a strong coupling between perception and action. Recent studies showed activation in posterior superior temporal gyrus (pSTS) and ventral premotor cortex during entrainment of motor responses with auditory cues in non-musicians.(53)

Combined visual and auditory cues:

Human visual and auditory systems are tele-receptive senses. This can be defined as a sensory system that receives and processes information from near and distant external environments and defines the origin of these sensations.(54)-(55) Perception of a scene depends on the concordance between the visual and auditory stimuli and should be temporally congruent in order to have a coherent percept, known as the Unity assumption. In other words, the more different sensory signals arise from a single source, the greater likelihood that the inputs would be combined to provide a coherent percept. For instance, women walkers produce a the sound of footsteps that suggest a feminine gender.(56-60) At the single neuron level, multisensory integration is defined as a statistically significant difference between the number of impulses evoked by a cross-modal combination of stimuli and the number evoked by the most effective of these stimuli individually.(61) The temporal congruency of visual and auditory stimuli is necessary for neural potentiation and is reflected in the magnitude of synaptic potential for congruent stimuli.(62) Thus, auditory and visual signals need to be meaningfully linked to each other and be temporally congruent for a coherent perception.

Sound signals can influence the perception of visual stimuli. A study presented variation in footstep sounds to young adults, resulting in an alternation of visual depth-perception of the point-light walker. The study showed that looming sounds paired with orthographic (facing the viewer) point-light walkers appear more looming, and similarly point-light walkers appear more receding (facing away from the viewer) when paired with receding sounds, compared to no-sound and stationary sound conditions.(55) Another study demonstrated decreased reaction time to the presence of a coherent point-light walker when auditory motion travelled in the same direction as the walker and increased reaction time when auditory and point-light walker motions travelled in the opposite direction.(59) Another group of researchers designed a task in which observers had to decide whether a periodically moving point-light walker had the same temporal frequency as a series of auditory beeps, which in some cases coincided with the footsteps of the walker. Performance in this task was consistently better for upright point-light walkers compared to inverted or scrambled walkers.(63) This suggests that individual auditory or visual stimuli can influence the the other simulus’s perception and that combined stimuli improve perception.

Among the various meaningful sounds in the environment, the sound of footsteps has a clear acoustic signal. Walking involves two distinct phases; a stance and swing phase, which are repeated periodically. The stance phase, which consists of heel strike and foot flat events has a distinct acoustic signal, while the swing phase has no acoustic signal.(64, 65) Human bipedal locomotion generates a stable regular footstep rhythm and a periodic motion of arms, trunk, and lower limb during normal gait.(66, 67) Besides, the sound of human footsteps carries a rich pattern of social information such as source - gender(68, 69), emotional state(68) and posture of the walker(70), walking surface(71), sole of footwear(68) as well as the temporal and spatial origin of sounds.(72) Also, footstep sound provides information about a dynamic human group, such as people walking in synchrony or one leading the other in pairs.(73) Given these qualities, the sound of footsteps is termed as footsteps’ acoustic signature in the literature.

Sensorimotor synchronization is a referential behaviour in which a motor action is performed in sync with an external predictable stimulus or event known as the ‘referent’.(74) There are two possible aspects to external cueing; first, the external stimuli are isochronous in time (fixed interstimulus interval) and therefore stable and predictable; second, the interstimulus interval is altered (progressive up or down ramp-like changes or random) and therefore unpredictable. In the variable frequency beats, there is an increase in synchronization error, and thus a deliberate and conscious action on the part of the individual is necessary in order to sustain a stable synchronization state.(75) Synchronized state is said to occur when the response sequence of movement has the same time interval as the external stimulation with no phase deviation.(75) However, during a sensory-motor synchronization, there exists variability in movement performance which necessitates a continuous adjustment to an external rhythm. The error-correction can be brought about by modulating movement in-phase and period duration.(75) Rhythmic auditory stimulation provides an effective timing mechanism, based on individuals’ ability to focus on the rhythm, detect time interval of external rhythm, process time information and consciously integrate it in on-going movement sequence. It also involves the ability to detect errors during synchronization and take corrective measures.(76) For example, walking to a fixed-paced metronome, which is isochronous, involves the ability to predict the subsequent beat, plan movement in anticipation of the beat, and closely match movement to external beat pace. In case of asynchrony or an error, a correction of the next movement is consciously considered in the subsequent movements. Furthermore, sensorimotor synchronization is also dependent on the motor effector. For instance, finger or foot tapping involves muscle force, movement amplitude, and degrees of freedom that are small and limited. In contrast, walking involves large muscle activity with many degrees of freedom that need to be constrained with additional postural and balance demands to match the external rhythm.

Section 2: Brain activation in response to sensory stimuli

The effect of the visual and auditory cue on movement is supported by neuroimaging research that has identified activation of certain brain regions in response to sensory stimulation.(77-89) Supra-spinal regions: Multisensory neurons, specifically in the superior colliculus and cerebral cortex, have a receptive field that needs the stimuli to be congruent and in a close time frame in order to have an accentuated or super-additive effect (non-linear enhancement) that is, stronger than the sum of unimodal response.(3) Perception of congruent audio-visual biological motion (point-light walker and sound of footsteps) stimuli is reflected as activation in the posterior extend of the posterior superior temporal sulcus (pSTS), inferior parietal sulcus (IPS), (77-83) and premotor cortex. (84, 85) A study by Wuerger et al. (2012) demonstrated the role of ventral premotor cortex activation for congruent (same motion direction in auditory and visual modalities) than incongruent (different motion direction) biological auditory (footsteps sound) and visual motion (point-light walker).(86) Alaerts et al. (2009) in healthy adults using auditory (crushing of plastic bottle) and visual (action of crushing) cues, using transcranial magnetic stimulation, showed an increase in motor evoked potentials in the primary motor cortex to congruent audio-visual stimuli compared to unimodal and incongruent actions.(87) In addition to these regions, neurons in the superior colliculus also display a super-additive effect for simultaneous visual and auditory stimuli in animals.(2, 88) There appears to be a continuum in pSTS regions where neurons that respond to vision, auditory, and combined sensory signals(89) are activated in response to action versus non-action related stimulus.(90) Area pSTS is delineated as a robust structure in the perception of biological motion across most human(83) and primate studies.(91) The super-additive effect is also seen for social context and complex visual and auditory signals.(92) This may indicate its role in deriving meaning from the biologically relevant events. The role of pSTS and premotor cortex in biological motion perception is also supported by a study in people with chronic unilateral stroke, which showed that lesions in superior temporal and inferior frontal (premotor) regions play a causal role in biological motion perception (walking, jogging, throwing activities) compared to age-matched controls.(77) The response in the premotor and parietal cortex is enhanced when the observer intends to perform the movement compared to passive observation.(93) There is yet another brain region that could explain the effect of sensory cues on movement: the mirror neuron network.

Mirror Neuron Network (MNN):

A group of visuomotor neurons in the primate ventral premotor cortex (f5) and inferior frontal cortex, called mirror neurons, are activated on observation and execution of goal-directed hand actions like grasping objects as well as listening to action-related sound.(94, 95) In humans, MNN is shown to be present in the ventral premotor cortex, precentral gyrus, posterior part of inferior frontal gyrus, Broca’s area, and intraparietal area, which activates in response to action observation and sound related to actions such as crushing of plastic bottle or ripping of paper.(79, 85, 96-100) This led to the hypothesis that perception-induced activation of movements may obey a ‘whole-or-nothing’ principle(96). For example, the perception of tearing paper evokes the action irrespective of whether it is heard, seen, or both, and is modality-independent(96). An alternate hypothesis is the shared ‘modality-dependent’ action representations, where the mechanism of perception-induced action retrieval is based on the simultaneous input of vision and sound describing the same movement.(101) Neuroimaging suggests that seeing actions activates the frontoparietal neural network, which is also active when performing those same actions(102, 103). There is evidence of motor cortex activation (increased motor evoked potentials) in the observer when viewing an action execution (for example, grasping an object) compared to seeing the object alone or movement alone.(104) The MNN activity occurs in response to action-associated sound, and this area is more likely to respond to ecological stimuli, such as the sound of footsteps than the metronome. The MNN is also hypothesized to be the site for self-agency, defined as a feeling of being in control and being the author of one’s movement.(105) The network is implicated in its ability to differentiate voluntary induced self-motion from external action in environment.(105),(106) Lastly, observing someone’s actions allows one to predict the possible movement pattern that one would have to generate in order to achieve a similar end outcome, which in other words, provides a first-person grasp of the motor goals(106). From the ideomotor perspective, the neural representation of action generated through observation is similar to that required to execute the same action.(107, 108) Moreover, mirror neurons and observation of gait share a similar neural substrate as motor execution, including the premotor cortex, SMA, basal ganglia, and cerebellum.(109-111) This suggests that sensorimotor tasks based on action observation, imitation, or execution involve complex activation patterns and goes beyond the activation of the cortex and subcortical structures. Therefore, by a mere act of observing an action, it is possible to bring about compatible effects in subsequent action execution.

The cerebellum plays an important role in motion detection and has been recently highlighted in neuroimaging studies. The rhythmic motor synchronization of finger tapping to progressively increased external rhythm shows activation in cerebellar regions (anterior cerebellar lobe, thalamus, cingulate area) and cerebral (intraparietal sulcus, lateral prefrontal, and bilateral dorsolateral prefrontal) regions in a positron emission tomography (PET) study(112). Also, finger tapping to random sequence rhythmic cues showed activation in parieto-thalamic and premotor activity.(112) This suggests that these cerebro-cerebellar connections play a possible role during rhythmic motor synchronization. Lastly, the cerebellum also has a connection to pSTS and possibly plays a role in motion perception.(113)

Section 3: Unintentional synchronization (spontaneous or uninstructed)

The studies that used external rhythmic auditory cueing strategy during walking tasks asked the participants to match their heel-strike to auditory cues as precisely as they could, thereby consciously engaging them to synchronize movement to the auditory stimuli.(21, 28, 30) As opposed to this conscious synchronization, there is literature supporting unintentional imitation in humans that is exhibited in daily life and thought to be necessary for social interaction and interpersonal communication.(114, 115) The unintentional imitation, mimicry or a sense of similarity occurs automatically without any intention or awareness.(116, 117) During an unintentional imitation of an action, the consequence of execution or feedback is not registered. Studies have shown that it is possible to influence the task performance during an unintentionally synchronized movement (finger tapping) by changing external rhythm (auditory cue that was modulated at 3%, 7%, and 20% of the interstimulus interval from baseline).(118, 119) In order for unintentional synchronization to remain below the level of conscious perception, the modulation of the external rhythm needs to be subtle and progressive, or else the modulated stimulus reaches a conscious level when the motor adaptations switch to the active response mode.(118) Neuroimaging using PET showed a switch in the activation areas from the ventral medial prefrontal cortex at 3% and dorsolateral prefrontal cortex at 20% modulation (when conscious perception occurs).(118) Similarly, a study by Oullier et al. (2008) involving spontaneous synchronization between partners during rhythmic finger movements showed that the movements become unintentionally coupled as soon as the visual information is available and that the synchronization persists when the visual input of the other’s movements is occluded.(120) Thus, subtle modulation in external sensory cues not only influences the ongoing task, but the effect continues even after the external cue is removed. Similarly, several other studies have demonstrated subconscious modulation in motor sequence. A visual-motor conflict study presented participants with their own full-body real-time images as an avatar. The avatar was programmed to deviate during a goal-directed locomotion task. This manipulation showed that the individuals induce compensatory correction without conscious perception when the deviation was within 10-15° but beyond this threshold, conscious correction kicked in.(105, 121) Another study using auditory cues manipulated footsteps and footsteps-related sound by introducing a temporal delay in real-time and demonstrated that the stepping correction occurred without conscious awareness with delays less than 120 milliseconds.(122) Furthermore, it has been suggested that energy cost may play a role in unintentional entrainment.(123) The subconscious negotiation of a shared stepping pattern between partners was demonstrated during a side-by-side treadmill walking task in healthy adults. This supports unintentional interpersonal synchronization in the absence of conscious effort compared to forced entrainment; however, the effect was transient and weak.(123) A similar effect was demonstrated during an over-ground walking task involving pairs that walked side-by-side while holding hands (tactile), visual (presence of another person), and auditory (sound of heel strike).(124, 125) Thus, external cue appears to affect one’s behaviour without conscious perception. Unintentional synchronization shows activation in the orbitofrontal, ventrolateral, ventral prefrontal cortex, and lateral cerebellar hemispheres; whereas conscious adaptation activates the dorsolateral prefrontal cortex, anterior cingulate gyrus and premotor cortex(118), and these sites share some common regions with the mirror network.(126)

Conclusion

This review summarised both the theoretical and brain mechanisms that underlie the effects of visual and auditory cues separately and when presented together. The use of combined visual and auditory cues that are biological in nature as well as unintentional synchronization may have a potential application in the field of rehabilitation.

References

1. Baumann, O. & Greenlee, M. W. Neural correlates of coherent audiovisual motion perception. Cerebral cortex (New York, N.Y. : 1991) 17, 1433-1443, doi:10.1093/cercor/bhl055 (2007).
2. Meredith, M. A. & Stein, B. E. Interactions among Converging Sensory Inputs in the Superior Colliculus. Science 221, 389-391, doi:10.2307/1691747 (1983).
3. Stein, B. E. & Stanford, T. R. Multisensory integration: current issues from the perspective of the single neuron. Nat Rev Neurosci 9, 255-266, doi:10.1038/nrn2331 (2008).
4. Perrott, D. R., Saberi, K., Brown, K. & Strybel, T. Z. Auditory psychomotor coordination and visual search performance. Perception & psychophysics 48, 214-226 (1990).
5. Hughes, H. C., Reuter-Lorenz, P. A., Nozawa, G. & Fendrich, R. Visual-auditory interactions in sensorimotor processing: saccades versus manual responses. Journal of experimental psychology. Human perception and performance 20, 131-153 (1994).
6. O'Hare, J. J. Perceptual integration. Journal of the Washington Academy of Sciences (1991).
7. Rossignol, S. Visuomotor regulation of locomotion. Canadian journal of physiology and pharmacology 74, 418-425 (1996).
8. Gibson, J. J. Reprinted from The British Journal of Psychology (1958), 49, 182-194: Visually controlled locomotion and visual orientation in animals. British journal of psychology (London, England : 1953) 100, 259-271, doi:10.1348/000712608x336077 (2009).
9. Patla, A. E. Understanding the roles of vision in the control of human locomotion. Gait & Posture 5, 54-69 (1997).
10. Patla, A. E. How is human gait controlled by vision. Ecological Psychology 10, 287-302 (1998).
11. Johansson, G. Visual perception of biological motion and a model for its analysis. Perception & psychophysics 14, 201-211, doi:10.3758/BF03212378 (1973).
12. Beauchamp, M. S., Lee, K. E., Haxby, J. V. & Martin, A. Parallel visual motion processing streams for manipulable objects and human movements. Neuron 34, 149-159 (2002).
13. Jacobs, A. & Shiffrar, M. Walking perception by walking observers. Journal of experimental psychology. Human perception and performance 31, 157-169, doi:10.1037/0096-1523.31.1.157 (2005).
14. Lim, I. et al. Effects of external rhythmical cueing on gait in patients with Parkinson's disease: a systematic review. Clin Rehabil 19, 695-713 (2005).
15. Hollands, K. L. et al. Visual cue training to improve walking and turning after stroke: a study protocol for a multi-centre, single blind randomised pilot trial. Trials 14, 276, doi:10.1186/1745-6215-14-276 (2013).
16. Heeren, A. et al. Step by step: a proof of concept study of C-Mill gait adaptability training in the chronic phase after stroke. Journal of rehabilitation medicine : official journal of the UEMS European Board of Physical and Rehabilitation Medicine 45, 616-622, doi:10.2340/16501977-1180 (2013).
17. Bank, P. J., Roerdink, M. & Peper, C. E. Comparing the efficacy of metronome beeps and stepping stones to adjust gait: steps to follow! Exp Brain Res 209, 159-169, doi:10.1007/s00221-010-2531-9 (2011).
18. Spidalieri, G., Busby, L. & Lamarre, Y. Fast ballistic arm movements triggered by visual, auditory, and somesthetic stimuli in the monkey. II. Effects of unilateral dentate lesion on discharge of precentral cortical neurons and reaction time. J Neurophysiol 50, 1359-1379 (1983).
19. Chapman, C. E., Spidalieri, G. & Lamarre, Y. Activity of dentate neurons during arm movements triggered by visual, auditory, and somesthetic stimuli in the monkey. J Neurophysiol 55, 203-226 (1986).
20. Thaut, M. H. et al. Rhythmic auditor Y stimulation improves gait more than NDT/Bobath training in near-ambulatory patients early poststroke: a single-blind, randomized trial. Neurorehabilitation & Neural Repair 21, 455-459 (2007).
21. Prassas, S., Thaut, M., McIntosh, G. & Rice, R. Effect of auditory rhythmic cuing on gait kinematic parameters of stroke patients. Gait and Posture 6, 218-223 (1997).
22. Soon Hyun, L., Kyoung Jin, L. & Chang Ho, S. Effects of Rhythmic Auditory Stimulation (RAS) on Gait Ability and Symmetry after Stroke. Journal of Physical Therapy Science 24, 311-314 (2012).
23. Bradt, J., Magee, W. L., Dileo, C., Wheeler, B. L. & McGilloway, E. Music therapy for acquired brain injury. Cochrane database of systematic reviews (Online) 7, CD006787 (2010).
24. Suteerawattananon, M., Morris, G. S., Etnyre, B. R., Jankovic, J. & Protas, E. J. Effects of visual and auditory cues on gait in individuals with Parkinson's disease. J Neurol Sci 219, 63-69, doi:10.1016/j.jns.2003.12.007 (2004).
25. Leman, M. et al. Activating and relaxing music entrains the speed of beat synchronized walking. PLoS One 8, e67932, doi:10.1371/journal.pone.0067932 (2013).
26. Conklyn, D. et al. A home-based walking program using rhythmic auditory stimulation improves gait performance in patients with multiple sclerosis: a pilot study. Neurorehabil Neural Repair 24, 835-842, doi:10.1177/1545968310372139 (2010).
27. Powell, W., Stevens, B., Hand, S. & Simmonds, M. Sounding better: fast audio cues increase walk speed in treadmill-mediated virtual rehabilitation environments. Studies in health technology and informatics 154, 202-207 (2010).
28. Roerdink, M., Lamoth, C. J. C., Kwakkel, G., van Wieringen, P. C. W. & Beek, P. J. Gait coordination after stroke: benefits of acoustically paced treadmill walking. Physical therapy 87, 1009-1022 (2007).
29. Pelton, T. A., Johannsen, L., Huiya, C. & Wing, A. M. Hemiparetic stepping to the beat: asymmetric response to metronome phase shift during treadmill gait. Neurorehabilitation & Neural Repair 24, 428-434 (2010).
30. Roerdink, M. et al. Rhythm perturbations in acoustically paced treadmill walking after stroke. Neurorehabilitation and Neural Repair 23, 668-678 (2009).
31. Thaut, M. H. Rhythm, Music, and the Brain: Scientific Foundations and Clinical Applications. (Routledge, 2005).
32. Thaut, M. & Miller, R. Multiple synchronization strategies in tracking of rhythmic auditory stimulation. Proceedings of the Society for Neuroscience 146 (1994).
33. Styns, F., van Noorden, L., Moelants, D. & Leman, M. Walking on music. Hum Mov Sci 26, 769-785, doi:10.1016/j.humov.2007.07.007 (2007).
34. Wittwer, J. E., Webster, K. E. & Hill, K. Music and metronome cues produce different effects on gait spatiotemporal measures but not gait variability in healthy older adults. Gait Posture 37, 219-222, doi:10.1016/j.gaitpost.2012.07.006 (2013).
35. Rodriguez-Fornells, A. et al. The involvement of audio-motor coupling in the music-supported therapy applied to stroke patients. Annals of the New York Academy of Sciences 1252, 282-293 (2012).
36. Zatorre, R. J., Chen, J. L. & Penhune, V. B. When the brain plays music: auditory-motor interactions in music perception and production. Nature reviews. Neuroscience 8, 547-558, doi:10.1038/nrn2152 (2007).
37. Bangert, M. & Altenmuller, E. O. Mapping perception to action in piano practice: a longitudinal DC-EEG study. BMC neuroscience 4, 26, doi:10.1186/1471-2202-4-26 (2003).
38. Bangert, M., Haeusler, U. & Altenmüller, E. On practice: how the brain connects piano keys and piano sounds. Annals of the New York Academy of Sciences 930, 425-428 (2001).
39. Pascual-Leone, A. et al. Modulation of muscle responses evoked by transcranial magnetic stimulation during the acquisition of new fine motor skills. J Neurophysiol 74, 1037-1045 (1995).
40. Classen, J. et al. Multimodal output mapping of human central motor representation on different spatial scales. The Journal of physiology 512 ( Pt 1), 163-179 (1998).
41. Bangert, M. et al. Shared networks for auditory and motor processing in professional pianists: evidence from fMRI conjunction. Neuroimage 30, 917-926, doi:10.1016/j.neuroimage.2005.10.044 (2006).
42. Haueisen, J. & Knosche, T. R. Involuntary motor activity in pianists evoked by music perception. J Cogn Neurosci 13, 786-792, doi:10.1162/08989290152541449 (2001).
43. Popescu, M., Otsuka, A. & Ioannides, A. A. Dynamics of brain activity in motor and frontal cortical areas during music listening: a magnetoencephalographic study. Neuroimage 21, 1622-1638, doi:10.1016/j.neuroimage.2003.11.002 (2004).
44. Baumann, S. et al. A network for audio-motor coordination in skilled pianists and non-musicians. Brain Res 1161, 65-78, doi:10.1016/j.brainres.2007.05.045 (2007).
45. Hickok, G., Buchsbaum, B., Humphries, C. & Muftuler, T. Auditory-motor interaction revealed by fMRI: speech, music, and working memory in area Spt. J Cogn Neurosci 15, 673-682, doi:10.1162/089892903322307393 (2003).
46. Lotze, M. et al. Comparison of representational maps using functional magnetic resonance imaging and transcranial magnetic stimulation. Clinical neurophysiology : official journal of the International Federation of Clinical Neurophysiology 114, 306-312 (2003).
47. Bunzeck, N., Wuestenberg, T., Lutz, K., Heinze, H. J. & Jancke, L. Scanning silence: mental imagery of complex sounds. Neuroimage 26, 1119-1127, doi:10.1016/j.neuroimage.2005.03.013 (2005).
48. Zatorre, R. J., Halpern, A. R., Perry, D. W., Meyer, E. & Evans, A. C. Hearing in the Mind's Ear: A PET Investigation of Musical Imagery and Perception. J Cogn Neurosci 8, 29-46, doi:10.1162/jocn.1996.8.1.29 (1996).
49. Halpern, A. R. & Zatorre, R. J. When that tune runs through your head: a PET investigation of auditory imagery for familiar melodies. Cerebral cortex (New York, N.Y. : 1991) 9, 697-704 (1999).
50. Roth, M. et al. Possible involvement of primary motor cortex in mentally simulated movement: a functional magnetic resonance imaging study. Neuroreport 7, 1280-1284 (1996).
51. Lotze, M. et al. Activation of cortical and cerebellar motor areas during executed and imagined hand movements: an fMRI study. J Cogn Neurosci 11, 491-501 (1999).
52. Porro, C. A. et al. Primary motor and sensory cortex activation during motor performance and motor imagery: a functional magnetic resonance imaging study. The Journal of neuroscience : the official journal of the Society for Neuroscience 16, 7688-7698 (1996).
53. Chen, J. L., Penhune, V. B. & Zatorre, R. J. The role of auditory and premotor cortex in sensorimotor transformations. Annals of the New York Academy of Sciences 1169, 15-34 (2009).
54. Aglioti, S. M. & Pazzaglia, M. Sounds and scents in (social) action. Trends Cogn Sci 15, 47-55, doi:10.1016/j.tics.2010.12.003 (2011).
55. Schouten, B., Troje, N. F., Vroomen, J. & Verfaillie, K. The effect of looming and receding sounds on the perceived in-depth orientation of depth-ambiguous biological motion figures. PLoS One 6, e14725, doi:10.1371/journal.pone.0014725 (2011).
56. Thomas, J. P. & Shiffrar, M. I can see you better if I can hear you coming: action-consistent sounds facilitate the visual detection of human gait. Journal of vision 10, 14, doi:10.1167/10.12.14 (2010).
57. van der Zwan, R. et al. Gender bending: auditory cues affect visual judgements of gender in biological motion displays. Exp Brain Res 198, 373-382, doi:10.1007/s00221-009-1800-y (2009).
58. Blake, R. & Shiffrar, M. Perception of human motion. Annu Rev Psychol 58, 47-73, doi:10.1146/annurev.psych.57.102904.190152 (2007).
59. Brooks, A. et al. Auditory motion affects visual biological motion processing. Neuropsychologia 45, 523-530, doi:10.1016/j.neuropsychologia.2005.12.012 (2007).
60. Zuniga, E. & Leavitt, L. Analysis of gait: A method of measurement. Biomechanics IV, University Park Press, Baltimore, MD, 85-90 (1974).
61. Meredith, M. A. & Stein, B. E. Interactions among converging sensory inputs in the superior colliculus. Science (1983).
62. Bi, G. & Poo, M. Synaptic modification by correlated activity: Hebb's postulate revisited. Annual review of neuroscience 24, 139-166, doi:10.1146/annurev.neuro.24.1.139 (2001).
63. Saygin, A. P., Driver, J. & de Sa, V. R. In the footsteps of biological motion and multisensory perception: judgments of audiovisual temporal relations are enhanced for upright walkers. Psychological science 19, 469-475, doi:10.1111/j.1467-9280.2008.02111.x (2008).
64. Cutting, J. E. Generation of synthetic male and female walkers through manipulation of a biomechanical invariant. Perception 7, 393-405 (1978).
65. Cutting, J. E., Proffitt, D. R. & Kozlowski, L. T. A biomechanical invariant for gait perception. J Exp Psychol Hum Percept Perform 4, 357-372 (1978).
66. Ekimov, A. & Sabatier, J. M. Rhythm analysis of orthogonal signals from human walking. The Journal of the Acoustical Society of America 129, 1306-1314, doi:10.1121/1.3533694 (2011).
67. Sutherland, D., Kaufman, K. & Moitoza, J. Kinematics of normal human walking. Human walking, 23-44 (1994).
68. Giordano, B. L. & McAdams, S. Material identification of real impact sounds: effects of size variation in steel, glass, wood, and plexiglass plates. J Acoust Soc Am 119, 1171-1181 (2006).
69. Li, X. F., Logan, R. J. & Pastore, R. E. Perception of acoustic source characteristics: walking sounds. The Journal of the Acoustical Society of America 90, 3036-3049 (1991).
70. Pastore, R. E., Flint, J. D., Gaston, J. R. & Solomon, M. J. Auditory event perception: the source-perception loop for posture in human gait. Percept Psychophys 70, 13-29 (2008).
71. Giordano, B. L. et al. Identification of walked-upon materials in auditory, kinesthetic, haptic, and audio-haptic conditions. J Acoust Soc Am 131, 4002-4012, doi:10.1121/1.3699205 (2012).
72. Deouell, L. Y., Heller, A. S., Malach, R., D'Esposito, M. & Knight, R. T. Cerebral responses to change in spatial location of unattended sounds. Neuron 55, 985-996 (2007).
73. Saarela, M. V. & Hari, R. Listening to humans walking together activates the social brain circuitry. Social neuroscience 3, 401-409, doi:10.1080/17470910801897633 (2008).
74. Repp, B. H. Sensorimotor synchronization: a review of the tapping literature. Psychon Bull Rev 12, 969-992 (2005).
75. Thaut, M. H., Miller, R. A. & Schauer, L. M. Multiple synchronization strategies in rhythmic sensorimotor tasks: phase vs period correction. Biological cybernetics 79, 241-250 (1998).
76. Molinari, M., Leggio, M. G., De Martin, M., Cerasa, A. & Thaut, M. Neurobiology of rhythmic motor entrainment. Ann N Y Acad Sci 999, 313-321 (2003).
77. Saygin, A. P. Superior temporal and premotor brain areas necessary for biological motion perception. Brain 130, 2452-2461, doi:10.1093/brain/awm162 (2007).
78. Saygin, A. P., Wilson, S. M., Hagler, D. J., Jr., Bates, E. & Sereno, M. I. Point-light biological motion perception activates human premotor cortex. The Journal of neuroscience : the official journal of the Society for Neuroscience 24, 6181-6188, doi:10.1523/jneurosci.0504-04.2004 (2004).
79. Grezes, J. et al. Does perception of biological motion rely on specific brain regions? Neuroimage 13, 775-785, doi:10.1006/nimg.2000.0740 (2001).
80. Grossman, E. D. & Blake, R. Brain Areas Active during Visual Perception of Biological Motion. Neuron 35, 1167-1175 (2002).
81. Pelphrey, K. A. et al. Brain activity evoked by the perception of human walking: controlling for meaningful coherent motion. The Journal of neuroscience : the official journal of the Society for Neuroscience 23, 6819-6825 (2003).
82. Ptito, M., Faubert, J., Gjedde, A. & Kupers, R. Separate neural pathways for contour and biological-motion cues in motion-defined animal shapes. Neuroimage 19, 246-252 (2003).
83. Puce, A. & Perrett, D. Electrophysiology and brain imaging of biological motion. Philosophical transactions of the Royal Society of London. Series B, Biological sciences 358, 435-445, doi:10.1098/rstb.2002.1221 (2003).
84. Kaplan, J. T. & Iacoboni, M. Multimodal action representation in human left ventral premotor cortex. Cognitive processing 8, 103-113, doi:10.1007/s10339-007-0165-z (2007).
85. Schubotz, R. I. & Von Cramon, D. Y. Sequences of abstract nonbiological stimuli share ventral premotor cortex with action observation and imagery. The Journal of Neuroscience 24, 5467-5474 (2004).
86. Wuerger, S. M. et al. Premotor cortex is sensitive to auditory-visual congruence for biological motion. J Cogn Neurosci 24, 575-587, doi:10.1162/jocn_a_00173 (2012).
87. Alaerts, K., Swinnen, S. P. & Wenderoth, N. Interaction of sound and sight during action perception: evidence for shared modality-dependent action representations. Neuropsychologia 47, 2593-2599, doi:10.1016/j.neuropsychologia.2009.05.006 (2009).
88. Meredith, M. A. & Stein, B. E. Spatial factors determine the activity of multisensory neurons in cat superior colliculus. Brain Res 365, 350-354 (1986).
89. Beauchamp, M. S., Argall, B. D., Bodurka, J., Duyn, J. H. & Martin, A. Unraveling multisensory integration: patchy organization within human STS multisensory cortex. Nat Neurosci 7, 1190-1192, doi:10.1038/nn1333 (2004).
90. Pizzamiglio, L. et al. Separate neural systems for processing action- or non-action-related sounds. Neuroimage 24, 852-861, doi:10.1016/j.neuroimage.2004.09.025 (2005).
91. Oram, M. W. & Perrett, D. I. Integration of form and motion in the anterior superior temporal polysensory area (STPa) of the macaque monkey. J Neurophysiol 76, 109-129 (1996).
92. Bushara, K. O. et al. Neural correlates of cross-modal binding. Nat Neurosci 6, 190-195, doi:10.1038/nn993 (2003).
93. Grezes, J., Costes, N. & Decety, J. The effects of learning and intention on the neural network involved in the perception of meaningless actions. Brain 122 ( Pt 10), 1875-1887 (1999).
94. Rizzolatti, G., Fadiga, L., Gallese, V. & Fogassi, L. Premotor cortex and the recognition of motor actions. Brain Res Cogn Brain Res 3, 131-141 (1996).
95. Gallese, V., Fadiga, L., Fogassi, L. & Rizzolatti, G. Action recognition in the premotor cortex. Brain 119 ( Pt 2), 593-609 (1996).
96. Kohler, E. et al. Hearing sounds, understanding actions: action representation in mirror neurons. Science 297, 846-848, doi:10.1126/science.1070311 (2002).
97. Rizzolatti, G. & Craighero, L. The mirror-neuron system. Annual review of neuroscience 27, 169-192, doi:10.1146/annurev.neuro.27.070203.144230 (2004).
98. Lahav, A., Saltzman, E. & Schlaug, G. Action representation of sound: audiomotor recognition network while listening to newly acquired actions. The Journal of neuroscience : the official journal of the Society for Neuroscience 27, 308-314, doi:10.1523/jneurosci.4822-06.2007 (2007).
99. Iacoboni, M. et al. Grasping the intentions of others with one's own mirror neuron system. PLoS biology 3, e79 (2005).
100. Aziz‐Zadeh, L., Iacoboni, M., Zaidel, E., Wilson, S. & Mazziotta, J. Left hemisphere motor facilitation in response to manual action sounds. European Journal of Neuroscience 19, 2609-2612 (2004).
101. Keysers, C. et al. Audiovisual mirror neurons and action recognition. Exp Brain Res 153, 628-636, doi:10.1007/s00221-003-1603-5 (2003).
102. Rizzolatti, G. & Sinigaglia, C. The functional role of the parieto-frontal mirror circuit: interpretations and misinterpretations. Nature Reviews Neuroscience 11, 264-274 (2010).
103. Iacoboni, M. Imitation, empathy, and mirror neurons. Annual review of psychology 60, 653-670 (2009).
104. Fadiga, L., Fogassi, L., Pavesi, G. & Rizzolatti, G. Motor facilitation during action observation: a magnetic stimulation study. J Neurophysiol 73, 2608-2611 (1995).
105. Kannape, O. A. & Blanke, O. Agency, gait and self-consciousness. International journal of psychophysiology : official journal of the International Organization of Psychophysiology 83, 191-199, doi:10.1016/j.ijpsycho.2011.12.006 (2012).
106. Salomon, R., Lim, M., Kannape, O., Llobera, J. & Blanke, O. "Self pop-out": agency enhances self-recognition in visual search. Exp Brain Res 228, 173-181, doi:10.1007/s00221-013-3549-6 (2013).
107. Hommel, B. Action control according to TEC (theory of event coding). Psychological Research PRPF 73, 512-526 (2009).
108. Prinz, W. Perception and action planning. European journal of cognitive psychology 9, 129-154 (1997).
109. Grezes, J. & Decety, J. Does visual perception of object afford action? Evidence from a neuroimaging study. Neuropsychologia 40, 212-222 (2002).
110. Hanakawa, T. et al. Functional properties of brain areas associated with motor execution and imagery. J Neurophysiol 89, 989-1002, doi:10.1152/jn.00132.2002 (2003).
111. Iseki, K., Hanakawa, T., Shinozaki, J., Nankaku, M. & Fukuyama, H. Neural mechanisms involved in mental imagery and observation of gait. Neuroimage 41, 1021-1031, doi:10.1016/j.neuroimage.2008.03.010 (2008).
112. Thaut, M. H. et al. Distinct cortico-cerebellar activations in rhythmic auditory motor synchronization. Cortex 45, 44-53 (2009).
113. Sokolov, A. A. et al. Biological motion processing: the left cerebellum communicates with the right superior temporal sulcus. Neuroimage 59, 2824-2830, doi:10.1016/j.neuroimage.2011.08.039 (2012).
114. Bernieri, F. J., Reznick, J. S. & Rosenthal, R. Synchrony, pseudosynchrony, and dissynchrony: Measuring the entrainment process in mother-infant interactions. Journal of personality and social psychology 54, 243 (1988).
115. Meltzoff, A. N. & Decety, J. What imitation tells us about social cognition: a rapprochement between developmental psychology and cognitive neuroscience. Philosophical Transactions of the Royal Society of London. Series B: Biological Sciences 358, 491-500 (2003).
116. Chartrand, T. L. & Dalton, A. N. Mimicry: Its ubiquity, importance, and functionality. Oxford handbook of human action, 458-483 (2009).
117. Scheflen, A. The significance of posture in communication systems. Communication Theory 293 (2007).
118. Stephan, K. M. et al. Conscious and subconscious sensorimotor synchronization--prefrontal cortex and the influence of awareness. Neuroimage 15, 345-352 (2002).
119. Thaut, M. H. Neural basis of rhythmic timing networks in the human brain. Ann N Y Acad Sci 999, 364-373 (2003).
120. Oullier, O., de Guzman, G. C., Jantzen, K. J., Lagarde, J. & Kelso, J. A. Social coordination dynamics: measuring human bonding. Social neuroscience 3, 178-192, doi:10.1080/17470910701563392 (2008).
121. Kannape, O. A., Schwabe, L., Tadi, T. & Blanke, O. The limits of agency in walking humans. Neuropsychologia 48, 1628-1636, doi:10.1016/j.neuropsychologia.2010.02.005 (2010).
122. Menzer, F. et al. Feeling in control of your footsteps: Conscious gait monitoring and the auditory consequences of footsteps. Cognitive neuroscience 1, 184-192 (2010).
123. Nessler, J. A. & Gilliland, S. J. Interpersonal synchronization during side by side treadmill walking is influenced by leg length differential and altered sensory feedback. Hum Mov Sci 28, 772-785, doi:10.1016/j.humov.2009.04.007 (2009).
124. Zivotofsky, A. Z., Gruendlinger, L. & Hausdorff, J. M. Modality-specific communication enabling gait synchronization during over-ground side-by-side walking. Hum Mov Sci 31, 1268-1285, doi:10.1016/j.humov.2012.01.003 (2012).
125. Zivotofsky, A. Z. & Hausdorff, J. M. The sensory feedback mechanisms enabling couples to walk synchronously: an initial investigation. J Neuroeng Rehabil 4, 28, doi:10.1186/1743-0003-4-28 (2007).
126. Vogeley, K. et al. Neural correlates of first-person perspective as one constituent of human self-consciousness. J Cogn Neurosci 16, 817-827, doi:10.1162/089892904970799 (2004).

This work is licensed under a Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International License.