The finding of a multisensory representation of actions in a premotor area of the monkey brain suggests that similar multimodal action-matching mechanisms may also be present in humans. Based on the existence of an audiovisual mirror system, we investigated whether sounds referring to actions that can be performed by the perceiver underlie different processing in the human brain. We recorded multichannel ERPs in a visuoauditory version of the repetition suppression paradigm to study the time course and the locus of the semantic processing of action-related sounds. Results show that the left posterior superior temporal and premotor areas are selectively modulated by action-related sounds; in contrast, the temporal pole is bilaterally modulated by non-action-related sounds. The present data, which support the hypothesis of distinctive action sound processing, may contribute to recent theories about the evolution of human language from a mirror system precursor.