4 New Functions of Sound-Image Coupling in Contemporary Computer Games

Central functions of the interplay between sound and image levels in games, which were specific to the medium, crystallized in interaction with these technical innovations. Sounds animate the fictitious game world and locate the players within the action, the auditive level expands the spatial dimension beyond the limited field of vision. In addition, sounds provide feedback and information about the dynamic situations in the game. The connection between image and sound in conjunction with the actions of the players generates a specific rhythm. Here sound and music open up an additional affective-emotional dimension. Against this background, the link between sound and image in games appears as a complex and dynamic interconnection. Whereas in film there is a fixed soundtrack, with which the link between images and sounds is fully defined, in computer games this is regulated through the programming of the game engine and from then on essentially dependent on various factors within the game situation.

Apart from pre-defined background sounds, sound effects and ambient noises relevant to the situation are dynamically generated by the game action. Although the relationships between visual objects and sounds are determined in the program, control over the time when they are triggered remains in the hands of the players.

Different sounds can be assigned to a single visual object, for example, corresponding to the changing dynamic states of this object. The actions of the visual representation of the adversary in a classical first person shooter, for example, can be signalized by different sounds: movement by the sound of steps, a shot by a bang, or a hit by a scream. The adversary does not necessarily have to remain in the field of vision, as the sound level can convey information relevant to the game about the location and the damage status of the adversary.

Sounds can be attributed not only to actions, but also to objects and locations in the game, their auditive qualities defined by the respective fields of use. Noises attributed to actions or objects often have a signal character that shifts them into the foreground. Sounds attributed to locations, on the other hand, create a background and therefore tend to be designed more atmospherically. These different attributions between image and sound can be described as audiovisual functions that fulfill different tasks related to the game events.