3 The Transmutability of Data: Mapping Input Signals to Sounds and Images

A significant theme in many audiovisual software artworks is the transmutability of digital data, as expressed by the mapping of some input data stream into sound and graphics. For these works, the premise that any information can be algorithmically sonified or visualized is the starting point for a conceptual transformation and/or aesthetic experience. Such projects may or may not reveal the origin of their input data in an obvious way, and, indeed, the actual source of the transformed data may not even matter. This proposition is made particularly evident in Data Diaries (2002) by Cory Arcangel, in which the artist has used Apple’s Quicktime movie player to straightforwardly interpret his computer’s entire hard drive as if it were an ordinary movie file.^[9] As Alex Galloway writes in the project’s introductory notes, [Arcangel’s] discovery was this: take a huge data file—in this case his computer’s memory file—and fool Quicktime into thinking it’s a video file. Then press play.^[10] Although Arcangel’s process in Data Diaries posits a near total rejection of artistic craft, the results of his readymade technique nonetheless delineate a pure glitch aesthetic with a colorful and surprisingly musical quality.

Most commonly, the transmutability of data per se is not itself the primary subject of a work, but is rather used as a means to an end, in enabling some data stream of interest to be understood, experienced, or made perceptible in a new way. In such cases, the artist typically gives special attention to the aesthetics (and sometimes the legibility) of the audiovisually rendered information. The software artworks in the Emergent City series by the British artist Stanza are representative of this approach; these projects employ data collected from urban spaces as the basis for generating audiovisual experiences themed around cities. In Datacity (2004), a browser-based Shockwave application, sounds and video are collected in real time from multiple cameras around the city of Bristol, and are then collaged and manipulated to produce a painterly interpretation of the landscape;^[11] in Sensity (2004–2009), measurement signals from a network of wireless environmental sensors, deployed by the artist throughout his neighborhood, are used to generate audiovisual layers in an interactive map display.^[12] The user of both projects is provided with various interfaces that allow further personalization of the audio mix and visual experience. Other artists have developed software art based on audiovisual mappings derived from weather data, network traffic (Carnivore, 2001, by Alex Galloway and the Radical Software Group),^[13] seismic activity (Mori, 1999, by Ken Goldberg et al.),^[14] Ebay user data (The Sound of Ebay, 2008, by Ubermorgen),^[15] topographic data (G-Player, 2004, by Jens Brand),^[16] and casualty statistics from the U.S. military action in Iraq (Hard Data, 2009, by R. Luke DuBois),^[17] to name just a few examples.

The voyeuristic software installation Listening Post (2001), by Mark Hansen and Ben Rubin, produces a particularly moving audiovisual experience by mapping voice sounds and typographic images to text fragments culled in real time from thousands of unrestricted Internet chat rooms, bulletin boards and other public forums. The texts are read (or sung) by a voice synthesizer, and simultaneously displayed across a suspended grid of more than two hundred small electronic screens.^[18] By rendering these otherwise disembodied texts into sounds and animated typography, this project literally gives voice to the unspoken words of thousands of people, placing its viewer at the center of a maelstrom of desires, opinions, chatter, and solicitations collected from around the world.

The design space of data-mapping projects has been humorously summarized in Jim Campbell’s Formula for Computer Art (1996–2003), an animated cartoon diagram, which mischievously implies that the inputs to many data-mapping artworks may be fundamentally arbitrary and thus interchangeable.^[19]