DAC Home   Real-Time Sound/Image Manipulation and Mapping in a Performance Setting
Thomas Ciufo, Brown University
<back to
"So strongly do these color phenomena appeal to me that I venture to predict that in the not very distant future there may be a color art analogous to the art of music--a color music." 
Albert Michelson, Light Waves and Their Uses.  1899

Although the idea of combining sound and image is not new, recent developments in computing have made this idea more tangible.  Real-time image and sound processing systems enable artists to affect sound and image generation, playback, and processing, based on expressive performance attributes. One unique potential of these new performance works is that of direct sound to image and image to sound mapping, making sound and image structurally integrated. To achieve this integration, specific aspects of the audio can be analyzed and used to directly control the manipulation of specific aspects of the image, and visa versa.

The real efficacy of this medium comes from combining the strength of interaction, real-time processing and sound/image linking and mapping into a singular work that exploits intermodal forms of expression. In this paper/demonstration, I will present my research on real-time sound/image mapping and processing, and address several of the primary concerns encountered in these types of works. A brief history of the field will be outlined and related fields that use sound and image will be contrasted with real-time sound/image works. I will discuss and demonstrate various aspects of processing, manipulation, and most importantly, how these processes can be controlled, linked and mapped. Image/sound composition has great potential and could be utilized in many different types of creative works. My recent compositions (including Incantation, for Native American flute and video) serve as a laboratory for exploring these ideas.

DAC2001 Home