Evaluating media equality in generative audiovisual composition using modality constraint and replay-based reveal

Abstract

Coupled audiovisual instruments, in which sound and image are produced from a shared generative process, promise media equality by design. But does causal coupling actually deliver perceptual parity? This article asks how composers experience media equality and intermedial interference when working with such an instrument. We report a practice-based study of the Cellular Au-Tonnetz, a system in which audio and LED light are parallel projections of the same evolving cellular-automaton state. Nine participants composed short pieces under three modality conditions: visual-only, audio-only and audiovisual. Each was followed by replay with both modalities revealed. By withholding one medium during composition and reinstating it at replay, the protocol makes the construction of audiovisual experience observable. Audiovisual access was preferred overall (eight of nine participants), but the audio-only condition most often produced the largest perceived mismatch (six of nine), and replay frequently turned apparent mismatch into coherent re-reading. Mismatch typically took the form of cue conflict: temporal persistence, density correspondence and perceptual masking. Productive interference emerged at replay through rebinding. We position this as foundational practice-based work, offering a method other practitioners can adopt, and design priorities for instruments where sound and image share a single generative substrate.