Rodrigo Carvalho’s PhD thesis, S+V+M: Relationships between the Sound, Visual and Movement domains in Interactive Systems, which I had the pleasure of cosupervising, is now available online at the repository of the University of Porto.

This dissertation explores the relations between the Sound, Visual and Movement domains in real-time interactive systems. The targeted systems are dance performances, performing interfaces, installations and immersive spaces, where the three domains are present, and where processes of transformation and translation occur between them in real-time (movement into sound, sound into visuals, movement into visuals, etc.). This research proposes a new framework , S+V+M (Sound+Visual+Movement), that aims to classify interactive artworks based on the flow of data through their system.

This approach provides a high level of abstraction by looking at the inputs and outputs, transformations, and data flows occurring inside the interactive system, which helps to establish common parameters in artworks with different characteristics, so they can be evaluated and discussed under a unified perspective.

This research is split in two interrelated and parallel paths:

— A theoretical path, where a series of interactive artworks were collected and catalogued according to their processes of transformation. Also a series of related classification systems from the literature were analyzed. Using as baseline the collected artworks, the methods and processes from the analyzed classifications systems as the inputs from the practice based research, a new framework (S+V+M) for the classification of interactive artworks was proposed;

— A practice-based research path, focused on the development of artworks exploring the relations between Sound, Visuals, and Movement, and this way giving inputs and shaping the development of the framework. A group of thirteen projects were developed and their documentation is presented, providing insights on techniques, software, hardware and critical analysis for the development of interactive artworks.

Through the S+V+M framework different data-flow patterns and system typologies are highlighted within the interactive systems of the case-studies artworks. The framework offers a high level of abstraction, helping to recognize similarities and establish common parameters between different systems. Providing this way a language for analysis and critique of the interactive artworks, both for creators and experts as well as for the audiences.

The S+V+M framework also proved to be a catalyst for new interactive artworks, since when observing the emerging patterns and relations between domains new research questions and artistic paths emerge, triggering this way new projects with different interactions and Sound, Visual and Movement connections.

Keywords: Sound, Visuals, Movement, Interaction, Transmutability.