S+V+M: Relationships between the Sound, Visual and Movement domains in Interactive Systems

Rodrigo Carvalho’s PhD thesis, S+V+M: Relationships between the Sound, Visual and Movement domains in Interactive Systems, which I had the pleasure of cosupervising, is now available online at the repository of the University of Porto.

This dissertation explores the relations between the Sound, Visual and Movement domains in real-time interactive systems. The targeted systems are dance performances, performing interfaces, installations and immersive spaces, where the three domains are present, and where processes of transformation and translation occur between them in real-time (movement into sound, sound into visuals, movement into visuals, etc.). This research proposes a new framework , S+V+M (Sound+Visual+Movement), that aims to classify interactive artworks based on the flow of data through their system.

This approach provides a high level of abstraction by looking at the inputs and outputs, transformations, and data flows occurring inside the interactive system, which helps to establish common parameters in artworks with different characteristics, so they can be evaluated and discussed under a unified perspective.

This research is split in two interrelated and parallel paths:

— A theoretical path, where a series of interactive artworks were collected and catalogued according to their processes of transformation. Also a series of related classification systems from the literature were analyzed. Using as baseline the collected artworks, the methods and processes from the analyzed classifications systems as the inputs from the practice based research, a new framework (S+V+M) for the classification of interactive artworks was proposed;

— A practice-based research path, focused on the development of artworks exploring the relations between Sound, Visuals, and Movement, and this way giving inputs and shaping the development of the framework. A group of thirteen projects were developed and their documentation is presented, providing insights on techniques, software, hardware and critical analysis for the development of interactive artworks.

Through the S+V+M framework different data-flow patterns and system typologies are highlighted within the interactive systems of the case-studies artworks. The framework offers a high level of abstraction, helping to recognize similarities and establish common parameters between different systems. Providing this way a language for analysis and critique of the interactive artworks, both for creators and experts as well as for the audiences.

The S+V+M framework also proved to be a catalyst for new interactive artworks, since when observing the emerging patterns and relations between domains new research questions and artistic paths emerge, triggering this way new projects with different interactions and Sound, Visual and Movement connections.

Keywords: Sound, Visuals, Movement, Interaction, Transmutability.

Anotações Sonoras: Espaço, Pausa, Repetição

Anotações Sonoras: Espaço, Pausa, Repetição on Vimeo.

“Anotações Sonoras: Espaço, Pausa, Repetição” was commissioned by oMuseu and Exhibitions – Faculty of Fine Arts, University of Porto, where it was shown from March 24 to June 30 2018. The installation included sound objects by: Alex FX, Alexander Rishaug, André Gonçalves, Andreas Trobollowitsch, Artificial Memory Trace, Arturas Bumšteinas, Attilio Novellino, Carlos Santos, Carlos Zíngaro, Cem Güney, Dan Powell, David Lee Myers, Diana Combo, Durán Vázquez, Emídio Buchinho, Emmanuel Mieville, Enrico Ascoli, Enrico Coniglio, Astrid & Ephraim Wegner, Haarvöl, Freiband, Gintas K, Graeme Truslove, Hugo Paquete, Ifs, James Eck Rippie, Janek Schaefer, Jazznoize, Jim Haynes, Jonathan Uliel Saldanha, Jörg Piringer, Jos Smolders, Lawrence English, Luca Forcucci, Maile Colbert, Marc Behrens, Martijn Tellinga, Mikel R. Nieto, Mise_en_Scene, Monty Adkins, Morten Riis, paL, Ran Slavin, Richard Eigner, Saverio Rosi, Simon Whetham, Sound Meccano, Stephen Vitiello, Sturqen, Tamtam, Tuulikki Bartosik, Ulrich Mitzlaff, Vitor Joaquim, Yiorgis Sakellariou. The wonderful video was shot and directed by Patrícia Viana Almeida.

On Serendipity in the Digital Medium: Towards a Framework for Valuable Unpredictability in Interaction Design

Ricardo Melo’s PhD thesis, On Serendipity in the Digital Medium: Towards a Framework for Valuable Unpredictability in Interaction Design, which I had the pleasure of supervising, is now available online at the repository of the University of Porto.

This research proposes the creation of a methodological framework for the design of interactive systems that allow and encourage valuable and unpredictable—viz serendipitous—encounters in the digital medium, expanding on its ability for the discovery of novel information, for enabling unfamiliar experiences, and supporting creative practices.

The current trajectory of the design of interactive systems aims for increased optimisation and personalisation of digital experiences which, while situationally apropos, when universally adopted hamper the scope and potential of the digital medium, limiting its reach to the systems’ constructed persona of the interactor. Considering the role digital media plays in contemporary life, we believe that by designing towards serendipity, we permit not just optimum interactions in the digital medium but also the wide breadth of human experience. To that end we establish a theoretically-grounded understanding of serendipity, identifying its fundamental concepts and domains. Grounding serendipity in both information discovery and interaction design, we propose a pattern for the serendipitous experience consisting of four stages: Prepared Mind, Trigger, Epiphany, and Value. Furthermore, observing the feasibility of designing for serendipity through an analysis of its accidental nature, we propose a distinction between serendipity that occurs naturally and that which may be designed. Following an exploration of the serendipitous approach of existing interactive systems, we proceed to the proposal of a framework composed by three identified vectors: Human Factors, Human Activities, and Heuristics which, through their articulation and interplay, define both a model for the development of serendipitous systems, as well as the analysis of existing ones.

Through the established understanding of serendipity and the developed framework, we not only aim for enabling the design of serendipitous systems, but also to alert for the necessity and significance of serendipity itself, to both designers and interactors of the digital medium.

Keywords: Serendipity, Digital Medium, Interaction Design, Serendipitous Pattern, Human Factors, Human Activities, Heuristics.