Narrative Games in Ergodic Media

A paper with Pedro Cardoso at Communication Studies n. 27, vol. 2:

Computational media allow the development of very particular relationships with readers. Their nature allows them to register static information but also complex and contingent behaviours that they are capable to operationalise, thus becoming interactive and immersive. These media exist in a dual state between a surface layer and a subface layer. These two are inextricably connected, with the subface often becoming a black box that can only be peered at through surface effusions that both mediate and isolate it. The procedural layer of the subface can be discovered through a process of virtuosic interpretation that allows readers to form a theory of system, breeding empathy with it, and ultimately, transferring some of its processes to their minds. This paper focuses on how virtuosic interpretation is developed, and how from it stems the development of a unique kind of aesthetic experience. It explores how computational media, through anamorphosis and a dialectics of aporia and epiphany, become narrative games.


A short text on LIA’s new work, “Autumn”.

For a computer artist as LIA, inspiration is not only found within software and computers. The world outside her computer, and nature, have often inspired her, and Autumn, produced at a time when the seasons change and the leaves fall, is a good example of such a case. LIA looks at the world computationally, using drawing as a surveying device to probe for form’s structures, and distil them to their structural essences. These elements are then used as starting points for a process of experimental programming. Experimental not in the sense that new techniques are developed or tested — although they may be, because influences arise from many other fields, and sometimes a mathematical formula or an algorithm can be as poetic as a street paved with fall leaves — but rather experimental as in experimental music, because it is a constructive process that is not planned in advance but that is defined and discovered as it is developed, a process whose goals can only be unearthed through doing. This is how LIA programs morphology, behaviour, and an emergent intricate choreography that spawns the artwork, or better still, that becomes the artwork.

LIA builds systems where everything is in motion, where everything is transient and fleeting, and also illusory. LIA usually works from the simplest, most fundamental, building blocks: points, lines, rotations, translations, etc., with prochronism often building the work’s surface. Forms are not drawn but rather built, from traces of behaviours and from the actions they originated. As such, in Autumn LIA did not actually program autumn leaves, or autumn trees, or what we may also perceive as autumn flowers. She did not draw them or program the piece to draw any such forms. She developed behaviours whose trails may be reminiscent of leaves, trees, or flowers. And what we see are not shapes but chronologies of those movements, vestiges of a dance that once was. We follow behind the action in the work, much as the fallen leaves of autumn follow summer.

S+V+M: Relationships between the Sound, Visual and Movement domains in Interactive Systems

Rodrigo Carvalho’s PhD thesis, S+V+M: Relationships between the Sound, Visual and Movement domains in Interactive Systems, which I had the pleasure of cosupervising, is now available online at the repository of the University of Porto.

This dissertation explores the relations between the Sound, Visual and Movement domains in real-time interactive systems. The targeted systems are dance performances, performing interfaces, installations and immersive spaces, where the three domains are present, and where processes of transformation and translation occur between them in real-time (movement into sound, sound into visuals, movement into visuals, etc.). This research proposes a new framework , S+V+M (Sound+Visual+Movement), that aims to classify interactive artworks based on the flow of data through their system.

This approach provides a high level of abstraction by looking at the inputs and outputs, transformations, and data flows occurring inside the interactive system, which helps to establish common parameters in artworks with different characteristics, so they can be evaluated and discussed under a unified perspective.

This research is split in two interrelated and parallel paths:

— A theoretical path, where a series of interactive artworks were collected and catalogued according to their processes of transformation. Also a series of related classification systems from the literature were analyzed. Using as baseline the collected artworks, the methods and processes from the analyzed classifications systems as the inputs from the practice based research, a new framework (S+V+M) for the classification of interactive artworks was proposed;

— A practice-based research path, focused on the development of artworks exploring the relations between Sound, Visuals, and Movement, and this way giving inputs and shaping the development of the framework. A group of thirteen projects were developed and their documentation is presented, providing insights on techniques, software, hardware and critical analysis for the development of interactive artworks.

Through the S+V+M framework different data-flow patterns and system typologies are highlighted within the interactive systems of the case-studies artworks. The framework offers a high level of abstraction, helping to recognize similarities and establish common parameters between different systems. Providing this way a language for analysis and critique of the interactive artworks, both for creators and experts as well as for the audiences.

The S+V+M framework also proved to be a catalyst for new interactive artworks, since when observing the emerging patterns and relations between domains new research questions and artistic paths emerge, triggering this way new projects with different interactions and Sound, Visual and Movement connections.

Keywords: Sound, Visuals, Movement, Interaction, Transmutability.