Inter-play
Inter-play was an interactive, cross-media installation shown at the UCSB Media Arts and Technology End of Year Show in 2023, by myself, Yuxi Lin, and Alexis Story Crawshaw. Inter-play incorporates computationally generative systems, live performance, as well as non-real-time content, and was conceived as a model for future exhibitions which seek to make conversation amongst works of different media, including new forms of media such as AI generated content. We thought of Inter-play as a semi-improvisational jam session between different forms of media.
System Diagram
Short video of two walls in the installation, containing the slime mold simulation and AI generated video. Sounds generated by a system coded in Max MSP can be heard in the background.
One screen in the installation showed a Processing sketch that I implemented of a slime mold simulation inspired by work by Sage Jenson. The simulation was connected to other elements on the installation and changed based on audience interaction. Installation visitors were able to seed new spores in the simulation by drawing, and the zoomed-in view allowed them to see new regions of migrating black blobs bloom from the path of their sketch. Audio parameters controlled the density of the grid, the strength of the trails left by spores, and could also trigger the death of collections of spores.
The slime mold algorithm in the installation.
The particle system projected in the installation, with the shadow of a participant while they draw on the provided drawing pad. Another module features a particle system impacted by data from the slime mold algorithm and the drawing program. It was created in the 3D modeling and animation program Blender using a generative modeling feature called Geometry Nodes.
Each system in our installation was, in one way or another, sending to and/or receiving data from the others and reinterpreting the data it received to conform to its internal logic. While we had implemented all the other data-interpreting logic for our other systems, we sought to establish a pipeline to a system which would translate incoming information in a manner relatively opaque to us and make conversation with such a system. We turned to Runway’s new Gen-2 AI tool, which generates videos based on text and image prompts. We prompted the tool with images of our existing installation elements, as well as poetic text generated by another AI tool, Chat GPT, when prompted with technical descriptions of our works. For example, we primed Chat GPT to write poetic text, then prompted with the phrase “write a short description of a black and white pixelated slime mold algorithm.” Chat GPT responded “In digital pixels, Black and white slime mold thrives. Paths emerge, wisdom shines.” We then prompted Runway with this text and received a 4 second video, a screen capture of which is shown here.
Still from Runway generated video. Using the workflow between ChatGPT and Runway, we compiled a collection of videos which drew imagery from the other installation elements, combining and re-imagining the data in both novel and recognizable ways. Although the video was the only element of our installation unable to be generated in real time, in the future iterations we plan to incorporate the visuals being shown in real time as data sent to other systems.
Still from Runway generated video.
Still from Runway generated video.
Still from Runway generated video.