This repository contains the computational material used for a visual essay series inspired by the sonic universe. While embracing static mediums, it assembles a collection of artworks of distinct interpretations of data-matrix, a composition from the audiovisual artist Ryoji Ikeda.

Selected visual essays

Despite the use of mostly deterministic systems based on the analysis of sound components such as amplitude and frequency, the generated outputs link up as a series of abstract and nonconventional portraits that link the fields of music, image, computing and mathematics. It seeks to broaden imagination and provide guidelines for future work. This is the result of an academic project developed for the Computational Creativity for Design course unit of the master's degree in Design and Multimedia (Faculty of Sciences and Technology of the University of Coimbra).

./src/* 
  ├─ src/sketches/* - # code for generated outputs
  │  ├─ S1 - # outputs 1, 4, 5, 7
  │  ├─ S2 - # output 9
  │  ├─ S3 - # output 14
  │  ├─ S4 - # outputs 16, 17
  │  ├─ S5 - # output 10, 11, 27
  │  ├─ S6 - # outputs 12, 13
  │  ├─ S7 - # output 8
  │  ├─ S8 - # outputs 2, 3
  │  ├─ S9 - # output 6
  │  ├─ S10 - # output 15
  │  ├─ S11 - # output 22
  │  ├─ S12 - # output 21
  │  ├─ S13 - # outputs 18, 19, 20
  │  ├─ S14 - # output 23
  │  ├─ S15 - # output 24
  │  ├─ S16 - # output 25
  │  ├─ S17 - # outputs not selected 
  │  ├─ S18 - # output 26
  │  ├─ S19 - # outputs not selected
  │  ├─ S20 - # output 28
  │  ├─ S21 - # output 29
  │  ├─ S22 - # outputs not selected
  │  └─ S23 - # outputs not selected
  └─  src/visual essays/* - # generated outputs
     ├─ selected/* - # selected outputs images
     └─ not selected/* - # not selected outputs images

We started by writing a list of possible systems and sketches to be developed. Before starting to implement them, we had to standardise the artworks proportion, analyse the mentioned list and sort it out. We used the following criteria:

1. Type (sequential or global approach);
2. Similarity with its precedent;
3. Time-consuming level.

The goal was to have printable outputs on a short term so they could guide the remaining work. This ordering also ended up by defining the compendium structure as we organised the artworks by different labels or types, consequently respecting and displaying the chronological procedures.

In the first stage of implementation we focused on finding patterns originated by the composition structure itself. To make them visible, different grids composed of distinct glyphs were tested. By analysing the sound more or less regularly it was possible to adjust the number of elements in the artwork. A sequential approach also has the ability to show the flow of the composition — how the frequencies values vary from start to finish and where the volume peaks are. Some of the generated and selected results display this more effectively than others. We entitled these sketches sequential.

When we looked into the composition from a general standpoint, we envisioned new generation systems that use sound components to create abstract outputs. We considered this type of methods and level of abstraction to be our boundary since we did not intend the outputs to be on the verge of total ambiguity. We called this approach global.

From within all the generated output and coding errors we selected the most successful and discarded the lesser ones (mostly all 3D variants). We realised that, despite following the created list, some of the best artworks came out from analysing the work we were developing and redesigning it.

For the last stage we gathered our selection and started building the compendium. We created a symbol system that works like an index to identify and organise the outputs and we used grey paper to distinguish the book sections. After including an introduction we submitted the document for printing tests, followed by its final versions.

The compendium

• Ash, M. (2005). Simulation and visualization of a 3d fluid. Université d'Orléans in Orléans. link
• Baecker, R. (2021). floating codes - work in progress. link
• Basquin, K. (2020). Mary ellen bute: Pioneer animator. Personal and professional life of Mary Ellen Bute. EUA, Indiana University Press. link
• Bonnett, J. (2000). Design by Numbers (John Maeda). A pioneering graphic designer shows how to use the computer as an artistic medium in its own right. link
• Gardiner, I. (2007). Music, Sound, and the Moving Image. Liverpool University Press. link
• Ikeda, R. (2006). damatics. link
• Jewanski et al. (2020). Audiovisual hallucinations in the synesthetic films of Jordan Belson and James Whitney. Music and Synesthesia, p. 150. Universität Wien. link
• Lab, S. (2017). Ted g ̈ardestad. link
• Moritz, W. (2019). John Whitney: Technological visionary, animation pioneer. Co- Editors: Jillian Coorey, Andre Murnieks, Heather Shaw and Rebecca Tegtmeyer, p. 56. link
• Mrugala, D. (2018). Sound of space. link
• Synthestruct. (2021). Vast. link
• Rush, D. (2019). Dasha rush presents: Aurora cerebralis. link
• Sa, A. (2016). A Perceptual Approach to Audio-Visual Instrument Design, Composition and Performance. Goldsmiths, University of London. link
• Widmer, G. and Zanon. P. (2004). Automatic recognition of famous artists by machine. Recognising famous musicians (concert pianists), based on their style of playing. Johannes Kepler University Linz. link
• Zheng, R. (2022). Look/hear. link

A huge thanks to Manuel Lages for the careful hand-made bookbinding and to the teaching staff for all the advice and availability throughout every stage of this project — Tiago Martins, Sérgio Rebelo, João Cunha, Penousal Machado and Pedro Silva.