Visual Music Systems

What are the corollaries between visual and audio music?

Blog post by Rose Sebastian

Audio music does not translate directly to visual music, no more than a drum composition can dictate the piano solo. Yet, there are corollaries between many elements of musical composition and visual music. What follows aren’t rules but ideas of how to create visual music.

  • Rhythm: Rhythm is a physical feeling that moves the audio composition. In visual music this feeling is communicated by moving the viewpoint within the 3D environment so as to give the audience the sensation that they are moving through the scene. For a fast rhythm, the viewpoint might shift so that the viewer feels like she is hurtling down a hill or speeding across a flat area. For a slow rhythm, the viewpoint might shift so that she feels like she is inching up a hill or trudging slowly through space. If you’ve seen an OmniMax film where the camera was placed on a roller-coaster, you can appreciate how the movement of the camera translates into strong physical sensations. Visual musicians have control over these impressions, so they can convey subtle rhythms through slight changes in the direction or speed of the movement. As the viewer moves through the 3D world, the motion itself produces a constant influx of new sensations, in the same way that the rhythms are the low level drivers of audio compositions.
  • Melody: Melodies are a sequence of notes moving up and down on a one dimensional axis. In visual music, this is communicated by the movement of objects in 3 dimensions within the environment. While rhythm is conveyed by moving the viewpoint, the melody is the movement of the individual objects within the environment.
  • Harmonics: The meaning of individual musical notes is defined by their harmonic relationship with other notes. Notes are consonant or dissonant depending on the interaction of their audio frequencies. Visual music uses shapes and patterns to create similar sensations. Symmetry has a similar impact to harmony. Just as harmony is more common in music than the ‘real’ world, symmetry has a larger role in visual music.
  • Tone: The tone of an audio instrument is somewhat analogous to the color of a visual instrument. Just as a violin and trumpet can generate different feelings while playing the same notes, changing the colors of objects can create different feelings within the same set of movements and harmonics.
  • Dynamics, , such as fortissimo, crescendo: This is how loudly the piece is played. While audio musicians control the volume of an instrument, visual musicians can control the brightness and pulsing of objects.
  • Articulation, such as legato, staccato: Articulation in music has to do with the transitions between notes and between sounds. Legato notes are played smoothly and connected together. In visual music, this effect is more dramatic as the visual shapes can become increasingly interconnected. Staccato notes are played for a short duration. Visual artists can control whether objects appear as flashes or stay steady and whether moving objects leave trails as pixels decay or shift without a trace.