Visual Music Systems

Inventing Colors in VR: How to Create Colors That Don't Exist in the the Real World

In environments where an artist has total control over the optics, colors can be created which do not exist in normal environments. Blues can be created that are more blue; reds that are more red; and other colors synthesized that lack names. Because the range of sensation exceeds what is possible outside these artificial environments, we call them “hyper-colors”. Visual musicians working in virtual reality can actually create—invent new colors.

Hyper colors are based both on the relativity of color discussed earlier and on the ephemeral nature of color perception . Just as the suddenly-turned-on room light seemed so much brighter than normal, it is possible to perceive colors during transition intervals that do not exist once the retinal chemicals have balanced. The colors that can appear during these transition intervals can provide a palette that is far more stimulating and exciting than the colors we can perceive in the normal environments. If for example the eye is adjusted for dark reds, the sudden appearance of a blue creates a sensation that can be far more “blue” than any shade than can exist in a static environment. Since the perception of colors is also a function of slowly-adapting chemical balances just as for brightness, then it follows that a sudden change in the color balances can produce color sensations that do not exist once the retina has properly balanced.

One of the first differences in working with hyper-colors is that you are working with a dynamic chemical process rather than with a static palette. A color that is exciting when first introduced can lose its magic within a few seconds as the retina adapts to the new environment. The goal is to keep shifting to new combinations, so that we can stay one step ahead of the chemical adaptive process. In these totally controlled virtual reality environments, what you perceive at any moment is a function of the current state of the retinal chemicals rather than any rules about color theory. Because of the differences, artists need to adjust when working with this palette. For example, the tendency is to introduce brighter colors, but they lose their brightness as the eye adapts and the artist can run out of space at the top of the palette. So the visual musician has to develop techniques for ramping down the brightness levels without appearing dull, such as using smaller regions of bright colors or doing things to make the shadow world interesting. Because the hyper-colors are so fleeting and dependent on what a viewer has just seen, the visual musician also needs to be able to control their virtual world instantaneously and to create new colors spontaneously to stay ahead of the retina’s attempts to rebalance and adapt.

Modern virtual reality interfaces like the Oculus have rich possibilities for polychromatic textures and effects. Unfortunately, this system makes creating hyper-colors more challenging. The OVC had only six monochromatic object elements, each with a specified RGB color. Controlling the spectrum in this context was relatively easy. The new immersive system supports large numbers of simultaneous polychromatic textures and objects. While this can create a richer scene at any instant, it can prevent the generation of hyper-colors. The presence of multiple shades tends to create more of an overall chromatic balance, especially if any of the hues are unsaturated, so that an element is stimulating all three types of filters in the eyes. This prevents the retinal chemical imbalances that we manipulate to create hyper-colors.

The creation of hyper colors is also affected by trade offs in how the visual music system itself is designed. If a system is designed to produce complex and interesting scenes, it is often not designed to give artists the precise spectral control they need to create hyper-colors. The OVC used only a small number of chromatically pure elements which the artist could spontaneously control to create new and exciting colors. The new visual music system that we use was designed to create complex polychromatic scenes. While it attained that goal, it has been unable to create some of the color effects from the old system. We are currently working on re-designing some of the control systems in order to give modern visual musicians a chance to also experiment with hyper colors.

As a technical detail, Bill has found that RGB controls are more intuitive for working with hyper-colors than HSI. Artists working in static environments often find HSI to be more intuitive – but the RGB levels seems to match the chemical processes in the retina better. The RGB values are a good proxy for the three color filters in the cones, even though the center poles of the retinal filters are actually at different frequencies than the standard RGB frequencies. If the blues start to get dull in a visual piece because of retinal adaptation, the B level can be reduced to give those chemicals a chance to calm down. And once the eye has adapted to a blue-less world, we have the opportunity to easily create a hyper blue by ramping up the B level in the scene.