Visual Music Systems
Anyone who has put on a head mounted display like the Oculus has probably gotten motion sickness at one point or another. Smart designing however can create movement without motion sickness in virtual reality.
Provided their HMD is working correctly and getting an adequate frame rate, most people aren’t going to get sick just sitting still in virtual reality. The problem comes when the application tries to adjust the view position or direction. Normally when you are moving in the real world you are getting feedback from your body with every movement. Every time you step your muscles gives your brain feedback. Try turning your head left to right. Feels totally normal right? Now imagine that someone came up from behind you and forced your head side to side unexpectedly. You’ve lost control of your head and lost that muscle feedback that comes with being in control. Suddenly the whole world looks like it is moving around you. When you are moving your own body, that constant feedback helps you construct reality as continuous even when you are making abrupt movement and keeps you from getting sick.
Imagine wearing a GoPro and racing down a steep hill and taking a sharp left turn. Now imagine being at home, trying to watch that GoPro video with the crazy down and sudden turns out of nowhere. While racing down that hill in reality might not make you sick, watching that video would be pretty head jerking, nausea inducing. That’s what moving in virtual reality is like. You aren’t getting any muscle feedback and can’t see around the corner so it constantly feels like the world is moving and jerking around you.
Even the Oculus developers have struggled with this. The initial Oculus development kit came with an application that used the mouse to move around in a virtual world. Doing this while in the headset would pretty much get you sick right away. When you moved the mouse your body had no feedback mechanism, so as soon as things shifted your brain had to recalculate and figure out what was happening. It induced both headaches and nausea- the fact that this was sent around as a demo of the headset shows how hard an issue this is to solve.
This is particularly an issue for visual music because the sensation of rhythm comes from the movement of the viewer within the environment—if you want there to be sensation of beats, the viewer need to move. So we had to figure out how to do this effectively . Over the past few years we have found a few things that help prevent motion sickness.
The first thing is to keep the direction changes to within an acceptable range of sensations—meaning no sudden left turns with your GoPro. On its own however, this isn’t enough.
Another trick is to give the viewer a sense of what is coming next by creating some sort of a visual path that lets them anticipate changes in direction. The rate at which direction can change increases as the path is better demarcated. When we initially created the Palmeiri piece, we tried to create the second half of the piece without a path. Viewers were uncomfortable so we sprinkled in some visual markers of where the viewer was going to be going. That got rid of almost all of the viewers’ uncomfortable sensations. Setting the sceneto include clues as to what the viewer will be seeing and where she will be going, allows the viewer’s brain to anticipate and adjust. We have found this to reduce motion sickness within our visual music pieces. A third trick is to point the default view direction at a location in advance of the current position. Imagine that the viewer is sitting with their HMD on, not moving their head at all and we are going to move them through the scene in ways that change direction. One option might be to set the view direction in the direction we are moving, as if a camera is fixed on the dashboard of a car. We tried this at first with disastrous results. For example, if we want to make a sharp left turn, at some point the world would suddenly jerk ninety degrees to the left. Instead of doing that, we point the view direction ahead on the path, so that the view direction starts rotating toward the left long before we enter the turn. The viewer gets the sensation of movement and turns because the view position is changing, but the anticipatory changes in view direction reduce the discomfort.
ece, Toward the end of the Yeah Man piece, the viewer is on a pretty intense roller coaster, with a strong sensation of moving up and down, but the viewers haven’t gotten ill. They are moving along a clear path and the field of view is set to always let them see slightly into the future so they know what is coming. While there are some people who struggle with motion sickness as soon as they put on a headset, for most people giving the brain cues as to what is coming ahead and constructing a patch seems to be enough to eradicate motion sickness.