Composing video game music for Virtual Reality: Comfort versus performance

In this article series for video game composers, Winifred Phillips is depicted in this photo working in her music production studio.

By Winifred Phillips | Contact | Follow

Delighted you’re here!  I’m videogame composer Winifred Phillips, and I’m happy to welcome you back to this four-part article series exploring the role of music in VR games! These articles are based on the presentation I gave at this year’s game Developer’s Conference in San Francisco, entitled Music in Virtual Reality (I’ve included the official description of my talk at this end of this article). If you haven’t read the previous three articles, you’ll find them here:

During my GDC presentation, I focused on three important questions for VR game music composers:

  • Do we compose our music in 3D or 2D?
  • Do we structure our music to be Diegetic or Non-Diegetic?
  • Do we focus our music on enhancing player Comfort or Performance?

In the course of exploring these questions during my GDC presentation, I discussed my work on four of my own VR game projects –the Bebylon: Battle Royale arena combat game from Kite & Lightning, the Dragon Front strategy game from High Voltage Software, the Fail Factory comedy game from Armature Studio, and the Scraper: First Strike shooter/RPG from Labrodex Inc.

In these articles, I’ve been sharing the discussions and conclusions that formed the basis of my talk, including the examples from these four VR game projects.  So now let’s look at the last of our three questions for VR video game composers:

Do we focus our music on enhancing player comfort or performance?

Okay, so we’re now going to spend some time talking about one of the top issues in VR game design – player comfort versus player performance. It’s a tricky problem.  By this time, we’ve probably all heard about VIMS – visually induced motion sickness. Unfortunately, it’s a pretty famous obstacle in VR game design, causing tons of consternation for game audio experts. So, can video game composers help? Can music be used to combat motion sickness?

Actually, yes. The US Center for Disease Control and Prevention recommends the use of music to relieve motion sickness. Music has been shown to appreciably decrease symptoms of nausea for people in moving vehicles.

Going further, according to a study at the Toronto Rehabilitation Institute, music significantly reduces nausea induced by visual stimulus alone. The conclusion is – music can be an awesome treatment for VIMS – but it can’t be just any music.

According to this research, in order for music to effectively ease VIMS, it has to be considered ‘pleasant’ by listeners. Pleasant music is nice, friendly and agreeable. That’s a real problem for an intense futuristic action game like Scraper: First StrikeAn illustration for the Scraper game designed for multiple popular VR platforms, from the article for video game composers by Winifred Phillips (game music composer). – we don’t expect pleasant music when we’re mowing down enemies.  According to studies from both Memory & Cognition Journal and the Journal of Marketing, when the mood of background music is incongruent with the situation in which we hear it, the music can actually impede our ability to absorb information. In the context of a game, that can potentially impact player performance.

When talking over style choices, the project director and I decided that it would be best to concentrate on uplifting heroic music for the Scraper: First Strike game. That style would keep the score feeling positive, but would still allow it to make sense during action sequences. Plus, the positive, inspirational music can potentially alleviate some VIMS symptoms. Here’s an example from the Scraper VR game:

Depiction of the official Fail Factory VR game logo, from the article for video game composers by Winifred Phillips.This technique also came into play in the Fail Factory project. When players are whisked off from one minigame to another, they travel through some twisty tunnels that might induce a touch of VIMS. However, the tunes during these journeys are probably the most pleasant in the game, as they’re designed to feel like really cheery elevator music. Here’s an example of that:

This cheery musical approach can have a palliative effect on the symptoms of VIMS, helping to reduce some nausea that might otherwise be experienced by players in the VR environment.

Conclusion

So now we’ve come to the conclusion of this four-article series, during which we’ve explored some of the issues facing game music composers creating music for virtual reality. As we discussed, 3D audio has had a long and complex history in game development, and the popular emergence of VR has breathed new life into the discipline.

Over the course of these four articles, we’ve talked about ways to combine stereo and positional music into a VR environment. We’ve explored how non-diegetic music can be effectively enfolded into virtual reality, and we’ve looked at ways in which music can help to alleviate the symptoms of visually induced motion sickness without impacting player performance.

I hope you’ll try some of the techniques we’ve discussed in these articles. Each VR game offers its own unique challenges, and music for VR can be a real trial-and-error process – but that’s what makes working in VR so great for video game composers! I hope you’ve enjoyed this four article series, and please feel free to share your thoughts in the comments section below!

 

 


 

Music in Virtual Reality

Illustration of the VR projects featuring music by game composer Winifred Phillips, to be discussed in a talk , presented by Winifred Phillips for video game composers - taking place at the famous Game Developers Conference.This lecture will present ideas for creating a musical score that complements an immersive VR experience. Composer Winifred Phillips will share tips from several of her VR projects. Beginning with a historical overview of positional audio technologies, Phillips will address several important problems facing composers in VR.

Topics will include 3D versus 2D music implementation, and the role of spatialized audio in a musical score for VR. The use of diegetic and non-diegetic music will be explored, including methods that blur the distinction between the two categories.

The discussion will also include an examination of the VIMS phenomenon (Visually Induced Motion Sickness), and the role of music in alleviating its symptoms.  Phillips’ talk will offer techniques for composers and audio directors looking to utilize music in the most advantageous way within a VR project.

Takeaway

Through examples from several VR games, Phillips will provide an analysis of music composition strategies that help music integrate successfully in a VR environment. The talk will include concrete examples and practical advice that audience members can apply to their own games.

Intended Audience

This session will provide composers and audio directors with strategies for designing music for VR. It will include an overview of the history of positional sound and the VIMS problem (useful knowledge for designers.)

The talk will be approachable for all levels (advanced composers may better appreciate the specific composition techniques discussed).

 

 

Photo of video game composer Winifred Phillips in her music production studio.Winifred Phillips is an award-winning video game music composer whose most recent projects are the triple-A first person shooter Homefront: The Revolution and the Dragon Front VR game for Oculus Rift. Her credits include games in five of the most famous and popular franchises in gaming: Assassin’s Creed, LittleBigPlanet, Total War, God of War, and The Sims. She is the author of the award-winning bestseller A COMPOSER’S GUIDE TO GAME MUSIC, published by the MIT Press. As a VR game music expert, she writes frequently on the future of music in virtual reality games. Follow her on Twitter @winphillips.

Composing video game music for Virtual Reality: The role of music in VR

In this article for video game composers, Winifred Phillips is pictured working in her music production studio.

By Winifred Phillips | Contact | Follow

Hey everybody!  I’m video game composer Winifred Phillips.  At this year’s Game Developers Conference in San Francisco, I was pleased to give a presentation entitled Music in Virtual Reality (I’ve included the official description of my talk at the end of this article). While I’ve enjoyed discussing the role of music in virtual reality in previous articles that I’ve posted here, the talk I gave at GDC gave me the opportunity to pull a lot of those ideas together and present a more concentrated exploration of the practice of music composition for VR games.  It occurred to me that such a focused discussion might be interesting to share in this forum as well. So, with that in mind, I’m excited to begin a four-part article series based on my GDC 2018 presentation!

Continue reading

Video Game Composers: The Art of Music in Virtual Reality (GDC 2018)

Video game composer Winifred Phillips, pictured in her music production studio.

 

By Winifred Phillips | Contact | Follow

Once again, the Game Developers Conference is almost upon us!  GDC 2018 promises to be an awesome event, chock full of great opportunities for us to learn and grow as video game music composers.  I always look forward to the comprehensive sessions on offer in the popular GDC audio track, and for the past few years I’ve been honored to be selected as a GDC speaker.  Last year I presented a talk that explored how I built suspense and tension through music I composed for such games as God of War and Homefront: The Revolution.  This year, I’m tremendously excited that I’ll be presenting the talk, “Music in Virtual Reality.” The subject matter is very close to my heart!  Throughout 2016 and 2017, I’ve composed music for many virtual reality projects, some of which have hit retail over the past year, and some of which will be released very soon.  I’ve learned a lot about the process of composing music for a VR experience, and I’ve given a lot of thought to what makes music for VR unique.  During my GDC talk in March, I’ll be taking my audience through my experiences composing music for four very different VR games –the Bebylon: Battle Royale arena combat game from Kite & Lightning, the Dragon Front strategy game from High Voltage Software, the Fail Factory comedy game from Armature Studio, and the Scraper: First Strike Shooter/RPG from Labrodex Inc.  I’ll talk about some of the top problems that came up, the solutions that were tried, and the lessons that were learned.  Virtual Reality is a brave new world for game music composers, and there will be a lot of ground for me to cover in my presentation!

In preparing my talk for GDC, I kept my focus squarely on composition techniques for VR music creation, while making sure to supply an overview of the technologies that would help place these techniques in context.  With these considerations in mind, I had to prioritize the information I intended to offer, and some interesting topics simply wouldn’t fit within the time constraints of my GDC presentation.  With that in mind, I thought it would be worthwhile to include some of these extra materials in a couple of articles that would precede my talk in March.  In this article, I’ll explore some theoretical ideas from experts in the field of VR, and I’ll include some of my own musings about creative directions we might pursue with VR music composition.  In the next article, I’ll talk about some practical considerations relating to the technology of VR music.

Continue reading