Tips, Tools and Guidance: VR Assets for the Game Music Composer

 

Video game music composer Winifred Phillips hard at work on the Audioshield Fitness VR project in her video game music production studio.

By Winifred Phillips | Contact Follow

Welcome!  I’m videogame composer Winifred Phillips, and my projects lately have included music composition for a lot of great virtual reality games.  It’s been fascinating work!  Last year, when my VR work started to really pick up, I wrote an article with lots of resources to help video game music composers become more comfortable in the world of VR audio development.  Since this discipline progresses rapidly, I thought it would be best to post an update article now that adds additional resources to address new developments in the field.

VR development is continuously innovative and cutting-edge, and I’ve been fortunately to experience this first-hand.  As an example: one of my more recent virtual reality game projects was music for Audioshield Fitness, developed by the creator of the famous Audiosurf music-rhythm game. I was asked to compose the new official Audioshield Theme for release with the Audioshield Fitness game, which takes the core game mechanics of Audioshield and pumps up the challenge with obstacles that make players dodge and duck to the music.  The result is an intense workout that was named as one of the top 5 VR Fitness Games of 2018 by PerfectBodyMate.com. To maximize the power of the Audioshield procedural system, my composition had to attune itself to the system’s powerful music analysis algorithm and deliver moments of both challenge and spectacle. I composed and mixed the music with specifically-targeted EQ frequency ranges where I placed rhythmic elements and punchy crescendoes.  The Audioshield music analysis system then reacted to this audio content and changed the pacing and content of gameplay to match these variables.  It was a fun challenge!  Here’s a video showing how that worked:

 

Composing for virtual reality is its own unique discipline, requiring a specialized set of skills and tools.  In this article, let’s collect some resources that explore the techniques, tools, and technologies associated with VR audio development.  Let’s also take a look at the professional community of VR developers that are there to help each other through the rough spots.  Ready?  Let’s go!

Technology and tools

An illustration of famous technology and tools for game composers - section of the article by video game music composer Winifred Phillips.First, let’s consider some of the tools and technology that has been designed to help audio folks create awesome virtual reality sound, starting first with new innovations over the past year:

Google Releases Resonance Audio

Google has entered the VR business in a big way, first with Google Cardboard, then with the Google Daydream device.  Now, Google has released its own multiplatform software development kit for VR audio.  This article sums up some of the features.  Below, I’ve also included a video demonstration of a binaural and ambisonic demo for Resonance Audio:

Let’s Test: 3D Audio Spatialization Plugins

In this article from the Designing Sound site, three spatializer plugins are tested for their performance in localizing sound sources under an assortment of conditions: circular movement, acoustic shadowing, vertical position, occlusion, moving occlusion, modeled attenuation, and near-field effect.  Similarities and differences between the plugins are discussed, and multiple audio samples are provided for comparison. 

MixOnline: The dearVR 3D Audio Reality Engine

This system from the Dear Reality company combines a plugin suitable for integration with Digital Audio Workstations, and a virtual reality application that allows users to instantly test their DAW mixes within a VR environment.  The MixOnline article (linked above) argues that such a system is long overdue and necessary for effective VR audio production workflows.  Below I’ve also included the demo video for this technology, produced by Dear Reality.

 

Envelop for Ableton Live

This suite of spatial sound tools is designed to integrate into the Ableton Live Digital Audio Workstation.  For Ableton Live users, the Envelop suite of tools allows effective mixing for a virtual space.  The article (linked above) discusses many practical applications for this software, including virtual reality. 

Image illustrating the concept of popular technology tools useful to game audio pros, from the article by Winifred Phillips for video game composersBelow, I’m including the links that were offered in last year’s article, since the information is still relevant:

The “Works” 3D Audio plugin for Pro Tools

This article explores the 3D rendering technology of the G’Audio Works plugin, which supports multi-channel, object-based and ambisonic spatialization within the Pro Tools application.

The Steam Audio Software Development Kit

This article focuses on the spatialized audio solution for VR developed by the famous Steam software distribution platform.  Available as a free download without any royalty requirements, Steam Audio is designed to assist both Unity and Unreal developers in creating and implementing spatialized audio in their projects.

Facebook 360 Spatial Workstation

The Two Big Ears audio company is known for its 3Dception software enabling audio folks to author spatialized audio for VR applications.  Now that Facebook has purchased the company, the software has been rebranded as the Facebook 360 Spatial Workstation with added compatibility for 360 videos hosted on the Facebook platform. The software is free for everyone to use in their projects. However, the previous plugin compatibility with Unity, Wwise and FMOD is no longer offered for new users.

Google’s Omnitone, the open source project for spatialized sound in VR

This article describes the Omnitone application, developed by Google to combine ambisonic decoding with binaural rendering.  Omnitone was designed to deliver spatialized audio for browser-based experiences and apps designed for Android and iOS.

NVIDIA VRWorks Audio

For Windows games and applications, NVIDIA now offers the VRWorks Audio Software Development Kit for implementing spatialized audio in VR for 64 bit Windows apps.  For developers working in Unreal Engine 4, the VRWorks Audio game engine plugin can be added directly to the UE4 engine, while future plugins are promised for other game and audio platforms.

 

Methods and techniques

Now, let’s check out some recent articles that offer methods and techniques for creating immersive virtual reality sound:

An image accompanying a discussion of popular methods and techniques for game composers - section from the article by Winifred Phillips, video game music composer.

How to Build Audio for VR Games

In this article, the audio director of CCP Games delves into how human beings perceive sound, and how that extends to VR.  Included: the virtues of hard panning stereo in VR, and the importance of adaptive music in VR. 

Oculus’ Audio SDK: Localization and the Human Auditory System

While this article is primarily a component of the Oculus Audio SDK documentation for VR audio development, it includes a concise and thorough breakdown of the components of sound that enable human beings to localize sound sources.

Immersive Audio for VR Workflow

This is a video recorded at the VR LA conference, in which a panel of game audio experts discusses a variety of topics connected to VR audio development.  Included: an exploration of the ways in which audio pros can lobby for the importance of audio content within a VR development team, and what features would be most welcome in audio production tools to make them more useful in VR development.  Watch here:

Audio Design for Interactive Narrative VR Experiences

In this audio postmortem of the VR project The Price of Freedom, the audio director explores how to prioritize audio implementation tasks during development, and how to effectively place audio elements within VR environments.  Mixing in VR is also discussed. 

9 Things You Should Know When Creating Sound for Virtual Reality

This article published by the GameSoundCon organization lays out several issues to consider when creating and implementing sound content in VR titles.  Among the topics addressed: equalization issues pertaining to HRTF processing, and natural human barriers to sound localization that can be addressed with careful spatialization choices.

An image accompanying a discussion of professional software solutions for VR audio developers, from the article for video game composers by Winifred Phillips (game music composer).As before, I’m including the links that were offered in last year’s article on this topic below, since the articles share a wealth of professional solutions to VR audio problems, and their insights can still be useful:

3D Audio formats for VR

3D Sound Labs takes us through the three most popular sound formats for spatial audio in VR: Multi-channel, Object-based, and Ambisonic.

How 3D Spatialized Audio Bottlenecks Virtual Reality Video

VR audio can consume enormous computational resources, resulting in a resource war between audio and video content.  This article discusses how audio demands in VR consume memory bus bandwidth, and proposes a short-cut that manipulates the frequency response of the audio content to enhance spatialization while consuming fewer resources.

Adapting Your DAW for VR Audio

A VR sound editor discusses his methods and workflow for virtual reality in this article.  Topics include capturing audio in the ambisonic format, and customizing the Pro Tools environment for spatialized audio projects.

Simple Spatial Audio for Beginners

This article summarizes the available entry-level techniques and equipment that can help a newcomer jump into the world of spatial audio for virtual reality.

An audio post production house shares techniques for VR Audio

In this article, we learn about some of the top equipment and methods used by the VR-focused division of an experienced audio post production facility.

 

Communities & Organizations

In this article for video game composers, Winifred Phillips discusses the importance of communities where game audio practitioners can offer advice and seek guidance.

Virtual reality audio development can be daunting.  Fortunately, there are lots of online communities and organizations that can provide advice and encouragement when needed.  I’ve assembled links to some of these below:

An illustration of popular communities and organizations for game composers - section of the article by Winifred Phillips (video game music composer).Here are the communities and organizations that were included in the article from last year:

Conferences

An illustration accompanying the discussion of famous conferences of interest to game composers who want to learn about VR audio - section of the article by video game music composer Winifred Phillips.

Finally, here’s a list of the virtual reality conferences, summits and conventions that have featured audio discussions and demonstrations in their past program schedules:

Here are the conferences that were included in the article from last year:

Conclusion

I hope you find the above resources interesting and helpful!  If you have any suggestions for additions to this list, please let me know in the comments section below!

 

Photo of video game composer Winifred Phillips in her game composers production studio.Popular music from composer Winifred Phillips’ award-winning Assassin’s Creed Liberation score will be performed live by a top 80-piece orchestra and choir as part of the Assassin’s Creed Symphony World Tour, which kicks off in 2019 with its Los Angeles premiere at the famous Dolby Theatre. As an accomplished video game composer, Phillips is best known for composing music for games in five of the most famous and popular franchises in gaming: Assassin’s Creed, LittleBigPlanet, Total War, God of War, and The Sims.  Phillips’ other notable projects include the triple-A first person shooter Homefront: The Revolution, and numerous virtual reality games, including Scraper: First Strike, Dragon Front, and many more.   She is the author of the award-winning bestseller A COMPOSER’S GUIDE TO GAME MUSIC, published by the MIT Press. As a VR game music expert, she writes frequently on the future of music in virtual reality games. Follow her on Twitter @winphillips.

Resources for Video Game Music Composers: The Big List

Video game music composer Winifred Phillips creating music in her video game music production studio.

By Winifred Phillips | Contact | Follow

Hey everybody!  I’m videogame composer Winifred Phillips.  Every year, between working in my studio creating music for some awesome games, I like to take a little time to gather together some of the top online resources and guidance available for newbies in the field of video game music.  What follows in this article is an updated and expanded collection of links on a variety of topics pertinent to our profession.  We begin with the concert tours and events where we can get inspired by seeing game music performed live.  Then we’ll move on to a discussion of online communities that can help us out when we’re trying to solve a problem.  Next, we’ll see a collection of software tools that are commonplace in our field.  Finally, we’ll check out some conferences and academic organizations where we can absorb new ideas and skills.

Continue reading

VR for the Game Music Composer: Audio for VR Platforms

In this article written for video game composers, Winifred Phillips (video game composer) is here pictured working in her music production studio on the music for the Scraper: First Strike game, developed for popular VR gaming platforms (PSVR, Oculus Rift, HTC Vive).

By Winifred Phillips | Contact | Follow

Hello there!  I’m video game music composer Winifred Phillips.  Lately, I’ve been very busy in my production studio composing music for a lot of awesome virtual reality games, including the upcoming Scraper: First Strike first person VR shooter (pictured above) that’s coming out next Wednesday (November 21st) for the Oculus Rift, HTC Vive and Windows Mixed Reality Devices, and will be released on December 18th for the Playstation VR.  My work on this project has definitely stoked my interest in everything VR!  Since the game will be released very soon, here’s a trailer video released by the developers Labrodex Studios, featuring some of the music I composed for the game:

Continue reading

Video Game Music Composers: New VR Headphone Tech (2018)

By Winifred Phillips | Contact | Follow

In this article for and about the craft of video game composers, Winifred Phillips is pictured in this photo from her lecture on Virtual Reality given at the popular Game Developers Conference in 2018.Hey, everyone!  I’m videogame composer Winifred Phillips, and my work has included the musical scores for top games on all sorts of popular gaming platforms, from handhelds and mobile, all the way up to the latest consoles and PCs.  Lately, I’ve been doing a lot of video game music composition for virtual reality.  I had the pleasure of presenting a lecture on Music in Virtual Reality (pictured left) at the most recent Game Developers Conference in San Francisco.

My experience as a composer for VR includes many VR games, including the Scraper: First Strike shooter (set to be released for the PSVR, Oculus Rift and HTC Vive in December 2018), and the recently released VR experience The Haunted Graveyard, which is now available on Steam and in VR Arcades around the world.  Since we’re in the Halloween season, and this VR experience is designed specifically for your Halloween pleasure, here’s a trailer that features my music from The Haunted Graveyard:

By virtue of all the experiences I’ve had recently creating music for VR, I’ve become keenly aware of the importance of sound fidelity in VR.  If the experience doesn’t sound real, it loses the chance to actually feel like a fully-convincing, thoroughly awesome virtual reality experience.  With that in mind, I’ve been writing periodic articles about new technologies in connection with headphones for VR.

Continue reading

VR Headphones Update 2018: Video Game Music Composers

In this article written for video game composers, Winifred Phillips (composer of music for God of War) is here pictured working in her music production studio.

By Winifred Phillips | Contact | Follow

Glad you’re here!  I’m videogame composer Winifred Phillips.  My work as a game music composer has included music for projects released on nearly all of the gaming platforms, from one of my most recent projects (a Homefront game released on all the latest consoles and PCs) to one of my earliest projects (a God of War game released on PlayStation 2, PlayStation 3, and PlayStation Vita, pictured above).  An image of the September 2018 cover of Music Connection Magazine featuring the article "Video Game Composers Speak!" - features interviews of famous game music composers, including popular game music composer Winifred Phillips.You can read about my work as a video game composer in an interview I gave to Music Connection Magazine for this month’s issue (pictured right).

Lately, I’ve also been creating lots of video game music for awesome virtual reality games developed for the Oculus Rift, Oculus Go, HTC Vive, Samsung Gear VR, PlayStation VR, and lots of other top VR platforms.  One of the things I’ve noticed while working in VR is the immense importance of the audio delivery mechanism.

When audio is painstakingly spatialized, it becomes crucial to convey that carefully-crafted spatialization to the player with as little fidelity loss as possible.  With the importance of this issue in mind, for the past few years I’ve been periodically writing about headphones in relation to their use in virtual reality.

Continue reading

Composing video game music for Virtual Reality: Comfort versus performance

In this article series for video game composers, Winifred Phillips is depicted in this photo working in her music production studio.

By Winifred Phillips | Contact | Follow

Delighted you’re here!  I’m videogame composer Winifred Phillips, and I’m happy to welcome you back to this four-part article series exploring the role of music in VR games! These articles are based on the presentation I gave at this year’s game Developer’s Conference in San Francisco, entitled Music in Virtual Reality (I’ve included the official description of my talk at this end of this article). If you haven’t read the previous three articles, you’ll find them here:

During my GDC presentation, I focused on three important questions for VR game music composers:

  • Do we compose our music in 3D or 2D?
  • Do we structure our music to be Diegetic or Non-Diegetic?
  • Do we focus our music on enhancing player Comfort or Performance?

In the course of exploring these questions during my GDC presentation, I discussed my work on four of my own VR game projects –the Bebylon: Battle Royale arena combat game from Kite & Lightning, the Dragon Front strategy game from High Voltage Software, the Fail Factory comedy game from Armature Studio, and the Scraper: First Strike shooter/RPG from Labrodex Inc.

Continue reading

Composing video game music for Virtual Reality: Diegetic versus Non-diegetic

In this article for and about the craft of video game composers, Winifred Phillips is pictured in this photo working in her music production studio.

By Winifred Phillips | Contact | Follow

So happy you’ve joined us!  I’m videogame composer Winifred Phillips.  Welcome back to our four part discussion of the role that music plays in Virtual Reality video games! These articles are based on the presentation I gave at this year’s gathering of the famous Game Developer’s Conference in San Francisco.  My talk was entitled Music in Virtual Reality (I’ve included the official description of my talk at this end of this article). If you haven’t read the previous two articles, you’ll find them here:

During my GDC presentation, I focused on three important questions for VR video game composers:

  • Do we compose our music in 3D or 2D?
  • Do we structure our music to be Diegetic or Non-Diegetic?
  • Do we focus our music on enhancing player Comfort or Performance?

While attempting to answer these questions during my GDC talk, I discussed my work on four of my own VR game projects – the Bebylon: Battle Royale arena combat game from Kite & Lightning, the Dragon Front strategy game from High Voltage Software, the Fail Factory comedy game from Armature Studio, and the Scraper: First Strike shooter/RPG from Labrodex Inc.

In these articles, I’ve been sharing the discussions and conclusions that formed the basis of my GDC talk, including numerous examples from these four VR game projects.  So now let’s look at the second of our three questions:

Continue reading