Understanding Audio in VR – A Game Music Composer’s Resource Guide

Video game music composer Winifred Phillips working in her game composers production studio.

By Winifred Phillips | Contact | Follow

When I’m not at work in my studio making music for games, I like to keep up with new developments in the field of interactive entertainment, and I’ll often share what I learn here in these articles.  Virtual reality is an awesome subject for study for a video game composer, and several of my recent projects have been in the world of VR.  Since I’m sure that most of us are curious about what’s coming next in virtual reality, I’ve decided to devote this article to a collection of educational resources.  I’ve made a point of keeping our focus general here, with the intent of understanding the role of audio in VR and the best resources available to audio folks.  As a component of the VR soundscape, our music must fit into the entire matrix of aural elements, so we’ll spend this article learning about what goes into making expert sound for a virtual reality experience. Let’s start with a few articles that discuss methods and techniques for VR audio practitioners.

Methods and techniques

3D Audio formats for VRAn illustration of popular methods and techniques for game composers - section from the article by Winifred Phillips, video game music composer.

3D Sound Labs takes us through the three most popular sound formats for spatial audio in VR: Multi-channel, Object-based, and Ambisonic.

How 3D Spatialized Audio Bottlenecks Virtual Reality Video

VR audio can consume enormous computational resources, resulting in a resource war between audio and video content.  This article discusses how audio demands in VR consume memory bus bandwidth, and proposes a short-cut that manipulates the frequency response of the audio content to enhance spatialization while consuming fewer resources.

Adapting Your DAW for VR Audio

A VR sound editor discusses his methods and workflow for virtual reality in this article.  Topics include capturing audio in the ambisonic format, and customizing the Pro Tools environment for spatialized audio projects.

Simple spatial audio for beginners

This article summarizes the available entry-level techniques and equipment that can help a newcomer jump into the world of spatial audio for virtual reality.

An audio post production house shares techniques for VR Audio

In this article, we learn about some of the top equipment and methods used by the VR-focused division of an experienced audio post production facility.

Technology and tools

Now that we’ve garnered some insight about what goes into creating VR audio, let’s take a look at some great technology and tools designed specifically for creating virtual reality sound.

The “Works” 3D Audio plugin for Pro ToolsAn illustration of famous technology and tools for game composers - section of the article by video game music composer Winifred Phillips.

This article explores the 3D rendering technology of the G’Audio Works plugin, which supports multi-channel, object-based and ambisonic spatialization within the Pro Tools application.

The Steam Audio Software Development Kit

This article focuses on the spatialized audio solution for VR developed by the famous Steam software distribution platform.  Available as a free download without any royalty requirements, Steam Audio is designed to assist both Unity and Unreal developers in creating and implementing spatialized audio in their projects.

Facebook 360 Spatial Workstation

The Two Big Ears audio company is known for its 3Dception software enabling audio folks to author spatialized audio for VR applications.  Now that Facebook has purchased the company, the software has been rebranded as the Facebook 360 Spatial Workstation with added compatibility for 360 videos hosted on the Facebook platform. The software is free for everyone to use in their projects. However, the previous plugin compatibility with Unity, Wwise and FMOD is no longer offered for new users.

Google’s Omnitone, the open source project for spatialized sound in VR

This article describes the Omnitone application, developed by Google to combine ambisonic decoding with binaural rendering.  Omnitone was designed to deliver spatialized audio for browser-based experiences and apps designed for Android and iOS.

NVIDIA VRWorks Audio

For Windows games and applications, NVIDIA now offers the VRWorks Audio Software Development Kit for implementing spatialized audio in VR for 64 bit Windows apps.  For developers working in Unreal Engine 4, the VRWorks Audio game engine plugin can be added directly to the UE4 engine, while future plugins are promised for other game and audio platforms.

The Google VR Audio Software Development Kit

This article runs through the capabilities of the Google VR Audio SDK, which was designed for use in developing applications for Google Daydream and Cardboard.  The article includes helpful suggestions for using the audio system to simulate indoor and outdoor environments, advice for creating spatialized sounds that users can easily pinpoint, and tips for ensuring that aural and visual content reinforce each other within the VR space.

Communities & Organizations

An illustration of popular communities and organizations for game composers - section of the article by Winifred Phillips (video game music composer).
Working in virtual reality can be a complicated and intimidating process.  Thankfully, there are plenty of online communities and organizations to help us.  I’ve assembled some of these below:

Conferences

An illustration for the famous conferences available for game composers - section of the article by video game music composer Winifred Phillips.

Finally, here’s a list of the virtual reality conferences, summits and conventions that have featured audio discussions and demonstrations in their past program schedules:

Conclusion

I hope you find the above resources interesting and helpful!  If you have any suggestions for additions to this list, please let me know in the comments section below!

 

Photo of video game composer Winifred Phillips in her game composers production studio.Winifred Phillips is an award-winning video game music composer whose most recent project is the triple-A first person shooter Homefront: The Revolution. Her credits include games in five of the most famous and popular franchises in gaming: Assassin’s Creed, LittleBigPlanet, Total War, God of War, and The Sims. She is the author of the award-winning bestseller A COMPOSER’S GUIDE TO GAME MUSIC, published by the MIT Press. As a VR game music expert, she writes frequently on the future of music in virtual reality games. Follow her on Twitter @winphillips.

Resources For Video Game Music Composers

Video game music composer Winifred Phillips, at work in her music production studio.

By Winifred Phillips | Contact | Follow

I’m pleased to announce that my book, A Composer’s Guide to Game Music, is now available its new paperback edition! I’m excited that my book has done well enough to merit a paperback release, and I’m looking forward to getting to know a lot of new readers!  The paperback is much lighter and more portable than the hardcover.  Here’s a view of the front and back covers of the new paperback edition of my book (click the image for a bigger version if you’d like to read the back cover):

award-winning video game music composer Winifred Phillips' book, A Composer's Guide to Game Music, is now available in paperback.

From the article by Winifred Phillips (composer of video game music) - depiction of the book cover of A COMPOSER'S GUIDE TO GAME MUSIC.As you might expect, many aspiring game composers read my book, and I’m honored that my book is a part of their hunt for the best resources to help them succeed in this very competitive business.  When I’m not working in my music studio, I like to keep up with all the great new developments in the game audio field, and I share a lot of what I learn in these articles. Keeping in mind how many of my readers are aspiring composers, I’ve made a point of devoting an article once a year to gathering the top online guidance currently available for newcomers to the game music profession.  In previous years I’ve focused solely on recommendations gleaned from the writings of game audio pros, but this time I’d like to expand that focus to include other types of resources that could be helpful.  Along the way, we’ll be taking a look at some nuggets of wisdom that have appeared on these sites.  So, let’s get started!

Continue reading

Video game music systems at GDC 2017: tools and tips for composers

Photo of video game composer Winifred Phillips, working in her music production studio on the music of the SimAnimals video game.

By video game composer Winifred Phillips | Contact | Follow

Welcome back to this three article series that’s bringing together the ideas that were discussed in five different GDC 2017 audio talks about interactive music!  These five speakers explored discoveries they’d made while creating interactivity in the music of their own game projects.  We’re looking at these ideas side-by-side to broaden our viewpoint and gain a sense of the “bigger picture” when it comes to the leading-edge thinking for music interactivity in games. We’ve been looking at five interactive music systems discussed in these five GDC 2017 presentations:

In the first article, we examined the basic nature of these interactive systems. In the second article, we contemplated why those systems were used, with some of the inherent pros and cons of each system discussed in turn.  So now, let’s get into the nitty gritty of tools and tips for working with such interactive music systems.  If you haven’t read parts one and two of this series, please go do so now and then come back:

  1. Video game music systems at GDC 2017: what are composers using?
  2. Video game music systems at GDC 2017: pros and cons for composers

Ready?  Great!  Here we go!

Continue reading

Video game music systems at GDC 2017: pros and cons for composers

Video game composer Winifred Phillips, pictured in her music production studio working on the music of LittleBigPlanet 2 Cross Controller

By Winifred Phillips | Contact | Follow

Welcome back to our three article series dedicated to collecting and exploring the ideas that were discussed in five different GDC 2017 audio talks about interactive music!  These five speakers shared ideas they’d developed in the process of creating interactivity in the music of their own game projects.  We’re looking at these ideas side-by-side to cultivate a sense of the “bigger picture” when it comes to the leading-edge thinking for music interactivity in games. In the first article, we looked at the basic nature of five interactive music systems discussed in these five GDC 2017 presentations:

If you haven’t read part one of this article series, please go do that now and come back.

Okay, so let’s now contemplate some simple but important questions: why were those systems used?  What was attractive about each interactive music strategy, and what were the challenges inherent in using those systems?

Continue reading

More Business Advice for the Game Music Composer

Winifred Phillips (video game composer) working in her music studio.Every so often, I like to grab some time between music composition gigs to gather together the current general wisdom regarding career strategies for game music composers (since so many of my readers are new to the industry and looking for guidance).  In this article, I’ve included some of the stand-out ideas garnered from three online resources – a Gamasutra article by a former audio designer for Rockstar North, an awesome discussion thread on Reddit about effective communication strategies (found in the GameAudio subreddit), and a roundtable discussion at GameSoundCon about best business practices for game audio pros.

Make some noise! Getting a job creating sound and music for videogames

Audio Director Will Morton of Solid Audioworks (formerly a senior audio designer and dialogue supervisor at the famous Rockstar North development studio), has written a comprehensive article for the game industry site Gamasutra about getting jobs in the game audio field.  The article, entitled “Make Some Noise! Getting a Job Creating Sound and Music for Videogames,” focuses on the importance of experience, networking and a polished presentation in order to sufficiently impress a potential employer/client.  While much of the article is solid advice that might apply to a job seeker in any industry, a few areas impressed me as particularly interesting for game composers to bear in mind.

Continue reading

Communication Tips for the Video Game Composer

Video game composer Winifred Phillips, working in her music studio.A successful career as a video game composer involves much more than our day-to-day challenges in our music studios. In addition to our role as music experts, we need to be well-rounded business people and great members of a creative team.  As a speaker in the audio track of the Game Developers Conference this year, I had a chance to take in a wide variety of GDC sessions, and I noticed how often teamwork was discussed.  Along the way, a common idea emerged from many of these talks — good communication is key. This is a concept that I explored in my book (A Composer’s Guide to Game Music), so I was delighted to see a further discussion of the issue at GDC this year.  Far from just a valuable personality asset, the ability to communicate well must be considered a top priority: as intrinsically valuable as rock-solid competency, awesome artistry or compelling vision. Good communication amongst team members can make or break the development of a game. As game audio pros, we share this in common with our coworkers in other segments of the game development community. However, it becomes especially important for us to focus and emphasize good communication when we’re working remotely as independent contractors. With that in mind, I thought I’d use this article to briefly highlight some GDC 2016 sessions in the game audio track that discussed this popular topic, so we can think about more ways to enhance and improve our communication skills.  And later we’ll discuss a practical example from my work on the music of the SimAnimals game from Electronic Arts.

Continue reading

A Grammy Category for Game Music Composers

grammys-650pxIt’s Grammy Awards time!  This coming Monday, the National Academy of Recording Arts and Sciences (NARAS) will throw its annual party, and many golden gramophones will be awarded to the popular recordings that were deemed most worthy this year. This is one of the most prestigious honors for any musician. Each year, Grammy nominees are selected as representing the top of their field: the very best in professional music.  There are 83 Grammy categories this year, ranging from famous categories like Album of the Year to lesser known categories such as Best Children’s Album.  Among those 83 categories, we don’t find a Best Video Game Music category listed… but we do see the category Best Score Soundtrack for Visual Media (Includes Film, TV, Video Games and Other Visual Media).  The Best Score Soundtrack for Visual Media category has gone through some interesting transformations during the long history of the Grammy Awards. In this article, we’ll be taking a walk down memory lane, exploring the ways in which the Visual Media category has changed to accommodate video game soundtracks.

Continue reading