Video Game Music Composers: New VR Headphone Tech (2018)

In this article for and about the craft of video game composers, Winifred Phillips is pictured in this photo from her lecture on Virtual Reality given at the popular Game Developers Conference in 2018.Hey, everyone!  I’m videogame composer Winifred Phillips, and my work has included the musical scores for top games on all sorts of popular gaming platforms, from handhelds and mobile, all the way up to the latest consoles and PCs.  Lately, I’ve been doing a lot of video game music composition for virtual reality.  I had the pleasure of presenting a lecture on Music in Virtual Reality (pictured left) at the most recent Game Developers Conference in San Francisco.

My experience as a composer for VR includes many VR games, including the Scraper: First Strike shooter (set to be released for the PSVR, Oculus Rift and HTC Vive in December 2018), and the recently released VR experience The Haunted Graveyard, which is now available on Steam and in VR Arcades around the world.  Since we’re in the Halloween season, and this VR experience is designed specifically for your Halloween pleasure, here’s a trailer that features my music from The Haunted Graveyard:

By virtue of all the experiences I’ve had recently creating music for VR, I’ve become keenly aware of the importance of sound fidelity in VR.  If the experience doesn’t sound real, it loses the chance to actually feel like a fully-convincing, thoroughly awesome virtual reality experience.  With that in mind, I’ve been writing periodic articles about new technologies in connection with headphones for VR.

In the previous article from last month, we discussed developments with technologies that were introduced years ago and have made recent advances.  We also noted the disappearance of products that failed to live up to their promises and were never released to the public.  Now, let’s turn to some brand new developments in the area of headphone audio for virtual reality projects.  In this article, we’ll look at some newly announced products, and we’ll also expand our discussion to encompass new technologies that promise to enhance existing stereo headphones for VR uses.  So, let’s start first with some headphones that could come in handy not only for VR gamers, but for indie VR audio developers as well!

The Ambeo Smart Headset

In this article discussing popular VR issues for video game composers, Winifred Phillips discusses the technology of the Sennheiser Ambeo Smart Headset.Sennheiser is an expert manufacturer of high-end headphones and microphones for audio professionals and consumers.  Their catalog includes a product line designed for gamers, but nothing in the specs of these headphones makes them especially suitable for VR use (beyond their virtues as quality headphones). So instead, let’s take a look at a new Sennheiser product design that takes headphones in a surprising direction with relevance to VR.

At the beginning of 2017, Sennheiser debuted an innovative new headphone product at the Consumer Electronics Show.  At the end of that year, their new product hit retail.  Usually, headphones live or die by their innate sound quality, but that isn’t the primary selling-point of Sennheiser’s new product.  With standard dynamic drivers and some noise-cancellation features, the headphones are capable of delivering an aural experience in VR that’s perfectly acceptable – but not particularly exceptional.  Add to that the limitation imposed by the Apple Lightning connector (which is only compatible with iOS devices), and these headphones start to seem less than desirable. Admittedly, the Ambeo Smart Headset could serve as a satisfactory set of standard headphones for gamers who use their iPhones in combination with iOS compatible VR headsets such as Google Cardboard, Merge, VR One Plus, and other similar devices.  That said, there’s nothing special on offer here… unless you’re an indie VR developer.  In that case, you’ll find a great additional feature that could make these headphones worth their $299.95 price tag.

In this article exploring VR-suitable headphone technologies of interest to video game composers, Winifred Phillips explains the unique functionality of the Ambeo Smart Headset from the famous Sennheiser audio manufacturing company.What differentiates the Ambeo Smart Headset from any other prospective set of VR headphones is its capability to not only deliver convincing binaural sound for VR, but capture it as well.  The headset includes a pair of directional electret capsule microphones – one mounted on each earpiece, facing outward. When a user wears the in-ear headset, each microphone is positioned so that it captures the audio exactly as it reaches that ear, including all of its associated Head Related Transfer Functions.  In essence, the user has now become a living, breathing, full-featured binaural dummy head microphone.  Since the famous Neumann Binaural Dummy Head microphone retails for nearly eight thousand dollars, having a fully-functioning substitute that only costs $299.95 is a dramatic new development.

Sennheiser promises that a version of this headset with Android connectors will be released sometime in 2019, so we’ll be keeping tabs on that.  In the meantime, here’s a video produced by the B & H Photo Video site that demonstrates the Ambeo Smart Headset in action:

 

Giving your headphones a VR make-over

In several previous articles, we’ve looked at headphone models designed specifically to maximize the aural experience of a VR gamer.  Now that the initial wave of VR excitement has given way to the more steady growth necessary for long-term success, I’ve noticed that fewer headphones are entering the market touting VR-specific features.  Instead, we’re seeing audio technology companies introducing ways to make the headphones we already own sound better in VR.  Since better-sounding headphones will mean that VR users will be able to better appreciate our video game music, let’s take a look at a couple of these sound-enhancing technologies:

Cingo

As we know, any pair of headphones can deliver a fully spatialized, three-dimensional audio experience (as demonstrated memorably by the famous Virtual Barber Shop recording).  But the audio of a VR game doesn’t consist of a single spatialized recording.  Instead, interactive VR audio can include numerous audio components that are combined by the game engine and delivered to the player’s ears.  The resulting mix, when successful, will enfold the player in the game’s aural universe… but getting that mix just right can be tricky.  In this article about headphone tech for the popular VR platforms (by a video game composer for video game composers) Winifred Phillips is pictured demoing the Cingo engine as implemented in the famous Samsung Gear VR.This is where Cingo comes in.  It’s a sound processing solution developed specifically for the challenges of optimizing mobile VR audio for headphones.  I got to try Cingo out during the Audio Engineering Society Convention in 2015 (pictured right).

Cingo can be used by a VR developer as a complete audio solution for mobile platforms.  As an all-in-one solution, Cingo includes creation software (the Cingo Composer authoring plug-in for VST and AAX) and a post-processing engine that can be integrated either directly into the operating system of the VR device, or into the software of the VR game (wherein it would act essentially as audio middleware).  Cingo’s audio enhancements for headphones in VR include loudness and equalization correction on-the-fly, and other aural adjustments based on head-tracking data.

The Cingo technology is currently incorporated directly into the Google Nexus operating system for Android, the Google Pixel line of laptops and smartphones, the Samsung Gear VR (pictured), and the LG 360 VR device.  Let’s see the technology demonstrated in a video produced by the Fraunhofer-Gesellschaft organization (creators of Cingo):

Creative Super X-Fi

We all probably remember the ill-fated OSSIC X headphones, which recently met an ignominious end when the manufacturer shut its doors due to lack of funds.  One of the promised features of these headphones was “individual anatomy calibration,” which would have allowed the headphones to formulate a map of the wearer’s unique ears.  This map would then have been used to construct a set of customized Head Related Transfer Functions that would purportedly have given the headphone-delivered sound a sense of unparalleled realism and spatial depth.  While we’ll never know if the OSSIC X headphones would have delivered on this promise, we can now turn our attention to another device claiming to achieve the same feat.

A photo of the SuperX-Fi dongle - in this article for video game composers, Winifred Phillips explores headphone tech appropriate for use in the popular Virtual Reality platform.Creative Technology made a big splash at the Consumer Electronics Show 2018 with their Super X-Fi device (pictured left), which earned enthusiastic reviews and a Best of CES Award.  The Super X-Fi resembles a slim black dongle, with input jacks for the sound source at one end, and an output jack to carry the sound from the Super X-Fi to a pair of standard headphones.  According to Creative Technology, when the product is released this summer, users will be able to take photos of their ears and face, and then the Super X-Fi will use these photos to match the anatomy of the user’s head and ears to one of a library of anatomical profiles. Once this match is made, the Super X-Fi will apply the associated Head Related Transfer Functions (HRTFs) from the profile to the sound passing through the dongle.  When the sound reaches the user, Creative Technology promises that the aural experience will be extraordinarily convincing and natural.  No need to purchase special headphones when a simple dongle can make any decent headphones sound like a million bucks, right?

While the product is receiving pretty widespread acclaim right now, it’s only available in Singapore, while consumers from other countries can only add themselves to a reservation list.  Also, there’s been no discussion of how this technology might interact with binaural sound sources.  Would the introduction of customized HRTFs along the signal path enhance the binaural content, or distort it?  This issue remains unclear, but it’s an intriguing new technology, and one worth keeping an eye on.  Here’s a video demonstration of the prototype model as it was shown at this year’s Consumer Electronics Show:

And then there were none

Finally, let’s take a look at a new technology that seems lifted right out of a high-tech spy movie, or some futuristic sci-fi flick.  It’s called Sowlo, created by Noveto Systems.  Traditionally, when we want to enjoy audio privately, we’ve always needed either a set of headphones, or some privacy and a locked door.  But Sowlo requires none of these things. That’s because Sowlo directs sound precisely into our ears.  No earbuds or headphones required.  An illustration of the Noveto Sowlo technology -- in this article for video game music composers, Winifred Phillips delves into an alternative to the popular headphone solution for personal VR audio delivery.The system tracks our eyes, ears and face to get a read on where we are in the room, and then sends the sound directly to us.  The audio signal proceeds in such an ultra-narrow, precisely-targeted beam that there can be other people in the room, hearing nothing at all, while we’re getting a full-force bombastic aural soundscape directed with laser-focus right into our ears.

One of Sowlo’s advantages for VR applications is its ability to free users from headphones.  With a VR headset already taking up valuable head-mounted real-estate, it could well be a welcome relief for some gamers to get rid of the dangling cables and weight of headphones.  Plus, there have certainly been many gamers who just detest headphones as a general rule, and wish that they could wander virtual worlds using a surround speaker system instead.  Up until now, that desire has been thwarted by the inability of a surround-sound system to track with the user’s head movements and deliver faithful spatialized audio.  But the makers of Sowlo claim that their system can track head movements and convey accurate positional audio for VR.

The technology is certainly intriguing.  Noveto currently estimates that the Sowlo devices will be available to purchase by fall 2019.  It’s worth keeping track of this thoroughly unique approach to audio delivery, and we’ll check in with Sowlo periodically to see how things are progressing.  In the meantime, here’s a video produced by VR Focus about Noveto’s Sowlo technology.

 

Conclusion

So we’ve taken a look at some new technologies for VR audio.  Some promise to improve the experience of headphone wearers in VR by virtue of aural enhancement or user customization.  Another does away with headphones entirely in favor of sound delivered in a private beam directly to the user.  And one technology transforms headphones into a tool for not only experiencing VR audio, but recording it as well.  These are interesting times in what we hope will be a long life cycle for virtual reality gaming.  We’ll continue to revisit this topic from year to year, to see what advances are being made in the field of VR headphone audio.  I hope you enjoyed the article, and please feel free to leave a comment in the space below!

 

Photo of video game composer Winifred Phillips in her music production studio.Winifred Phillips is an award-winning video game music composer whose projects include the triple-A first person shooter Homefront: The Revolution. She has composed music for games in five of the most famous and popular franchises in gaming: Assassin’s Creed, LittleBigPlanet, Total War, God of War, and The Sims.  Her latest video game credits include numerous Virtual Reality games, including The Haunted Graveyard, AudioshieldScraper: First Strike, Bebylon: Battle Royale, Fail Factory, Dragon Front, and many more. She is the author of the award-winning bestseller A COMPOSER’S GUIDE TO GAME MUSIC, published by the MIT Press. As a VR game music expert, she writes frequently on the future of music in virtual reality games. Follow her on Twitter @winphillips.

VR Headphones Update 2018: Video Game Music Composers

In this article written for video game composers, Winifred Phillips (composer of music for God of War) is here pictured working in her music production studio.

By Winifred Phillips | Contact | Follow

Glad you’re here!  I’m videogame composer Winifred Phillips.  My work as a game music composer has included music for projects released on nearly all of the gaming platforms, from one of my most recent projects (a Homefront game released on all the latest consoles and PCs) to one of my earliest projects (a God of War game released on PlayStation 2, PlayStation 3, and PlayStation Vita, pictured above).  An image of the September 2018 cover of Music Connection Magazine featuring the article "Video Game Composers Speak!" - features interviews of famous game music composers, including popular game music composer Winifred Phillips.You can read about my work as a video game composer in an interview I gave to Music Connection Magazine for this month’s issue (pictured right).

Lately, I’ve also been creating lots of video game music for awesome virtual reality games developed for the Oculus Rift, Oculus Go, HTC Vive, Samsung Gear VR, PlayStation VR, and lots of other top VR platforms.  One of the things I’ve noticed while working in VR is the immense importance of the audio delivery mechanism.

When audio is painstakingly spatialized, it becomes crucial to convey that carefully-crafted spatialization to the player with as little fidelity loss as possible.  With the importance of this issue in mind, for the past few years I’ve been periodically writing about headphones in relation to their use in virtual reality.

Continue reading

Composing video game music for Virtual Reality: Comfort versus performance

In this article series for video game composers, Winifred Phillips is depicted in this photo working in her music production studio.

By Winifred Phillips | Contact | Follow

Delighted you’re here!  I’m videogame composer Winifred Phillips, and I’m happy to welcome you back to this four-part article series exploring the role of music in VR games! These articles are based on the presentation I gave at this year’s game Developer’s Conference in San Francisco, entitled Music in Virtual Reality (I’ve included the official description of my talk at this end of this article). If you haven’t read the previous three articles, you’ll find them here:

During my GDC presentation, I focused on three important questions for VR game music composers:

  • Do we compose our music in 3D or 2D?
  • Do we structure our music to be Diegetic or Non-Diegetic?
  • Do we focus our music on enhancing player Comfort or Performance?

In the course of exploring these questions during my GDC presentation, I discussed my work on four of my own VR game projects –the Bebylon: Battle Royale arena combat game from Kite & Lightning, the Dragon Front strategy game from High Voltage Software, the Fail Factory comedy game from Armature Studio, and the Scraper: First Strike shooter/RPG from Labrodex Inc.

Continue reading

Composing video game music for Virtual Reality: Diegetic versus Non-diegetic

In this article for and about the craft of video game composers, Winifred Phillips is pictured in this photo working in her music production studio.

By Winifred Phillips | Contact | Follow

So happy you’ve joined us!  I’m videogame composer Winifred Phillips.  Welcome back to our four part discussion of the role that music plays in Virtual Reality video games! These articles are based on the presentation I gave at this year’s gathering of the famous Game Developer’s Conference in San Francisco.  My talk was entitled Music in Virtual Reality (I’ve included the official description of my talk at this end of this article). If you haven’t read the previous two articles, you’ll find them here:

During my GDC presentation, I focused on three important questions for VR video game composers:

  • Do we compose our music in 3D or 2D?
  • Do we structure our music to be Diegetic or Non-Diegetic?
  • Do we focus our music on enhancing player Comfort or Performance?

While attempting to answer these questions during my GDC talk, I discussed my work on four of my own VR game projects – the Bebylon: Battle Royale arena combat game from Kite & Lightning, the Dragon Front strategy game from High Voltage Software, the Fail Factory comedy game from Armature Studio, and the Scraper: First Strike shooter/RPG from Labrodex Inc.

In these articles, I’ve been sharing the discussions and conclusions that formed the basis of my GDC talk, including numerous examples from these four VR game projects.  So now let’s look at the second of our three questions:

Continue reading

Composing video game music for Virtual Reality: 3D versus 2D

In this article written for video game composers, Winifred Phillips is here pictured working in her music production studio.

Welcome!  I’m videogame composer Winifred Phillips, and this is the continuation of our four-part discussion of the role that music can play in Virtual Reality video games.  These articles are based on the presentation I gave at this year’s Game Developer’s Conference in San Francisco, entitled Music in Virtual Reality (I’ve included the official description of my talk at this end of this article).  If you missed the first article exploring the history and significance of positional audio, please go check that article out first.

Are you back?  Great!  Let’s continue!

During my GDC talk, I addressed three questions which are important to video game music composers working in VR:

  • Do we compose our music in 3D or 2D?
  • Do we structure our music to be Diegetic or Non-Diegetic?
  • Do we focus our music on enhancing player Comfort or Performance?

Continue reading

Composing video game music for Virtual Reality: The role of music in VR

In this article for video game composers, Winifred Phillips is pictured working in her music production studio.

By Winifred Phillips | Contact | Follow

Hey everybody!  I’m video game composer Winifred Phillips.  At this year’s Game Developers Conference in San Francisco, I was pleased to give a presentation entitled Music in Virtual Reality (I’ve included the official description of my talk at the end of this article). While I’ve enjoyed discussing the role of music in virtual reality in previous articles that I’ve posted here, the talk I gave at GDC gave me the opportunity to pull a lot of those ideas together and present a more concentrated exploration of the practice of music composition for VR games.  It occurred to me that such a focused discussion might be interesting to share in this forum as well. So, with that in mind, I’m excited to begin a four-part article series based on my GDC 2018 presentation!

Continue reading

Video Game Composers: The Art of Music in Virtual Reality (GDC 2018)

Video game composer Winifred Phillips, pictured in her music production studio.

 

By Winifred Phillips | Contact | Follow

Once again, the Game Developers Conference is almost upon us!  GDC 2018 promises to be an awesome event, chock full of great opportunities for us to learn and grow as video game music composers.  I always look forward to the comprehensive sessions on offer in the popular GDC audio track, and for the past few years I’ve been honored to be selected as a GDC speaker.  Last year I presented a talk that explored how I built suspense and tension through music I composed for such games as God of War and Homefront: The Revolution.  This year, I’m tremendously excited that I’ll be presenting the talk, “Music in Virtual Reality.” The subject matter is very close to my heart!  Throughout 2016 and 2017, I’ve composed music for many virtual reality projects, some of which have hit retail over the past year, and some of which will be released very soon.  I’ve learned a lot about the process of composing music for a VR experience, and I’ve given a lot of thought to what makes music for VR unique.  During my GDC talk in March, I’ll be taking my audience through my experiences composing music for four very different VR games –the Bebylon: Battle Royale arena combat game from Kite & Lightning, the Dragon Front strategy game from High Voltage Software, the Fail Factory comedy game from Armature Studio, and the Scraper: First Strike Shooter/RPG from Labrodex Inc.  I’ll talk about some of the top problems that came up, the solutions that were tried, and the lessons that were learned.  Virtual Reality is a brave new world for game music composers, and there will be a lot of ground for me to cover in my presentation!

In preparing my talk for GDC, I kept my focus squarely on composition techniques for VR music creation, while making sure to supply an overview of the technologies that would help place these techniques in context.  With these considerations in mind, I had to prioritize the information I intended to offer, and some interesting topics simply wouldn’t fit within the time constraints of my GDC presentation.  With that in mind, I thought it would be worthwhile to include some of these extra materials in a couple of articles that would precede my talk in March.  In this article, I’ll explore some theoretical ideas from experts in the field of VR, and I’ll include some of my own musings about creative directions we might pursue with VR music composition.  In the next article, I’ll talk about some practical considerations relating to the technology of VR music.

Continue reading