Video Game Music Composers: New VR Headphone Tech (2018)

In this article for and about the craft of video game composers, Winifred Phillips is pictured in this photo from her lecture on Virtual Reality given at the popular Game Developers Conference in 2018.Hey, everyone!  I’m videogame composer Winifred Phillips, and my work has included the musical scores for top games on all sorts of popular gaming platforms, from handhelds and mobile, all the way up to the latest consoles and PCs.  Lately, I’ve been doing a lot of video game music composition for virtual reality.  I had the pleasure of presenting a lecture on Music in Virtual Reality (pictured left) at the most recent Game Developers Conference in San Francisco.

My experience as a composer for VR includes many VR games, including the Scraper: First Strike shooter (set to be released for the PSVR, Oculus Rift and HTC Vive in December 2018), and the recently released VR experience The Haunted Graveyard, which is now available on Steam and in VR Arcades around the world.  Since we’re in the Halloween season, and this VR experience is designed specifically for your Halloween pleasure, here’s a trailer that features my music from The Haunted Graveyard:

By virtue of all the experiences I’ve had recently creating music for VR, I’ve become keenly aware of the importance of sound fidelity in VR.  If the experience doesn’t sound real, it loses the chance to actually feel like a fully-convincing, thoroughly awesome virtual reality experience.  With that in mind, I’ve been writing periodic articles about new technologies in connection with headphones for VR.

In the previous article from last month, we discussed developments with technologies that were introduced years ago and have made recent advances.  We also noted the disappearance of products that failed to live up to their promises and were never released to the public.  Now, let’s turn to some brand new developments in the area of headphone audio for virtual reality projects.  In this article, we’ll look at some newly announced products, and we’ll also expand our discussion to encompass new technologies that promise to enhance existing stereo headphones for VR uses.  So, let’s start first with some headphones that could come in handy not only for VR gamers, but for indie VR audio developers as well!

The Ambeo Smart Headset

In this article discussing popular VR issues for video game composers, Winifred Phillips discusses the technology of the Sennheiser Ambeo Smart Headset.Sennheiser is an expert manufacturer of high-end headphones and microphones for audio professionals and consumers.  Their catalog includes a product line designed for gamers, but nothing in the specs of these headphones makes them especially suitable for VR use (beyond their virtues as quality headphones). So instead, let’s take a look at a new Sennheiser product design that takes headphones in a surprising direction with relevance to VR.

At the beginning of 2017, Sennheiser debuted an innovative new headphone product at the Consumer Electronics Show.  At the end of that year, their new product hit retail.  Usually, headphones live or die by their innate sound quality, but that isn’t the primary selling-point of Sennheiser’s new product.  With standard dynamic drivers and some noise-cancellation features, the headphones are capable of delivering an aural experience in VR that’s perfectly acceptable – but not particularly exceptional.  Add to that the limitation imposed by the Apple Lightning connector (which is only compatible with iOS devices), and these headphones start to seem less than desirable. Admittedly, the Ambeo Smart Headset could serve as a satisfactory set of standard headphones for gamers who use their iPhones in combination with iOS compatible VR headsets such as Google Cardboard, Merge, VR One Plus, and other similar devices.  That said, there’s nothing special on offer here… unless you’re an indie VR developer.  In that case, you’ll find a great additional feature that could make these headphones worth their $299.95 price tag.

In this article exploring VR-suitable headphone technologies of interest to video game composers, Winifred Phillips explains the unique functionality of the Ambeo Smart Headset from the famous Sennheiser audio manufacturing company.What differentiates the Ambeo Smart Headset from any other prospective set of VR headphones is its capability to not only deliver convincing binaural sound for VR, but capture it as well.  The headset includes a pair of directional electret capsule microphones – one mounted on each earpiece, facing outward. When a user wears the in-ear headset, each microphone is positioned so that it captures the audio exactly as it reaches that ear, including all of its associated Head Related Transfer Functions.  In essence, the user has now become a living, breathing, full-featured binaural dummy head microphone.  Since the famous Neumann Binaural Dummy Head microphone retails for nearly eight thousand dollars, having a fully-functioning substitute that only costs $299.95 is a dramatic new development.

Sennheiser promises that a version of this headset with Android connectors will be released sometime in 2019, so we’ll be keeping tabs on that.  In the meantime, here’s a video produced by the B & H Photo Video site that demonstrates the Ambeo Smart Headset in action:

 

Giving your headphones a VR make-over

In several previous articles, we’ve looked at headphone models designed specifically to maximize the aural experience of a VR gamer.  Now that the initial wave of VR excitement has given way to the more steady growth necessary for long-term success, I’ve noticed that fewer headphones are entering the market touting VR-specific features.  Instead, we’re seeing audio technology companies introducing ways to make the headphones we already own sound better in VR.  Since better-sounding headphones will mean that VR users will be able to better appreciate our video game music, let’s take a look at a couple of these sound-enhancing technologies:

Cingo

As we know, any pair of headphones can deliver a fully spatialized, three-dimensional audio experience (as demonstrated memorably by the famous Virtual Barber Shop recording).  But the audio of a VR game doesn’t consist of a single spatialized recording.  Instead, interactive VR audio can include numerous audio components that are combined by the game engine and delivered to the player’s ears.  The resulting mix, when successful, will enfold the player in the game’s aural universe… but getting that mix just right can be tricky.  In this article about headphone tech for the popular VR platforms (by a video game composer for video game composers) Winifred Phillips is pictured demoing the Cingo engine as implemented in the famous Samsung Gear VR.This is where Cingo comes in.  It’s a sound processing solution developed specifically for the challenges of optimizing mobile VR audio for headphones.  I got to try Cingo out during the Audio Engineering Society Convention in 2015 (pictured right).

Cingo can be used by a VR developer as a complete audio solution for mobile platforms.  As an all-in-one solution, Cingo includes creation software (the Cingo Composer authoring plug-in for VST and AAX) and a post-processing engine that can be integrated either directly into the operating system of the VR device, or into the software of the VR game (wherein it would act essentially as audio middleware).  Cingo’s audio enhancements for headphones in VR include loudness and equalization correction on-the-fly, and other aural adjustments based on head-tracking data.

The Cingo technology is currently incorporated directly into the Google Nexus operating system for Android, the Google Pixel line of laptops and smartphones, the Samsung Gear VR (pictured), and the LG 360 VR device.  Let’s see the technology demonstrated in a video produced by the Fraunhofer-Gesellschaft organization (creators of Cingo):

Creative Super X-Fi

We all probably remember the ill-fated OSSIC X headphones, which recently met an ignominious end when the manufacturer shut its doors due to lack of funds.  One of the promised features of these headphones was “individual anatomy calibration,” which would have allowed the headphones to formulate a map of the wearer’s unique ears.  This map would then have been used to construct a set of customized Head Related Transfer Functions that would purportedly have given the headphone-delivered sound a sense of unparalleled realism and spatial depth.  While we’ll never know if the OSSIC X headphones would have delivered on this promise, we can now turn our attention to another device claiming to achieve the same feat.

A photo of the SuperX-Fi dongle - in this article for video game composers, Winifred Phillips explores headphone tech appropriate for use in the popular Virtual Reality platform.Creative Technology made a big splash at the Consumer Electronics Show 2018 with their Super X-Fi device (pictured left), which earned enthusiastic reviews and a Best of CES Award.  The Super X-Fi resembles a slim black dongle, with input jacks for the sound source at one end, and an output jack to carry the sound from the Super X-Fi to a pair of standard headphones.  According to Creative Technology, when the product is released this summer, users will be able to take photos of their ears and face, and then the Super X-Fi will use these photos to match the anatomy of the user’s head and ears to one of a library of anatomical profiles. Once this match is made, the Super X-Fi will apply the associated Head Related Transfer Functions (HRTFs) from the profile to the sound passing through the dongle.  When the sound reaches the user, Creative Technology promises that the aural experience will be extraordinarily convincing and natural.  No need to purchase special headphones when a simple dongle can make any decent headphones sound like a million bucks, right?

While the product is receiving pretty widespread acclaim right now, it’s only available in Singapore, while consumers from other countries can only add themselves to a reservation list.  Also, there’s been no discussion of how this technology might interact with binaural sound sources.  Would the introduction of customized HRTFs along the signal path enhance the binaural content, or distort it?  This issue remains unclear, but it’s an intriguing new technology, and one worth keeping an eye on.  Here’s a video demonstration of the prototype model as it was shown at this year’s Consumer Electronics Show:

And then there were none

Finally, let’s take a look at a new technology that seems lifted right out of a high-tech spy movie, or some futuristic sci-fi flick.  It’s called Sowlo, created by Noveto Systems.  Traditionally, when we want to enjoy audio privately, we’ve always needed either a set of headphones, or some privacy and a locked door.  But Sowlo requires none of these things. That’s because Sowlo directs sound precisely into our ears.  No earbuds or headphones required.  An illustration of the Noveto Sowlo technology -- in this article for video game music composers, Winifred Phillips delves into an alternative to the popular headphone solution for personal VR audio delivery.The system tracks our eyes, ears and face to get a read on where we are in the room, and then sends the sound directly to us.  The audio signal proceeds in such an ultra-narrow, precisely-targeted beam that there can be other people in the room, hearing nothing at all, while we’re getting a full-force bombastic aural soundscape directed with laser-focus right into our ears.

One of Sowlo’s advantages for VR applications is its ability to free users from headphones.  With a VR headset already taking up valuable head-mounted real-estate, it could well be a welcome relief for some gamers to get rid of the dangling cables and weight of headphones.  Plus, there have certainly been many gamers who just detest headphones as a general rule, and wish that they could wander virtual worlds using a surround speaker system instead.  Up until now, that desire has been thwarted by the inability of a surround-sound system to track with the user’s head movements and deliver faithful spatialized audio.  But the makers of Sowlo claim that their system can track head movements and convey accurate positional audio for VR.

The technology is certainly intriguing.  Noveto currently estimates that the Sowlo devices will be available to purchase by fall 2019.  It’s worth keeping track of this thoroughly unique approach to audio delivery, and we’ll check in with Sowlo periodically to see how things are progressing.  In the meantime, here’s a video produced by VR Focus about Noveto’s Sowlo technology.

 

Conclusion

So we’ve taken a look at some new technologies for VR audio.  Some promise to improve the experience of headphone wearers in VR by virtue of aural enhancement or user customization.  Another does away with headphones entirely in favor of sound delivered in a private beam directly to the user.  And one technology transforms headphones into a tool for not only experiencing VR audio, but recording it as well.  These are interesting times in what we hope will be a long life cycle for virtual reality gaming.  We’ll continue to revisit this topic from year to year, to see what advances are being made in the field of VR headphone audio.  I hope you enjoyed the article, and please feel free to leave a comment in the space below!

 

Photo of video game composer Winifred Phillips in her music production studio.Winifred Phillips is an award-winning video game music composer whose projects include the triple-A first person shooter Homefront: The Revolution. She has composed music for games in five of the most famous and popular franchises in gaming: Assassin’s Creed, LittleBigPlanet, Total War, God of War, and The Sims.  Her latest video game credits include numerous Virtual Reality games, including The Haunted Graveyard, AudioshieldScraper: First Strike, Bebylon: Battle Royale, Fail Factory, Dragon Front, and many more. She is the author of the award-winning bestseller A COMPOSER’S GUIDE TO GAME MUSIC, published by the MIT Press. As a VR game music expert, she writes frequently on the future of music in virtual reality games. Follow her on Twitter @winphillips.

VR Headphones Update: Video Game Music Composers

Video game composer Winifred Phillips, pictured in her music production studio working on the music of the Dragon Front virtual reality game for Oculus Rift and Samsung Gear VR.

By Winifred Phillips | Contact | Follow

Last year while working on the music of the Dragon Front virtual reality game for Oculus Rift (as pictured above), I gave a lot of consideration to the listening environment in which VR gamers would be hearing my video game music.  Since then I’ve served as the video game composer for several more virtual reality games (which will be released in the next few months).  I’ve also written a number of articles on this subject in order to share what I’ve learned with other game composers.  Last September I devoted two articles to a discussion of audio headphones designed specifically for the demands of virtual reality applications.  You can read those here:

In addition, two years ago I wrote an article that focused on some of the top difficulties associated with choosing the right headphones for VR.  You can read that article here:

Music Composers and Sound Designers in VR: The Headphones Problem

Now, I’d like to revisit the ideas discussed in those articles, so that we can see how the art of VR audio for headphones has progressed.

Continue reading

Video Game Music Composers: New VR Headphones

Video game composer Winifred Phillips, pictured in her music production studio.

By Winifred Phillips | Contact | Follow

As a video game composer, I’ve been working in my studio composing music for quite a few virtual reality projects lately (as pictured above), so I’ve been thinking a lot about issues related to audio in the VR environment.  Those issues include how gamers experience the audio content through various headphone models.  In this article, I thought we’d take a look at three newly-announced headphone models that are targeting the VR marketplace, and see what new technologies are being proposed to facilitate the best and most awesome VR audio experiences.  So, let’s get started!

Continue reading

Understanding Audio in VR – A Game Music Composer’s Resource Guide

Video game music composer Winifred Phillips working in her game composers production studio.

By Winifred Phillips | Contact | Follow

When I’m not at work in my studio making music for games, I like to keep up with new developments in the field of interactive entertainment, and I’ll often share what I learn here in these articles.  Virtual reality is an awesome subject for study for a video game composer, and several of my recent projects have been in the world of VR.  Since I’m sure that most of us are curious about what’s coming next in virtual reality, I’ve decided to devote this article to a collection of educational resources.  I’ve made a point of keeping our focus general here, with the intent of understanding the role of audio in VR and the best resources available to audio folks.  As a component of the VR soundscape, our music must fit into the entire matrix of aural elements, so we’ll spend this article learning about what goes into making expert sound for a virtual reality experience. Let’s start with a few articles that discuss methods and techniques for VR audio practitioners.

Continue reading

VRDC 2017 takeaways: VR music for the game composer

Video game music composer Winifred Phillips, at work in her music production studio - from the article about music for virtual reality / VR.The Game Developers Conference is always an awesome opportunity for game audio experts to learn and share experiences.  I’ve given presentations at GDC for a few years now, and I’m always excited to hear about what’s new and notable in game audio.  This year, the hot topic was virtual reality.  In fact, the subject received its own dedicated sub-conference that took place concurrently with the main GDC show.  The VRDC (Virtual Reality Developers Conference) didn’t focus particularly on the audio and music side of VR, but there were a couple of notable talks on that subject.  In this article, let’s take a look at some of the more intriguing VR game music takeaways from those two talks.  Along the way, I’ll also share some of my related experience as the composer of the music of the Dragon Front VR game for the Oculus Rift (pictured above).

Inside and outside

The talks we’ll be discussing in this article are entitled “Audio Adventures in VR Worlds” and “The Sound Design of Star Wars: Battlefront VR.”  Here’s a common issue that popped up in both talks:

An illustration of music in the popular VR platform, from the article by Winifred Phillips (video game composer).Where should video game music be in a VR game?  Should it feel like it exists inside the VR world, weaving itself into the immersive 3D atmosphere surrounding the player?  Or should it feel like it’s somehow outside of the VR environment and is instead coasting on top of the experience, being conveyed directly to the player?  The former approach suggests a spacious and expansive musical soundscape, and the latter would feel much closer and more personal.  Is one of these approaches more effective in VR than the other?  Which choice is best?

Continue reading

Video game music systems at GDC 2017: tools and tips for composers

Photo of video game composer Winifred Phillips, working in her music production studio on the music of the SimAnimals video game.

By video game composer Winifred Phillips | Contact | Follow

Welcome back to this three article series that’s bringing together the ideas that were discussed in five different GDC 2017 audio talks about interactive music!  These five speakers explored discoveries they’d made while creating interactivity in the music of their own game projects.  We’re looking at these ideas side-by-side to broaden our viewpoint and gain a sense of the “bigger picture” when it comes to the leading-edge thinking for music interactivity in games. We’ve been looking at five interactive music systems discussed in these five GDC 2017 presentations:

In the first article, we examined the basic nature of these interactive systems. In the second article, we contemplated why those systems were used, with some of the inherent pros and cons of each system discussed in turn.  So now, let’s get into the nitty gritty of tools and tips for working with such interactive music systems.  If you haven’t read parts one and two of this series, please go do so now and then come back:

  1. Video game music systems at GDC 2017: what are composers using?
  2. Video game music systems at GDC 2017: pros and cons for composers

Ready?  Great!  Here we go!

Continue reading