Video Game Music Composer: Music and Sound in VR Headphones (Part Two)

Photo of game composer Winifred Phillips in her music production studio, from the article "Video Game Music Composer: Music and Sound in VR Headphones (Part Two)"My work as a video game composer has lately included some projects for virtual reality games (more info on that in the coming months), and as a result I’ve been thinking a lot about the awesome potential of VR, and have also been writing lots of articles on the subject.  Earlier this month I began a two-part article that focuses on the experience of the end user, and the gear with which they’ll be enjoying our video game music and audio content (you can read part one here). So, let’s now continue our discussion about the new generation of headphones designed specifically for VR!

In this article, we’ll be discussing two headphone models:

  • Entrim 4D
  • Plantronics RIG 4VR

So let’s get underway!

Entrim 4D headphones

Photo of the Entrim 4D, from the VR headphones article by Winifred Phillips (award-winning game music composer)This March at the famous SXSW convention in Austin, Samsung showed off a piece of experimental technology promising to bring a new dimension of immersion to virtual reality.  It’s designed specifically to complement their popular Samsung Gear VR device, and it works by virtue of electrodes that send electrical signals right into the wearer’s head!  As if virtual reality itself weren’t futuristic enough, now we’re talking about a device that zaps us to make the VR feel more real!  It’s called Entrim 4D (pictured right).  We’re talking about it here because (among other things) Entrim 4D is a pair of audio headphones built specifically for VR.

The Entrim 4D headphones work by virtue of Galvanic Vestibular Stimulation, a process wherein low level electrical impulses are directed into the wearer’s inner ear.  Reaching the nerve in the ear that enables humans to regulate their sense of balance, this subtle electrical pulse can create specific sensations of movement for the wearer, as well as alterations in the wearer’s sense of personal orientation.  In other words, the pulses can make us feel like we’re leaning or moving in any direction.

In an interview with tech journalist Ben Popper of The Verge, Entrim 4D’s creative lead Steve Jung describes the top goals of the project. “Virtual reality shouldn’t be experienced only with the eyes,” says Jung. “With Entrim 4D, we hope that people can experience VR the way it was meant to be—with their whole bodies.”

Photo illustrating a racing simulation, from game composer Winifred Phillips' article on audio headphones built for virtual realityAt SXSW, journalist Karissa Bell of Mashable.com gave the Entrim 4D headphones a test drive.  “I started the demo, which was a first person view of driving around a race track,” she writes. “The demo switched between turning the motion effects on and off and the difference was dramatic. When the effects were on, I found that not only did it feel more realistic, I was far better at steering myself around the track.”

This same technology has been in experimental use for many years, and hasn’t always created the most positive experiences for the wearer.  In 2005, Associated Press correspondent Yuri Kagayama describes her experience wearing a similar headset developed by the Nippon Telegraph & Telephone Corp. in Japan.  “I found the experience unnerving and exhausting: I sought to step straight ahead but kept careening from side to side. Those alternating currents literally threw me off.”  Later, she describes wearing the headphones while watching a game demo.  “I watched a simple racing-car game demonstration on a large screen while wearing a device programmed to synchronize the curves with galvanic vestibular stimulation. It accentuated the swaying as an imaginary racing car zipped through a virtual course, making me wobbly.”

Image of the Entrim 4D, from the article by game music composer Winifred Phillips (Music and Sound in VR Headphones)For the Entrim 4D headset, the project team at Samsung have been hard at work refining this technology.  Their staff of engineers and biomedical experts have run multiple experiments on over 1,500 test subjects.  The results of those experiments have allowed the team to refine the electrical signals to deliver the sensation of 30 distinct patterns of movement.  They’re currently working to add additional electrodes that can induce the sense of rotation – as if the Entrim 4D wearer had started spinning.

One of the ultimate goals behind the Entrim 4D system is the cure for the motion sickness problem in VR.  By reconciling visually perceived motion with physically perceived motion, the Entrim 4D team hopes to eliminate the cognitive dissonance that can lead to the woozy symptoms experienced by some VR users.  In a press release, Samsung expressed its goals in regards to the motion sickness dilemma.  “Unsurprisingly, the mental discrepancy of seeing yourself riding a roller coaster or zooming around a race track but not actually experiencing the movement can leave you nauseous,” Samsung observes.  “Entrim 4D, though still in a developmental phase, may well be able to present a new and practical solution.”

Unfortunately, the team behind the Entrim 4D has revealed nothing about the audio capabilities of its headphones, concentrating solely on the Galvanic Vestibular Stimulation system.  Since the headphones are still in development with no projected release date as of yet, we can expect to hear more about the audio side of the Entrim 4D headphones in the months ahead.  In the meantime, here is a video showing users experiencing the Entrim 4D headphones first hand:

Plantronics RIG 4VR

After the above discussion of a VR headset in a highly experimental stage, let’s now take a look at a much more down-to-earth product that will be hitting retail this Holiday 2016 – the Plantronics RIG 4VR headphones.

Plantronics logo, from the article by Winifred Phillips (game music composer) about audio headphones designed for virtual realityPlantronics is well known as a military contractor for audio technology.  Its headphones equipment has a long history, is currently used heavily by air traffic controllers and aviators, and most notably by NASA during Neil Armstrong’s famous “One small step for man” message from the moon.  However, Plantronics has a strong presence in the consumer headphones industry and a large consumer product catalog, which includes their RIG line of gaming headsets.  The newest entry in this product line, the RIG 4VR, is designed to accommodate one specific model of VR hardware – the PlayStation VR system.

The PlayStation VR headset includes earbuds, but according to CNET’s David Carnoy, some gamers will want more. “A lot of people will want to use better headphones that deliver a more immersive audio experience — or at least that’s what Plantronics is hoping.”

Photo of the Plantronics RIG 4VR headphones, from video game music composer Winifred Phillips' article on audio headphones designed for virtual reality

While the previous headset systems we’ve discussed (both in this article and in the previous one) are all endowed with special aural or electrical features designed specifically for VR, the Plantronics RIG 4VR (pictured right) is customized in a much simpler way.  Plantronics has focused on the form factor of the headphones, optimizing the ergonomics to allow the headphones to be comfortably worn by a gamer who is also wearing the PlayStation VR headset.  It is worth noting that the other headphones we’ve previously discussed, while impressive in their features, can also be fairly large in both their earcups and their headbands.  Plantronics observed that headphones with a bulky form factor have the potential to bump against a VR device, so they took steps to best address the problem.

Image illustrating the Plantronics RIG 4VR, from the article by Winifred Phillips (composer of video game music) about audio headphones designed for virtual realityThe earcups of the RIG 4VR are oval rather than round, in order to keep them out of the way of the PlayStation VR hardware.  The earcups also allow the ambient sound of the room to filter through.  Plantronics has an interesting theory about how this aural permeability may help alleviate VR sickness. “What we’ve found, this may actually help VR users for maintaining balance,” says Corey Rosemond, the head of global Esports for Plantronics, during an interview with UploadVR.com. “People don’t realize that subconsciously, what you hear and how noise sounds, that affects your balance. Being able to allow some of that in allows us to have some better balance from a VR perspective. It lessens the motion sickness just a bit.”

Plantronics also says that the vented earcup design helps to keep ears cooler, while also enabling players to conduct conversations when playing with other VR gamers in the same room.  In addition, Plantronics hinted to CNET that their RIG 4VR headphones have been “tuned for PlayStation VR,” but more specifics about that aural tuning were not made available.

When the RIG 4VR is released concurrently with the Playstation VR system this holiday season, we may learn more about this specific aural tuning for the PlayStation VR.  For now, here’s a “first look” video that shows the form factor of the RIG 4VR:

Conclusion

So that concludes this look at headphones designed specifically to accommodate the needs of virtual reality!  I hope you enjoyed these two articles, and please feel free to let me know your thoughts in the comments below!

 


Photo of video game music composer Winifred Phillips in her music production studio.Winifred Phillips is an award-winning video game music composer whose most recent project is the triple-A first person shooter Homefront: The Revolution. Her credits include five of the most famous and popular franchises in video gaming: Assassin’s Creed, LittleBigPlanet, Total War, God of War, and The Sims. She is the author of the award-winning bestseller A COMPOSER’S GUIDE TO GAME MUSIC, published by the Massachusetts Institute of Technology Press. As a VR game music expert, she writes frequently on the future of music in virtual reality video games. Follow her on Twitter @winphillips.

Communication Tips for the Video Game Composer

Video game composer Winifred Phillips, working in her music studio.A successful career as a video game composer involves much more than our day-to-day challenges in our music studios. In addition to our role as music experts, we need to be well-rounded business people and great members of a creative team.  As a speaker in the audio track of the Game Developers Conference this year, I had a chance to take in a wide variety of GDC sessions, and I noticed how often teamwork was discussed.  Along the way, a common idea emerged from many of these talks — good communication is key. This is a concept that I explored in my book (A Composer’s Guide to Game Music), so I was delighted to see a further discussion of the issue at GDC this year.  Far from just a valuable personality asset, the ability to communicate well must be considered a top priority: as intrinsically valuable as rock-solid competency, awesome artistry or compelling vision. Good communication amongst team members can make or break the development of a game. As game audio pros, we share this in common with our coworkers in other segments of the game development community. However, it becomes especially important for us to focus and emphasize good communication when we’re working remotely as independent contractors. With that in mind, I thought I’d use this article to briefly highlight some GDC 2016 sessions in the game audio track that discussed this popular topic, so we can think about more ways to enhance and improve our communication skills.  And later we’ll discuss a practical example from my work on the music of the SimAnimals game from Electronic Arts.

Continue reading

Video Game Music Composer: How To Break Into the Business

Video game composer Winifred Phillips, working on the music of Homefront: The Revolution in her production studio.As a video game composer and author of the book A Composer’s Guide to Game Music, I’m frequently asked for advice on how a young composer can gain entry into this business.  I dedicated a chapter of my book to this topic (Chapter 14: Acting Like a Business and Finding Work), so I’ve certainly thought a great deal about the issue.  From my very first project (God of War) all the way to my most recent game (Homefront The Revolution, pictured right), one thing has always been abundantly clear: landing gigs can be a complex journey.  That’s especially true for newcomers, and there are no easy signposts pointing the way. While I tried to use my own experiences and insights to provide useful guidance in my book, I know that everyone’s experience is different, and multiple points of view can be very helpful.  So in this article, I’ll be offering resources from articles and community discussions on how to face down the awesome challenges of breaking into the industry as a composer of music for games.

First, I’ll be sharing a video from my presentation at the Society of Composers and Lyricists seminar, in which I answered the question about how I got my start in the games industry.  Then, we’ll be exploring highlights from a collection of online articles that offer helpful tips for how to break in and establish a career as a game composer.  Finally, at the end of this article I’ll be including a full list of links for further reading and reference.

Continue reading

VR Game Composer: Music Beyond the Virtual

Photo of video game music composer Winifred Phillips, from the article entitled "VR Game Composer: Music Beyond the Virtual."Welcome to the third installment in our series on the fascinating possibilities created by virtual reality motion tracking, and how the immersive nature of VR may serve to inspire us as video game composers and afford us new and innovative tools for music creation.  As modern composers, we work with a lot of technological tools, as I can attest from the studio equipment that I rely on daily (pictured left). Many of these tools communicate with each other by virtue of the Musical Instrument Digital Interface protocol, commonly known as MIDI – a technical standard that allows music devices and software to interact.

Image depicting VR apps from the article by Winifred Phillips, Game Music Composer.In order for a VR music application to control and manipulate external devices, the software must be able to communicate by way of the MIDI protocol – and that’s an exciting development in the field of music creation in VR!

This series of articles focuses on what VR means for music composers and performers. In previous installments, we’ve had some fun exploring new ways to play air guitar and air drums, and we’ve looked at top VR applications that provide standalone virtual instruments and music creation tools.  Now we’ll be talking about the most potentially useful application of VR for video game music composers – the ability to control our existing music production tools from within a VR environment.

We’ll explore three applications that employ MIDI to connect music creation in VR to our existing music production tools. But first, let’s take a look at another, much older gesture-controlled instrument that in ways is quite reminiscent of these motion-tracking music applications for VR:

Continue reading

VR Game Composer: Music Inside the Machine

Illustration for an article in the VR Game Composer series, written by video game composer Winifred PhillipsWelcome to part two of our ongoing exploration of some interesting possibilities created by the motion tracking capabilities of VR, and how this might alter our creative process as video game composers.

In part one we discussed how motion tracking lets us be awesome air guitarists and drummers inside the virtual space.  In this article, we’ll be taking a look at how the same technology will allow us to make interesting music using more serious tools that are incorporated directly inside the VR environment – musical instruments that exist entirely within the VR ‘machine.’

Our discussion to follow will concentrate on three software applications: Soundscape, Carillon, and Lyra.  Later, in the third article of this ongoing series, we’ll take a look at applications that allow our VR user interfaces to harness the power of MIDI to control some of the top music devices and software that we use in our external production studios. But first, let’s look at the ways that VR apps can function as fully-featured musical instruments, all on their own!

Soundscape

Let’s start with something simple – a step sequencer with a sound bank and signal processing tools, built for the mobile virtual reality experience of the Samsung Gear VR.

Video game composer Winifred Phillips demonstrating the Samsung Gear VR headset during the AES convention in NYC.I got a chance to demo the Samsung Gear VR during the Audio Engineering Society Convention in NYC last year, and while it doesn’t offer the best or most mind-blowing experience in VR (such as what we can experience from products like the famous Oculus Rift), it does achieve a satisfying level of immersion. Plus, it’s great fun!  The Soundscape VR app was built for Samsung Gear VR by developer Sander Sneek of the Netherlands.  It’s a simple app designed to enable users to create dance loops using three instruments from a built-in electro sound library, a pentatonic step sequencer that enables the user to create rhythm and tone patterns within the loops, and a collection of audio signal processing effects that let the user warp and mold the sounds as the loops progress, adding variety to the performance.

Continue reading

VR Game Composer: Music in the Air

Illustration for the article by game music composer Winifred Phillips, entitled "VR Game Composer: Music in the Air"Since I’ve been working recently on music for a Virtual Reality project (more info in the coming months), I’ve been thinking a lot about VR technology and its effect on the creative process.  Certainly, VR is going to be a great environment in which to be creative and perform tasks and skills with enhanced focus, according to this article from the VR site SingularityHub.  I’ve written in this blog before about the role that music and sound will play in the Virtual Reality gaming experience.  It’s clear that music will have an impact on the way in which we experience VR, not only during gaming experiences, but also when using the tools of VR to create and be productive.  With that in mind, let’s consider if the opposite statement may also be true – will VR impact the way in which we experience music, not only as listeners, but also as video game composers?

Game composer Winifred Phillips tries out the VR experience of Google Cardboard (pictured here in her music production studio).Simple VR technologies like the popular Google Cardboard headset can be a lot of fun – as I personally experienced recently (photo to the left).  However, they offer only the rudimentary visual aspects, which omits some of the most compelling aspects of the VR experience.  When motion tracking (beyond simple head movement) is added to the mix, the potential of VR explodes.  Over the next three articles, we’ll be exploring some interesting possibilities created by the motion tracking capabilities of VR, and how this might alter our creative process.  In the first article, we’ll have some fun exploring new ways to play air guitars and air drums in the VR environment. In the second article, we’ll take a look at ways to control virtual instruments and sound modules that are folded into the VR software.  And finally, in the third article we’ll explore the ways in which VR motion tracking is allowing us to immersively control our existing real-world instruments using MIDI. But first, let’s take a look at the early days of VR musical technology!

Continue reading