Video Game Composers: How Music Enhances Virtual Presence (The Theory of Flow)

Video game music composer Winifred Phillips in her video game music production studio working on the music of "The Haunted Graveyard" VR game.

By Winifred Phillips | ContactFollow

Hello there!  I’m video game composer Winifred Phillips.  At this year’s Game Developers Conference in San Francisco, I was pleased to give a presentation entitled How Music Enhances Virtual Presence (I’ve included the official description of my talk at the end of this article). The talk I delivered at GDC gave me the opportunity to pull a lot of ideas about virtual reality together and present a concentrated exploration of how music can increase a sensation of presence for VR gamers.  It occurred to me that such a discussion might be interesting to share in this forum as well. So, with that in mind, I’m excited to begin a four-part article series based on my GDC 2019 presentation!

An illustration of famous virtual reality headset design, in the article for game composers by video game music composer Winifred Phillips.Over the past couple of years, a lot of development studios have hired me to create music for virtual reality games. It’s fascinating work!  In my GDC talk, I discussed virtual presence in connection with seven of the virtual reality games and experiences that I’ve scored, which have either released within the past year or will be released within the coming months.  These include the PSVR version of the Scraper: First Strike VR shooter that was released just last week by Labrodex Inc, and numerous other VR titles including  Audioshield (Audiosurf LLC), Bebylon Battle Royale (Kite & Lightning), Fail Factory (Armature Studio), The Haunted Graveyard (Holospark), Life Hutch VR (Next Stop Willoughby), and Shattered State (Supermassive Games).

Gaming in virtual reality involves an engulfing 360 degree spherical environment and a sense of physical agency never-before possible with traditional gaming. Unlike other forms of video gaming, players in VR have the best chance to feel bodily present inside the virtual world. This sense of Virtual Presence offers an awesome opportunity for expert development teams to create powerful gaming experiences.

During this four-part article series, we’ll explore what Virtual Presence is. We’ll take a look at some creative composition strategies we can use to induce Virtual Presence.  We’ll discuss some of the possible drawbacks of Virtual Presence, and what we video game composers can do to support players and keep the fun going during their adventures in VR. I hope you’ll find these creative strategies to be useful in your own VR projects. Keep in mind that, while all these ideas are primarily designed to make gaming more fun in the popular VR gaming platform, they are essentially meant to get players more viscerally involved in any kind of gameplay, which means they can be applied to traditional game projects too.

Let’s start with the basics.

What is Virtual Presence?

Image illustrating the concept of Virtual Presence, from the article by Winifred Phillips for video game composersAccording to Professor Thomas B. Sheridan of the Massachusetts Institute of Technology, Virtual Presence is defined as the sensation of being “present in the environment generated by the computer.” In order to feel virtually present, we have to accept the in-game location as a tangible place. According to Professor Sheridan, there are lots of methods that VR designers can use to accentuate the reality of their virtual environments. They can shower players with sensory stimuli. They can encourage players to move about, changing their relative viewpoint and altering their binaural soundscape. They can place objects in the environment that can be manipulated and changed by players, leading to a great sense of material engagement.

Logo for The Haunted Graveyard VR game developed by Holospark for VR Arcades (from the article about Virtual Presence by video game composer Winifred Phillips).Here’s an example – one of my most recent projects, The Haunted Graveyard, from developers Holospark. As a Halloween-inspired VR experience designed primarily to be accessible to a wide audience in top VR arcades, The Haunted Graveyard focuses on an involving, atmospheric landscape for players to explore. Virtual Presence is extremely important, so the game provides the necessary sensory stimuli, player movement and usable objects that Professor Sheridan described. However, the game also includes lots of music to keep players emotionally stimulated. Here’s a ten-minute gameplay video of The Haunted Graveyard:

IImage illustrating the concept of Virtual Presence, from the article by Winifred Phillips for video game composersn order for true Virtual Presence to be attained, players have to let go of their natural incredulity and get emotionally involved. They have to forget about the fact that they’re in VR, and just live the adventure.

As you saw from the gameplay video, the music in The Haunted Graveyard is an important part of the game’s design.

Music can be inspiring – it can make you feel more committed to what you’re doing, and more invested in the world around you. The question is – how does music enable Virtual Presence? And what tools can video game music composers use to make that happen?

In this four-part article series, we’re going to explore three ways wherein music enables Virtual Presence:

  • Music empowers Flow.
  • Music promotes psychological attachment.
  • Music provides an avenue for mood attenuation.

In this article, we’ll start with the first (and most famous) mechanism on our list:

The Theory of Flow

Cover of the book "Flow: The psychology of optimal experience" - section from the article by Winifred Phillips, video game music composer.First introduced in the book Flow: The Psychology of Optimal Experience, the Theory of Flow tries to explain what it means when we get so absorbed by some interesting task that we forget everything else – all distractions fall away and our minds sharpen to a laser-point of focus. Sometimes it’s called being “in the zone.” The feeling is an incredible rush. Moreover, it’s a useful tool when trying to instill the sensation of Virtual Presence in our players.

In a study published in the journal Computers in Human Behavior, researchers from Indiana University found that the Flow phenomenon, when experienced in a virtual world, can enable and enhance the sensation of Virtual Presence. In other words, when we’re feeling “in the zone” during VR, we also tend to feel more like we’re actually inside an alternate world.

With that in mind, how can we game music composers use this phenomenon to our advantage? Well, science has shown us that music has a direct correlation with the Flow state. In an experiment conducted at Brunel University, researchers studied athletes who listened to music during their exercise routines. The researchers found that music had a strong beneficial effect on the athletes’ flow state, helping them to get “into the zone.”

An image depicting the logo of the Audioshield VR rhythm game - section from the article by Winifred Phillips, video game music composer.Here’s an example – one of my most recent projects was the main theme for Audioshield – a music rhythm game for VR. Using a shield in each hand, players block glowing orbs that fly towards them in synch with the music. The game analyzes the unique qualities of the music to determine the type and frequency of orbs coming towards the player.

Because Audioshield analyzes music and then constructs gameplay around it, composing music for Audioshield was a real trial-and-error process. All during music production, the game designer and I both repeatedly loaded the music into the game and played through it, looking for any orb activity that felt jarring, or choppy, or rough – anything that would interrupt Flow. I’d go back into the music studio, make changes – then we’d play through the track in Audioshield again. Were the orbs showing up in ways that were too surprising? Or not surprising enough?

We were looking for a sweet spot, when Flow would grab hold, when the player would be firmly in the zone. And when that happened, it felt like everything came together. The gameplay was fun, the music was driving, and you really felt like you were there. Everything seemed tangible and real. Flow and Virtual Presence kicked in together. Here’s a video showing how gameplay worked during the main theme of Audioshield:

So we’ve now discussed the relationship between Flow Theory and the concept of Virtual Presence.  In our next article, we’ll examine the second mechanism by which music enables Virtual Presence – psychological attachment.  Thanks for reading!


 

How Music Enhances Virtual Presence

Compilation of images depicting popular game titles for VR platforms that are included in the GDC 2019 lecture of game composer Winifred Phillips.

(Game Developers Conference Session Description)

Virtual Presence is defined as a state in which gamers fully accept the virtual world around them and their existence within it. This talk, “How Music Enhances Virtual Presence,” will explore how highly effective game music can enhance the sensation of Virtual Presence in VR gaming.

The talk will begin with an exploration of both the Flow Theory of Mihaly Csikszentmihalyi and the research of Dr. Paul Cairns on psychological engagement in video gaming. By understanding how the mental activity of players interacts with the way a game is designed, composers can create music intended to induce psychological states conducive with the formation of Virtual Presence.

The talk will include a discussion of techniques aimed at drawing attention to mission objectives, facilitating effective concentration, enhancing emotional empathy and intensifying player focus. The discussion will also include an exploration of some inherent drawbacks to Virtual Presence, including its fragility when exposed to negative emotional states, and its possible susceptibility to inducing the “event boundary” phenomenon. Musical solutions to these problems will be explored.

Phillips’ talk will offer techniques for composers and audio directors who seek to employ music as a tool to enhance Virtual Presence for their players.

Takeaway

Using examples from several games, Phillips will explore how music can influence the mental states of players through specific effects documented in scientific research. Study data will be discussed in regards to the interaction between music and cognition. Phillips will offer strategies and tips for composers seeking to use their music to influence the player’s mental state, thus facilitating the formation of Virtual Presence.

Intended Audience

This session is intended to inspire and stimulate composers seeking to employ their music towards enhancing player engagement and enjoyment, with a particular emphasis on VR games. Includes overview of Flow Theory and the psychological components of Virtual Presence, which may be useful to other disciplines within game development. Talk will be approachable for all levels (advanced composers may better appreciate the specific composition techniques discussed).

 

Photo depicting video game composer Winifred Phillips working in her music production studio.

Popular music from composer Winifred Phillips’ award-winning Assassin’s Creed Liberation score is currently being performed live by a top 80-piece orchestra and choir as part of the Assassin’s Creed Symphony World Tour, which kicked off in June 2019 with its Paris premiere. As an accomplished video game composer, Phillips is best known for composing music for games in five of the most famous and popular franchises in gaming: Assassin’s Creed, LittleBigPlanet, Total War, God of War, and The Sims.  Phillips’ other notable projects include the triple-A first person shooter Homefront: The Revolution, and numerous virtual reality games, including Sports Scramble, Audioshield, Scraper: First Strike, Dragon Front, and many more.   She is the author of the award-winning bestseller A COMPOSER’S GUIDE TO GAME MUSIC, published by the MIT Press. As a VR game music expert, she writes frequently on the future of music in virtual reality games. Phillips’ is a sought-after public speaker, and she has been invited to speak about her work as a game composer at the Library of Congress, the Game Developers Conference, the Audio Engineering Society, the Society of Composers and Lyricists, and many more.  Follow her on Twitter @winphillips.

Composer Winifred Phillips answers Reddit’s questions in viral Ask-Me-Anything about video game music

Photo of popular video game composer Winifred Phillips, taken as 'proof photo' for her recent viral Reddit Ask-Me-Anything that hit the Reddit front page, receiving 14.8 thousand upvotes and garnering Reddit's gold and platinum awards.

By Winifred Phillips | Contact | Follow

Glad you’re here!  I’m video game music composer Winifred Phillips, and I’m the author of the book A Composer’s Guide to Game Music.  Recently my publisher The MIT Press requested that I host a question and answer session on Reddit’s famous Ask Me Anything forum, to share my knowledge about game music and spread the word about my book on that topic.  I’d be answering questions from a community consisting of thousands of gamers, developers and aspiring composers.  It sounded like fun, so last Thursday and Friday I logged onto Reddit and answered as many questions as I possibly could.  It was an awesome experience!  Over the course of those two days, my Reddit AMA went viral.  It ascended to the Reddit front page, receiving 14.8 thousand upvotes and garnering Reddit’s gold and platinum awards.  My AMA has now become one of the most engaged and popular Reddit gaming AMAs ever hosted on the Ask-Me-Anything subreddit.  I’m so grateful to the Reddit community for their amazing support and enthusiasm!!  During the course of those two days, the community posed some wonderful questions, and I thought it would be great to gather together some of those questions and answers that might interest us here.  Below you’ll find a discussion focused on the art and craft of game music composition.  The discussion covered the gamut of subjects, from elementary to expert, and I’ve arranged the discussion below under topic headings for the sake of convenience.  I hope you enjoy this excerpted Q&A from my Reddit Ask-Me-Anything!  If you’d like to read the entire AMA (which also includes lots of discussion of my past video game music projects), you’ll find the whole Reddit AMA here.

Continue reading

Video Game Composers: How Music Enhances Virtual Presence (GDC 2019)

In this article about Virtual Presence in VR written for video game composers, Winifred Phillips (video game composer) is here pictured working in her music production studio.

By Winifred Phillips | Contact | Follow

Delighted you’re here!  I’m very pleased to share that over the next two months I’ll be speaking at two fantastic events focusing on music in video games!  My two presentations will explore the unique structure and character of video game music, and how it helps to better envelop players in the worlds that game designers have created.  I thought that this article might be a good opportunity to delve into some of the ideas that form the basis of my two upcoming talks.  First, I’d like to share some details about the presentations I’ll be giving.

The Library of Congress logo, included in an article discussing popular game conferences, from the article for video game composers by Winifred Phillips (game music composer).The Library of Congress has invited me to speak this April as a part of their “Augmented Realities” video game music festival. My presentation, “The Interface Between Music Composition and Game Design,” will take place at the Library of Congress in Washington DC. I’m very excited to participate in this event, which will be the first of its kind hosted by the “Concerts from the Library” series at the Library of Congress! The “Augmented Realities” video game music festival will also include panels on video game music history and preservation presented by distinguished curators and archivists at the Library of Congress, a special documentary screening that explores the ChipTunes movement, and a live “game creation lab.” My presentation will be the concluding lecture of the festival, and I’m honored to speak at such an illustrious event!  If you find yourself in the Washington DC area on April 6th 2019, you’re very welcome to come to my lecture at the Library of Congress!  Tickets are free (first come, first served), and they’re available now via EventBrite.

The GDC logo, accompanying the discussion of networking at such famous game conferences, from the article for video game composers by Winifred Phillips (game music composer).But before my lecture at the Library of Congress, I’ll be making a trip to San Francisco for the famous Game Developers Conference that takes place this month. For the past few years I’ve been excited and honored to be selected as a Game Developers Conference speaker in the Game Audio track, and I’m happy to share that I’ll be speaking again this month in San Francisco at GDC 2019! My talk this year is entitled “How Music Enhances Virtual Presence.

Continue reading

Tips, Tools and Guidance: VR Assets for the Game Music Composer

 

Video game music composer Winifred Phillips hard at work on the Audioshield Fitness VR project in her video game music production studio.

By Winifred Phillips | Contact Follow

Welcome!  I’m videogame composer Winifred Phillips, and my projects lately have included music composition for a lot of great virtual reality games.  It’s been fascinating work!  Last year, when my VR work started to really pick up, I wrote an article with lots of resources to help video game music composers become more comfortable in the world of VR audio development.  Since this discipline progresses rapidly, I thought it would be best to post an update article now that adds additional resources to address new developments in the field.

VR development is continuously innovative and cutting-edge, and I’ve been fortunately to experience this first-hand.  As an example: one of my more recent virtual reality game projects was music for Audioshield Fitness, developed by the creator of the famous Audiosurf music-rhythm game. I was asked to compose the new official Audioshield Theme for release with the Audioshield Fitness game, which takes the core game mechanics of Audioshield and pumps up the challenge with obstacles that make players dodge and duck to the music.  The result is an intense workout that was named as one of the top 5 VR Fitness Games of 2018 by PerfectBodyMate.com. To maximize the power of the Audioshield procedural system, my composition had to attune itself to the system’s powerful music analysis algorithm and deliver moments of both challenge and spectacle. I composed and mixed the music with specifically-targeted EQ frequency ranges where I placed rhythmic elements and punchy crescendoes.  The Audioshield music analysis system then reacted to this audio content and changed the pacing and content of gameplay to match these variables.  It was a fun challenge!  Here’s a video showing how that worked:

 

Composing for virtual reality is its own unique discipline, requiring a specialized set of skills and tools.  In this article, let’s collect some resources that explore the techniques, tools, and technologies associated with VR audio development.  Let’s also take a look at the professional community of VR developers that are there to help each other through the rough spots.  Ready?  Let’s go!

Continue reading

VR for the Game Music Composer: Audio for VR Platforms

In this article written for video game composers, Winifred Phillips (video game composer) is here pictured working in her music production studio on the music for the Scraper: First Strike game, developed for popular VR gaming platforms (PSVR, Oculus Rift, HTC Vive).

By Winifred Phillips | Contact | Follow

Hello there!  I’m video game music composer Winifred Phillips.  Lately, I’ve been very busy in my production studio composing music for a lot of awesome virtual reality games, including the upcoming Scraper: First Strike first person VR shooter (pictured above) that’s coming out next Wednesday (November 21st) for the Oculus Rift, HTC Vive and Windows Mixed Reality Devices, and will be released on December 18th for the Playstation VR.  My work on this project has definitely stoked my interest in everything VR!  Since the game will be released very soon, here’s a trailer video released by the developers Labrodex Studios, featuring some of the music I composed for the game:

Continue reading

Video Game Music Composers: New VR Headphone Tech (2018)

By Winifred Phillips | Contact | Follow

In this article for and about the craft of video game composers, Winifred Phillips is pictured in this photo from her lecture on Virtual Reality given at the popular Game Developers Conference in 2018.Hey, everyone!  I’m videogame composer Winifred Phillips, and my work has included the musical scores for top games on all sorts of popular gaming platforms, from handhelds and mobile, all the way up to the latest consoles and PCs.  Lately, I’ve been doing a lot of video game music composition for virtual reality.  I had the pleasure of presenting a lecture on Music in Virtual Reality (pictured left) at the most recent Game Developers Conference in San Francisco.

My experience as a composer for VR includes many VR games, including the Scraper: First Strike shooter (set to be released for the PSVR, Oculus Rift and HTC Vive in December 2018), and the recently released VR experience The Haunted Graveyard, which is now available on Steam and in VR Arcades around the world.  Since we’re in the Halloween season, and this VR experience is designed specifically for your Halloween pleasure, here’s a trailer that features my music from The Haunted Graveyard:

By virtue of all the experiences I’ve had recently creating music for VR, I’ve become keenly aware of the importance of sound fidelity in VR.  If the experience doesn’t sound real, it loses the chance to actually feel like a fully-convincing, thoroughly awesome virtual reality experience.  With that in mind, I’ve been writing periodic articles about new technologies in connection with headphones for VR.

Continue reading

VR Headphones Update 2018: Video Game Music Composers

In this article written for video game composers, Winifred Phillips (composer of music for God of War) is here pictured working in her music production studio.

By Winifred Phillips | Contact | Follow

Glad you’re here!  I’m videogame composer Winifred Phillips.  My work as a game music composer has included music for projects released on nearly all of the gaming platforms, from one of my most recent projects (a Homefront game released on all the latest consoles and PCs) to one of my earliest projects (a God of War game released on PlayStation 2, PlayStation 3, and PlayStation Vita, pictured above).  An image of the September 2018 cover of Music Connection Magazine featuring the article "Video Game Composers Speak!" - features interviews of famous game music composers, including popular game music composer Winifred Phillips.You can read about my work as a video game composer in an interview I gave to Music Connection Magazine for this month’s issue (pictured right).

Lately, I’ve also been creating lots of video game music for awesome virtual reality games developed for the Oculus Rift, Oculus Go, HTC Vive, Samsung Gear VR, PlayStation VR, and lots of other top VR platforms.  One of the things I’ve noticed while working in VR is the immense importance of the audio delivery mechanism.

When audio is painstakingly spatialized, it becomes crucial to convey that carefully-crafted spatialization to the player with as little fidelity loss as possible.  With the importance of this issue in mind, for the past few years I’ve been periodically writing about headphones in relation to their use in virtual reality.

Continue reading