Video Game Composers: How Music Enhances Virtual Presence (The Theory of Flow)

Video game music composer Winifred Phillips in her video game music production studio working on the music of "The Haunted Graveyard" VR game.

By Winifred Phillips | ContactFollow

Hello there!  I’m video game composer Winifred Phillips.  At this year’s Game Developers Conference in San Francisco, I was pleased to give a presentation entitled How Music Enhances Virtual Presence (I’ve included the official description of my talk at the end of this article). The talk I delivered at GDC gave me the opportunity to pull a lot of ideas about virtual reality together and present a concentrated exploration of how music can increase a sensation of presence for VR gamers.  It occurred to me that such a discussion might be interesting to share in this forum as well. So, with that in mind, I’m excited to begin a four-part article series based on my GDC 2019 presentation!

An illustration of famous virtual reality headset design, in the article for game composers by video game music composer Winifred Phillips.Over the past couple of years, a lot of development studios have hired me to create music for virtual reality games. It’s fascinating work!  In my GDC talk, I discussed virtual presence in connection with seven of the virtual reality games and experiences that I’ve scored, which have either released within the past year or will be released within the coming months.  These include the PSVR version of the Scraper: First Strike VR shooter that was released just last week by Labrodex Inc, and numerous other VR titles including  Audioshield (Audiosurf LLC), Bebylon Battle Royale (Kite & Lightning), Fail Factory (Armature Studio), The Haunted Graveyard (Holospark), Life Hutch VR (Next Stop Willoughby), and Shattered State (Supermassive Games).

Gaming in virtual reality involves an engulfing 360 degree spherical environment and a sense of physical agency never-before possible with traditional gaming. Unlike other forms of video gaming, players in VR have the best chance to feel bodily present inside the virtual world. This sense of Virtual Presence offers an awesome opportunity for expert development teams to create powerful gaming experiences.

During this four-part article series, we’ll explore what Virtual Presence is. We’ll take a look at some creative composition strategies we can use to induce Virtual Presence.  We’ll discuss some of the possible drawbacks of Virtual Presence, and what we video game composers can do to support players and keep the fun going during their adventures in VR. I hope you’ll find these creative strategies to be useful in your own VR projects. Keep in mind that, while all these ideas are primarily designed to make gaming more fun in the popular VR gaming platform, they are essentially meant to get players more viscerally involved in any kind of gameplay, which means they can be applied to traditional game projects too.

Let’s start with the basics.

What is Virtual Presence?

Image illustrating the concept of Virtual Presence, from the article by Winifred Phillips for video game composersAccording to Professor Thomas B. Sheridan of the Massachusetts Institute of Technology, Virtual Presence is defined as the sensation of being “present in the environment generated by the computer.” In order to feel virtually present, we have to accept the in-game location as a tangible place. According to Professor Sheridan, there are lots of methods that VR designers can use to accentuate the reality of their virtual environments. They can shower players with sensory stimuli. They can encourage players to move about, changing their relative viewpoint and altering their binaural soundscape. They can place objects in the environment that can be manipulated and changed by players, leading to a great sense of material engagement.

Logo for The Haunted Graveyard VR game developed by Holospark for VR Arcades (from the article about Virtual Presence by video game composer Winifred Phillips).Here’s an example – one of my most recent projects, The Haunted Graveyard, from developers Holospark. As a Halloween-inspired VR experience designed primarily to be accessible to a wide audience in top VR arcades, The Haunted Graveyard focuses on an involving, atmospheric landscape for players to explore. Virtual Presence is extremely important, so the game provides the necessary sensory stimuli, player movement and usable objects that Professor Sheridan described. However, the game also includes lots of music to keep players emotionally stimulated. Here’s a ten-minute gameplay video of The Haunted Graveyard:

IImage illustrating the concept of Virtual Presence, from the article by Winifred Phillips for video game composersn order for true Virtual Presence to be attained, players have to let go of their natural incredulity and get emotionally involved. They have to forget about the fact that they’re in VR, and just live the adventure.

As you saw from the gameplay video, the music in The Haunted Graveyard is an important part of the game’s design.

Music can be inspiring – it can make you feel more committed to what you’re doing, and more invested in the world around you. The question is – how does music enable Virtual Presence? And what tools can video game music composers use to make that happen?

In this four-part article series, we’re going to explore three ways wherein music enables Virtual Presence:

  • Music empowers Flow.
  • Music promotes psychological attachment.
  • Music provides an avenue for mood attenuation.

In this article, we’ll start with the first (and most famous) mechanism on our list:

The Theory of Flow

Cover of the book "Flow: The psychology of optimal experience" - section from the article by Winifred Phillips, video game music composer.First introduced in the book Flow: The Psychology of Optimal Experience, the Theory of Flow tries to explain what it means when we get so absorbed by some interesting task that we forget everything else – all distractions fall away and our minds sharpen to a laser-point of focus. Sometimes it’s called being “in the zone.” The feeling is an incredible rush. Moreover, it’s a useful tool when trying to instill the sensation of Virtual Presence in our players.

In a study published in the journal Computers in Human Behavior, researchers from Indiana University found that the Flow phenomenon, when experienced in a virtual world, can enable and enhance the sensation of Virtual Presence. In other words, when we’re feeling “in the zone” during VR, we also tend to feel more like we’re actually inside an alternate world.

With that in mind, how can we game music composers use this phenomenon to our advantage? Well, science has shown us that music has a direct correlation with the Flow state. In an experiment conducted at Brunel University, researchers studied athletes who listened to music during their exercise routines. The researchers found that music had a strong beneficial effect on the athletes’ flow state, helping them to get “into the zone.”

An image depicting the logo of the Audioshield VR rhythm game - section from the article by Winifred Phillips, video game music composer.Here’s an example – one of my most recent projects was the main theme for Audioshield – a music rhythm game for VR. Using a shield in each hand, players block glowing orbs that fly towards them in synch with the music. The game analyzes the unique qualities of the music to determine the type and frequency of orbs coming towards the player.

Because Audioshield analyzes music and then constructs gameplay around it, composing music for Audioshield was a real trial-and-error process. All during music production, the game designer and I both repeatedly loaded the music into the game and played through it, looking for any orb activity that felt jarring, or choppy, or rough – anything that would interrupt Flow. I’d go back into the music studio, make changes – then we’d play through the track in Audioshield again. Were the orbs showing up in ways that were too surprising? Or not surprising enough?

We were looking for a sweet spot, when Flow would grab hold, when the player would be firmly in the zone. And when that happened, it felt like everything came together. The gameplay was fun, the music was driving, and you really felt like you were there. Everything seemed tangible and real. Flow and Virtual Presence kicked in together. Here’s a video showing how gameplay worked during the main theme of Audioshield:

So we’ve now discussed the relationship between Flow Theory and the concept of Virtual Presence.  In our next article, we’ll examine the second mechanism by which music enables Virtual Presence – psychological attachment.  Thanks for reading!


 

How Music Enhances Virtual Presence

Compilation of images depicting popular game titles for VR platforms that are included in the GDC 2019 lecture of game composer Winifred Phillips.

(Game Developers Conference Session Description)

Virtual Presence is defined as a state in which gamers fully accept the virtual world around them and their existence within it. This talk, “How Music Enhances Virtual Presence,” will explore how highly effective game music can enhance the sensation of Virtual Presence in VR gaming.

The talk will begin with an exploration of both the Flow Theory of Mihaly Csikszentmihalyi and the research of Dr. Paul Cairns on psychological engagement in video gaming. By understanding how the mental activity of players interacts with the way a game is designed, composers can create music intended to induce psychological states conducive with the formation of Virtual Presence.

The talk will include a discussion of techniques aimed at drawing attention to mission objectives, facilitating effective concentration, enhancing emotional empathy and intensifying player focus. The discussion will also include an exploration of some inherent drawbacks to Virtual Presence, including its fragility when exposed to negative emotional states, and its possible susceptibility to inducing the “event boundary” phenomenon. Musical solutions to these problems will be explored.

Phillips’ talk will offer techniques for composers and audio directors who seek to employ music as a tool to enhance Virtual Presence for their players.

Takeaway

Using examples from several games, Phillips will explore how music can influence the mental states of players through specific effects documented in scientific research. Study data will be discussed in regards to the interaction between music and cognition. Phillips will offer strategies and tips for composers seeking to use their music to influence the player’s mental state, thus facilitating the formation of Virtual Presence.

Intended Audience

This session is intended to inspire and stimulate composers seeking to employ their music towards enhancing player engagement and enjoyment, with a particular emphasis on VR games. Includes overview of Flow Theory and the psychological components of Virtual Presence, which may be useful to other disciplines within game development. Talk will be approachable for all levels (advanced composers may better appreciate the specific composition techniques discussed).

 

Photo depicting video game composer Winifred Phillips working in her music production studio.

Popular music from composer Winifred Phillips’ award-winning Assassin’s Creed Liberation score is currently being performed live by a top 80-piece orchestra and choir as part of the Assassin’s Creed Symphony World Tour, which kicked off in June 2019 with its Paris premiere. As an accomplished video game composer, Phillips is best known for composing music for games in five of the most famous and popular franchises in gaming: Assassin’s Creed, LittleBigPlanet, Total War, God of War, and The Sims.  Phillips’ other notable projects include the triple-A first person shooter Homefront: The Revolution, and numerous virtual reality games, including Sports Scramble, Audioshield, Scraper: First Strike, Dragon Front, and many more.   She is the author of the award-winning bestseller A COMPOSER’S GUIDE TO GAME MUSIC, published by the MIT Press. As a VR game music expert, she writes frequently on the future of music in virtual reality games. Phillips’ is a sought-after public speaker, and she has been invited to speak about her work as a game composer at the Library of Congress, the Game Developers Conference, the Audio Engineering Society, the Society of Composers and Lyricists, and many more.  Follow her on Twitter @winphillips.

Video Game Composers: How Music Enhances Virtual Presence (GDC 2019)

In this article about Virtual Presence in VR written for video game composers, Winifred Phillips (video game composer) is here pictured working in her music production studio.

By Winifred Phillips | Contact | Follow

Delighted you’re here!  I’m very pleased to share that over the next two months I’ll be speaking at two fantastic events focusing on music in video games!  My two presentations will explore the unique structure and character of video game music, and how it helps to better envelop players in the worlds that game designers have created.  I thought that this article might be a good opportunity to delve into some of the ideas that form the basis of my two upcoming talks.  First, I’d like to share some details about the presentations I’ll be giving.

The Library of Congress logo, included in an article discussing popular game conferences, from the article for video game composers by Winifred Phillips (game music composer).The Library of Congress has invited me to speak this April as a part of their “Augmented Realities” video game music festival. My presentation, “The Interface Between Music Composition and Game Design,” will take place at the Library of Congress in Washington DC. I’m very excited to participate in this event, which will be the first of its kind hosted by the “Concerts from the Library” series at the Library of Congress! The “Augmented Realities” video game music festival will also include panels on video game music history and preservation presented by distinguished curators and archivists at the Library of Congress, a special documentary screening that explores the ChipTunes movement, and a live “game creation lab.” My presentation will be the concluding lecture of the festival, and I’m honored to speak at such an illustrious event!  If you find yourself in the Washington DC area on April 6th 2019, you’re very welcome to come to my lecture at the Library of Congress!  Tickets are free (first come, first served), and they’re available now via EventBrite.

The GDC logo, accompanying the discussion of networking at such famous game conferences, from the article for video game composers by Winifred Phillips (game music composer).But before my lecture at the Library of Congress, I’ll be making a trip to San Francisco for the famous Game Developers Conference that takes place this month. For the past few years I’ve been excited and honored to be selected as a Game Developers Conference speaker in the Game Audio track, and I’m happy to share that I’ll be speaking again this month in San Francisco at GDC 2019! My talk this year is entitled “How Music Enhances Virtual Presence.

Continue reading

Video game music composer: Getting your big break

In this article for video game composers, popular game composer Winifred Phillips is depicted in this photo working in her music production studio.

By Winifred Phillips | Contact | Follow

So happy you’ve joined us!  I’m videogame composer Winifred Phillips (pictured above working on my career breakthrough project, God of War). Today I’ll be discussing a hot topic that we’ve previously explored, but that definitely deserves to be revisited periodically.  This is one of the most popular subjects that I’ve addressed in my previous articles here: How does a newcomer get hired as a game composer?

I’m asked this question frequently, and while I offered quite a lot of advice on this topic in my book A Composer’s Guide to Game Music, I’m keenly aware of how urgent the need is for updated guidance on this issue for aspiring video game composers.  Game music newcomers often feel adrift and alone in the game industry, and some good advice can be a welcome lifeline.  In my book, I described the career path that led me into the game industry and allowed me to land my first gigs, but I’m well aware that my experience was pretty unique.  With that in mind, I’ve collated some recent research and insights from some top game industry professionals in this article, in the hopes that some of these expert observations might prove helpful.  There are lots of original and provocative viewpoints presented here, so we should feel free to pick and choose the strategies and tips that will work best for us.

Also, later in the article you’ll find my presentation for the Society of Composers and Lyricists seminar, in which I answered the question about how I personally got my start in the games industry (for those who might be curious).  Finally, at the end of the article I have included a full list of links for further reading and reference.

Continue reading

Resources for Video Game Music Composers: The Big List

Video game music composer Winifred Phillips creating music in her video game music production studio.

By Winifred Phillips | Contact | Follow

Hey everybody!  I’m videogame composer Winifred Phillips.  Every year, between working in my studio creating music for some awesome games, I like to take a little time to gather together some of the top online resources and guidance available for newbies in the field of video game music.  What follows in this article is an updated and expanded collection of links on a variety of topics pertinent to our profession.  We begin with the concert tours and events where we can get inspired by seeing game music performed live.  Then we’ll move on to a discussion of online communities that can help us out when we’re trying to solve a problem.  Next, we’ll see a collection of software tools that are commonplace in our field.  Finally, we’ll check out some conferences and academic organizations where we can absorb new ideas and skills.

Continue reading

VR for the Game Music Composer: Audio for VR Platforms

In this article written for video game composers, Winifred Phillips (video game composer) is here pictured working in her music production studio on the music for the Scraper: First Strike game, developed for popular VR gaming platforms (PSVR, Oculus Rift, HTC Vive).

By Winifred Phillips | Contact | Follow

Hello there!  I’m video game music composer Winifred Phillips.  Lately, I’ve been very busy in my production studio composing music for a lot of awesome virtual reality games, including the upcoming Scraper: First Strike first person VR shooter (pictured above) that’s coming out next Wednesday (November 21st) for the Oculus Rift, HTC Vive and Windows Mixed Reality Devices, and will be released on December 18th for the Playstation VR.  My work on this project has definitely stoked my interest in everything VR!  Since the game will be released very soon, here’s a trailer video released by the developers Labrodex Studios, featuring some of the music I composed for the game:

Continue reading

Video Game Music Composers: New VR Headphone Tech (2018)

By Winifred Phillips | Contact | Follow

In this article for and about the craft of video game composers, Winifred Phillips is pictured in this photo from her lecture on Virtual Reality given at the popular Game Developers Conference in 2018.Hey, everyone!  I’m videogame composer Winifred Phillips, and my work has included the musical scores for top games on all sorts of popular gaming platforms, from handhelds and mobile, all the way up to the latest consoles and PCs.  Lately, I’ve been doing a lot of video game music composition for virtual reality.  I had the pleasure of presenting a lecture on Music in Virtual Reality (pictured left) at the most recent Game Developers Conference in San Francisco.

My experience as a composer for VR includes many VR games, including the Scraper: First Strike shooter (set to be released for the PSVR, Oculus Rift and HTC Vive in December 2018), and the recently released VR experience The Haunted Graveyard, which is now available on Steam and in VR Arcades around the world.  Since we’re in the Halloween season, and this VR experience is designed specifically for your Halloween pleasure, here’s a trailer that features my music from The Haunted Graveyard:

By virtue of all the experiences I’ve had recently creating music for VR, I’ve become keenly aware of the importance of sound fidelity in VR.  If the experience doesn’t sound real, it loses the chance to actually feel like a fully-convincing, thoroughly awesome virtual reality experience.  With that in mind, I’ve been writing periodic articles about new technologies in connection with headphones for VR.

Continue reading