Game Music and Psychological Attachment: How Game Composers Can Enhance Virtual Presence (Pt. 2)

Photo of video game music composer Winifred Phillips working in her music production studio on the musical score of the Shattered State VR game from Supermassive Games.

By Winifred Phillips | ContactFollow

So happy you’ve joined us!  I’m video game composer Winifred Phillips, and this is the continuation of our four-part discussion of how music can enhance presence in virtual reality.  These articles are based on the presentation I gave at this year’s Game Developer’s Conference in San Francisco, entitled How Music Enhances Virtual Presence (I’ve included the official description of my talk at this end of this article). In my GDC talk, I discussed Virtual Presence in connection with seven of the virtual reality games and experiences that I’ve scored, which have either released within the past year or will be released within the coming months.  These include Audioshield (Audiosurf LLC), Bebylon Battle Royale (Kite & Lightning), Fail Factory (Armature Studio), The Haunted Graveyard (Holospark), Life Hutch VR (Next Stop Willoughby), Scraper: First Strike (Labrodex Inc), and Shattered State (Supermassive Games).  If you missed the first article exploring how Flow can support Virtual Presence in VR gaming, please go check that article out first.

Are you back?  Great!  Let’s continue!

Now that we’ve taken a look at how Flow can best enable Virtual Presence in VR, let’s look at the second mechanism by which music enables Virtual Presence:

Psychological Attachment

Image illustrating the three states contributing to the attainment of Virtual Presence (Engagement, Engrossment, and Empathy) -- from the article by video game composer Winifred Phillips.In a paper presented at the Computer-Human Interaction conference, a research team from Carnegie-Mellon defined Virtual Presence as “the extent to which a person’s Cognitive and perceptual systems are tricked into believing they are somewhere other than their physical location.” This assertion formed the jumping-off point for two researchers from University College London, who set out to define what specific circumstances could lead to Virtual Presence in gaming. They developed a model for how gamers developed the psychological attachment necessary to achieve Virtual Presence.  Their model consists of three stages:

  • Engagement
  • Engrossment
  • Empathy

So let’s start with the first stage.

Engagement

We can define engagement as the commitment of the player to remain focused on the game. Note that this is all about energy and attention – it’s not about emotion. Engagement is a motivational tool. Since music has long been one of the top methods to help motivate people, it makes sense to use music as a tool for helping players achieve engagement. This becomes especially important when striving for Virtual Presence. Let’s look at an example from one of my VR projects.

From the article discussing Virtual Presence (by video game composer Winifred Phillips), this image depicts the logo of the virtual reality game Shattered State.In Shattered State, developed by Supermassive Games, players are thrown into the job of Director of the National Intelligence Agency during a crisis situation involving terrorists, bomb threats, and a possible military coup. The player must make quick decisions with profound repercussions, then watch the results of those choices. It was important for players to feel committed to the action. So the audio director at Supermassive and I developed an interactive music system that scaled in energy and intensity as the stakes grew higher.

The music system of Shattered State featured abrupt changes during the interval in which decisions must be made, to draw the player’s attention toward the in-game decision-making menu, and to provide a conducive atmosphere for considering hard choices. The design of the music system also focused on keeping the player’s attention while the consequences of their decisions played out before them. The thrust of the system was about motivation, not about emotional reaction, and I composed the music to emphasize shifts in energy.

Let’s take a look at how that worked. Notice how the music adjusts whenever the player makes a significant choice during this six-minute gameplay video from the Shattered State VR game:

The music system of Shattered State focuses on maintaining player engagement, and this helps to reinforce psychological attachment to the experience.  So now that we’ve considered engagement, let’s move to the second of the three stages of psychological attachment:

Engrossment

An iconic depiction of an engrossed video gamer, from the article by game composer Winifred Phillips about the importance of Virtual Presence in VR gaming.Engrossment goes beyond simple motivation or focus. When players are engrossed, they reach this state because, according to the aforementioned researchers from University College London, “game features combine in such a way that the gamers’ emotions are directly affected by the game.” The researchers cite various popular game characteristics that can achieve this, including the visuals, the tasks, and the plot.

Music has the ability to effect all three of these characteristics. For the visual elements in a game, music not only influences what we see, but how much we see. At the University of Groningen in the Netherlands, a study showed that the overall mood of music can influence what visuals we notice while listening.  Happy music causes us to see pleasant details, while sad music brings more grim sights to our attention.

An image accompanying a discussion of the components needed to provoke adequate levels of engrossment from VR players, from the article for video game composers by Winifred Phillips (game music composer).In regards to tasks, music has long been known to make tasks more enjoyable and rewarding. A study at the University of Windsor showed that listening to music while working not only increases enjoyment of the work, but also increases the quality of the work as well.

Finally, music can intensify our appreciation for and understanding of plot. At the Universität Hildesheim in Germany, researchers conducted a study that paired vastly different musical scores with the same short film, to see if study subjects would be influenced by the music to interpret the plot of the film in different ways. They concluded that when the music was altered, the interpretation of the plot was dramatically altered as well, even though the events in the short film remained unchanged.

The common thread here is that players need to feel a sense of emotional connection – to visuals, to tasks, and to plot. When we’re trying to stimulate player engrossment, we’re specifically reaching out to their emotions. Music is well known for enhancing the inherent emotion of a situation, but I’d like to direct our attention to a specific example in which music is used to introduce emotional states to situations where the mood is otherwise undefined.

An image depicting the Life Hutch VR logo, from the article by composer Winifred Phillips discussing the importance of Virtual Presence in VR game design.One of my most recent projects is the Life Hutch VR game, which will be released on September 1st 2019 for the Valve Index, the Oculus Rift and the HTC Vive.  Life Hutch was created by the expert development team at Next Stop Willoughby, and is based on the famous short story written by world-renowned science fiction author Harlan Ellison. Life Hutch is an outer-space saga with a weird, surreal quality. As a story told out-of-sequence, many of the events in the game convey layers of elusive double-meaning, without any context to interpret the strangeness. This is a part of what makes Life Hutch compelling.

My job as the video game composer for this project was to project an undercurrent of raw emotion into the mix, to make events feel strange or disquieting, even when objective circumstances didn’t seem to warrant those feelings. The goal was to stir up disconcerting emotional states. Here’s a gameplay sequence in which the player is tasked with rapidly crossing a hostile planet surface of steaming rock and lava flows. Rather than conveying the objective momentum of urgency and risk, the music expresses subjective, unsettling emotions instead:

As we just saw in that gameplay sequence, the music of Life Hutch sought to keep players emotionally engrossed by challenging them with unexpected moods. By keeping players on edge, the musical score sought to stir up the strong emotions that could help players feel a more powerful and awesome sensation of Virtual Presence.

So we’ve now discussed two mechanisms by which psychological attachment can help to support Virtual Presence.  In our next article, we’ll examine the third component of psychological attachment – empathy.  Looking forward to seeing you then!

 


 

How Music Enhances Virtual Presence

Compilation of images depicting popular game titles for VR platforms that are included in the GDC 2019 lecture of game composer Winifred Phillips.

(Game Developers Conference Session Description)

Virtual Presence is defined as a state in which gamers fully accept the virtual world around them and their existence within it. This talk, “How Music Enhances Virtual Presence,” will explore how highly effective game music can enhance the sensation of Virtual Presence in VR gaming.

The talk will begin with an exploration of both the Flow Theory of Mihaly Csikszentmihalyi and the research of Dr. Paul Cairns on psychological engagement in video gaming. By understanding how the mental activity of players interacts with the way a game is designed, composers can create music intended to induce psychological states conducive with the formation of Virtual Presence.

The talk will include a discussion of techniques aimed at drawing attention to mission objectives, facilitating effective concentration, enhancing emotional empathy and intensifying player focus. The discussion will also include an exploration of some inherent drawbacks to Virtual Presence, including its fragility when exposed to negative emotional states, and its possible susceptibility to inducing the “event boundary” phenomenon. Musical solutions to these problems will be explored.

Phillips’ talk will offer techniques for composers and audio directors who seek to employ music as a tool to enhance Virtual Presence for their players.

Takeaway

Using examples from several games, Phillips will explore how music can influence the mental states of players through specific effects documented in scientific research. Study data will be discussed in regards to the interaction between music and cognition. Phillips will offer strategies and tips for composers seeking to use their music to influence the player’s mental state, thus facilitating the formation of Virtual Presence.

Intended Audience

This session is intended to inspire and stimulate composers seeking to employ their music towards enhancing player engagement and enjoyment, with a particular emphasis on VR games. Includes overview of Flow Theory and the psychological components of Virtual Presence, which may be useful to other disciplines within game development. Talk will be approachable for all levels (advanced composers may better appreciate the specific composition techniques discussed).

 

Photo of video game composer Winifred Phillips in her game composers production studio.Popular music from composer Winifred Phillips’ award-winning Assassin’s Creed Liberation score is currently being performed live by a top 80-piece orchestra and choir as part of the Assassin’s Creed Symphony World Tour, which kicked off in June 2019 with its Paris premiere. As an accomplished video game composer, Phillips is best known for composing music for games in five of the most famous and popular franchises in gaming: Assassin’s Creed, LittleBigPlanet, Total War, God of War, and The Sims.  Phillips’ other notable projects include the triple-A first person shooter Homefront: The Revolution, and numerous virtual reality games, including Sports Scramble, Audioshield, Scraper: First Strike, Dragon Front, and many more.   She is the author of the award-winning bestseller A COMPOSER’S GUIDE TO GAME MUSIC, published by the MIT Press. As a VR game music expert, she writes frequently on the future of music in virtual reality games. Phillips is a sought-after public speaker, and she has been invited to speak about her work as a game composer at the Library of Congress, the Game Developers Conference, the Audio Engineering Society, the Society of Composers and Lyricists, and many more.  Follow her on Twitter @winphillips.

Game Music and The Theory of Flow: How Game Composers Can Enhance Virtual Presence (Pt. 1)

Video game music composer Winifred Phillips in her video game music production studio working on the music of "The Haunted Graveyard" VR game.

By Winifred Phillips | ContactFollow

Hello there!  I’m video game composer Winifred Phillips.  At this year’s Game Developers Conference in San Francisco, I was pleased to give a presentation entitled How Music Enhances Virtual Presence (I’ve included the official description of my talk at the end of this article). The talk I delivered at GDC gave me the opportunity to pull a lot of ideas about virtual reality together and present a concentrated exploration of how music can increase a sensation of presence for VR gamers.  It occurred to me that such a discussion might be interesting to share in this forum as well. So, with that in mind, I’m excited to begin a four-part article series based on my GDC 2019 presentation!

Continue reading

Video: The first Library of Congress lecture given by a video game music composer

Popular video game composer Winifred Phillips giving her Library of Congress lecture -- the first presentation by a video game composer at the Library of Congress.

By Winifred Phillips | Contact | Follow

Hey everyone!  I’m video game music composer Winifred Phillips.  This past April, I gave a lecture on video game music composition techniques at the invitation of The Library of Congress in Washington DC. It was the first speech on game music composition given at The Library of Congress, and I was tremendously honored to be able to represent the field of video game music!  My presentation was entitled “The Interface Between Music Composition and Game Design,” and was supported by a full house in the Whittall Pavilion of the Thomas Jefferson Building at the Library of Congress. In a previous article, I posted a partial transcript of the Q&A portion from my Library of Congress session, including some of the best questions from the Q&A.  Since then, The Library of Congress has included a video of my entire presentation as a part of their permanent archival collection for future generations.  I’m very pleased to be able to share the entire video with you!

Continue reading

Video game music composer: Getting your big break

In this article for video game composers, popular game composer Winifred Phillips is depicted in this photo working in her music production studio.

By Winifred Phillips | Contact | Follow

So happy you’ve joined us!  I’m videogame composer Winifred Phillips (pictured above working on my career breakthrough project, God of War). Today I’ll be discussing a hot topic that we’ve previously explored, but that definitely deserves to be revisited periodically.  This is one of the most popular subjects that I’ve addressed in my previous articles here: How does a newcomer get hired as a game composer?

I’m asked this question frequently, and while I offered quite a lot of advice on this topic in my book A Composer’s Guide to Game Music, I’m keenly aware of how urgent the need is for updated guidance on this issue for aspiring video game composers.  Game music newcomers often feel adrift and alone in the game industry, and some good advice can be a welcome lifeline.  In my book, I described the career path that led me into the game industry and allowed me to land my first gigs, but I’m well aware that my experience was pretty unique.  With that in mind, I’ve collated some recent research and insights from some top game industry professionals in this article, in the hopes that some of these expert observations might prove helpful.  There are lots of original and provocative viewpoints presented here, so we should feel free to pick and choose the strategies and tips that will work best for us.

Also, later in the article you’ll find my presentation for the Society of Composers and Lyricists seminar, in which I answered the question about how I personally got my start in the games industry (for those who might be curious).  Finally, at the end of the article I have included a full list of links for further reading and reference.

Continue reading

Resources for Video Game Music Composers: The Big List

Video game music composer Winifred Phillips creating music in her video game music production studio.

By Winifred Phillips | Contact | Follow

Hey everybody!  I’m videogame composer Winifred Phillips.  Every year, between working in my studio creating music for some awesome games, I like to take a little time to gather together some of the top online resources and guidance available for newbies in the field of video game music.  What follows in this article is an updated and expanded collection of links on a variety of topics pertinent to our profession.  We begin with the concert tours and events where we can get inspired by seeing game music performed live.  Then we’ll move on to a discussion of online communities that can help us out when we’re trying to solve a problem.  Next, we’ll see a collection of software tools that are commonplace in our field.  Finally, we’ll check out some conferences and academic organizations where we can absorb new ideas and skills.

Continue reading

Video Game Music Composers: New VR Headphone Tech (2018)

By Winifred Phillips | Contact | Follow

In this article for and about the craft of video game composers, Winifred Phillips is pictured in this photo from her lecture on Virtual Reality given at the popular Game Developers Conference in 2018.Hey, everyone!  I’m videogame composer Winifred Phillips, and my work has included the musical scores for top games on all sorts of popular gaming platforms, from handhelds and mobile, all the way up to the latest consoles and PCs.  Lately, I’ve been doing a lot of video game music composition for virtual reality.  I had the pleasure of presenting a lecture on Music in Virtual Reality (pictured left) at the most recent Game Developers Conference in San Francisco.

My experience as a composer for VR includes many VR games, including the Scraper: First Strike shooter (set to be released for the PSVR, Oculus Rift and HTC Vive in December 2018), and the recently released VR experience The Haunted Graveyard, which is now available on Steam and in VR Arcades around the world.  Since we’re in the Halloween season, and this VR experience is designed specifically for your Halloween pleasure, here’s a trailer that features my music from The Haunted Graveyard:

By virtue of all the experiences I’ve had recently creating music for VR, I’ve become keenly aware of the importance of sound fidelity in VR.  If the experience doesn’t sound real, it loses the chance to actually feel like a fully-convincing, thoroughly awesome virtual reality experience.  With that in mind, I’ve been writing periodic articles about new technologies in connection with headphones for VR.

Continue reading

VR Headphones Update 2018: Video Game Music Composers

In this article written for video game composers, Winifred Phillips (composer of music for God of War) is here pictured working in her music production studio.

By Winifred Phillips | Contact | Follow

Glad you’re here!  I’m videogame composer Winifred Phillips.  My work as a game music composer has included music for projects released on nearly all of the gaming platforms, from one of my most recent projects (a Homefront game released on all the latest consoles and PCs) to one of my earliest projects (a God of War game released on PlayStation 2, PlayStation 3, and PlayStation Vita, pictured above).  An image of the September 2018 cover of Music Connection Magazine featuring the article "Video Game Composers Speak!" - features interviews of famous game music composers, including popular game music composer Winifred Phillips.You can read about my work as a video game composer in an interview I gave to Music Connection Magazine for this month’s issue (pictured right).

Lately, I’ve also been creating lots of video game music for awesome virtual reality games developed for the Oculus Rift, Oculus Go, HTC Vive, Samsung Gear VR, PlayStation VR, and lots of other top VR platforms.  One of the things I’ve noticed while working in VR is the immense importance of the audio delivery mechanism.

When audio is painstakingly spatialized, it becomes crucial to convey that carefully-crafted spatialization to the player with as little fidelity loss as possible.  With the importance of this issue in mind, for the past few years I’ve been periodically writing about headphones in relation to their use in virtual reality.

Continue reading