Game Music and Psychological Attachment: How Game Composers Can Enhance Virtual Presence (Pt. 2)

Photo of video game music composer Winifred Phillips working in her music production studio on the musical score of the Shattered State VR game from Supermassive Games.

By Winifred Phillips | ContactFollow

So happy you’ve joined us!  I’m video game composer Winifred Phillips, and this is the continuation of our four-part discussion of how music can enhance presence in virtual reality.  These articles are based on the presentation I gave at this year’s Game Developer’s Conference in San Francisco, entitled How Music Enhances Virtual Presence (I’ve included the official description of my talk at this end of this article). In my GDC talk, I discussed Virtual Presence in connection with seven of the virtual reality games and experiences that I’ve scored, which have either released within the past year or will be released within the coming months.  These include Audioshield (Audiosurf LLC), Bebylon Battle Royale (Kite & Lightning), Fail Factory (Armature Studio), The Haunted Graveyard (Holospark), Life Hutch VR (Next Stop Willoughby), Scraper: First Strike (Labrodex Inc), and Shattered State (Supermassive Games).  If you missed the first article exploring how Flow can support Virtual Presence in VR gaming, please go check that article out first.

Are you back?  Great!  Let’s continue!

Now that we’ve taken a look at how Flow can best enable Virtual Presence in VR, let’s look at the second mechanism by which music enables Virtual Presence:

Psychological Attachment

Image illustrating the three states contributing to the attainment of Virtual Presence (Engagement, Engrossment, and Empathy) -- from the article by video game composer Winifred Phillips.In a paper presented at the Computer-Human Interaction conference, a research team from Carnegie-Mellon defined Virtual Presence as “the extent to which a person’s Cognitive and perceptual systems are tricked into believing they are somewhere other than their physical location.” This assertion formed the jumping-off point for two researchers from University College London, who set out to define what specific circumstances could lead to Virtual Presence in gaming. They developed a model for how gamers developed the psychological attachment necessary to achieve Virtual Presence.  Their model consists of three stages:

  • Engagement
  • Engrossment
  • Empathy

So let’s start with the first stage.

Engagement

We can define engagement as the commitment of the player to remain focused on the game. Note that this is all about energy and attention – it’s not about emotion. Engagement is a motivational tool. Since music has long been one of the top methods to help motivate people, it makes sense to use music as a tool for helping players achieve engagement. This becomes especially important when striving for Virtual Presence. Let’s look at an example from one of my VR projects.

From the article discussing Virtual Presence (by video game composer Winifred Phillips), this image depicts the logo of the virtual reality game Shattered State.In Shattered State, developed by Supermassive Games, players are thrown into the job of Director of the National Intelligence Agency during a crisis situation involving terrorists, bomb threats, and a possible military coup. The player must make quick decisions with profound repercussions, then watch the results of those choices. It was important for players to feel committed to the action. So the audio director at Supermassive and I developed an interactive music system that scaled in energy and intensity as the stakes grew higher.

The music system of Shattered State featured abrupt changes during the interval in which decisions must be made, to draw the player’s attention toward the in-game decision-making menu, and to provide a conducive atmosphere for considering hard choices. The design of the music system also focused on keeping the player’s attention while the consequences of their decisions played out before them. The thrust of the system was about motivation, not about emotional reaction, and I composed the music to emphasize shifts in energy.

Let’s take a look at how that worked. Notice how the music adjusts whenever the player makes a significant choice during this six-minute gameplay video from the Shattered State VR game:

The music system of Shattered State focuses on maintaining player engagement, and this helps to reinforce psychological attachment to the experience.  So now that we’ve considered engagement, let’s move to the second of the three stages of psychological attachment:

Engrossment

An iconic depiction of an engrossed video gamer, from the article by game composer Winifred Phillips about the importance of Virtual Presence in VR gaming.Engrossment goes beyond simple motivation or focus. When players are engrossed, they reach this state because, according to the aforementioned researchers from University College London, “game features combine in such a way that the gamers’ emotions are directly affected by the game.” The researchers cite various popular game characteristics that can achieve this, including the visuals, the tasks, and the plot.

Music has the ability to effect all three of these characteristics. For the visual elements in a game, music not only influences what we see, but how much we see. At the University of Groningen in the Netherlands, a study showed that the overall mood of music can influence what visuals we notice while listening.  Happy music causes us to see pleasant details, while sad music brings more grim sights to our attention.

An image accompanying a discussion of the components needed to provoke adequate levels of engrossment from VR players, from the article for video game composers by Winifred Phillips (game music composer).In regards to tasks, music has long been known to make tasks more enjoyable and rewarding. A study at the University of Windsor showed that listening to music while working not only increases enjoyment of the work, but also increases the quality of the work as well.

Finally, music can intensify our appreciation for and understanding of plot. At the Universität Hildesheim in Germany, researchers conducted a study that paired vastly different musical scores with the same short film, to see if study subjects would be influenced by the music to interpret the plot of the film in different ways. They concluded that when the music was altered, the interpretation of the plot was dramatically altered as well, even though the events in the short film remained unchanged.

The common thread here is that players need to feel a sense of emotional connection – to visuals, to tasks, and to plot. When we’re trying to stimulate player engrossment, we’re specifically reaching out to their emotions. Music is well known for enhancing the inherent emotion of a situation, but I’d like to direct our attention to a specific example in which music is used to introduce emotional states to situations where the mood is otherwise undefined.

An image depicting the Life Hutch VR logo, from the article by composer Winifred Phillips discussing the importance of Virtual Presence in VR game design.One of my most recent projects is the Life Hutch VR game, which will be released on September 1st 2019 for the Valve Index, the Oculus Rift and the HTC Vive.  Life Hutch was created by the expert development team at Next Stop Willoughby, and is based on the famous short story written by world-renowned science fiction author Harlan Ellison. Life Hutch is an outer-space saga with a weird, surreal quality. As a story told out-of-sequence, many of the events in the game convey layers of elusive double-meaning, without any context to interpret the strangeness. This is a part of what makes Life Hutch compelling.

My job as the video game composer for this project was to project an undercurrent of raw emotion into the mix, to make events feel strange or disquieting, even when objective circumstances didn’t seem to warrant those feelings. The goal was to stir up disconcerting emotional states. Here’s a gameplay sequence in which the player is tasked with rapidly crossing a hostile planet surface of steaming rock and lava flows. Rather than conveying the objective momentum of urgency and risk, the music expresses subjective, unsettling emotions instead:

As we just saw in that gameplay sequence, the music of Life Hutch sought to keep players emotionally engrossed by challenging them with unexpected moods. By keeping players on edge, the musical score sought to stir up the strong emotions that could help players feel a more powerful and awesome sensation of Virtual Presence.

So we’ve now discussed two mechanisms by which psychological attachment can help to support Virtual Presence.  In our next article, we’ll examine the third component of psychological attachment – empathy.  Looking forward to seeing you then!

 


 

How Music Enhances Virtual Presence

Compilation of images depicting popular game titles for VR platforms that are included in the GDC 2019 lecture of game composer Winifred Phillips.

(Game Developers Conference Session Description)

Virtual Presence is defined as a state in which gamers fully accept the virtual world around them and their existence within it. This talk, “How Music Enhances Virtual Presence,” will explore how highly effective game music can enhance the sensation of Virtual Presence in VR gaming.

The talk will begin with an exploration of both the Flow Theory of Mihaly Csikszentmihalyi and the research of Dr. Paul Cairns on psychological engagement in video gaming. By understanding how the mental activity of players interacts with the way a game is designed, composers can create music intended to induce psychological states conducive with the formation of Virtual Presence.

The talk will include a discussion of techniques aimed at drawing attention to mission objectives, facilitating effective concentration, enhancing emotional empathy and intensifying player focus. The discussion will also include an exploration of some inherent drawbacks to Virtual Presence, including its fragility when exposed to negative emotional states, and its possible susceptibility to inducing the “event boundary” phenomenon. Musical solutions to these problems will be explored.

Phillips’ talk will offer techniques for composers and audio directors who seek to employ music as a tool to enhance Virtual Presence for their players.

Takeaway

Using examples from several games, Phillips will explore how music can influence the mental states of players through specific effects documented in scientific research. Study data will be discussed in regards to the interaction between music and cognition. Phillips will offer strategies and tips for composers seeking to use their music to influence the player’s mental state, thus facilitating the formation of Virtual Presence.

Intended Audience

This session is intended to inspire and stimulate composers seeking to employ their music towards enhancing player engagement and enjoyment, with a particular emphasis on VR games. Includes overview of Flow Theory and the psychological components of Virtual Presence, which may be useful to other disciplines within game development. Talk will be approachable for all levels (advanced composers may better appreciate the specific composition techniques discussed).

 

Photo of video game composer Winifred Phillips in her game composers production studio.Popular music from composer Winifred Phillips’ award-winning Assassin’s Creed Liberation score is currently being performed live by a top 80-piece orchestra and choir as part of the Assassin’s Creed Symphony World Tour, which kicked off in June 2019 with its Paris premiere. As an accomplished video game composer, Phillips is best known for composing music for games in five of the most famous and popular franchises in gaming: Assassin’s Creed, LittleBigPlanet, Total War, God of War, and The Sims.  Phillips’ other notable projects include the triple-A first person shooter Homefront: The Revolution, and numerous virtual reality games, including Sports Scramble, Audioshield, Scraper: First Strike, Dragon Front, and many more.   She is the author of the award-winning bestseller A COMPOSER’S GUIDE TO GAME MUSIC, published by the MIT Press. As a VR game music expert, she writes frequently on the future of music in virtual reality games. Phillips is a sought-after public speaker, and she has been invited to speak about her work as a game composer at the Library of Congress, the Game Developers Conference, the Audio Engineering Society, the Society of Composers and Lyricists, and many more.  Follow her on Twitter @winphillips.

Game Music and The Theory of Flow: How Game Composers Can Enhance Virtual Presence (Pt. 1)

Video game music composer Winifred Phillips in her video game music production studio working on the music of "The Haunted Graveyard" VR game.

By Winifred Phillips | ContactFollow

Hello there!  I’m video game composer Winifred Phillips.  At this year’s Game Developers Conference in San Francisco, I was pleased to give a presentation entitled How Music Enhances Virtual Presence (I’ve included the official description of my talk at the end of this article). The talk I delivered at GDC gave me the opportunity to pull a lot of ideas about virtual reality together and present a concentrated exploration of how music can increase a sensation of presence for VR gamers.  It occurred to me that such a discussion might be interesting to share in this forum as well. So, with that in mind, I’m excited to begin a four-part article series based on my GDC 2019 presentation!

Continue reading

Video game composers can make you smarter! (The music of Dragon Front) Pt. 3

Winifred Phillips, video game music composer, pictured at the GDC 2016 display for the Dragon Front virtual reality game.

By Winifred Phillips | Contact | Follow

Welcome to the third (and final) article in this three-part discussion of how video game composers (like us) can make strategy gamers smarter!  We’ve been exploring the best ways that the music of game composers can help strategy gamers to better concentrate while making more sound tactical decisions. During this discussion, I’ve shared my personal perspective as the composer for the popular Dragon Front strategy game for VR.

In part one, we discussed the concept of ‘music-message congruency,’ so if you haven’t read that article yet, you can read it here.  In part two, we explored the meaning of ‘cognition-enhancing tempo’ – you can read that article here.  Please make sure to read both those articles first and then come back.

Are you back?  Awesome!  Let’s launch into a discussion of the third technique for increasing the smarts of strategy gamers!

Tension-regulating affect

From the article by game composer Winifred Phillips, an illustration of 'psychological affect.'In psychology, the term ‘affect’ refers to emotion, particularly in terms of the way in which such emotional content is displayed.  Whether by visual or aural means, an emotion can not be shared without some kind of ‘affect’ that serves as its mode of communication from one person to another.  When we’re happy, we smile.  When we’re angry, we frown.

Continue reading

Music Game Plan: Tactics for the Video Game Composer (Part Two)

Composer Winifred Phillips working on the music of the popular Spore Hero video game from Electronic Arts.

Welcome back to my four-part article series presenting videos and helpful references to aid aspiring game music composers in understanding how interactive music works. In Part One of this series, we took a look at a simple example demonstrating the Horizontal Re-Sequencing model of musical interactivity, as it was used in the music I composed for the Speed Racer Videogame from Warner Bros. Interactive.  Now let’s turn our attention to a more complex example of horizontal re-sequencing as demonstrated by the interactive music of the Spore Hero game from Electronic Arts.

Continue reading

Music Game Plan: Tactics for the Video Game Composer (Part One)

Video game composer Winifred Phillips, working on the music of Speed Racer the Video Game.

Interactive music is always a hot topic in the game audio community, and newcomers to game music composition can easily become confused by the structure and process of creating non-linear music for games.  To address this issue, I produced four videos that introduce aspiring video game composers to some of the most popular tactics and procedures commonly used by game audio experts in the structuring of musical interactivity for games.  Over the next four articles, I’ll be sharing these videos with you, and I’ll also be including some supplemental information and accompanying musical examples for easy reference.  Hopefully these videos can answer some of the top questions about interactive music composition.  Music interactivity can be awesome, but it can also seem very abstract and mysterious when we’re first learning about it. Let’s work together to make the process feel a bit more concrete and understandable!

Continue reading

First Look: Beep Documentary interview with game music composer Winifred Phillips

Beep-Headshot_Winifred-Phillips

The Beep documentary is an awesome upcoming crowdfunded film consisting of interviews with top game composers and sound designers from around the world.  Leading up to the film’s world premiere in Spring 2016, the Beep Documentary team has been releasing webisodes of interview footage with selected composers and sound designers who will be featured in the documentary.  I’m pleased to share that a webisode of my interview has just been posted by the Beep documentary team!

Beep has been described as “the most comprehensive documentary of game music/audio history ever made,” and “a huge and culturally significant undertaking to document the history of video game sound and music through interviews with composers and other game audio professionals from around the globe.”  The Beep documentary is described best on the project’s website: “Relive the moments of your childhood, and hear the stories behind the songs and sounds of your favorite games from the people who created them. Help us to give the composers and sound designers throughout game history a chance to tell their own stories, to share the truly amazing things that they achieved.”

Continue reading

A Composer’s Guide to Game Music, now in Japanese!

 

A Composer’s Guide to Game Music by Winifred Phillips, now on sale in Japanese! Published by O’Reilly Japan.

I’m excited to share that my book, A Composer’s Guide to Game Music, was released today in Japan in its newly-published Japanese-language edition!  O’Reilly Japan has published the Japanese softcover of my book in Japan under the title, “Game Sound Production Guide: Composer Techniques for Interactive Music.”

This is the Japanese cover of the book. In Japanese, A Composer's Guide to Game Music is titled "Game sound production guide - composer techniques for interactive music," by Winifred Phillips.

Side-by-side, these are the covers of the two editions of the book. In Japanese, A Composer’s Guide to Game Music is titled “Game sound production guide – composer techniques for interactive music,” by Winifred Phillips.

I’m very excited that the Japanese language edition of my book has already hit #1 on the “Most Wished For” list on Amazon Japan!

The Amazon Japan "Most Wished For" list.

The “Most Wished For” list on Amazon.co.jp.

Coincidentally, the English-language version of A Composer’s Guide to Game Music is now #1 on the Kindle Top Rated list, too!

The Kindle "Top Rated" list on Amazon.com.

The Kindle “Top Rated” list on Amazon.com.

O’Reilly Japan is located in Tokyo, and is dedicated to translating books about technological innovation for Japanese readers.  They are a division of O’Reilly Media, a California publishing company that acts as “a chronicler and catalyst of leading-edge development, homing in on the technology trends that really matter and galvanizing their adoption by amplifying “faint signals” from the alpha geeks who are creating the future.  O’Reilly publishes definitive books on computer technologies for developers, administrators, and users. Bestselling series include the legendary “animal books,” Missing Manuals, Hacks, and Head First.”

oreilly

From what I’ve gathered, my book – A Composer’s Guide to Game Music – is the first English language book about game music to be translated into Japanese and sold in Japan.  There are a few other books available in Japan on the subject – but they were all originally written in Japanese.  These include a book exploring game sound by the audio hardware designer and sound developer Shiomi Toshiyukia text on creating sound for games with the CRI ADX2 middleware by Uchida Tomoya, and a book on producing game music and sound design by the artist “polymoog” of the dance music duo ELEKETL (pictured below, from left to right).

books-ama-jp

I’m tremendously excited about the Japanese edition of my book, and my excitement comes in large part from the venerable tradition of outstanding music in Japanese games.  From the most celebrated classic scores of such top game composers as Koji Kondo (Super Mario Bros.) and Nobuo Uematsu (Final Fantasy), to the excellent modern scores of such popular composers as Masato Kouda (Monster Hunter) and Yoko Shimomura (Kingdom Hearts), Japanese video game composers have set the creative bar very high.  I’m incredibly honored that my book will be read by both established and aspiring game composers in Japan!  I hope they’ll find some helpful information in my book, and I’m excited to contribute to the ongoing conversation about game music in the Japanese development community.

I’ve always loved Japanese game music.  In 2008, I participated in a compilation album in which successful game composers created cover versions of celebrated video game songs from classic games.  The album was called “Best of the Best: A Tribute to Game Music.”  I chose the music by Koji Kondo from Super Mario Bros., and recorded an a cappella vocal version.  It’s currently available for sale from the Sumthing Else Music Works record label, and can also be downloaded on iTunes.  You can hear the track on YouTube here:

If you’d like to learn more about the rich legacy of game music composition in Japan, you can watch an awesome free documentary series produced by the Red Bull Music Academy, entitled “Diggin’ in the Carts: A Documentary Series About Japanese Video Game Music.”  The series interviews famous game composers of Japan, which means that the interviews and narration are both in Japanese (with English subtitles).  Here’s an episode that focuses on modern accomplishments by Japanese game composers:

border-159926_640_white

Studio1_GreenWinifred Phillips is an award-winning video game music composer whose most recent project is the triple-A first person shooter Homefront: The Revolution. Her credits include five of the most famous and popular franchises in video gaming: Assassin’s Creed, LittleBigPlanet, Total War, God of War, and The Sims. She is the author of the award-winning bestseller A COMPOSER’S GUIDE TO GAME MUSIC, published by the Massachusetts Institute of Technology Press. As a VR game music expert, she writes frequently on the future of music in virtual reality video games. Follow her on Twitter @winphillips.