Video game music systems at GDC 2017: pros and cons for composers

Video game composer Winifred Phillips, pictured in her music production studio working on the music of LittleBigPlanet 2 Cross Controller

By Winifred Phillips | Contact | Follow

Welcome back to our three article series dedicated to collecting and exploring the ideas that were discussed in five different GDC 2017 audio talks about interactive music!  These five speakers shared ideas they’d developed in the process of creating interactivity in the music of their own game projects.  We’re looking at these ideas side-by-side to cultivate a sense of the “bigger picture” when it comes to the leading-edge thinking for music interactivity in games. In the first article, we looked at the basic nature of five interactive music systems discussed in these five GDC 2017 presentations:

If you haven’t read part one of this article series, please go do that now and come back.

Okay, so let’s now contemplate some simple but important questions: why were those systems used?  What was attractive about each interactive music strategy, and what were the challenges inherent in using those systems?

The Pros and Cons

Illustration of a GDC 2017 presentation, from the article by Winifred Phillips (video game composer).In this discussion of the advantages and disadvantages of musical interactivity, let’s start with the viewpoint of Sho Iwamoto, audio programmer of Final Fantasy XV for Square Enix.  He articulates a perspective on interactive music that’s rarely given voice in the game audio community.  “So first of all,” Iwamoto says, “I want to clarify that the reason we decided to implement interactive music is not to reduce repetition.”

From the article by game composer Winifred Phillips - an illustration of the game Final Fantasy XV.Those of us who have been in the game audio community for years have probably heard countless expert discussions of how crucial it is for video game composers to reduce musical repetition, and how powerful interactivity can be in eliminating musical recurrences in a game.  But for Iwamoto, this consideration is entirely beside the point.  “Repeating music is not evil,” he says. “Of course, it could be annoying sometimes, but everyone loves to repeat their favorite music, and also, repetition makes the music much more memorable.”  So, if eliminating repetition was not at the top of Iwamoto’s list of priorities, then what was?

“We used (musical interactivity) to enhance the user’s emotional experience by playing music that is more suitable to the situation,” Iwamoto explains, also adding that he wanted “to make transitions musical, as much as possible.”  So, if the best advantage of musical interactivity for Iwamoto was an enhanced emotional experience for gamers, then what was the biggest drawback?

For Iwamoto, the most awesome struggle arose from the desire to focus on musicality and melodic content, with the intent to present a traditionally epic musical score that maintained its integrity within an interactive framework. Often, these two imperatives seemed to smash destructively into each other.  “At first it was like a crash of the epic music and the interactive system,” he says.  “How can I make the music interactive while maintaining its epic melodies? Making music interactive could change or even screw up the music itself, or make the music not memorable enough.”

 

My perspective on epic interactive music

A photo of video game composer Winifred Phillips working in her music production studio on the music of LittleBigPlanet Cross Controller.Sho Iwamoto makes a very good point about the difficulty of combining epic musicality with an interactive structure.  For the popular LittleBigPlanet Cross Controller game for Sony Europe, I dealt with a very similar conundrum.  The development team asked me to create an epic orchestral action-adventure track that would be highly melodic but also highly interactive.  Balancing the needs of the interactivity with the needs of an expressive action-adventure orchestral score proved to be very tricky.  I structured the music around a six-layer system of vertical layering, wherein the music was essentially disassembled by the music engine and reassembled in different instrument combinations depending on the player’s progress.  Here’s a nine-minute gameplay video in which this single piece of music mutates and changes to accommodate the gameplay action:


 

Illustration of a GDC 2017 presentation, from the article by Winifred Phillips (video game composer).Leonard J. Paul’s work on the platformer Vessel also hinged on a vertical layering music system. However, the biggest advantage of the vertical layering music system for Paul was in its ability to adapt existing music into an interactive framework.  From the article by game composer Winifred Phillips - an illustration of the game Vessel.Working with multiple licensing agencies, the development team for Vessel was able to obtain a selection of songs for their game project while it was still early in development.  The songs became rich sources of inspiration for the development team.  “They had made the game listening to those songs so the whole entire game was steeped in that music,” Paul observes.

Nevertheless, the situation also presented some distinct disadvantages.  “The licensing for those ten tracks took eight months,” Paul admits, then he goes on to describe some of the other problems inherent in adapting preexisting music for interactivity.  “It’s really hard to remix someone else’s work so that it has contour yet it stays consistent,” Paul says, “So it doesn’t sound like, oh, I figured out something new in the puzzle or I did something wrong, just because there’s something changing in the music.” In order to make the music convey a single, consistent atmosphere, Paul devoted significant time and energy to making subtle, unnoticeable adjustments to the songs.  “It’s very hard to make your work transparent,” Paul points out.


 

Illustration of a GDC 2017 presentation, from the article by Winifred Phillips (video game composer).For sound designer Steve Green’s work on the music of the underwater exploration game ABZU, the main advantage of their use of an interactive music system was in the system’s ability to customize the musical content to the progress of the player by calling up location-specific tracks during exploration, without needing the make any significant changes to the content of those music files.  “So its mainly not the fact that we’re changing the music itself as you’re playing it, we’re just helping the music follow you along,” Green explains.  This enabled the music to “keep up with you as you’re playing the game, so it’s still interactive in a sense in that it’s changing along with the player.”

From the article by game composer Winifred Phillips - an illustration of the game ABZU.While this was highly-desirable, it also created some problems when one piece of music ended and another began, particularly if the contrast between the two tracks was steep.  “The dilemma we faced was going in from track one to track two,” Green observes.  For instance, if an action-oriented piece of music preceded a more relaxed musical composition, then “there was a high amount of energy that you just basically need to get in and out of.”

 

My perspective on interactive transitions

Photo of video game composer Winifred Phillips working in her music production studio on the music of the Speed Racer video game.Steve Green makes a great point about the need for transitions when moving between different energy levels in an interactive musical score.  I encountered a similar problem regarding disparate energy levels that required transitions when I composed the music for the Speed Racer video game (published by Warner Bros Interactive).  During races, the player would have the option to enter a special mode called “Zone Mode” in which their vehicle would travel much faster and would become instantly invincible.  During those sequences, the music switched from the main racing music to a much-more energetic track, and it became important for me to build a transition into that switch-over so that the change wouldn’t be jarring to the player.  I describe the process in this tutorial video:


 

Illustration of a GDC 2017 presentation, from the article by Winifred Phillips (video game composer).While sometimes a game audio team will choose an interactive music system strictly based on its practical advantages, there are also times in which the decision may be influenced by more emotional factors.  “We love MIDI,” confesses Becky Allen, audio director for the Plants vs. Zombies: Heroes game for mobile devices.  In fact, the development team, PopCap Games, has a long and distinguished history of innovative musical interactivity using the famous MIDI protocol.  During the Plants vs. Zombies: Heroes project, MIDI was a powerful tool for the audio team.  “It really was flexible, it was something you really could work with,” Allen says.

From the article by game composer Winifred Phillips - an illustration of the game Planets vs. Zombies: Heroes.However, that didn’t mean that the MIDI system didn’t create some problems for the audio team.  Early on in development for Plants vs. Zombies: Heroes, the team decided to record their own library of 24 musical instrument sounds for the game.  But during initial composition, those instruments weren’t yet available. This led to an initial reliance on a pre-existing library (East West Symphonic Orchestra).  “We were undergoing this sample library exercise, knowing that we’d be moving over to those samples eventually,” Allen says. Although the East West sample libraries had been initially used, they were fundamentally different. “Our PopCap sample library is fantastic too, but it’s totally different,” Allen adds.  “So the sounds were not the same, and the music, even though they were the same cues, just felt wrong.”  Allen advises, “I think it’s very important, if you can, to write to the sample library that you’ll be using ultimately at the end.”


 

Illustration of a GDC 2017 presentation, from the article by Winifred Phillips (video game composer).For Paul Weir’s work on the space exploration game No Man’s Sky, the motivation to use a procedural music system was also partly influenced by emotional factors.  “I really enjoy ceding control to the computer, giving it rules and letting it run,” Weir confides.  But there were other motivating influences as well. According to Weir, the advantages of procedural music rest with its unique responsiveness to in-game changes.  “Procedural audio, to make it different, to make it procedural, it has to be driven by the game,” Weir says.  From the article by game composer Winifred Phillips - an illustration of the game No Man's Sky.“What are you doing, game? I’m going to react to that in some way, and that’s going to be reflected in the sound I’m producing. In order to do that,” Weir adds, “it has to use some form of real-time generated sound.”  According to Weir, “procedural audio is the creation of sound in real-time, using synthesis techniques such as physical modeling, with deep links into game systems.”

While this gives a procedural music system the potential to be the most pliable and reactive system available for modern game design, there are steep challenges inherent in its structure.  “Some of the difficulties of procedural generated content,” Weir explains, “is to give a sense of its meaningfulness, like it feels like it’s hand crafted.” In a moment of personal reflection, Weir shares, “One of my big issues, is that if you have procedural audio, that the perception of it has to be as good as traditional audio. It’s no good if you compromise.”

 


 

So, for each of these interactive music systems there were distinct advantages and disadvantages.  In the third and final article of this series, we’ll get down to some nitty-gritty details of how these interactive systems were put to use.  Thanks for reading, and please feel free to leave your comments in the space below!

 

Photo of video game composer Winifred Phillips in her music production studio.Winifred Phillips is an award-winning video game music composer whose most recent projects are the triple-A first person shooter Homefront: The Revolution and the Dragon Front VR game for Oculus Rift. Her credits include games in five of the most famous and popular franchises in gaming: Assassin’s Creed, LittleBigPlanet, Total War, God of War, and The Sims. She is the author of the award-winning bestseller A COMPOSER’S GUIDE TO GAME MUSIC, published by the MIT Press. As a VR game music expert, she writes frequently on the future of music in virtual reality games. Follow her on Twitter @winphillips.

Video game music systems at GDC 2017: what are composers using?

By video game music composer Winifred Phillips | Contact | Follow

Video game composer Winifred Phillips, presenting at the Game Developers Conference 2017.The 2017 Game Developers Conference could be described as a densely-packed deep-dive exploration of the state-of-the-art tools and methodologies used in modern game development.  This description held especially true for the game audio track, wherein top experts in the field offered a plethora of viewpoints and advice on the awesome technical and artistic challenges of creating great sound for games. I’ve given GDC talks for the past three years now (see photo), and every year I’m amazed at the breadth and diversity of the problem-solving approaches discussed by my fellow GDC presenters.  Often I’ll emerge from the conference with the impression that we game audio folks are all “doing it our own way,” using widely divergent strategies and tools.

This year, I thought I’d write three articles to collect and explore the ideas that were discussed in five different GDC audio talks.  During their presentations, these five speakers all shared their thoughts on best practices and methods for instilling interactivity in modern game music.  By absorbing these ideas side-by-side, I thought we might gain a sense of the “bigger picture” when it comes to the current leading-edge thinking for music interactivity in games. In the first article, we’ll look at the basic nature of these interactive systems.  We’ll devote the second article to the pros and cons of each system, and in the third article we’ll look at tools and tips shared by these music interactivity experts. Along the way, I’ll also be sharing my thoughts on the subject, and we’ll take a look at musical examples from some of my own projects that demonstrate a few ideas explored in these GDC talks:

So, let’s begin with the most obvious question.  What kind of interactive music systems are game audio folks using lately?

Continue reading

GDC 2017: How video game composers can use music to build suspense

Winifred Phillips, video game composer, giving a talk as part of the Game Developers Conference 2016 in San Francisco.

By Winifred Phillips | Contact | Follow

The Game Developers Conference is coming up soon!  Last year I presented a talk on music for mobile games (pictured above), and I’m pleased that this year I’ll be presenting the talk, “Homefront’ to ‘God of War’: Using Music to Build Suspense(Wednesday, March 1st at 11am in room 3006 West Hall, Moscone Center, San Francisco).  In my talk I’ll be focusing on practical applications of techniques for video game composers and game audio folks, using my own experiences as concrete examples for exploration.  Along the way, I’ll be discussing some very compelling scholarly research on the relationship between suspense, gameplay and musical expression.  In preparing my GDC 2017 presentation I did a lot of reading and studying about the nature of suspense in video games, the importance of suspense in gameplay design, and the role that video game music plays in regulating and elevating suspense.  There will be lots of ground to cover in my presentation!  That being said, the targeted focus of my presentation precluded me from incorporating some very interesting extra research into the importance of suspense in a more general sense… why human beings need suspense, and what purpose it serves in our lives.  I also couldn’t find the space to include everything I’d encountered regarding suspense as an element in the gaming experience.  It occurred to me that some of this could be very useful to us in our work as game makers, so I’d like to share some of these extra ideas in this article.

Continue reading

Video Game Music Composer: How To Break Into the Business

Video game composer Winifred Phillips, working on the music of Homefront: The Revolution in her production studio.As a video game composer and author of the book A Composer’s Guide to Game Music, I’m frequently asked for advice on how a young composer can gain entry into this business.  I dedicated a chapter of my book to this topic (Chapter 14: Acting Like a Business and Finding Work), so I’ve certainly thought a great deal about the issue.  From my very first project (God of War) all the way to my most recent game (Homefront The Revolution, pictured right), one thing has always been abundantly clear: landing gigs can be a complex journey.  That’s especially true for newcomers, and there are no easy signposts pointing the way. While I tried to use my own experiences and insights to provide useful guidance in my book, I know that everyone’s experience is different, and multiple points of view can be very helpful.  So in this article, I’ll be offering resources from articles and community discussions on how to face down the awesome challenges of breaking into the industry as a composer of music for games.

First, I’ll be sharing a video from my presentation at the Society of Composers and Lyricists seminar, in which I answered the question about how I got my start in the games industry.  Then, we’ll be exploring highlights from a collection of online articles that offer helpful tips for how to break in and establish a career as a game composer.  Finally, at the end of this article I’ll be including a full list of links for further reading and reference.

Continue reading

Video Game Music Production Tips from GDC 2016

Game Composer Winifred Phillips during her game music presentation at the Game Developers Conference 2016I was pleased to give a talk about composing music for games at the 2016 Game Developers Conference (pictured left).  GDC took place this past March in San Francisco – it was an honor to be a part of the audio track again this year, which offered a wealth of awesome educational sessions for game audio practitioners.  So much fun to see the other talks and learn about what’s new and exciting in the field of game audio!  In this blog, I want to share some info that I thought was really interesting from two talks that pertained to the audio production side of game development: composer Laura Karpman’s talk about “Composing Virtually, Sounding Real” and audio director Garry Taylor’s talk on “Audio Mastering for Interactive Entertainment.”  Both sessions had some very good info for video game composers who may be looking to improve the quality of their recordings.  Along the way, I’ll also be sharing a few of my own personal viewpoints on these music production topics, and I’ll include some examples from one of my own projects, the Ultimate Trailers album for West One Music, to illustrate ideas that we’ll be discussing.  So let’s get started!

Continue reading

From Total War to Assassin’s Creed: Music from my GDC Talk

From Total War to Assassin's Creed: Music from my GDC Talk (article by Winifred Phillips, video game composer)Last week, it was my honor and pleasure to give a presentation at the Game Developers Conference in San Francisco. My talk was entitled “From Total War to Assassin’s Creed: Music for Mobile Games.” The talk focused on the best and most effective methods for composition and implementation of music in portable gaming.  The talk was structured for the benefit of video game composers and game audio pros, and as a part of the presentation, I played short excerpts of music that I composed for several of my top mobile and handheld video game projects. Now that GDC is over, I thought I’d provide streaming links to some of the complete music tracks that I featured during my presentation, in case attendees were curious about the complete pieces of music. So, without further ado, here are tracks from my GDC 2016 talk!

Assassin’s Creed Liberation

The Assassin’s Creed Liberation game was released by Ubisoft for the PlayStation Vita, and delivered an immersive experience from the popular Assassin’s Creed franchise. The game was designed specifically for a portable system, and as such, all aspects of the design were adjusted to cater specifically to a portable gaming experience, including the music.

Game composer Winifred Phillips speaking about the music of Assassin's Creed Liberation at GDC 2016

Continue reading

Strategies in Audio & Music for Portable Games

Portable Game Audio and Music (article by award winning video game composer Winifred Phillips)
I’ll be talking about effective music composition for mobile and portable gaming platforms during my talk, “From Total War to Assassin’s Creed: Music for Mobile Games,” which will take place on March 16th at the upcoming Game Developers Conference at the Moscone Center in San Francisco.  With that in mind, I thought I’d use this blog entry to share some resources that explore current strategies and trends in regards to sound and music for mobile – resources that could be useful to the video game composer and sound designer.
Audio and Music for Portable Games (blog written by Winifred Phillips, video game composer)While my talk at GDC will focus specifically on music composition and implementation for handheld devices, the resources that will follow in this blog offer assistance with the more general technical issues that face audio pros creating sound assets for a mobile gaming environment.  I’ve included links to the original articles, as well as a summation of some of the best points that I thought were particularly interesting:

Continue reading