Video game music systems at GDC 2017: pros and cons for composers

Video game composer Winifred Phillips, pictured in her music production studio working on the music of LittleBigPlanet 2 Cross Controller

By Winifred Phillips | Contact | Follow

Welcome back to our three article series dedicated to collecting and exploring the ideas that were discussed in five different GDC 2017 audio talks about interactive music!  These five speakers shared ideas they’d developed in the process of creating interactivity in the music of their own game projects.  We’re looking at these ideas side-by-side to cultivate a sense of the “bigger picture” when it comes to the leading-edge thinking for music interactivity in games. In the first article, we looked at the basic nature of five interactive music systems discussed in these five GDC 2017 presentations:

If you haven’t read part one of this article series, please go do that now and come back.

Okay, so let’s now contemplate some simple but important questions: why were those systems used?  What was attractive about each interactive music strategy, and what were the challenges inherent in using those systems?

The Pros and Cons

Illustration of a GDC 2017 presentation, from the article by Winifred Phillips (video game composer).In this discussion of the advantages and disadvantages of musical interactivity, let’s start with the viewpoint of Sho Iwamoto, audio programmer of Final Fantasy XV for Square Enix.  He articulates a perspective on interactive music that’s rarely given voice in the game audio community.  “So first of all,” Iwamoto says, “I want to clarify that the reason we decided to implement interactive music is not to reduce repetition.”

From the article by game composer Winifred Phillips - an illustration of the game Final Fantasy XV.Those of us who have been in the game audio community for years have probably heard countless expert discussions of how crucial it is for video game composers to reduce musical repetition, and how powerful interactivity can be in eliminating musical recurrences in a game.  But for Iwamoto, this consideration is entirely beside the point.  “Repeating music is not evil,” he says. “Of course, it could be annoying sometimes, but everyone loves to repeat their favorite music, and also, repetition makes the music much more memorable.”  So, if eliminating repetition was not at the top of Iwamoto’s list of priorities, then what was?

“We used (musical interactivity) to enhance the user’s emotional experience by playing music that is more suitable to the situation,” Iwamoto explains, also adding that he wanted “to make transitions musical, as much as possible.”  So, if the best advantage of musical interactivity for Iwamoto was an enhanced emotional experience for gamers, then what was the biggest drawback?

For Iwamoto, the most awesome struggle arose from the desire to focus on musicality and melodic content, with the intent to present a traditionally epic musical score that maintained its integrity within an interactive framework. Often, these two imperatives seemed to smash destructively into each other.  “At first it was like a crash of the epic music and the interactive system,” he says.  “How can I make the music interactive while maintaining its epic melodies? Making music interactive could change or even screw up the music itself, or make the music not memorable enough.”

 

My perspective on epic interactive music

A photo of video game composer Winifred Phillips working in her music production studio on the music of LittleBigPlanet Cross Controller.Sho Iwamoto makes a very good point about the difficulty of combining epic musicality with an interactive structure.  For the popular LittleBigPlanet Cross Controller game for Sony Europe, I dealt with a very similar conundrum.  The development team asked me to create an epic orchestral action-adventure track that would be highly melodic but also highly interactive.  Balancing the needs of the interactivity with the needs of an expressive action-adventure orchestral score proved to be very tricky.  I structured the music around a six-layer system of vertical layering, wherein the music was essentially disassembled by the music engine and reassembled in different instrument combinations depending on the player’s progress.  Here’s a nine-minute gameplay video in which this single piece of music mutates and changes to accommodate the gameplay action:


 

Illustration of a GDC 2017 presentation, from the article by Winifred Phillips (video game composer).Leonard J. Paul’s work on the platformer Vessel also hinged on a vertical layering music system. However, the biggest advantage of the vertical layering music system for Paul was in its ability to adapt existing music into an interactive framework.  From the article by game composer Winifred Phillips - an illustration of the game Vessel.Working with multiple licensing agencies, the development team for Vessel was able to obtain a selection of songs for their game project while it was still early in development.  The songs became rich sources of inspiration for the development team.  “They had made the game listening to those songs so the whole entire game was steeped in that music,” Paul observes.

Nevertheless, the situation also presented some distinct disadvantages.  “The licensing for those ten tracks took eight months,” Paul admits, then he goes on to describe some of the other problems inherent in adapting preexisting music for interactivity.  “It’s really hard to remix someone else’s work so that it has contour yet it stays consistent,” Paul says, “So it doesn’t sound like, oh, I figured out something new in the puzzle or I did something wrong, just because there’s something changing in the music.” In order to make the music convey a single, consistent atmosphere, Paul devoted significant time and energy to making subtle, unnoticeable adjustments to the songs.  “It’s very hard to make your work transparent,” Paul points out.


 

Illustration of a GDC 2017 presentation, from the article by Winifred Phillips (video game composer).For sound designer Steve Green’s work on the music of the underwater exploration game ABZU, the main advantage of their use of an interactive music system was in the system’s ability to customize the musical content to the progress of the player by calling up location-specific tracks during exploration, without needing the make any significant changes to the content of those music files.  “So its mainly not the fact that we’re changing the music itself as you’re playing it, we’re just helping the music follow you along,” Green explains.  This enabled the music to “keep up with you as you’re playing the game, so it’s still interactive in a sense in that it’s changing along with the player.”

From the article by game composer Winifred Phillips - an illustration of the game ABZU.While this was highly-desirable, it also created some problems when one piece of music ended and another began, particularly if the contrast between the two tracks was steep.  “The dilemma we faced was going in from track one to track two,” Green observes.  For instance, if an action-oriented piece of music preceded a more relaxed musical composition, then “there was a high amount of energy that you just basically need to get in and out of.”

 

My perspective on interactive transitions

Photo of video game composer Winifred Phillips working in her music production studio on the music of the Speed Racer video game.Steve Green makes a great point about the need for transitions when moving between different energy levels in an interactive musical score.  I encountered a similar problem regarding disparate energy levels that required transitions when I composed the music for the Speed Racer video game (published by Warner Bros Interactive).  During races, the player would have the option to enter a special mode called “Zone Mode” in which their vehicle would travel much faster and would become instantly invincible.  During those sequences, the music switched from the main racing music to a much-more energetic track, and it became important for me to build a transition into that switch-over so that the change wouldn’t be jarring to the player.  I describe the process in this tutorial video:


 

Illustration of a GDC 2017 presentation, from the article by Winifred Phillips (video game composer).While sometimes a game audio team will choose an interactive music system strictly based on its practical advantages, there are also times in which the decision may be influenced by more emotional factors.  “We love MIDI,” confesses Becky Allen, audio director for the Plants vs. Zombies: Heroes game for mobile devices.  In fact, the development team, PopCap Games, has a long and distinguished history of innovative musical interactivity using the famous MIDI protocol.  During the Plants vs. Zombies: Heroes project, MIDI was a powerful tool for the audio team.  “It really was flexible, it was something you really could work with,” Allen says.

From the article by game composer Winifred Phillips - an illustration of the game Planets vs. Zombies: Heroes.However, that didn’t mean that the MIDI system didn’t create some problems for the audio team.  Early on in development for Plants vs. Zombies: Heroes, the team decided to record their own library of 24 musical instrument sounds for the game.  But during initial composition, those instruments weren’t yet available. This led to an initial reliance on a pre-existing library (East West Symphonic Orchestra).  “We were undergoing this sample library exercise, knowing that we’d be moving over to those samples eventually,” Allen says. Although the East West sample libraries had been initially used, they were fundamentally different. “Our PopCap sample library is fantastic too, but it’s totally different,” Allen adds.  “So the sounds were not the same, and the music, even though they were the same cues, just felt wrong.”  Allen advises, “I think it’s very important, if you can, to write to the sample library that you’ll be using ultimately at the end.”


 

Illustration of a GDC 2017 presentation, from the article by Winifred Phillips (video game composer).For Paul Weir’s work on the space exploration game No Man’s Sky, the motivation to use a procedural music system was also partly influenced by emotional factors.  “I really enjoy ceding control to the computer, giving it rules and letting it run,” Weir confides.  But there were other motivating influences as well. According to Weir, the advantages of procedural music rest with its unique responsiveness to in-game changes.  “Procedural audio, to make it different, to make it procedural, it has to be driven by the game,” Weir says.  From the article by game composer Winifred Phillips - an illustration of the game No Man's Sky.“What are you doing, game? I’m going to react to that in some way, and that’s going to be reflected in the sound I’m producing. In order to do that,” Weir adds, “it has to use some form of real-time generated sound.”  According to Weir, “procedural audio is the creation of sound in real-time, using synthesis techniques such as physical modeling, with deep links into game systems.”

While this gives a procedural music system the potential to be the most pliable and reactive system available for modern game design, there are steep challenges inherent in its structure.  “Some of the difficulties of procedural generated content,” Weir explains, “is to give a sense of its meaningfulness, like it feels like it’s hand crafted.” In a moment of personal reflection, Weir shares, “One of my big issues, is that if you have procedural audio, that the perception of it has to be as good as traditional audio. It’s no good if you compromise.”

 


 

So, for each of these interactive music systems there were distinct advantages and disadvantages.  In the third and final article of this series, we’ll get down to some nitty-gritty details of how these interactive systems were put to use.  Thanks for reading, and please feel free to leave your comments in the space below!

 

Photo of video game composer Winifred Phillips in her music production studio.Winifred Phillips is an award-winning video game music composer whose most recent projects are the triple-A first person shooter Homefront: The Revolution and the Dragon Front VR game for Oculus Rift. Her credits include games in five of the most famous and popular franchises in gaming: Assassin’s Creed, LittleBigPlanet, Total War, God of War, and The Sims. She is the author of the award-winning bestseller A COMPOSER’S GUIDE TO GAME MUSIC, published by the MIT Press. As a VR game music expert, she writes frequently on the future of music in virtual reality games. Follow her on Twitter @winphillips.

More Business Advice for the Game Music Composer

Winifred Phillips (video game composer) working in her music studio.Every so often, I like to grab some time between music composition gigs to gather together the current general wisdom regarding career strategies for game music composers (since so many of my readers are new to the industry and looking for guidance).  In this article, I’ve included some of the stand-out ideas garnered from three online resources – a Gamasutra article by a former audio designer for Rockstar North, an awesome discussion thread on Reddit about effective communication strategies (found in the GameAudio subreddit), and a roundtable discussion at GameSoundCon about best business practices for game audio pros.

Make some noise! Getting a job creating sound and music for videogames

Audio Director Will Morton of Solid Audioworks (formerly a senior audio designer and dialogue supervisor at the famous Rockstar North development studio), has written a comprehensive article for the game industry site Gamasutra about getting jobs in the game audio field.  The article, entitled “Make Some Noise! Getting a Job Creating Sound and Music for Videogames,” focuses on the importance of experience, networking and a polished presentation in order to sufficiently impress a potential employer/client.  While much of the article is solid advice that might apply to a job seeker in any industry, a few areas impressed me as particularly interesting for game composers to bear in mind.

Continue reading

Communication Tips for the Video Game Composer

Video game composer Winifred Phillips, working in her music studio.A successful career as a video game composer involves much more than our day-to-day challenges in our music studios. In addition to our role as music experts, we need to be well-rounded business people and great members of a creative team.  As a speaker in the audio track of the Game Developers Conference this year, I had a chance to take in a wide variety of GDC sessions, and I noticed how often teamwork was discussed.  Along the way, a common idea emerged from many of these talks — good communication is key. This is a concept that I explored in my book (A Composer’s Guide to Game Music), so I was delighted to see a further discussion of the issue at GDC this year.  Far from just a valuable personality asset, the ability to communicate well must be considered a top priority: as intrinsically valuable as rock-solid competency, awesome artistry or compelling vision. Good communication amongst team members can make or break the development of a game. As game audio pros, we share this in common with our coworkers in other segments of the game development community. However, it becomes especially important for us to focus and emphasize good communication when we’re working remotely as independent contractors. With that in mind, I thought I’d use this article to briefly highlight some GDC 2016 sessions in the game audio track that discussed this popular topic, so we can think about more ways to enhance and improve our communication skills.  And later we’ll discuss a practical example from my work on the music of the SimAnimals game from Electronic Arts.

Continue reading

A Grammy Category for Game Music Composers

grammys-650pxIt’s Grammy Awards time!  This coming Monday, the National Academy of Recording Arts and Sciences (NARAS) will throw its annual party, and many golden gramophones will be awarded to the popular recordings that were deemed most worthy this year. This is one of the most prestigious honors for any musician. Each year, Grammy nominees are selected as representing the top of their field: the very best in professional music.  There are 83 Grammy categories this year, ranging from famous categories like Album of the Year to lesser known categories such as Best Children’s Album.  Among those 83 categories, we don’t find a Best Video Game Music category listed… but we do see the category Best Score Soundtrack for Visual Media (Includes Film, TV, Video Games and Other Visual Media).  The Best Score Soundtrack for Visual Media category has gone through some interesting transformations during the long history of the Grammy Awards. In this article, we’ll be taking a walk down memory lane, exploring the ways in which the Visual Media category has changed to accommodate video game soundtracks.

Continue reading

A Composer’s Guide to Game Music, now in Japanese!

 

A Composer's Guide to Game Music by Winifred Phillips, now on sale in Japanese! Published by O'Reilly Japan.

A Composer’s Guide to Game Music by Winifred Phillips, now on sale in Japanese!  Published by O’Reilly Japan.

I’m excited to share that my book, A Composer’s Guide to Game Music, was released today in Japan in its newly-published Japanese-language edition!  O’Reilly Japan has published the Japanese softcover of my book in Japan under the title, “Game Sound Production Guide: Composer Techniques for Interactive Music.”

This is the Japanese cover of the book. In Japanese, A Composer's Guide to Game Music is titled "Game sound production guide - composer techniques for interactive music," by Winifred Phillips.

Side-by-side, these are the covers of the two editions of the book. In Japanese, A Composer’s Guide to Game Music is titled “Game sound production guide – composer techniques for interactive music,” by Winifred Phillips.

I’m very excited that the Japanese language edition of my book has already hit #1 on the “Most Wished For” list on Amazon Japan!

The Amazon Japan "Most Wished For" list.

The “Most Wished For” list on Amazon.co.jp.

Coincidentally, the English-language version of A Composer’s Guide to Game Music is now #1 on the Kindle Top Rated list, too!

The Kindle "Top Rated" list on Amazon.com.

The Kindle “Top Rated” list on Amazon.com.

O’Reilly Japan is located in Tokyo, and is dedicated to translating books about technological innovation for Japanese readers.  They are a division of O’Reilly Media, a California publishing company that acts as “a chronicler and catalyst of leading-edge development, homing in on the technology trends that really matter and galvanizing their adoption by amplifying “faint signals” from the alpha geeks who are creating the future.  O’Reilly publishes definitive books on computer technologies for developers, administrators, and users. Bestselling series include the legendary “animal books,” Missing Manuals, Hacks, and Head First.”

oreilly

From what I’ve gathered, my book – A Composer’s Guide to Game Music – is the first English language book about game music to be translated into Japanese and sold in Japan.  There are a few other books available in Japan on the subject – but they were all originally written in Japanese.  These include a book exploring game sound by the audio hardware designer and sound developer Shiomi Toshiyukia text on creating sound for games with the CRI ADX2 middleware by Uchida Tomoya, and a book on producing game music and sound design by the artist “polymoog” of the dance music duo ELEKETL (pictured below, from left to right).

books-ama-jp

I’m tremendously excited about the Japanese edition of my book, and my excitement comes in large part from the venerable tradition of outstanding music in Japanese games.  From the most celebrated classic scores of such top game composers as Koji Kondo (Super Mario Bros.) and Nobuo Uematsu (Final Fantasy), to the excellent modern scores of such popular composers as Masato Kouda (Monster Hunter) and Yoko Shimomura (Kingdom Hearts), Japanese video game composers have set the creative bar very high.  I’m incredibly honored that my book will be read by both established and aspiring game composers in Japan!  I hope they’ll find some helpful information in my book, and I’m excited to contribute to the ongoing conversation about game music in the Japanese development community.

I’ve always loved Japanese game music.  In 2008, I participated in a compilation album in which successful game composers created cover versions of celebrated video game songs from classic games.  The album was called “Best of the Best: A Tribute to Game Music.”  I chose the music by Koji Kondo from Super Mario Bros., and recorded an a cappella vocal version.  It’s currently available for sale from the Sumthing Else Music Works record label, and can also be downloaded on iTunes.  You can hear the track on YouTube here:

If you’d like to learn more about the rich legacy of game music composition in Japan, you can watch an awesome free documentary series produced by the Red Bull Music Academy, entitled “Diggin’ in the Carts: A Documentary Series About Japanese Video Game Music.”  The series interviews famous game composers of Japan, which means that the interviews and narration are both in Japanese (with English subtitles).  Here’s an episode that focuses on modern accomplishments by Japanese game composers:

border-159926_640_white

Studio1_GreenWinifred Phillips is an award-winning video game music composer whose most recent project is the triple-A first person shooter Homefront: The Revolution. Her credits include five of the most famous and popular franchises in video gaming: Assassin’s Creed, LittleBigPlanet, Total War, God of War, and The Sims. She is the author of the award-winning bestseller A COMPOSER’S GUIDE TO GAME MUSIC, published by the Massachusetts Institute of Technology Press. As a VR game music expert, she writes frequently on the future of music in virtual reality video games. Follow her on Twitter @winphillips.

How To Break Into Game Music Composing: Advice from GDC 2015

During the Game Developers Conference in San Francisco this year, attendees of the GDC Audio Track were treated to not one, but two sessions on how to break into the industry as a game audio professional.  Since this is always a topic of intense interest, I thought I’d use this blog entry to sum up some of the ideas from those two sessions. Hopefully some of this information will prove useful!

First, let’s look at the basic details about those two GDC talks.

Amongst the many different speakers at the two presentations, there were a lot of ideas and strategies put forward about ways to break into game audio development.  A few common themes definitely rose to the surface, and I’ll go over those here:

You gotta have good connections.

networking

The importance of friendships and relationships was an oft repeated idea at these talks.  When the members of the “Landing Your First Game” panel were asked about how they got their first gig, they all said that they owed it to a referral from a friend.  Likewise, the Audio Bootcamp talk stressed the importance of making friendships during in-person meetings and conferences.

According to the speakers, it’s important to simply hang out with game audio pros in a low-pressure environment in which prospective gigs aren’t discussed at all.  Taking the opportunity to form friendships without pursuing career objectives was considered the best approach for an aspiring game audio professional attending a conference or convention.  The speakers suggested that opportunities would eventually surface from these friendly relationships, but patience would be required, and years might pass before any of these relationships lead to a game audio gig.  I can confirm from my own experience that maintaining relationships is vitally important, and that a friendly acquaintance can lead to a game audio job years down the line.

At this point, I should probably mention that a certain aspect of my early career experience would be considered highly atypical – I landed my first gig by doing something that the speakers labeled as a “cold-call.”  According to the speakers, a “cold-call” is a phone call or an e-mail sent to someone who is essentially a stranger.  This isn’t supposed to be a successful approach, and the speakers strongly advised against cold-calling… but that’s what I did.  I sent an e-mail to a music supervisor who didn’t know me, and I introduced myself and asked about music opportunities at that company.  This led to my first game job, as a composer on the music composition team of God of War from Sony Computer Entertainment America.  My experience in respect to the “cold call” seems to be the exception that proves the rule, according to the speakers at these two presentations.

Be willing to negotiate, or to work for free

negotiation

One of the speakers told a story about breaking into the industry by donating music for free to a mod.  A “mod” is a modification to the program code of a retail game by an indie team, resulting in a new game that requires players to own the original retail game in order to play the modification.  Since a mod team typically consists of hobbyists with little or no budget, creating music for a mod is usually a charitable endeavor on the part of the composer.  At one point, an audience member asked how long any aspiring game composer should donate their services before finally resolving not to work for free anymore.  The response was that free work can continue throughout an audio professional’s career – this kind of free work was referred to as a “passion project.”  When we become uncomfortable with working for free, however, the time has come to negotiate for compensation.

Both presentations spent considerable time discussing compensation structures that diverged from traditional rate-per-minute work-for-hire agreements.  Licensing agreements were mentioned repeatedly during both talks, which might (or might not) include participation in the “back end” revenue that the video game would earn once it was released.  Also under discussion was the possible retention of the rights to release a soundtrack album and collect its earnings.  This trend seemed to indicate that offering an agreement that would require less money from the developer up-front might convince the developer to pay something, rather than nothing, and that this might help the aspiring audio pro to secure their first paying gig.

On a personal note, I donated music to a “mod” early in my career.  Although it allowed me to gain a bit of experience as a game composer, I didn’t enjoy any particular career advantages for having done so. However, the mod I scored ended up as vaporware (i.e. unreleased), which underlines the importance of evaluating the mod project in order to estimate whether the mod is likely to be finished or not. I talk about this experience, along with several of my other early game projects, in my book.

Lately, I’ve heard a lot of buzz about licensing agreements in the game industry, and I imagine that it might be a viable alternative for new game audio folks looking to transition from free gigs to paying work.  However, according to the latest Game Audio Industry Survey conducted by GameSoundCon, only 12% of professionally produced indie games allow composers to license their music rather than selling the rights on a work-for-hire basis.  Most of the game development teams open to music licensing would be characterized as “not professionally produced,” according to the language in the Game Audio Industry Survey.

Some final thoughts

The speakers mentioned several conferences and other resources for the aspiring game composer, which I’ve listed below:

Also, just as an addendum, audio designer Will Morton, best known for his work on the Grand Theft Auto games, has recently published an article on Gamasutra.com called “Make Some Noise! Getting a Job Creating Sound and Music for Videogames.”  The article offers lots of interesting advice and personal anecdotes from industry pros.

Montreal International Game Summit 2014

WP-Session-MIGS-2
Just came back from a fantastic experience speaking at the Montreal International Game Summit 2014!

Montreal is a beautiful city, and that’s reflected in the fantastic rainbow-tinted windows of the convention center where the summit was held – the Palais des congrès de Montréal.

MIGS-Colored-Glass

The weather was relatively warm while I was there, but I spent most of my time at the summit… although I did enjoy the city views from the enormous walls of windows.


MIGS-WindowWall

This year’s summit was more vibrant than ever, and the fun began in the wide hallways where attendees could test their video game trivia knowledge by taking part in “The Game Masters” quiz show.  I wasn’t brave enough to compete, but I had to get a picture of the set:

MIGS-Game-Masters The show floor was very exciting this year, with a lot of the activity centering around the two Oculus Rift stations.  My attention, though, was caught by two things.  First — the AudioKinetic booth, where the Wwise middleware was on display:

MIGS-AudioKinetic

And second, this big green guy who was hulking inside the Ubisoft booth.  He looks brutish, but don’t let that fool you — he’s a real charmer.

MIGS-Ubisoft-Mascot

Here’s the big schedule of sessions that was posted at the event.  My speech was towards the end of the second day of the summit, right before the MIGS Brain Dump (which is kind of similar to a GDC rant).

MIGS-Big-Schedule

My talk was titled, “Music, the Brain, and the Three Levels of Immersion.”  It was a great audience!

WP-Session-MIGS-1

I had a wonderful time sharing some ideas about the role that music can play in helping gamers to achieve immersion. I’d first explored these ideas in my book, A Composer’s Guide to Game Music, and it was such a joy to explore these ideas with such an enthusiastic audience!

MIGS-SpeechPhoto-YT-Thumb

I’ll be posting a video excerpt from my talk soon.  It was wonderful to speak at MIGS 2014, and thanks to all the creative and inspiring people I met this year in Montreal – it was a tremendous pleasure!