Workflow in Multiple Takes (for the Game Music Composer)

At this year’s Game Developers Conference, voice director Michael Csurics presented a terrific talk called “VO Session Live: Ensemble Edition.”  By soliciting the audience for volunteer voice actors, Csurics staged a live voice-over recording session that included both solo actor/performers and multiple actors running dialogue in a traditionally theatrical way.  The demonstration served to reveal the ins-and-outs of recording voice dialogue, both from an artistic and technical standpoint.  One of the portions of Csurics’ talk that I found particularly interesting was his exploration of the process of multiple takes in recording voice dialogue, and I thought his method might have some bearing on the process of recording multiple takes of a musician’s performance.  In this blog, I’ll be going over the basics of his methodology, as he described it during his talk, and I’ll also be sharing some of my thoughts regarding how his working method intersects with my experiences recording session performances with live musicians.

Csurics-GDC15

Michael Csurics giving his talk, “VO Session Live: Ensemble Edition”

For typical voice-over recording sessions, Csurics begins by having his voice performers record a short piece of dialogue at least two or three times.  This process “gets them comfortable, gets them past the cold read and into an actual read.”  During these voice sessions he will be assisted by an audio engineer running the Pro Tools session. Csurics will usually pause after the first two or three readings, sometimes giving direction and asking for additional takes. Once satisfied, he will then tell the audio engineer which take he liked, and the audio engineer will “pull down” this take.  In other words, the engineer will select the portion of the recorded waveform that represents the final take and copy/paste it into a blank audio track directly below the track that is currently recording the vocal session.  In this way, Csurics is able to save post-production time by making his final selections on-the-fly.

Michael Csurics' isolation booth for vocal recording

Michael Csurics’ isolation booth for vocal recording

When I’m recording live musicians (either my own performances or those of others), my own workflow resembles Csurics’ process in some respects.  I’ll have the backing music mix loaded into its own audio tracks in Pro Tools, and then I’ll set up a collection of blank audio tracks below the rest of the music mix. The number of blank tracks will largely depend on how many takes I expect will be required.  For especially difficult parts in which a large number of takes are anticipated, I may set up anywhere from four to fifteen blank tracks.  The musician (or myself, if that’s the case) will record a short section of the music, repeating that section for as many takes as may seem warranted.  Each take will be recorded into one of the separate audio tracks that I’d set up prior to the recording session. Once recorded, I’ll mute that audio track, record-enable the next one and then record the next take.  Once complete, I’ll have a fairly large assortment of takes that I can audition during post-production in order to edit together a version that will be used in the final recording.  During post, I employ the same “pull down” method that Csurics described during his talk – i.e. copy/pasting the best version of each performance into a blank audio track at the bottom of my Pro Tools edit window.

I admire Csurics’ on-the-fly method of selecting the best take, but personally I’m only comfortable with making an instant decision if it pertains to simply discarding a take completely and trying again.  In this case, I’ll delete the recording from the edit window, ensuring that I avoid any possibility of confusion later.  Using this method for recording live instrumental performances gives me a large amount of flexibility in post, allowing me to assemble the most ideal musical performance possible.  I can listen critically to every phrase in every take, selecting only the very best execution of every musical element for inclusion in the final recording.  I talk more about some of the technical aspects of my workflow in my book, A Composer’s Guide to Game Music.

The Bleeps 'n' Bloops game audio podcast

The Bleeps ‘n’ Bloops game audio podcast

There were lots more details regarding voice dialogue tracking and editing in Csurics’ talk, which is available in the GDC Vault.  If you don’t have access to the Vault, you can hear a lot of those same ideas in this great podcast interview with Csurics, recorded by the Bleeps ‘n’ Bloops podcast during last year’s GameSoundCon.  The interview includes some interesting details about workflow, track naming, post editing and processing.  I’ve embedded the SoundCloud recording of that podcast interview below.  The section regarding voice-over session logistics starts at 11:40:

There’s also an earlier interview with Csurics posted on the Game Audio Network Guild site, which reflects on some of Csurics’ experiences as the Dialogue Supervisor for 2K Marin:

Interview with 2K Marin’s Dialogue Supervisor Michael Csurics

Since leaving 2K Games in 2012, Michael Csurics has worked as an independent contractor and now runs a game audio production services company called The Brightskull Entertainment Group (he can be reached on Twitter at @mcsurics).  If you’d like to see and hear the great work going on at Brightskull, here’s a video demonstration from one of their latest projects, The Vanishing of Ethan Carter:

MIDI in Wwise for the Game Music Composer: Peggle Blast

PeggleBlastBanner

In a previous blog post, we took a look at a few tutorial resources for the latest version of the Wwise audio middleware.  One of the newest innovations in the Wwise software package is a fairly robust MIDI system.  This system affords music creators and implementers the opportunity to avail themselves of the extensive adaptive possibilities of the MIDI format from within the Wwise application.  Last month, during the Game Developers Conference in the Moscone Center in San Francisco, some members of the PopCap audio development team presented a thorough, step-by-step explanation of the benefits of this MIDI capability for one of their latest projects, Peggle Blast.  Since my talk during the Audio Bootcamp at GDC focused on interactive music and MIDI (with an eye on the role of MIDI in both the history and future of game audio development), I thought that we could all benefit from a summation of some of the ideas discussed during the Peggle Blast talk, particularly as they relate to dynamic MIDI music in Wwise.  In this blog, I’ve tried to convey some of the most important takeaways from this GDC presentation.

PeggleBlastSession

“Peggle Blast: Big Concepts, Small Project” was presented on Thursday, March 5th by three members of the PopCap audio team: technical sound designer RJ Mattingly, audio lead Jaclyn Shumate, and senior audio director Guy Whitmore.  The presentation began with a quote from Igor Stravinsky:

The more constraints one imposes, the more one frees oneself, and the arbitrariness of the constraint serves only to maintain the precision of the execution.

This idea became a running theme throughout the presentation, as the three audio pros detailed the constraints under which they worked, including:

  1. A 5mb memory limit for all audio assets
  2. Limited CPU
  3. 2.5mb memory allocation for the music elements

These constraints were a result of the mobile platforms (iOS and Android) for which Peggle Blast had been built.  For this reason, the music team focused their attention on sounds that could convey lots of emotion while also maintaining a very small file size.  Early experimentation with tracks structured around the use of a music box instrument led the team to realize that they still needed to replicate the musical experience from the full-fledged console versions of the game.  A simple music-box score was too unsatisfying, particularly for players who were familiar with the music from the previous installments in the franchise.  With that in mind, the team concentrated on very short orchestral samples taken from the previous orchestral session recordings for Peggle 2.  Let’s take a look at a video from those orchestral sessions:

Using those orchestral session recordings, the audio team created custom sample banks that were tailored specifically to the needs of Peggle Blast, focusing on lots of very short instrument articulations and performance techniques including:

  1. pizzicato
  2. marcato
  3. staccato
  4. mallets

A few instruments (including a synth pad and some orchestral strings) were edited to loop so that extended note performances became possible, but the large majority of instruments remained brief, punctuated sounds that did not loop.  These short sounds were arranged into sample banks in which one or two note samples would be used per octave of instrument range, and note tracking would transpose the sample to fill in the rest of the octave.  The sample banks consisted of a single layer of sound, which meant that the instruments did not adjust their character depending on dynamics/velocity.  In order to make the samples more musically pleasing, the built-in digital signal processing capability of Wwise was employed by way of a real-time reverb bus that allowed these short sounds to have more extended and natural-sounding decay times.

wwise-logo-empowers

The audio team worked with a beta version of Wwise 2014 during development of Peggle Blast, which allowed them to implement their MIDI score into the Unity game engine.  The composer, Guy Whitmore, composed the music in a style consisting of whimsically pleasant, non-melodic patterns that were structured into a series of chunks.  These chunks could be triggered according to the adaptive system in Peggle Blast, wherein the music went through key changes (invariably following the circle of fifths) in reaction to the player’s progress.  To better see how this works, let’s watch an example of some gameplay from Peggle Blast:

As you can see, very little in the way of a foreground melody existed in this game.  In the place of a melody, foreground musical tones would be emitted when the Peggle ball hit pegs during its descent from the top of the screen.  These tones would follow a predetermined scale, and would choose which type of scale to trigger (major, natural minor, harmonic minor, or mixolydian) depending on the key in which the music was currently playing.  Information about the key was dropped into the music using markers that indicated where key changes took place, so that the Peggle ball would always trigger the correct type of scale at any given time.  The MIDI system did not have to store unique MIDI data for scales in every key change, but would instead calculate the key transpositions for each of the scale types, based on the current key of the music that was playing.

The presentation ended with an emphasis on the memory savings and flexibility afforded by MIDI, and the advantages that MIDI presents to game composers and audio teams.  It was a very interesting presentation!  If you have access to the GDC Vault, you can watch a video of the entire presentation online.  Otherwise, there are plenty of other resources on the music of Peggle Blast, and I’ve included a few below:

Inside the Music of Peggle Blast – An Interview with Audio Director Guy Whitmore

Peggle Blast!  Peg Hits and the Music System, by RJ Mattingly

Real-Time Synthesis for Sound Creation in Peggle Blast, by Jaclyn Shumate

PopCap’s Guy Whitmore Talks Musical Trials And Triumphs On Peggle Blast

 

How To Break Into Game Music Composing: Advice from GDC 2015

During the Game Developers Conference in San Francisco this year, attendees of the GDC Audio Track were treated to not one, but two sessions on how to break into the industry as a game audio professional.  Since this is always a topic of intense interest, I thought I’d use this blog entry to sum up some of the ideas from those two sessions. Hopefully some of this information will prove useful!

First, let’s look at the basic details about those two GDC talks.

Amongst the many different speakers at the two presentations, there were a lot of ideas and strategies put forward about ways to break into game audio development.  A few common themes definitely rose to the surface, and I’ll go over those here:

You gotta have good connections.

networking

The importance of friendships and relationships was an oft repeated idea at these talks.  When the members of the “Landing Your First Game” panel were asked about how they got their first gig, they all said that they owed it to a referral from a friend.  Likewise, the Audio Bootcamp talk stressed the importance of making friendships during in-person meetings and conferences.

According to the speakers, it’s important to simply hang out with game audio pros in a low-pressure environment in which prospective gigs aren’t discussed at all.  Taking the opportunity to form friendships without pursuing career objectives was considered the best approach for an aspiring game audio professional attending a conference or convention.  The speakers suggested that opportunities would eventually surface from these friendly relationships, but patience would be required, and years might pass before any of these relationships lead to a game audio gig.  I can confirm from my own experience that maintaining relationships is vitally important, and that a friendly acquaintance can lead to a game audio job years down the line.

At this point, I should probably mention that a certain aspect of my early career experience would be considered highly atypical – I landed my first gig by doing something that the speakers labeled as a “cold-call.”  According to the speakers, a “cold-call” is a phone call or an e-mail sent to someone who is essentially a stranger.  This isn’t supposed to be a successful approach, and the speakers strongly advised against cold-calling… but that’s what I did.  I sent an e-mail to a music supervisor who didn’t know me, and I introduced myself and asked about music opportunities at that company.  This led to my first game job, as a composer on the music composition team of God of War from Sony Computer Entertainment America.  My experience in respect to the “cold call” seems to be the exception that proves the rule, according to the speakers at these two presentations.

Be willing to negotiate, or to work for free

negotiation

One of the speakers told a story about breaking into the industry by donating music for free to a mod.  A “mod” is a modification to the program code of a retail game by an indie team, resulting in a new game that requires players to own the original retail game in order to play the modification.  Since a mod team typically consists of hobbyists with little or no budget, creating music for a mod is usually a charitable endeavor on the part of the composer.  At one point, an audience member asked how long any aspiring game composer should donate their services before finally resolving not to work for free anymore.  The response was that free work can continue throughout an audio professional’s career – this kind of free work was referred to as a “passion project.”  When we become uncomfortable with working for free, however, the time has come to negotiate for compensation.

Both presentations spent considerable time discussing compensation structures that diverged from traditional rate-per-minute work-for-hire agreements.  Licensing agreements were mentioned repeatedly during both talks, which might (or might not) include participation in the “back end” revenue that the video game would earn once it was released.  Also under discussion was the possible retention of the rights to release a soundtrack album and collect its earnings.  This trend seemed to indicate that offering an agreement that would require less money from the developer up-front might convince the developer to pay something, rather than nothing, and that this might help the aspiring audio pro to secure their first paying gig.

On a personal note, I donated music to a “mod” early in my career.  Although it allowed me to gain a bit of experience as a game composer, I didn’t enjoy any particular career advantages for having done so. However, the mod I scored ended up as vaporware (i.e. unreleased), which underlines the importance of evaluating the mod project in order to estimate whether the mod is likely to be finished or not. I talk about this experience, along with several of my other early game projects, in my book.

Lately, I’ve heard a lot of buzz about licensing agreements in the game industry, and I imagine that it might be a viable alternative for new game audio folks looking to transition from free gigs to paying work.  However, according to the latest Game Audio Industry Survey conducted by GameSoundCon, only 12% of professionally produced indie games allow composers to license their music rather than selling the rights on a work-for-hire basis.  Most of the game development teams open to music licensing would be characterized as “not professionally produced,” according to the language in the Game Audio Industry Survey.

Some final thoughts

The speakers mentioned several conferences and other resources for the aspiring game composer, which I’ve listed below:

Also, just as an addendum, audio designer Will Morton, best known for his work on the Grand Theft Auto games, has recently published an article on Gamasutra.com called “Make Some Noise! Getting a Job Creating Sound and Music for Videogames.”  The article offers lots of interesting advice and personal anecdotes from industry pros.

Game Music Middleware, Part 2: FMOD

Middleware-Blackboard

 

Since third-party audio middleware in game development is becoming slowly more prevalent, I’m devoting two blog entries to some tutorials by game audio pros who have produced videos to demonstrate their process working with the software.  I posted the first blog entry in February — you can read it here.

This second blog is devoted to FMOD, and the tutorials were produced by two game composers who have generously shared their experiences.  The first video focuses on the creation of adaptive music for a demo competition hosted by the Game Audio Network Guild and taking place during the Game Developers Conference 2014 in San Francisco.  The tutorial was produced by composer Anastasia Devana, whose game credits include the recently released puzzle game Synergy and the upcoming roleplaying game Anima – Gate of Memories.

Adaptive music in Angry Bots using FMOD Studio and Unity

border-159926_640_white The next tutorials come to us from composer Matthew Pablo, who produced a series of videos on the implementation of game music via the FMOD middleware.  Matthew’s work as a game composer includes N-Dimensions, Arizona Rose and the Pharaoh’s Riddles, Cloud Spin, Micromon, and many more.  Here is the first video — the rest can be found in Matthew’s YouTube playlist.

FMOD Studio: Overview & Introduction

 

 

GDC 2015 Photos

The Game Developers Conference was a fantastic experience for me this year!  I gave two presentations — “Advanced Composition Techniques for Adaptive Systems” at the GDC Audio Bootcamp on Tuesday, and “LittleBigPlanet 3 and Beyond: Taking Your Score to Vertical Extremes” during the main conference on Friday.  I had a great time!  Here are a few of my photos from GDC week.  Just click on the first thumbnail image to open the full-sized gallery.

Interactive Game Music of LittleBigPlanet 3 (Concepts from my GDC Talk)

LittleBigPlanet 3 and Beyond: Taking Your Score to Vertical Extremes -- Speaker, Winifred Phillips

LittleBigPlanet 3 and Beyond: Taking Your Score to Vertical Extremes

I was honored to be selected by the Game Developers Conference Advisory Board to present two talks during this year’s GDC in San Francisco earlier this month.  On Friday March 6th I presented a talk on the music system of the LittleBigPlanet franchise.  Entitled LittleBigPlanet 3 and Beyond: Taking Your Score to Vertical Extremes,” the talk explored the Vertical Layering music system that has been employed in all of the LittleBigPlanet games (the soundtrack for that game is available here).  I’ve been on the LittleBigPlanet music composition team for six of their games so far, and my talk used many examples from musical compositions I created for all six of those projects.

After my talk, several audience members let me know that the section of my presentation covering the music system for the Pod menu of LittleBigPlanet 3 was particularly interesting – so I thought I’d share the concepts and examples from that part of my presentation in this blog.

GDC15_LBP3-Game-Music_Winifred-Phillips-1

That’s me, giving my GDC speech on the interactive music system of the LittleBigPlanet franchise.  Here I’m just starting the section about the Pod menu music.

The audio team at Media Molecule conceived the dynamic music system for the LittleBigPlanet franchise.  According to the franchise’s music design brief, all interactive tracks in LittleBigPlanet games must be arranged in a vertical layering system.  I discussed this type of interactive music in a blog I published last year, but I’ll recap the system briefly here as well.  In a vertical layering music system, the music is not captured in a single audio recording.  Instead, several audio recordings play in sync with one other.  Each layer of musical sound features unique content.  Each of the layers represents a certain percentage of the entire musical composition.  Played all together, we hear the full mix embodying the entire musical composition.  Played separately, we hear submixes that are still satisfying and entertaining for their own sake.  The music system can play all the layers either together or separately, or can combine the layers into different sets that represent a portion of the whole mix.

When implemented into gameplay, layers are often activated when the player moves into a new area.  This helps the music to feel responsive to the player’s actions.  The music seems to acknowledge the player’s progress throughout the game.  It’s important to think about the way in which individual layers may be activated, and the functions that the layers may be called upon to serve during the course of the game.

GDC15-Winifred-Phillips

In LittleBigPlanet 3, the initial menu system for the game is called “The Pod.”  The music for the Pod is arranged in vertical layers that are activated and deactivated according to where the player is in the menu hierarchy.  All the layers can be played simultaneously, and they play in multiple combinations… however, each of the individual layers is also associated with a specific portion of the menu system, and is activated when the player enters that particular part of the menu.

Let’s take a quick tour through the layers of the Pod menu music.  I’ve embedded some short musical excerpts of each layer.  You’ll find the SoundCloud players for each layer embedded below – just click the Play buttons to listen to each excerpt.  The first layer of the Pod menu music is associated with the Main Menu, and it features some floaty, science-fiction-inspired textures and effects:

The next layer is associated with a menu labeled “My Levels,” and the music for that layer is very different.  Now, woodwinds are accompanied by a gentle harp, combining to create a homey and down-to-earth mood:

Moving on to the music layer for the “Play” menu, we find that the instrumentation now features an ethereal choir and shimmering bells, expressing a much more celestial atmosphere:

Now let’s listen to the “Adventure” menu layer, in which plucked strings and bells combine to deliver a prominent melody line:

Finally, in the music layer associated with the “Community” and “Popit” menus, we hear a quirky mix of synths and effects that hearken back to menu music from previous games in the LittleBigPlanet franchise:

As the player navigates the Pod menu system, these various music layers are activated to correspond with the player’s location within the menu hierarchy.  This sort of dynamic music triggering lies at the very heart of the Vertical Layering interactive music mechanism.

GDC15_LBP3-Game-Music_Winifred-Phillips-2

Every layer in a Vertical Layering composition can have a very distinct musical identity.  When that layer is turned off, the entire mix changes in a noticeable way.  The mix can be changed subtly…

GDC15_LBP3-Game-Music_Winifred-Phillips-3

… or it can be altered radically, with large scale activations or deactivations of layers.  Even with these kinds of dramatic changes, the musical composition retains its identity.  The same piece of music continues to play, and the player is conscious of continuing to hear the same musical composition, even though it has just altered in reaction to the circumstances of gameplay and the player’s progress.

In the Pod menu music system, the layers would change in reaction to the player’s menu navigation, which could be either slow and leisurely or brisk and purposeful.  Layer activations and deactivations would occur with smooth crossfade transitions as the player moved from one menu to another.  Now let’s take a look at a video showing some navigation through the Pod menu system, so we can hear how these musical layers behaved during actual gameplay:

 As you can see, triggering unique musical layers for different portions of the menu system helps serve to define them.  I hope you found this explanation of the Pod music to be interesting!  If you attended GDC but missed my talk on the interactive music of LittleBigPlanet, you’ll be able to find the entire presentation posted as a video in the GDC Vault in just a few weeks.  In the meantime, please feel free to add any comments or questions below!

GDC 2015 Book Signing – A Composer’s Guide to Game Music

BookStoreBanner

It was wonderful to sign copies of my book last week at the book signing event organized by The MIT Press for my book, A COMPOSER’S GUIDE TO GAME MUSIC!  The book signing took place on March 6th at the official GDC Bookstore in the Moscone Center (South Hall) during this year’s Game Developers Conference in San Francisco!  It was a lot of fun!  So terrific to meet such a great group of composers and sound designers, and I loved hearing about your creative endeavors in the world of game audio!  I’m also very humbled and pleased that you’re using my book to help you with your projects!  Here’s a gallery with pictures of some of the folks who were at the book signing last week! Just click on the first thumbnail image to open the full-sized gallery.