Strategies in Audio & Music for Portable Games

Portable Game Audio and Music (article by award winning video game composer Winifred Phillips)
I’ll be talking about effective music composition for mobile and portable gaming platforms during my talk, “From Total War to Assassin’s Creed: Music for Mobile Games,” which will take place on March 16th at the upcoming Game Developers Conference at the Moscone Center in San Francisco.  With that in mind, I thought I’d use this blog entry to share some resources that explore current strategies and trends in regards to sound and music for mobile – resources that could be useful to the video game composer and sound designer.
Audio and Music for Portable Games (blog written by Winifred Phillips, video game composer)While my talk at GDC will focus specifically on music composition and implementation for handheld devices, the resources that will follow in this blog offer assistance with the more general technical issues that face audio pros creating sound assets for a mobile gaming environment.  I’ve included links to the original articles, as well as a summation of some of the best points that I thought were particularly interesting:

Memory Issues in Mobile Gaming:

Audio For Mobile Games by Jennifer Walden (Post Magazine)

Picture of Jennifer Walden (Strategies in Audio and Music for Portable Games, article by video game composer Winifred Phillips)Jennifer Walden is an experienced sound designer and an accomplished journalist.  Her writing focuses on audio post production, and her articles have appeared in such top industry publications as Mix Magazine, Sound & Picture, postPerspective and Post Magazine. In this article from March 2015, she explores the audio design of projects that include FIFA 15 Ultimate Team for Android, Eisenhorn – Xenos, and Wave Champions.  Taking a close look at the challenges posed by each project, Walden discusses the techniques and creative solutions that the audio teams developed along the way.  In so doing, it becomes apparent that memory limitations exert a strong guiding influence on the decision making process of audio pros working in mobile.

FIFA 15 Logo (blog written by Winifred Phillips, video game composer and author)In the popular FIFA 15 Ultimate Team for Android game, live recordings of authentic stadium cheering were transferred from the console version of the game to the mobile version. In order to accommodate the memory limitations, the number of recordings had to be limited.  This resulted in a somewhat diminished variety in audience reactions, but ensured that the robust presence of the stadium crowd would still be felt in the mobile version.  Likewise, the voice commentary in the console video game could not be included in the initial installation of the mobile game, since the 400MB size of the commentary far exceeded the 250MB initial download limit.  To address this, the team offered the commentary content to the player after installation as a separate download, which was triggered when the player chose their preferred commentary language (English, French, Spanish or German).

Eisenhorn: Xenos logo (article about portable game audio and music, written by award winning video game composer and author Winifred Phillips)In Eisenhorn – Xenos, the audio team worked hard to maximize the aural impact of this game in the famous Warhammer 40,000 universe, while also economizing on memory by removing some low end from their recordings. They anticipated that the mobile device’s speakers would not be able to convey bass with much fidelity.  The audio team also found ways to conserve their memory resources by limiting variation in repeating sounds (such as footsteps or sword swipes), favoring repetition of a small number of sounds over the use of lots of variations.

Wave Champions logo (article explores portable game audio and music, author: Winifred Phillips, video game composer)In the Wave Champions racing game, the team was able to introduce variety in the sounds of collisions by focusing on the materials of the objects involved in the collisions, and then creating a pool of ten variations for each type of material.  For instance, a plastic boat crashing into a concrete obstacle would make a noise consisting of a plastic, concrete, and water sound. Each of these sounds would be selected from a pool of ten variations – for instance, the plastic impact would come from a list of ten plastic impact sounds.  Combining this with a concrete impact and a water splash would create a unique sound effect, and these combinations would serve to increase sonic variety during gameplay.

The full article explores the audio issues facing these projects with plenty of useful details and observations.

Loudness Issues in Mobile Gaming:

Adaptive Audio for Mobile by Rob Bridgett (Gamasutra)

Photo of Rob Bridgett (blog explores portable game audio and music - by video game composer Winifred Phillips)Rob Bridgett is a G.A.N.G. award-winning writer and audio developer, currently serving as audio director of Eidos’ Montreal studio. In March 2015, while working at Clockwork Fox Studios, he wrote a detailed article for Gamasutra on the audio development of Zorbit’s Math Adventure – a game designed to teach kindergarten children the fundamentals of math.  In describing the audio design of this game, Bridgett explains that his primary focus for this project was centered on the issue of loudness in the mobile gaming environment.

Keenly aware of the sensitive nature of children’s hearing, Bridgett developed an “adaptive loudness model” that compensated for two factors:

  • Delivery method (headphones, external speakers, built in speakers)
  • Environment (noisy or quiet)

Zorbits Math Adventure logo (portable game audio and music article, by video game composer & author Winifred Phillips)This system attempted to create an optimal volume level by using various traditional compression techniques in tandem with an innovative system of adaptive compression based on the noise level detected by the device’s external microphone.  By measuring the “noisiness” of the room in which the game is being played, the adaptive compression system can adjust the perceived loudness of the game to compensate for any competing aural sources.  While some aspects of the system couldn’t be practically implemented in Zorbit’s Math Adventure and remain theoretical, the possibilities of such a system are intriguing.  Bridgett describes this “adaptive loudness” system in depth within his Gamasutra article, and it’s a fascinating read.

Audio Contractors in Mobile Development

Crank It Up!  Social Gaming Needs Great Sound! by Nicholas Day

Photo of Nicholas Day (blog discusses portable game music and audio, article by Winifred Phillips, award-winning author and video game composer)Nicholas Day is the Creative Director of Plarium Global Ltd., a developer of mobile and social games for hardcore gamers. In May 2015 on the Plarium web site, he wrote an enthusiastic article supporting audio excellence in mobile and social games.  After first making the case for an increased focus on audio quality in these types of projects, Day went on to describe the steps taken for various Plarium projects, including Total Domination and Soldiers Inc.  An important facet of the audio production workflow was the contracting of quality external audio professionals.

Plarium Games Logo (in a blog about audio and music in portable games, written by Winifred Phillips, video game composer / author)While Day acknowledges that audio resources can be created and implemented in-house, he stresses that contracting externally can prove to be more cost effective than acquiring internal personnel and equipment.  Moreover, Day emphasizes that external contractors need not be located within proximity of the developer, and can in fact collaborate effectively from great distances.  “There are quality sound studios throughout the world,” Day asserts.  “With the Internet, a global studio is certainly possible.”  The article explores the advantages and considerations involved in external contracting for mobile audio, and is well worth reading.

Extreme Memory Issues in Mobile Audio

Raising Hell in Mobile Audio: The Sounds of Dungeon Keeper, by Nick LaMartina (G.A.N.G.)

Photo of Nick LaMartina (in an article about audio in portable games, by Winifred Phillips, author / video game composer)Nick LaMartina is an experienced sound designer/composer who is currently working as an audio contractor for Magic Leap, Inc.  He previously held an audio director position at EA Mythic, and in January 2015 he wrote a comprehensive article for the Game Audio Network Guild web site about the audio development process of Mythic’s Dungeon Keeper game.  In the article, LaMartina emphasizes the artistic ambition of the game’s audio development, and the teams’ desire to attain AAA audio quality for their mobile game.

Dungeon Keeper logo (discussed in a blog exploring portable game music and audio, by video game composer Winifred Phillips, author of A COMPOSER'S GUIDE TO GAME MUSIC)However, as the article illustrates, the push to economize on memory resources was persistent throughout the project. “We used every compression format and optimization Wwise makes available,” says LaMartina.  He goes on to describe an audio tactic he describes as “a Level-of-Detail system,” designed to keep audio complexity under control by deactivating digital signal processing, altering voice limits, pausing submixes and adjusting volume levels as compensation for low-memory situations.  LaMartina includes some interesting technical detail for this system, and for the team’s decision to extensively use the Vorbis codec in the game.  The article is a great exploration of the audio workflow of a mobile game with AAA ambitions.

Conclusion

These four articles offers some awesome perspectives on the technical challenges of audio development for mobile platforms, as well as some innovative solutions for the most common problems.  In my upcoming GDC talk, I’ll be exploring issues relating specifically to music composition for mobile, which has its own unique challenges.  I’m looking forward to it, and I hope you’ll join me in San Francisco on March 16th!  You can find more information about my talk here: “From Total War to Assassin’s Creed: Music for Mobile Games.”

 


Photo of video game composer Winifred Phillips, author of the book A COMPOSER'S GUIDE TO GAME MUSIC (The MIT Press)Winifred Phillips is an award-winning video game music composer whose most recent project is the triple-A first person shooter Homefront: The Revolution. Her credits include five of the most famous and popular franchises in video gaming: Assassin’s Creed, LittleBigPlanet, Total War, God of War, and The Sims. She is the author of the award-winning bestseller A COMPOSER’S GUIDE TO GAME MUSIC, published by the Massachusetts Institute of Technology Press. As a VR game music expert, she writes frequently on the future of music in virtual reality video games. Follow her on Twitter @winphillips.

2 responses to “Strategies in Audio & Music for Portable Games

  1. Always fascinates me how game audio people, in spite of games having evolved tenfold, have basically had the same problem with memory allocation for decades!

    Reminds me of the clever ways composers in the 80s got round the limitations they had back then. Good to see we’re still finding creative solutions today 🙂 Very interesting as always, hope the talk goes well.

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s