The Great MIDI Comeback?

I recently read a great article by Bernard Rodrigue of Audiokinetic in Develop Magazine, heralding the return of MIDI to the field of video game music.  It was a very well-written article, filled with hopeful optimism about the capability of MIDI to add new musical capabilities to interactive video game scores, particularly in light of the memory and CPU resources of modern games consoles.

It also reminded me strongly of another article I read, from 2010.

Four years ago, Microsoft Sound Supervisor West Latta wrote for Shockwave-Sound.com that “we may see a sort of return to a hybrid approach to composing, using samples and some form of MIDI-like control data… the next Xbox or Playstation could, in fact, yield enough RAM and CPU power to load a robust (and highly compressed) orchestral sample library.”

So, it seems that the game audio sector has been anticipating a return to MIDI for awhile now (I wrote at length about the history and possible future of MIDI in my book, A Composer’s Guide to Game Music).  The question is – has the current generation of video game consoles evolved to the point that a quality orchestral sample library could be loaded and used by MIDI within a modern video game?  So far, I haven’t come across an answer to this question, and it’s a very intriguing mystery.

Certainly, the availability of an orchestral sample library in a MIDI-based interactive video game score would depend on factors that are not all hinged to the technical specs of the hardware.  Would the development teams be willing to devote that amount of memory to a quality orchestral sample library?  As games continue to participate in a visual arms race, development teams devote available hardware horsepower to pixels and polygons… so, would the music team be able to get a big enough slice of that pie to make a high-quality orchestral MIDI score possible?

I’m keeping my eyes open for developments in this area. Certainly, the return of MIDI could be a game changer for composers of interactive music, but only if the musical standards remain high, both in terms of the music compositions and the quality of the instruments used within them. Let me know in the comments if you’ve heard any news about the great MIDI comeback!

 

Breathing Music: The Role of Silence

Image by GagDonkey.com

 

The Designing Sound blog recently devoted a series of articles to the topic of silence, including an elaborately philosophical article on the nature of silence as a Zen state of altered consciousness (Silence is the Sound of Listening, by Miguel Isaza).

My main impression from the article was an emphasis on sound as the state of calm in which we (as listeners) become receptive to the world of aural phenomena constantly surrounding us.

 

The article brought to mind a few ideas that I thought I would share about the role of silence in the creative output of a game composer.

Sometimes when we as game composers receive creative guidance in regards to the musical style of a project, we’ll be instructed to do the following:

Let the music breathe.

The idea of “breathing music” can be interpreted in several ways.  It can mean that the music should dwindle intermittently into absolute silence so that the game’s soundscape can essentially “take over” for a few moments, before the music resumes.  It can also mean that the music should be written with sparse instrumentation and lots of unoccupied space in the frequency spectrum, resulting in the impression of lots of brief silent pauses that allow the sound design environment to filter through the lattice of musical elements.  Finally, it can mean that the music is composed of a series of crescendos and diminuendos, whereby the musical score swells dramatically and then recedes into a near-silent state on a regular basis.

All of these approaches share one aspect in common: the music is structured to allow the sound design to move regularly into the foreground, pushing the music further into the background of the player’s conscious awareness.  With this in mind, should we interpret this instruction to “let the music breathe” as a desire to deemphasize the music in favor of other aspects of the game’s aural design?

On page 52 of my book, A Composer’s Guide to Game Music, I discuss an interesting study conducted by Stanford University, which casts a very different light on the effect of silence on the experience of listening to music.  The study revealed that when listening to a piece of music, our minds become most attentive and filled with the most anticipatory focus when the music becomes silent for a moment.  For instance, in the short pauses between the movements of a symphony, the listener’s attention to the music peaks.

So, when we’re asked to let the music “breathe,” perhaps we can interpret this to mean that we should include those brief pauses that cause the player to pay more attention to our music than they had before.  As Miguel Isaza wrote in his article for Designing Sound, the act of becoming silent awakens our consciousness to the world of sound around us.  Perhaps by using silence as a tool in our game music, we can awaken gamers to the world of music we have created.

 

Game Music and Gamer Anticipation

DS_Icon

When the circumstances of a gameplay scenario create a sense of anticipation, this is usually engineered for a good reason.  The developers want the gamer to be tensely expectant of the monster attack, or the sniper ambush, or any other enemy-type that might leap out and yell “boo!”  In the article, “The Story in Surround – Game narrative and sound,” author Cormac Donnelly of Designing Sound makes a case for the role that audio can play in creating that sense of expectation.

The theory hinges on the idea of “mental modeling,” in which we develop expectations about the outcome of future events based on our past experiences.  Donnelly suggests that by initiating specific sound design elements in advance of certain in-game events, we can train the player to associate the sound with the impending event.  This can then be used to manipulate the emotions and even the subsequent actions of players who will respond to the sounds with immediate reactions based on their “mental model.”

It’s an interesting concept that we, as game composers, can also apply to our own work.  In cooperation with the audio teams, we can plan music systems that include music precursors to significant events. These anticipatory musical elements can then become built into the players’ “mental model” concerning the event in question.  This music becomes one more way in which the game can sculpt the emotional dynamic of gamers and nudge them in specific directions.

The article by Cormac Donnelly, “The Story in Surround – Game narrative and sound,” appeared on the Designing Sound web site – it can be found here.

Music and the Anxious Gamer

OUPblog

An interesting article from the Oxford University Press blog examines the importance of audio in video games, with an emphasis on its effect on player performance. Do gamers play more successfully with immersive music and sound effects, or is their performance improved in a more silent play-space?

According to studies conducted by the author Siu-Lan Tan, gamers perform better with game audio turned on rather than off, although this performance improvement seemed most pronounced among experienced gamers.  For newbies, the game audio and music seemed to increase initial anxiety levels at the start of the game, and these players benefited from turning the audio off initially and replacing it with unrelated music playing from a boombox in the room.

What does this tell us?  In my book, A Composer’s Guide to Game Music, I explore this idea along with a possible approach that game composers can take in order to alleviate the problem (Chapter 3, page 40):

“A player may be initially overwhelmed or even discouraged by the apparent complexity of the game.  This can pose a very large barrier to access.  As game composers, our job here is to offer emotional reassurance to the player.”

The chapter continues with an exploration of musical choices that can enhance player concentration and alleviate stress.  This is one of the many ways in which music can aid players in performing at their best throughout the course of a game.

The article by Siu-Lan Tan for the Oxford University Press is called, “What’s the secret to high scores on video games?” and you can find it here.

 

 

Music in Virtual Reality

DS_Banner

 

Virtual Reality has stepped out of the realm of science fiction and Hollywood pipe dreams.  Sooner than we think, we’ll be gaming with VR systems like Sony’s Project Morpheus and Facebook’s Oculus Rift.  As a user interface for gaming, the VR helmet promises to offer a spectacular aural environment by virtue of binaural audio (you’ll find a good explanation of the binaural system here).

I recently read a great article on the Designing Sound site about the audio possibilities inherent in the VR technology, and I highly recommend it.  The article discusses some of the profound differences between VR audio and traditional surround-sound audio for modern games.  The binaural technology involves the use of two microphones positioned carefully to mimic the natural placement of human ears. These two mono signals allow the resulting stereo recording to convey positional information with great accuracy, making it possible for the final result to deliver an immersive “surround-sound” experience without the use of multiple speakers.

Early in my career, when I was working as a composer/sound designer for a National Public Radio drama series, I captured some rudimentary quasi-binaural recordings by employing two microphones tied to my body while recording to a portable Digital Audio Tape recorder.  Thus equipped, I walked through various outdoor environments, rode on public transportation, situated myself in noisy crowds and let the people stream around me, all while recording with my two microphones.  I remember that the end result was powerfully immersive.  Technically, I didn’t fully achieve the true binaural effect.  My microphones were not at “head-level” but were instead at hip-height and spaced a bit wider than a human head would be… so perhaps it might be considered a true binaural recording designed for a hobbit with a huge noggin.  Nevertheless, the principle was very similar, and I remember how excited I was about the immersive realism of those recordings. The use of binaural recordings in VR should be an amazing contribution to the “reality” portion of the virtual reality experience.

One important question isn’t answered by the Designing Sound article – how will music be incorporated into such a system?  This question seems to pose similar difficulties to those faced when incorporating music into a surround-sound mix.  Do we mix the music in surround sound as well, so that the movements of players directly impacts the physical positioning of the music in the 3D world – or will this be too confusing?  Conversely, do we keep the music in traditional stereo, and if so, should the music always occupy a position directly in front of the players’ faces, no matter which way they may turn their heads?  It’s a complicated decision, and it looks like VR technology will only make this issue more complex.

The Oculus Rift is expected to hit the marketplace at the end of 2015, so we’ll have at least a little time to consider these problems before they’ll have to be solved.

 

GameFAQs Game Audio Poll

GameFAQ-Poll

On the community forums at GameFAQs.com, users are comparing notes regarding their favorite method of listening to game audio.  This includes an interesting poll, wherein users can vote for their preferred video game listening environment.  The winner?  Gaming headsets, which pulled 30% of the vote.  This was followed by 5.1 surround systems, TV speakers, 7.1 surround systems, with an external stereo system trailing the field.

Why does this matter to a game composer?  Our method of monitoring our mixes should reflect what the majority of listeners will experience in-game.  If we primarily hear our mixes through stereo monitors, we’re experiencing the music in a way that matches the listening environment of only a small percentage of our target audience.  We’ll have to make sure that we check our mixes through multiple audio delivery systems, with a particular emphasis on headsets.

Source: GameFAQs

E3 2014: What’s New in Audio Gear

e3-future

Next week, the game industry will gather together for their annual trade fair extravaganza. I’m curious about the anticipated game announcements and press conferences, but as a game audio professional, I’ll be very interested in the consumer products and services that will be demonstrated on the E3 show floor next week.  Here are some of the E3 exhibitors and products that may be of interest to game audio folks:

astro-logo

ASTRO Gaming / Skullcandy

ASTRO Gaming hasn’t given any indication of what they’ll be showing at E3 this year, but it’s a good bet that their “Watch_Dogs” A40 & A30 headsets with included speaker tags will be on display there.  “Watch_Dogs” is an action-adventure game from Ubisoft that just came out last month, and Ubisoft will be showing the game in its booth on the show floor.  The PC headphones come with branded “Watch_Dogs” speaker tags, which are decorative magnets that affix to headphones in order to make them more stylish.  E3 Booth Location:  Concourse Hall, Meeting Room 513.

logo_dreamgear

dreamGear

This company isn’t an audio specialist – instead, it offers a range of game accessories including controllers, headsets and power solutions.  In the audio category, they’ll be featuring their Universal Elite gaming headset at their booth.  E3 Booth Location: West Hall, Booth 5422.

Exeo_Logo

Exeo Entertainment

The product of interest to us here – the Psyko surround-sound headphones – claim to deliver  “the highest level of audio directionality and natural sound reproduction over any other 5.1 or 7.1 surround sound gaming headset.”  The headphones have five distinct speakers and a subwoofer embedded in each earcup, according to Exeo. The resulting effect is purported to emulate true surround sound more faithfully in the headphone monitoring environment. E3 Booth Location: West Hall, Booth 5336.

Gamestergear_logo

GamesterGear

The headset manufacturer GamesterGear will be debuting a new line of headsets in its booth at E3.  The Falcon console and PC gaming headset series will be put through its paces during the booth’s daily “Beat a Pro” Tournaments, which offer booth visitors an opportunity to compete against the professional gamers of TeamGamesterGear.  The Falcon headsets feature “the industry’s largest 57mm and 30mm drivers” and a force-feedback technology that the company has dubbed “BASS QUAKE.”  E3 Booth Location: South Hall, Booth 3047.

immerz-logo

Immerz

Based in Cambridge, Massachusetts, Immerz, Inc. is the developer of the KOR-FX game peripheral that will be officially announced at this upcoming E3.  The KOR-FX device is a peripheral that resembles a hi-tech vest that you wear while gaming.  The vest converts audio content into vibrations that are delivered to specific areas of the chest via transducers.  This is meant to render the audio content of a game more impactful and immersive throughout the course of play.  It should be interesting to see this device demonstrated on the show floor.  E3 Booth Location: South Hall, Booth 2855.

plantronics-gaming_logo

Plantronics

Plantronics is an audio communications equipment manufacturer known for supplying the headsets worn by astronauts during the first moon landing.  Their computer and gaming headsets line will be on display on the E3 show floor.  This will likely include their relatively new Plantronics Rig headset with swappable mics and EQ control.  Also, this will be an opportuntity to see and hear the newest RPG created by Richard Garriott (creator of the renowned Ultima series). Shroud of the Avatar: Forsaken Virtues will be on display here, and will give booth visitors an opportunity to test out the newest Plantronics headphones while listening to the soundscape of Garriott’s newest creation.  E3 Booth Location: South Hall, Booth 515.

polk-audio-logo-png

Polk Audio

This audio technology manufacturer specializing in speakers and headphones for audiophiles will be showing a new soundbar and two new headphone models specifically created for the Xbox One.  Polk describes the N1 Soundbar as having “four immersion modes for a tailored and immersive listening experience.”  It’s 4Shot for Xbox One and 133t for Xbox 360 are a pair of headphones that will “deliver individualized audio unparalleled in the category.”  All three products will not be available for purchase until this fall.  E3 Booth Location: West Hall, Booth 4012.

turtletbeach_logo

Turtle Beach

This headphone manufacturer has joined forces with Lucasfilm to create a line of headphones decorated with characters and artwork from the Star Wars sci-fi series.  The designs will be unveiled at E3, allowing booth visitors to get a look at the assortment of swappable speaker plates that will allow the owner to customize the look of these headsets.  E3 Booth Location: South Hall, Booth 1447.