Aleatoricism for the Game Music Composer

dices-160005_1280

I recently read a research paper about game music that I came across via Google Scholar (always an excellent resource if you’re looking for food for thought).  The paper was entitled “Game Scoring: Towards a Broader Theory,” written by Mack Enns of the University of Western Ontario towards his Master of Arts degree.  While the paper’s intention is to show how composing music for gaming is “distinct from other types of scoring,” the author repeatedly asserts that game music “remains always inherently ‘aleatoric.'”

This was the first I’d heard anyone put forward a theory such as this, and it’s definitely a bold statement.  Is all game music intrinsically aleatoric?  In my book, A Composer’s Guide to Game Music, I discuss aleatoricism as “music in which some elements are left to chance.” (Chapter 2, pg. 30).  This means that some components of the music are left to the metaphorical roll of the dice.  Mack Enns further applies this concept to games by asserting that all game music “involves both chance operations and a degree of improvisation… the “performer” of a game score is not a musical performer but a “ludal” one, that is, a “gamer”…”

So, according to this, the gamer is essentially performing the music.  Well, let’s think about that a moment.  The gamer does perform actions which trigger musical reactions, so one might imagine the gamer operating as a sort of “conductor” for the musical score, indirectly cueing the music to begin, or determining which musical segments will be triggered at any given time by virtue of gameplay choices. But a performer and a conductor aren’t really the same thing… so the idea of the improvising “gamer” as the chance element in a game music composition may not hold up in all games.  But does it hold up in some?

Just to take a look at what the improvising performer brings to aleatoric music, I’ll first give you an example from one of the more famous aleatoric compositions by John Cage, after which we’ll take a look at some top video game examples.  In John Cage’s “Living Room Music,” a group of performers are instructed to use household objects as rhythmic instruments, but no instruction is given regarding the best objects to choose, or the desired tonal quality to be produced by the objects.  That’s left entirely to chance.  Here are two performances of “Living Room Music.”  The first, performed by the MET Orchestra Percussionists, uses a combination of fingertips and glasses.

As a contrast, the second version, performed by Square Peg Round Hole, uses a collection of cardboard boxes (music begins at the 1 minute mark).

You can see that the choices of the performers have a dramatic impact on the sound of the resulting composition.  Does a gamer have a similar awesome impact on the musical content of a game by virtue of the choices made during gameplay?  That’s a harder idea to mentally digest, but Mack Enns proposes that if we also include the game’s sound effects as a part of the musical score, then we can more easily perceive the gamer as an aleatoric musical performer.  “Game scoring,” Enns writes, “includes a recognition, accommodation and, even, an orchestration of sound effects which ordinarily
might also be considered “extramusical” to the untrained ear.”

So, if we’re willing to accept the game’s sound effects as a part of its musical score, then the entire experience becomes an aleatoric performance… but that presupposes the idea that sound effects should be accepted as musical events.  If we don’t perceive the sound effects as musical, then are there still circumstances in which the gamer can be considered an aleatoric musical performer?  It seems to me that the role of the gamer as a musician would depend on the nature of the game being played… and there are certainly games in which the player fills the role of the performer in a composition governed by chance.  Let’s take a look at a few popular video game examples.

border-159926_640_white

Flow

In the game Flow, players become a small organism glowing in a watery universe.  In this game, the sound effects are musical.  When your organism eats a glowing mote in the water, a tone sounds. Progressing through the game, the tones create a musical latticework according to the player’s choices, essentially weaving the fabric of the game’s musical score as the gamer plays.  This seems like a good example of an aleatoric score in which the sound effects could rightly be considered musical, and the player can be identified as an aleatoric performer.

border-159926_640_white

Electroplankton

Electroplankton is the most overt example of a game that casts the player in the role of a musician.  The game consists of an assortment of minigames that allow the player to participate in music creation through many clever graphical interfaces and procedures.  For instance, in the “Beatnes” game, the player is introduced to the art of “Live Looping” by rhythmically tapping the screen to trigger sounds that thereafter loop repeatedly with the beat, allowing the player to create more complex repeating layers of sounds as the music progresses.

border-159926_640_white

Rez

Rez is a rail shooter in which the player navigates a virtual environment within a computer network, destroying viruses and firewalls in order to rescue an artificial intelligence named Eden.  There are no sound effects in Rez – every action of the player is punctuated with a musical tone or rhythmic effect that melds with the electronic soundtrack to create the overall musical score.  When players “shoot,” they’re essentially playing the music.

border-159926_640_white

Lumines

The gameplay of this popular puzzle game is less directly musical, but still has a very strong effect on the progress of the musical score.  Players of Lumines manipulate falling blocks (in a manner vaguely reminiscent of Tetris).  The music consists of looping segments, during which the overall composition seems to halt any forward progress and dwell on a short internal segment repeatedly until the player racks up enough points to trigger the music to move ahead.  Because the player’s actions influence the progress of the music rather than its content, it seems that this game casts the player in a role more similar to a conductor than a musician.

border-159926_640_white

Flower

In the Flower video game, the player collects flower petals, which emit bell-like tones as they fly up into the air.  These tones join with the game’s musical score with a delicate, “wind-chime” effect.  The player is able to sound these bells each time a new flower petal is collected, and these sounds serve to provide foreground interest for the music and a sense of shape and forward movement to the composition.

border-159926_640_white

So, after reading Mack Enn’s paper and considering his thesis, I think it’s certainly reasonable to identify certain games as having soundtracks structured in such a way as to allow the player to become an aleatoric musical performer.  I’m not sure I’d agree that all games are structured this way, and the idea of considering all sound effects as musical is a highly experimental concept.  Nevertheless, it’s interesting to consider!  Let me know what you think in the comments, and feel free to post any other “aleatoric” game scores you’d like to share!

border-159926_640_white

Studio1_GreenWinifred Phillips is an award-winning game music composer.  Her most well-known projects include such famous and popular games as Assassin’s Creed Liberation, God of War, the LittleBigPlanet franchise, and many others.  She is the author of the award-winning bestseller A COMPOSER’S GUIDE TO GAME MUSIC, published by the Massachusetts Institute of Technology Press.  Follow her on Twitter @winphillips.

Game Music Middleware, Part 5: psai

Middleware-Blackboard

 

This is a continuation of my blog series on the top audio middleware options for game music composers, this time focusing on the psai Interactive Music Engine for games, developed by Periscope Studio, an audio/music production house. Initially developed as a proprietary middleware solution for use by Periscope’s in-house musicians, the software is now being made available commercially for use by game composers.  In this blog I’ll take a quick look at psai and provide some tutorial resources that will further explore the utility of this audio middleware.  If you’d like to read the first four blog entries in this series on middleware for the game composer, you can find them here:

Game Music Middleware, Part 1: Wwise

Game Music Middleware, Part 2: FMOD

Game Music Middleware, Part 3: Fabric

Game Music Middleware, Part 4: Elias

What is psai?

The name “psai” is an acronym for “Periscope Studio Audio Intelligence,” and its lowercase appearance is intentional.  Like the Elias middleware (explored in a previous installment of this blog series), the psai application attempts to provide a specialized environment specifically tailored to best suit the needs of game composers.  The developers at Periscope Studio claim that psai’s “ease of use is unrivaled,” primarily because the middleware was “designed by videogame composers, who found that the approaches of conventional game audio middleware to interactive music were too complicated and not flexible enough.”  The psai music engine was originally released for PC games, with a version of the software for the popular Unity engine released in January 2015.

psai graphical user interface

psai graphical user interface

Both Elias and psai offer intuitive graphical user interfaces designed to ease the workflow of a game composer. However, unlike Elias, which focused exclusively on a vertical layering approach to musical interactivity, the psai middleware is structured entirely around horizontal re-sequencing, with no support for vertical layering.  As I described in my book, A Composer’s Guide to Game Music, “the fundamental idea behind horizontal re-sequencing is that when composed carefully and according to certain rules, the sequence of a musical composition can be rearranged.” (Chapter 11, page 188).

Music for the psai middleware is composed in what Periscope describes as a “snippets” format, in which short chunks of music are arranged into groups that can then be triggered semi-randomly by the middleware.  The overall musical composition is called a “theme,” and the snippets represent short sections of that theme.  The snippets are assigned numbers that best represent degrees of emotional intensity (from most intense to most relaxed), and these intensity numbers help determine which of the snippets will be triggered at any given time.  Other property assignments include whether a snippet is designated as an introductory or ending segment, or whether the snippet is bundled into a “middle” group with a particular intensity designation.  Periscope cautions, “The more Middle Segments you provide, the more diversified your Theme will be. The more Middle Segments you provide for a Theme, the less repetition will occur. For a highly dynamic soundtrack make sure to provide a proper number of Segments across different levels of intensity.”

Here’s an introductory tutorial video produced by Periscope for the psai Interactive Music Engine for videogames:

Because psai only supports horizontal re-sequencing, it’s not as flexible as the more famous tools such as Wwise or FMOD, which can support projects that alternate between horizontal and vertical interactivity models.  However, psai’s ease of use may prove alluring for composers who had already planned to implement a horizontal re-sequencing structure for musical interactivity.  The utility of the psai middleware also seems to depend on snippets that are quite short, as is demonstrated by the above tutorial video produced by Periscope Studio.  There could be some negative effects of this structure on a composer’s ability to develop melodic content (as is sometimes the case in a horizontal re-sequencing model).  It would be helpful if Periscope could demonstrate psai using longer snippets that might give us a better sense of how musical ideas might be developed within the confines of their dynamic music system.  One can imagine an awesome potential for creativity with this system, if the structure can be adapted to allow for more development of musical ideas over time.

The psai middleware has been used successfully in a handful of game projects, including Black Mirror III, Lost Chronicles of Zerzura, Legends of Pegasus, Mount & Blade II – Bannerlord, and The Devil’s Men.  Here’s some gameplay video that demonstrates the music system of Legends of Pegasus:

And here is some gameplay video that demonstrates the music system of Mount & Blade II – Bannerlord:

border-159926_640_white

Studio1_GreenWinifred Phillips is an award-winning game music composer.  Her most well-known projects include such famous and popular games as Assassin’s Creed Liberation, God of War, the LittleBigPlanet franchise, and many others.  She is the author of the award-winning bestseller A COMPOSER’S GUIDE TO GAME MUSIC, published by the Massachusetts Institute of Technology Press.  Follow her on Twitter @winphillips.

A Composer’s Guide to Game Music wins the Nonfiction Book Award!

NonFictionBookAwards-Photo

I’m excited to share some awesome news!  My book, A Composer’s Guide to Game Music, has been selected as a Gold winner of the Nonfiction Book Awards!

The Nonfiction Authors Association presents Bronze, Silver and Gold Awards in its Nonfiction Book Awards competition to honor the best book-length publications in an array of nonfiction genres.  A Composer’s Guide to Game Music was recognized with a Gold award (the top honor presented by the awards competition) in the “Arts, Music, and Photography” category.

Here’s how A Composer’s Guide to Game Music was described by Stephane Chandler, the founder of the Nonfiction Authors Association:

Winifred Phillips presents music composition for a specific genre and audience in an easy-to-understand way, whether for seasoned composers or self-taught music enthusiasts looking to create a beautifully composed work for the video game market. Phillips goes above and beyond, guiding her reader through not only the composition process, but everything else tied to producing music for video games, including but not limited to working with teams and how to understand key audience demographics.

My most sincere appreciation goes out to the judging panel of the Nonfiction Book Awards for this honor!

This is the fourth award presented to A Composer’s Guide to Game Music (The MIT Press).  To date, the book has also won a National Indie Excellence Book Award, a Global Music Award for an exceptional book in the field of music, and an Annual Game Music Award from the popular site Game Music Online in the category of “Best Publication.”

A Composer's Guide to Game Music won a National Indie Excellence Book Award in the genre of Performing Arts (Film, Theater, Dance & Music).

A Composer’s Guide to Game Music won a National Indie Excellence Book Award in the genre of Performing Arts (Film, Theater, Dance & Music).

Global-Music-Award_SM
The Global Music Awards presented a Gold Medal Award of Excellence as a GMA Book Award to A Composer’s Guide to Game Music, which was judged as exceptional in the field of music.

 

The staff of accomplished music journalists of Game Music Online has presented awards in many categories that acknowledge the diversity and range of the video game music genre.

The staff of accomplished music journalists of Game Music Online presented a “Best Publication” award to A Composer’s Guide to Game Music, acknowledging its “accessible yet deep insight into the process of making game music.”

border-159926_640_white

Studio1_GreenWinifred Phillips is an award-winning game music composer with more than 11 years of experience in the video game industry.  Her projects include such famous games as Assassin’s Creed Liberation, God of War, the LittleBigPlanet franchise, and many others.  She is the author of the award-winning bestseller A COMPOSER’S GUIDE TO GAME MUSIC, published by the Massachusetts Institute of Technology Press.  Follow her on Twitter @winphillips.

Game Music for the 4th of July

Happy Independence Day to all my fellow Americans!  This is a day to celebrate all the best, most awesome things we enjoy about being Americans – and that includes our love of video games!  So to celebrate, I’ve gathered together some of the top patriotic songs of the USA as they appeared in popular game soundtracks.  Enjoy!

Guitar Hero 5 –  My Country, Tis of Thee

border-159926_640_white

Civilization V – America the Beautiful

border-159926_640_white

Fallout 3 – Yankee Doodle

border-159926_640_white

Civil War 2: Generals – When Johnny Comes Marching Home

border-159926_640_white

BioShock Infinite – You’re A Grand Old Flag

border-159926_640_white

Fallout 3 – Hail Columbia

border-159926_640_white

Civil War 2: Generals – Battle Hymn of the Republic

border-159926_640_white

Civilization IV – Marines’ Hymn (The United States Marine Corps)

border-159926_640_white

Studio1_GreenWinifred Phillips is an award-winning game music composer with more than 11 years of experience in the video game industry.  Her projects include Assassin’s Creed Liberation, God of War, the LittleBigPlanet franchise, and many others.  She is the author of the award-winning bestseller A COMPOSER’S GUIDE TO GAME MUSIC, published by the Massachusetts Institute of Technology Press.  Follow her on Twitter @winphillips.

Music Composers and Sound Designers in VR: The Headphones Problem

headphones-690685_1280

Over the past few months I’ve taken several opportunities to blog about the role that music and sound may play in the virtual reality systems and games that have become famous in the media of late, and which will begin to hit retail during the holidays this year.  Today I encountered a very interesting research paper that warns of a possible problem that may face game developers as they attempt to deliver three-dimensional audio for virtual reality experiences.  I explored some issues regarding three-dimensional game audio in my book, A Composer’s Guide to Game Music, and as game composers, I thought that audio for VR might be of particular interest to us.  So, I’m going to share what I learned about the issue from this research paper, including a conclusion which may indicate an imminent problem for some VR gamers.

All of the popular VR systems rely on headphones for audio delivery, but only one (the Oculus Rift) will include built-in headphones as a part of the system.  The rest will allow the consumer to use their own headphones, and even the Oculus Rift allows for its attached headphones to be removed so that the user can replace them with their own “high quality” headphones.

The Oculus Rift, shipping with detachable headphones.

The Oculus Rift, shipping with detachable headphones.

So, here’s where things start to get tricky.

What do the words “high quality headphones” mean to the modern gamer?  Well, the gaming website Kotaku held a survey last year so that its hardcore gaming community could vote to determine the very best gaming headphones.  The winner (by a wide margin) was the Astro A50 7.1 Wireless Surround Sound headset, followed by the Logitech G930 Wireless 7.1-Channel Surround Sound headset.  Two surround-sound models had come out on top.  Of the other headsets in the survey, most were stereo rather than surround, and the only other surround-sound headset in the survey was wired rather than wireless.  Clearly, the community had told us what “high quality” meant – and that was a surround sound experience.

The Astro A50 Wireless 7.1 Surround Sound Headset.

The Astro A50 Wireless 7.1 Surround Sound Headset.

Now, here’s where we hit upon the problem, and it’s explored in the paper “Challenges of the Headphone Mix in Games,” written by Aristotel Digenis (lead audio programmer with FreeStyleGames), who presented his paper in February 2015 at the Audio Engineering Society’s International Conference on Audio for Games in London.  Virtual reality games will be offering binaural audio to simulate a fully three-dimensional listening environment.  While binaural audio can present an awesome level of immersion and realism, the technology of binaural sound isn’t the same as that of surround sound.  In fact, they’re fundamentally different.  If gamers have opted to use their own “high quality” surround sound headsets, then they may be experiencing a lower-quality sound environment than the game developers intended.

Many of the highest quality surround sound headphones include the ability to process an incoming non-surround audio signal into a compatible surround-sound mix (essentially imitating surround sound by virtue of some built-in digital signal processing).  If this processing were applied to the binaural soundscape of a virtual reality game, the effect would cause the immersive quality of the audio to deteriorate rather than improve.  Gamers would be left wondering why their stellar top-of-the-line headphones are making their VR game sound lousy.

So far I haven’t heard any reps from the three VR system manufacturers address this issue, and gamers should definitely be warned that “high quality” headphones for VR will need to be stereo, rather than surround.  VR enthusiasts who are hoping for the ultimate virtual reality experience may need to purchase some excellent stereo headphones, if they don’t already own them.  Without a warning about this issue, some VR gamers may be set up for a nasty sonic surprise.

border-159926_640_white

Studio1_GreenWinifred Phillips is an award-winning game music composer with more than 11 years of experience in the video game industry.  Her projects include Assassin’s Creed Liberation, God of War, the LittleBigPlanet franchise, and many others.  She is the author of the award-winning bestseller A COMPOSER’S GUIDE TO GAME MUSIC, published by the Massachusetts Institute of Technology Press.  Follow her on Twitter @winphillips.

Game Music Middleware, Part 4: Elias

Middleware-Blackboard

Welcome back to my blog series that offers tutorial resources exploring game music middleware for the game music composer. I initially planned to write two blog entries on the most popular audio middleware solutions (Wwise and FMOD), but since I started this blog series, I’ve been hearing buzz about other middleware solution and so I thought it best to expand the series to incorporate other interesting solutions to music implementation in games.  This blog will focus on a brand new middleware application called Elias, developed by Elias Software.  While not as famous as Wwise or FMOD, this new application offers some intriguing new possibilities for the creation of interactive music in games.

If you’d like to read the first three blog entries in this series, you can find them here:

Game Music Middleware, Part 1: Wwise

Game Music Middleware, Part 2: FMOD

Game Music Middleware, Part 3: Fabric

Elias-Logo

Elias stands for Elastic Lightweight Integrated Audio System.  It is developed by Kristofer Eng and Philip Bennefall for Microsoft Windows, with a Unity plugin for consoles, mobile devices and browser-based games.  What makes Elias interesting is the philosophy of its design.  Instead of designing a general audio middleware tool with some music capabilities, Eng and Bennefall decided to bypass the sound design arena completely and create a middleware tool specifically outfitted for the game music composer. The middleware comes with an authoring tool called Elias Composer’s Studio that “helps the composer to structure and manage the various themes in the game and bridges the gap between the composer and level designer to ease the music integration process.”

Here’s the introductory video for Elias, produced by Elias Software:

The interactive music system of the Elias middleware application seems to favor a Vertical Layering (or vertical re-orchestration) approach with a potentially huge number of music layers able to play in lots of combinations.  The system includes flexible options for layer triggering, including the ability to randomize the activation of the layers to keep the listening experience unpredictable during gameplay.

Elias has produced a series of four tutorial videos for the Composer’s Studio authoring tool.  Here’s the first of the four tutorials:

There’s also a two-part series of tutorials about Elias produced by Dale Crowley, the founder of the game audio services company Gryphondale Studios.  Here’s the first of the two videos:

As a middleware application designed specifically to address the top needs of game music composers, Elias is certainly intriguing!  The software has so far been used in only one published game – Gauntlet, which is the latest entry in the awesome video game franchise first developed by Atari Games for arcade cabinets in 1985.  This newest entry in the franchise was developed by Arrowhead Game Studios for Windows PCs.  We can hear the Elias middleware solution in action in this gameplay video from Gauntlet:

The music of Gauntlet was composed by Erasmus Talbot.  More of his music from Gauntlet is available on his SoundCloud page.

Elias Software recently demonstrated its Elias middleware application on the expo floor of the Nordic Game 2015 conference in Malmö, Sweden (May 20-22, 2015).  Here’s a look at Elias’ booth from the expo:

Elias-NordicGame2015

Since Elias is a brand new application, I’ll be curious to see how widely it is accepted by the game audio community.  A middleware solution that focuses solely on music is definitely a unique approach!  If audio directors and audio programmers embrace Elias, then it may have the potential to give composers better tools and an easier workflow in the creation of interactive music for games.

E3 2015 for the Game Music Composer

E32015

The Electronic Entertainment Expo is upon us once again, so I’ll be spending this blog exploring what we can expect to see and learn that’s most relevant to the field of game audio from this year’s big convention.

Virtual Reality

The impending releases of three virtual reality systems should make things especially interesting on the E3 show floor, and it will be awesome to see and hear what these systems have to offer.  Let’s take a look at what we might expect from the three top VR systems, as well as a possible surprise VR reveal that might happen next week.

Project Morpheus

PlayStation president of worldwide studios Shuhei Yoshida has already announced that several internally developed VR games for the Morpheus headset will be unveiled during E3, and we also learned during the 3D Audio in VR talk at the Shayla Games VR Jam in Denmark that the Morpheus will now incorporate an audio system involving Head-Related Transfer Function (HRTF was discussed in this blog during a previous post about audio in VR).  It should be interesting to see if this HRTF system is implemented into the hardware that will be demonstrating on the E3 show floor.  The latest model of the Morpheus doesn’t include built-in headphones, as you’ll see in this video demo that TechCrunch released last month.  The demo discusses the capabilities of the hardware, including its audio functionality:

Oculus Rift

The newest model of the Oculus Rift, the famous Crescent Bay, offers 3D audio through a set of built-in headphones.  Here’s an interview that Oculus CEO Brendan Iribe gave to Gamecrate during the ever-popular Consumer Electronics Show 2015 about the new VR audio features of the Oculus Rift.

HTC Vive

The HTC Vive doesn’t currently offer built-in headphones, but the developer assures us that the final consumer version will offer integrated 3D audio.  The current model offers the user the option to connect their own high-end headphones to the Vive.  E3 attendees may get to see how aurally immersive that can be by playing Arizona Sunshine, a game designed for the Vive and set in a genre so famous and pervasive that its appearance in the VR world was inevitable: the apocalyptic zombie shooter.  The game was announced on May 21st by its developer, Vertigo Games, and it’s a good bet that the game could be showing on the E3 exhibit floor.  Here’s a look at a trailer for Arizona Sunshine:

Microsoft VR System

Finally, the rumor mill is swirling around speculation that Microsoft may officially reveal its own virtual reality headset system during this year’s E3.

border-159926_640_white

Audio exhibitors at E3

The Electronic Entertainment Expo is an opportunity for consumer audio hardware manufacturers to show off most of their top products, so let’s take a look at what this year’s exhibitors are offering.

Astro Gaming

Game Audio Products:

  • A50 XBox One Edition Headset, 2nd Generation
  • A50 Astro Edition Headset & TX, 2nd Generation
  • A40 Xbox One Edition Headset + MixAmp M80
  • A40 Astro Edition Headset + Mixamp, 2nd Generation
  • A40 Astro Edition Headset + Mixamp, 2nd Generation
  • A40 PC Edition Headset, 2nd Generation
  • A38 Astro Bluetooth Wireless Headset
  • MixAmp Pro, 2nd Generation

border-159926_640_white

dreamGEAR

Game Audio Products:

  • Prime Wired Headset for PS4
  • Universal Elite Wired Headset

border-159926_640_white

Performance Designed Products

Game Audio Products:

  • Afterglow Karga Headset for Xbox One
  • Afterglow Fener Premium Wireless Headset for PS4
  • Afterglow Kral Wireless Headset for PS4
  • Afterglow Nur Headset
  • Afterglow PS4 Bluetooth Communicator

border-159926_640_white

Plantronics

Game Audio Products:

  • RIG Gaming Audio System
  • RIG Surround Sound Gaming System
  • RIG Flex
  • GameCom 788 Gaming Headset
  • GameCom 388 Gaming Headset

border-159926_640_white

Polk Audio

Game Audio Products:

  • Striker Zx Xbox One Gaming Headset
  • Striker P1 Multiplatform Gaming Headset
  • N1 Gaming Sound Bar
  • 4 Shot Xbox One Gaming Headset

border-159926_640_white

Turtle Beach

Game Audio Products:

  • Turtle Beach Elite 800X Gaming Headset for Xbox One
  • Turtle Beach Stealth 500X headset for Xbox One
  • Turtle Beach XO Seven Pro
  • Turtle Beach XO Four Stealth
  • Turtle Beach Call of Duty Online PC Gaming Headset