Media Synth Video Games
Content
Analyzing Modular Smoothness in Video Game Music
Elizabeth Medina-Gray (Oberlin College)
Society for Music Theory, Annual Meeting, November 2014
Figure 1 The modular process
Modules +
Rules
Assembly
Sounding
Music
Figure 2 Definitions
Sequential seam: One module sounds after another module with no overlap.
Simultaneous seam: Two modules sound at the same time (includes crossfades).
Smoothness: Two modules fit well together; agreement; merging/continuity.
Disjunction: Two modules do not fit well together; disagreement; separation.
Figure 3 Smoothness scale for modular seams
Entirely
Smooth
Primarily
Smooth
Half
Smooth
Partially
Smooth
Partially
Disjunct
Half
Disjunct
Primarily
Disjunct
Entirely
Disjunct
A single seam can be smooth in some ways and disjunct in others, because multiple aspects
of the music involved in the seam—including meter, timbre, pitch, volume, and
abruptness—can be either smooth or disjunct.
Figure 4 Analytical rubric for sequential seams
Smooth
Meter:
Same
(agrees)
(continuous)
Timbre:
Pitch:
Disjunct
Grouping or
displacement
dissonance
All instruments
same
Some instruments
same
All PCs present
in macroharmony
Most PCs present
in macroharmony
Volume:
Abruptness:
Grouping and
displacement
dissonance
Completely
discontinuous
Different
Very
different
Few PCs present
No PCs present
in macroharmony in macroharmony
Same or similar
throughout seam
Changes significantly during
seam (including to silence)
Gradual fade-in and/or
fade-out, or pitches seem to
enter and decay naturally
Abrupt entrance or
exit mid-sound;
cut off
Sample additional contributions:
Gradual change across seam
in meter, timbre, or pitch
Continuation of rhythmic patterns
Buffer during seam (silence > 250 ms,
or a masking sound effect)
Significant difference in trends
of consonance or dissonance
Significant difference in
textural thickness
Difference in sound location
Notes on using this rubric:
For each aspect (row) in each possible seam, compare the music in the 1 second after the
seam (i.e., at the beginning of the second module) with the music in the 5 seconds before the
seam (i.e., at the end of the first module). Show the results as probabilities in a grid.
Meter: Look for layers of pulse streams continuing across the seam. Grouping and
displacement dissonance (after Krebs 1999) involve continuity in at least one pulse stream
and conflict/discontinuity in others.
Timbre: Comparisons in this aspect can involve familiar instruments and/or synthesized
timbres. “Very different” = different attack and quality of sound; e.g., guitar, bowed strings,
and oboe are all “very different,” whereas guitar and pizz. strings are merely “different”.
Piano and flute after just piano (or vice versa) is mildly smooth (“some instruments same”).
Pitch: Look to see how many of the pitch classes after the seam are also present in the
macroharmony before the seam. (Macroharmony = all pitch classes in a given span of music,
after Tymoczko 2011.)
Medina-Gray, 2
Figure 5 Analysis for a single OceanOutset Island sequential seam in Wind Waker (Nintendo 2003)
5 seconds
1 second
(pizz.)
Figure 6 Analysis for all possible OceanOutset Island sequential seams in Wind Waker
Medina-Gray, 3
Figure 7 Analytical rubric for simultaneous seams
Smooth
Meter:
Same
(agrees)
Timbre:
Pitch:
No instruments
different
No new
intervals
(octaves only)
Volume:
Disjunct
Grouping or
displacement
dissonance
Some instruments
different
Consonance
only
Same or similar
(not counting rests or decay)
(Only w/
continuity)
Abruptness:
Gradual fade-in and/or
fade-out, or pitches seem to
enter and decay naturally
Grouping and
displacement
dissonance
Completely
disagrees
All instruments
different
Very
different
Soft
diss.
only
Three >Three
hard
hard
diss.
diss.
One Two
hard hard
diss. diss.
Significantly different
Abrupt entrance or
exit mid-sound;
cut off
Sample additional contributions:
Same rhythm/onset pattern
Additional (dynamic, ongoing)
process affects both modules equally
Additional (dynamic, ongoing)
process affects one module
but not the other
Difference in sound location
Notes on using this rubric:
For each aspect (row) in each possible simultaneous combination, examine the two
modules on a moment-by-moment basis. Show the results as probabilities in a grid.
Meter: Grouping and displacement dissonance (after Krebs 1999) involve
alignment/agreement in at least one pulse stream and conflict in others.
Timbre: Piano on top of piano and percussion is very smooth (“no instruments different”).
Piano and percussion on top of piano and flute is mildly smooth (“some instruments
different”).
Pitch: Examine the intervals created by the two modules’ pitch classes in combination (i.e.,
the new intervals that are not already present in either module by itself). M2 and m7 are
soft dissonances; m2, M7 and tritones are hard dissonances. E.g., {C,F#} + {C,G} = one hard
dissonance (between F# and G).
Medina-Gray, 4
Select Bibliography
Bregman, Albert S. 1990. Auditory Scene Analysis: The Perceptual Organization of Sound. Cambridge,
MA: The MIT Press.
Childs, IV, G. W. 2007. Creating Music and Sound for Games. Boston: Thomson Course Technology
PTR.
Collins, Karen. 2008. Game Sound: An Introduction to the History, Theory, and Practice of Video Game
Music and Sound Design. Cambridge, MA: The MIT Press.
Hasty, Christopher. 1981. “Segmentation and Process in Post-Tonal Music.” Music Theory Spectrum
3: 54–73.
Hoffert, Paul. 2007. Music for New Media: Composing for Videogames, Web Sites, Presentations, and
Other Interactive Media. Edited by Jonathan Feist. Boston: Berklee Press.
Huron, David. 2006. Sweet Anticipation: Music and the Psychology of Expectation. Cambridge, MA:
The MIT Press.
Jørgensen, Kristine. 2009. A Comprehensive Study of Sound in Computer Games: How Audio Affects
Player Action. Lewiston, NY: Edwin Mellen Press.
Krebs, Harald. 1999. Fantasy Pieces: Metrical Dissonance in the Music of Robert Schumann. New
York: Oxford University Press.
Marks, Aaron. 2009. The Complete Guide to Game Audio: For Composers, Musicians, Sound Designers,
Game Developers. 2nd ed. New York: Focal Press.
Paul, Leonard J. 2013. “Droppin’ Science: Video Game Audio Breakdown.” In Music and Game:
Perspectives on a Popular Alliance, edited by Peter Moormann, 63–80. Wiesbaden: Springer
VS.
Sheffield, Brandon. 2008. “Staying In Tune: Richard Jacques On Game Music’s Past, Present, And
Future.” Gamasutra.
http://www.gamasutra.com/view/feature/3695/staying_in_tune_richard_jacques_.php.
Tymoczko, Dmitri. 2011. A Geometry of Music: Harmony and Counterpoint in the Extended Common
Practice. New York: Oxford University Press.
Medina-Gray, 5
Elizabeth Medina-Gray (Oberlin College)
Society for Music Theory, Annual Meeting, November 2014
Figure 1 The modular process
Modules +
Rules
Assembly
Sounding
Music
Figure 2 Definitions
Sequential seam: One module sounds after another module with no overlap.
Simultaneous seam: Two modules sound at the same time (includes crossfades).
Smoothness: Two modules fit well together; agreement; merging/continuity.
Disjunction: Two modules do not fit well together; disagreement; separation.
Figure 3 Smoothness scale for modular seams
Entirely
Smooth
Primarily
Smooth
Half
Smooth
Partially
Smooth
Partially
Disjunct
Half
Disjunct
Primarily
Disjunct
Entirely
Disjunct
A single seam can be smooth in some ways and disjunct in others, because multiple aspects
of the music involved in the seam—including meter, timbre, pitch, volume, and
abruptness—can be either smooth or disjunct.
Figure 4 Analytical rubric for sequential seams
Smooth
Meter:
Same
(agrees)
(continuous)
Timbre:
Pitch:
Disjunct
Grouping or
displacement
dissonance
All instruments
same
Some instruments
same
All PCs present
in macroharmony
Most PCs present
in macroharmony
Volume:
Abruptness:
Grouping and
displacement
dissonance
Completely
discontinuous
Different
Very
different
Few PCs present
No PCs present
in macroharmony in macroharmony
Same or similar
throughout seam
Changes significantly during
seam (including to silence)
Gradual fade-in and/or
fade-out, or pitches seem to
enter and decay naturally
Abrupt entrance or
exit mid-sound;
cut off
Sample additional contributions:
Gradual change across seam
in meter, timbre, or pitch
Continuation of rhythmic patterns
Buffer during seam (silence > 250 ms,
or a masking sound effect)
Significant difference in trends
of consonance or dissonance
Significant difference in
textural thickness
Difference in sound location
Notes on using this rubric:
For each aspect (row) in each possible seam, compare the music in the 1 second after the
seam (i.e., at the beginning of the second module) with the music in the 5 seconds before the
seam (i.e., at the end of the first module). Show the results as probabilities in a grid.
Meter: Look for layers of pulse streams continuing across the seam. Grouping and
displacement dissonance (after Krebs 1999) involve continuity in at least one pulse stream
and conflict/discontinuity in others.
Timbre: Comparisons in this aspect can involve familiar instruments and/or synthesized
timbres. “Very different” = different attack and quality of sound; e.g., guitar, bowed strings,
and oboe are all “very different,” whereas guitar and pizz. strings are merely “different”.
Piano and flute after just piano (or vice versa) is mildly smooth (“some instruments same”).
Pitch: Look to see how many of the pitch classes after the seam are also present in the
macroharmony before the seam. (Macroharmony = all pitch classes in a given span of music,
after Tymoczko 2011.)
Medina-Gray, 2
Figure 5 Analysis for a single OceanOutset Island sequential seam in Wind Waker (Nintendo 2003)
5 seconds
1 second
(pizz.)
Figure 6 Analysis for all possible OceanOutset Island sequential seams in Wind Waker
Medina-Gray, 3
Figure 7 Analytical rubric for simultaneous seams
Smooth
Meter:
Same
(agrees)
Timbre:
Pitch:
No instruments
different
No new
intervals
(octaves only)
Volume:
Disjunct
Grouping or
displacement
dissonance
Some instruments
different
Consonance
only
Same or similar
(not counting rests or decay)
(Only w/
continuity)
Abruptness:
Gradual fade-in and/or
fade-out, or pitches seem to
enter and decay naturally
Grouping and
displacement
dissonance
Completely
disagrees
All instruments
different
Very
different
Soft
diss.
only
Three >Three
hard
hard
diss.
diss.
One Two
hard hard
diss. diss.
Significantly different
Abrupt entrance or
exit mid-sound;
cut off
Sample additional contributions:
Same rhythm/onset pattern
Additional (dynamic, ongoing)
process affects both modules equally
Additional (dynamic, ongoing)
process affects one module
but not the other
Difference in sound location
Notes on using this rubric:
For each aspect (row) in each possible simultaneous combination, examine the two
modules on a moment-by-moment basis. Show the results as probabilities in a grid.
Meter: Grouping and displacement dissonance (after Krebs 1999) involve
alignment/agreement in at least one pulse stream and conflict in others.
Timbre: Piano on top of piano and percussion is very smooth (“no instruments different”).
Piano and percussion on top of piano and flute is mildly smooth (“some instruments
different”).
Pitch: Examine the intervals created by the two modules’ pitch classes in combination (i.e.,
the new intervals that are not already present in either module by itself). M2 and m7 are
soft dissonances; m2, M7 and tritones are hard dissonances. E.g., {C,F#} + {C,G} = one hard
dissonance (between F# and G).
Medina-Gray, 4
Select Bibliography
Bregman, Albert S. 1990. Auditory Scene Analysis: The Perceptual Organization of Sound. Cambridge,
MA: The MIT Press.
Childs, IV, G. W. 2007. Creating Music and Sound for Games. Boston: Thomson Course Technology
PTR.
Collins, Karen. 2008. Game Sound: An Introduction to the History, Theory, and Practice of Video Game
Music and Sound Design. Cambridge, MA: The MIT Press.
Hasty, Christopher. 1981. “Segmentation and Process in Post-Tonal Music.” Music Theory Spectrum
3: 54–73.
Hoffert, Paul. 2007. Music for New Media: Composing for Videogames, Web Sites, Presentations, and
Other Interactive Media. Edited by Jonathan Feist. Boston: Berklee Press.
Huron, David. 2006. Sweet Anticipation: Music and the Psychology of Expectation. Cambridge, MA:
The MIT Press.
Jørgensen, Kristine. 2009. A Comprehensive Study of Sound in Computer Games: How Audio Affects
Player Action. Lewiston, NY: Edwin Mellen Press.
Krebs, Harald. 1999. Fantasy Pieces: Metrical Dissonance in the Music of Robert Schumann. New
York: Oxford University Press.
Marks, Aaron. 2009. The Complete Guide to Game Audio: For Composers, Musicians, Sound Designers,
Game Developers. 2nd ed. New York: Focal Press.
Paul, Leonard J. 2013. “Droppin’ Science: Video Game Audio Breakdown.” In Music and Game:
Perspectives on a Popular Alliance, edited by Peter Moormann, 63–80. Wiesbaden: Springer
VS.
Sheffield, Brandon. 2008. “Staying In Tune: Richard Jacques On Game Music’s Past, Present, And
Future.” Gamasutra.
http://www.gamasutra.com/view/feature/3695/staying_in_tune_richard_jacques_.php.
Tymoczko, Dmitri. 2011. A Geometry of Music: Harmony and Counterpoint in the Extended Common
Practice. New York: Oxford University Press.
Medina-Gray, 5
