作者：RetroModular

在本篇教程中我们将着眼于为游戏构造并排序动态音乐的一种技术。构造和排序是发生在运行时间中，这让游戏开发者可以修改音乐结构而对游戏世界中所发生的一切做出反应。

在分析技术细节前，你可能想要看看这一技术在运行过程中的示例。示例中的音乐是源自一些音频独立组块，即在运行时间中被排序并组合在一起去形成完整的音乐歌曲。

这一示例要求一个能够支持W3C Web Audio API和OGG音频的网页浏览器。谷歌Chrome便是是最佳选择，而Firefox Aurora也同样适用。

综述

该技术的使用方式非常直接，如果基于创造性的话它便具有足够的潜能能够添加一些优秀的动态音乐到游戏中。这也让我们能够通过较小的音频文件创造出无限长的音乐歌曲。

从本质上来看最初的音乐会被分解成一些组块，每个组块都有自己的长条，这些组块被储存在一个音频文件中。音序器会加载音频文件并提取所需要的原生音频样本去改造音乐。音乐的结构是由一些可变数组在播放音乐组块时传达音序器而诞生的。

你可以将这种技术当成是测序软件的简化版本，就像Reason，FL Studio或Dance EJay。你也可以认为这一技术是音乐版的乐高积木。

音频文件结构

就像之前所提到的，音序器要求将最初音乐分解成一些组块，并且这些组块必须储存在音频文件中。

该图像演示的便是组块如何被储存在音频文件中。

在图像中你可以看到音频文件中储存了5个独立组块，所有的这些组块都是等长。为了保持教程的简单化，我们让组块具有条块的长度。

音频文件中组块的次序很重要，因为它将影响组快被分配到哪个音序器渠道中。第一个组块（如鼓）将被分配到第一个音序器渠道，第二个组块（如打击乐）将被分配到第二个音序器渠道，如此类推。

音序器渠道

一个音序器渠道代表一排组块并包含指示被分配到渠道中的组块是否应该进行演奏的旗帜（每个音乐条块都有一面旗帜）。每面旗帜都是一个数值，有可能是0（不能演奏组块）也有可能是1（演奏组块）。

该图像演示的是组块和音序器渠道之间的关系。

上述图像最底部垂直对齐的数字代表的便是条块值。就像你所看到的，在第一个音乐条块（01）中只有吉他组块开始演奏，但是在第五个条块（05）中，鼓，打击乐，贝斯和吉他组块都开始演奏。

编程

在这一教程中，我们将不涉及完整运行的音序器代码；相反地，我们将着眼于让一个简单的音序器运行起来的核心代码。代码将作为伪代码而尽可能保持内容与语言无关。

在我们开始前，你需要记住你最终决定使用的编程语言需要一个API，即让你能够基于较低水平操纵音频。JavaScript中的Web Audio API便是一个有效的例子。

你可以下载这篇教程所附带的源文件去学习一个基本音序器的JavaScript执行（这是针对于本篇教程所创造的演示内容）。

扼要概述

我们拥有一个包含音乐组块的音频文件。每个音乐组块是基于条块的长度，音频文件中的组块顺序影响着组块被分配到怎样的音序器渠道中。

常数

在继续分析前，我们需要明确两个信息。我们需要知道音乐的拍子，每分钟几拍以及在每个条块中的拍子数。后者可以被当成音乐的拍号。该信息将作为常数值进行储存，因为在音序器运行期间它不会发生改变。

TEMPO     = 100 // beats per minute SIGNATURE = 4   // beats per bar

我们同样也需要清楚音频API所使用的抽样率。一般说来应该是44100赫兹，因为这是最适合音频的，但是有些人通过配置他们的硬件去使用一个更高的抽样率。你选择使用的音频API应该提供该信息，但是出于本篇教程的目的，我们将假设抽样率为44100赫兹。

SAMPLE_RATE = 44100 // Hertz

现在我们可以计算一个音乐条块的样本长度—-也就是在一个音乐组块中的音频样本的数量。这一数值很重要，因为它让音序器能够将音乐的独立组块以及每个组块中的音频样本放置在音频文件数据中。

BLOCK_SIZE = floor( SAMPLE_RATE * ( 60 / ( TEMPO / SIGNATURE ) ) )

音频流

你所选择的音频API将传达音频流（游戏邦注：音频样本的数组）是如何呈现在代码中。举个例子来说，Web Audio API便是使用AudioBuffer对象。

而在该教程中有2个音频流。第一个音频流是只读的，将包含所有加载自音频文件（包含音频组块）的音频样本，这便是“输入”音频流。

第二个音频流是只写，将用于把音频样本推向硬件；这是“输出”音频流。每个流都将作为一个一维数组。

input  = [ ... ]
output = [ ... ]

从文件中加载音频文件并提取音频样本的确切过程将通过你所使用的编程语言描述出来。记住这点，我们将假设输入音频流数组已经包含了来自音频文件的音频样本。

输出音频流将是基于固定的长度，大多数音频API将允许你选择频率，而音频样本便是基于该频率进行处理并发送到硬件上—-这也是援用更新函数的频率。频率经常与音频延迟直接维系在一起，较高的频率将要求更大的处理器能力，但也会因此减少延迟时间，反之亦然。

音序器数据

音序器数据是一个多维数组；每个子数组代表一个音序器渠道并包含指示被分配到渠道中的组块是否应该进行演奏的旗帜（每个音乐条块都有一面旗帜）。渠道数组的长度也影响着音乐的长度。

channels = [
[ 0,0,0,0, 0,0,0,0, 1,1,1,1, 1,1,1,1 ], // drums     [ 0,0,0,0, 1,1,1,1, 1,1,1,1, 1,1,1,1 ], // percussion     [ 0,0,0,0, 0,0,0,0, 1,1,1,1, 1,1,1,1 ], // bass     [ 1,1,1,1, 1,1,1,1, 1,1,1,1, 1,1,1,1 ], // guitar     [ 0,0,0,0, 0,0,1,1, 0,0,0,0, 0,0,1,1 ]  // strings ]

你所看到的数据代表的是一个基于16个条块长的音乐结构。它包含了5个渠道，每个渠道代表音频文件中的每个音乐组块，并且渠道与音频文件中的音乐组块一样是基于同样的顺序。渠道数组的旗帜让我们能够清楚分配到渠道中的组块是否应该演奏：0意味着数组将不会演奏，而1则意味着组块将进行演奏。

该数据结构是易变的，它可以随时改变，甚至在音序器运行时，这让你能够修改气质和音乐结构去反应游戏中所发生的事。

音频处理

大多数音频API将传播一个事件到一个事件处理程序函数中，或在需要推动更多音频样本到硬件中时直接调用一个函数。该函数就像游戏的祝更新循环，会时常被调用，但却不够有规律，所以我们应该花些时间去优化它。

该函数中发生了什么：

从输入音频流中提取多个音频样本。这些样本将被汇聚在一起形成一个音频样本。该音频样本奖被推向输出音频流中。在深入分析函数前，我们需要在代码中定义一些变量：

playing  = true // indicates whether the music (the sequencer) is playing position = 0    // the position of the sequencer playhead, in samples

这一playing布尔逻辑（游戏邦注：对象是一种数据类型，它可以使用true或false两个值中一个值）让我们知道了音乐是否在演奏。如果不在演奏，我们便需要将无声的音频样本推向输出音频流中。位置将记录播放头在音乐中的位置，所以这有点像典型的音乐或视频播放器的刷子。

现在让我们深入函数的内部：

function update() {
outputIndex = 0
outputCount = output.length
if( playing == false ) {
// silent samples need to be pushed to the output stream         while( outputIndex < outputCount ) {
output[ outputIndex++ ] = 0.0
}
// the remainder of the function should not be executed
return
}
chnCount = channels.length
// the length of the music, in samples
musicLength = BLOCK_SIZE * channels[ 0 ].length
while( outputIndex < outputCount ) {
chnIndex = 0
// the bar of music that the sequencer playhead is pointing at         barIndex = floor( position / BLOCK_SIZE )
// set the output sample value to zero (silent)
output[ outputIndex ] = 0.0
while( chnIndex < chnCount ) {
// check the channel flag to see if the block should be played             if( channels[ chnIndex ][ barIndex ] == 1 ) {
// the position of the block in the “input” stream                 inputOffset = BLOCK_SIZE * chnIndex
// index into the “input” stream
inputIndex = inputOffset + ( position % BLOCK_SIZE )                   // add the block sample to the output sample
output[ outputIndex ] += input[ inputIndex ]
}
chnIndex++
}
// advance the playhead position
position++
if( position >= musicLength ) {
// reset the playhead position to loop the music             position = 0
}
outputIndex++
}
}

就像你所看到的，需要处理音频样本的代码非常简单，但因为该代码每秒将运行多次，所以你应该寻找某种方法在函数中优化代码并尽可能地预先计算更多数值。你可以用于优化代码的方法是基于你所使用的编程语言。

如果你想要看看使用Web Audio API在JavaScript中执行基本的音序器的方法，你可以下载本篇教程所附带的源文件。

注：

你所使用的音频文件格式应该让音频能够无缝地循环。换句话说，用于生成音频文件的编码器应该不能注入任何填充内容（如无声的音频组块）到音频文件中。不幸的是基于该原因我们便不能使用MP3和MP4文件。不过我们能够使用OGG文件。如果你想要的话也可以使用WAV文件，但是考虑到规格，它们并不是网页游戏或应用的有效选择。

如果你正在编写一款游戏，如果你所使用的编程语言也支持游戏的并发性，那么你便需要考虑基于其自身的线程去运行音频处理代码。这么做将缓解游戏的任何音频处理的主更新循环开支。

受欢迎的游戏中的动态音乐

以下是一些受欢迎游戏基于某种方式所使用的动态音乐。这些游戏在执行动态音乐的方法上可能有所不同，但是最终结果却都是一样的：游戏玩家拥有更多沉浸式的游戏体验。

《Journey》：thatgamecompany.com

《Flower》：thatgamecompany.com

《小小大星球》：littlebigplanet.com

《传送门2》：thinkwithportals.com

《像素垃圾：射击》：pixeljunk.jp

《荒野大镖客：救赎》：rockstargames.com

《神秘海域》：naughtydog.com

结论

这下你便知道—-一次简单的动态序列音乐执行能够强化游戏的情感属性。不管如何使用该技术或音序器变得多复杂都是完全取决于你。这一简单的执行还可以基于许多其它的方法，我们也将在今后的教程中谈到这些方法。

（本文为游戏邦/gamerboom.com编译，拒绝任何不保留版权的转载，如需转载请联系：游戏邦）

Dynamic, Sequential Soundtracks for Games

By RetroModular

In this tutorial we will be taking a look at one technique for constructing and sequencing dynamic music for games. The construction and sequencing happens at runtime, allowing game developers to modify the structure of the music to reflect what is happening in the game world.

Before we jump into the technical details, you may want to take a look at a working demonstration of this technique in action. The music in the demonstration is constructed from a collection of individual blocks of audio which are sequenced and mixed together at runtime to form the full music track.

This demonstration requires a web browser that supports the W3C Web Audio API and OGG audio. Google Chrome is the best browser to use to view this demonstration with, but Firefox Aurora can also be used.

If you can’t view the above demo in your browser, you can watch this YouTube video instead:

Overview

The way this technique works is fairly straightforward, but it has the potential to add some really nice dynamic music to games if it is used creatively. It also allows infinitely long music tracks to be created from a relatively small audio file.

The original music is essentially deconstructed into a collection of blocks, each of which is one bar in length, and those blocks are stored in a single audio file. The music sequencer loads the audio file and extracts the raw audio samples it needs to reconstruct the music. The structure of the music is dictated by a collection of mutable arrays that tell the sequencer when to play the blocks of music.

You can think of this technique as a simplified version of sequencing software such as Reason, FL Studio, or Dance EJay. You can also think of this technique as the musical equivalent of Lego bricks.

Audio File Structure

As mentioned previously, the music sequencer requires the original music to be deconstructed into a collection of blocks, and those blocks need to be stored in an audio file.

This image demonstrates how the blocks might be stored in an audio file.

In that image you can see there are five individual blocks stored in the audio file, and all of the blocks are of equal length. To keep things simple for this tutorial the blocks are all one bar long.

The order of the blocks in the audio file is important because it dictates which sequencer channels the blocks are assigned to. The first block (e.g. drums) will be assigned to the first sequencer channel, the second block (e.g. percussion) will be assigned to the second sequencer channel, and so on.

Sequencer Channels

A sequencer channel represents a row of blocks and contains flags (one for each bar of music) that indicate whether the block assigned to the channel should be played. Each flag is a numerical value and is either zero (do not play the block) or one (play the block).

This image demonstrates the relationship between the blocks and the sequencer channels.

The numbers aligned horizontally along the bottom of the above image represent bar numbers. As you can see, in the first bar of music (01) only the Guitar block will be played, but in the fifth bar (05) the Drums, Percussion, Bass and Guitar blocks will be played.

Programming

In this tutorial we will not step through the code of a full working music sequencer; instead, we will look at the core code required to get a simple music sequencer running. The code will be presented as pseudo-code to keep things as language-agnostic as possible.

Before we begin, you need to bear in mind the programming language that you ultimately decide to use will require an API that allows you to manipulate audio at a low level. A good example of this is the Web Audio API available in JavaScript.

You can also download the source files attached to this tutorial to study a JavaScript implementation of a basic music sequencer that was created as a demonstration for this tutorial.

Quick Recap

We have a single audio file that contains blocks of music. Each block of music is one bar in length, and the order of the blocks in the audio file dictates the sequencer channel the blocks are assigned to.

Constants

There are two pieces of information that we will need before we can proceed. We need to know the tempo of the music, in beats per minute, and the the number of beats in each bar. The latter can be thought of as the time signature of the music. This information should be stored as constant values because it does not change while the music sequencer is running.

TEMPO     = 100 // beats per minute SIGNATURE = 4   // beats per bar

We also need to know the sample rate that the audio API is using. Typically this will be 44100 Hz, because it is perfectly fine for audio, but some people have their hardware configured to use a higher sample rate. The audio API you choose to use should provide this information, but for the purpose of this tutorial we will assume the sample rate is 44100 Hz.

SAMPLE_RATE = 44100 // Hertz

We can now calculate the sample length of one bar of music – that is, the number of audio samples in one block of music. This value is important because it allows the music sequencer to locate the individual blocks of music, and the audio samples within each block, in the audio file data.

BLOCK_SIZE = floor( SAMPLE_RATE * ( 60 / ( TEMPO / SIGNATURE ) ) )

Audio Streams

The audio API you choose to use will dictate how audio streams (arrays of audio samples) are represented in your code. For example, the Web Audio API uses AudioBuffer objects.

For this tutorial there will be two audio streams. The first audio stream will be read-only and will contain all of the audio samples loaded from the audio file containing the music blocks, this is the “input” audio stream.

The second audio stream will be write-only and will be used to push audio samples to the hardware; this is the “output” audio stream. Each of these streams will be represented as a one-dimensional array.

input  = [ ... ]
output = [ ... ]

The exact process required to load the audio file and extract the audio samples from the file will be dictated by the programming language that you use. With that in mind, we will assume the input audio stream array already contains the audio samples extracted from the audio file.

The output audio stream will usually be a fixed length because most audio APIs will allow you to choose the frequency at which the audio samples need to be processed and sent to the hardware – that is, how often an update function is invoked. The frequency is normally tied directly to the latency of the audio, high frequencies will require more processor power but they result in lower latencies, and vice-versa.

Sequencer Data

The sequencer data is a multi-dimensional array; each sub-array represents a sequencer channel and contains flags (one for each bar of music) that indicate whether the music block assigned to the channel should be played or not. The length of the channel arrays also dictates the length of the music.

channels = [
[ 0,0,0,0, 0,0,0,0, 1,1,1,1, 1,1,1,1 ], // drums     [ 0,0,0,0, 1,1,1,1, 1,1,1,1, 1,1,1,1 ], // percussion     [ 0,0,0,0, 0,0,0,0, 1,1,1,1, 1,1,1,1 ], // bass     [ 1,1,1,1, 1,1,1,1, 1,1,1,1, 1,1,1,1 ], // guitar     [ 0,0,0,0, 0,0,1,1, 0,0,0,0, 0,0,1,1 ]  // strings ]

The data you see there represents a music structure that is sixteen bars long. It contains five channels, one for each block of music in the audio file, and the channels are in the same order as the blocks of music in the audio file. The flags in the channel arrays let us know whether the block assigned to the channels should be played or not: the value 0 means a block will not be played; the value 1 means a block will be played.

This data structure is mutable, it can be changed at any time even when the music sequencer is running, and this allows you to modify the flags and structure of the music to reflect what is happening in a game.

Audio Processing

Most audio APIs will either broadcast an event to an event handler function, or invoke a function directly, when it needs to push more audio samples to the hardware. This function is usually invoked constantly like the main update loop of a game, but not as frequently, so time should be spent optimizing it.

Basically what happens in this function is:

Multiple audio samples are pulled from the input audio stream.
Those samples are added together to form a single audio sample.
That audio sample is pushed into the output audio stream.
Before we get to the guts of the function, we need to define a couple more variables in the code:

playing  = true // indicates whether the music (the sequencer) is playing position = 0    // the position of the sequencer playhead, in samples

The playing Boolean simply lets us know whether the music is playing; if it is not playing we need to push silent audio samples into the output audio stream. The position keeps track of where the playhead is within the music, so it’s a bit like a scrubber on a typical music or video player.

Now for the guts of the function:

function update() {
outputIndex = 0
outputCount = output.length
if( playing == false ) {
// silent samples need to be pushed to the output stream         while( outputIndex < outputCount ) {
output[ outputIndex++ ] = 0.0
}
// the remainder of the function should not be executed
return
}
chnCount = channels.length
// the length of the music, in samples
musicLength = BLOCK_SIZE * channels[ 0 ].length
while( outputIndex < outputCount ) {
chnIndex = 0
// the bar of music that the sequencer playhead is pointing at         barIndex = floor( position / BLOCK_SIZE )
// set the output sample value to zero (silent)
output[ outputIndex ] = 0.0
while( chnIndex < chnCount ) {
// check the channel flag to see if the block should be played             if( channels[ chnIndex ][ barIndex ] == 1 ) {
// the position of the block in the “input” stream                 inputOffset = BLOCK_SIZE * chnIndex
// index into the “input” stream
inputIndex = inputOffset + ( position % BLOCK_SIZE )                   // add the block sample to the output sample
output[ outputIndex ] += input[ inputIndex ]
}
chnIndex++
}
// advance the playhead position
position++
if( position >= musicLength ) {
// reset the playhead position to loop the music             position = 0
}
outputIndex++
}
}

As you can see, the code required to process the audio samples is fairly simple, but as this code will be run numerous times a second you should look at ways to optimize the code within the function and pre-calculate as many values as possible. The optimizations that you can apply to the code depend solely on the programming language you use.

Don’t forget you can download the source files attached to this tutorial if you want to look at one way of implementing a basic music sequencer in JavaScript using the Web Audio API.

Notes

The format of the audio file you use must allow the audio to loop seamlessly. In other words, the encoder used to generate the audio file should not inject any padding (silent chunks of audio) into the audio file. Unfortunately MP3 and MP4 files cannot be used for that reason. OGG files (used by the JavaScript demonstration) can be used. You could also use WAV files if you wanted to, but they are not a sensible choice for web based games or applications due to their size.

If you are programming a game, and if the programming language you are using for the game supports concurrency (threads or workers) then you may want to consider running the audio processing code in its own thread or worker if it is possible to do so. Doing that will relieve the game’s main update loop of any audio processing overhead that may occur.

Dynamic Music in Popular Games

The following is a small selection of popular games that take advantage of dynamic music in one way or another. The implementation these games use for their dynamic music may vary, but the end result is the same: the game’s players have a more immersive gaming experience.

Journey: thatgamecompany.com
Flower: thatgamecompany.com
LittleBigPlanet: littlebigplanet.com
Portal 2: thinkwithportals.com
PixelJunk Shooter: pixeljunk.jp
Red Dead Redemption: rockstargames.com
Uncharted: naughtydog.com

Conclusion

So, there you go – a simple implementation of dynamic sequential music that can really enhance the emotive nature of a game. How you decide to use this technique, and how complex the sequencer becomes, is entirely up to you. There are a lot of directions that this simple implementation can take and we will cover some of those directions in a future tutorial.(source:tutsplus)