游戏邦在:
杂志专栏:
gamerboom.com订阅到鲜果订阅到抓虾google reader订阅到有道订阅到QQ邮箱订阅到帮看

Peter Drescher解说手机游戏音频制作过程

发布时间:2011-06-03 00:51:19 Tags:,,

游戏邦注:本文原作者是Peter Drescher,现任Danger, Inc音效设计师(Danger, Inc是手机设备T-Mobile Sidekick的制造商),很早之前他曾经在街头弹奏钢琴曲卖艺,但是在15年前,这种生活发生了转变。从那时起他开始为多媒体制作音频,并逐渐成为制作小型精巧音频文件的行家。

刚开始我是为《After Dark》的屏幕保护程序制作音乐。后来也为General Magic设备制作音乐邮票以及56K调制解调器的网络音频。而最近,我开始为手机设备制作铃声和游戏音频了。这也正是我将在本文提到的内容。

如今我们提到的“移动设备”指的是手机或者“智能”手机,而不是PSP或者掌上游戏机。在这种情况下,移动设备被定义为一种带有收音机,并能够通过移动网络传播或接收声音和数据的设备。这种设备具有更复杂的操作系统,例如诺基亚和摩托罗拉手机,或者Treo(游戏邦注:美国Palm公司生产的智能手机系列)和Sidekick等设备,因为我们很关心基于不同移动设备操作系统所制作出来的交互式游戏音效。

Peter Dresche正在使用他那台定制的黑色nanoHiptop边玩游戏边听歌

Peter Dresche正在使用他那台定制的黑色nanoHiptop边玩游戏边听歌

操作点

当我们在为游戏制作音效和背景音乐时,我们同时也在加强并完善游戏设置,为用户创造有趣,令人兴奋的音频体验而努力。最后我们还必须避免那些让玩家感到烦恼,愤怒,或者不快的音乐出现在游戏中。

玩家并不喜欢那些反复播放的音乐。

玩家并不喜欢冗长的音乐。

玩家并不喜欢反复播放同一个音乐。

如果能够发现玩家的这些问题,你便能够明白如何去避免这些问题。

但是不幸的是,当你的所有音频制作只能够限制在100K储存空间时,上述几种问题将不可避免会出现在你所制作的音频中,这种资源匮乏的情况与当今手机市场中的某些情况类似。假设CD音频播放1秒钟需要耗费86K的话,这时如果你希望游戏音效能为你带来更多的利益,你就必须使用一些特别的技巧。

你必须尽可能地采取一些有效的手段。你希望能够尽可能地压缩音频中的每一个字节,以达到最合理的使用。重复播放是大忌,最大压缩很有益,创造性地使用有限资源则是有效动力。这些标语如果用在音频制作中,便意味着你必须应用各种方法制作出低分辨率且高压缩的声音样本。这种情况如果出现在音乐创作中,就会涉及音乐设备数字接口(MIDI)的概念。下面,我将为你们展现如何使用较小的文件制作出有趣,交互性强且非重复的手机游戏音频。但是如果你想要让这些小块的数据音频变得更好听,就需要多费些脑子了。以下是我们采取的做法。

《Cheese Racer》是一款基于《迷魂车》(Rally X)而开发的游戏,游戏通过运用各种技巧在狭小的储存空间中制造出了各种声音和音乐,包括:

MIDI文件中静音和非静音多重音轨所产生的不同混音。

以不同的音调演奏同一个音乐样本使其产生多种变体。

通过调高或调低音调技巧节省储存空间。

MIDI序列图中包含了《Cheese Racer》这款游戏中使用的所有音轨

MIDI序列图中包含了《Cheese Racer》这款游戏中使用的所有音轨

在这款游戏中,播放工具是嵌在通用MIDI的声音库中,同时也包含有一些RMF文件中的特定样本文件。

首先我们来看看原声音乐。图2是游戏中MIDI序列的截图,为我们呈现出了完整的音轨布局。因为这款游戏主要是针对Hiptop这款移动设备而设计,该设备操作系统使用了Beatnik音频引擎(BAE),所以我们可以利用RMF文件创作游戏的原声音乐。也就是说,我们可以从通用MIDI声音库的工具,或者专为该款游戏定制的软件工具看到上图的MIDI序列。

例子:位于前列的两个打击乐音轨样本:

打击乐1(56K MP3)

打击乐2(68K MP3)

然后是包括低音,旋律和和弦的音轨:

低音(92K MP3)

旋律(92K MP3)

和弦(88K MP3)

在第一阶段的核心部分,我们将定义十种不同的音轨组合,包括打击乐+低音,打击乐+低音+旋律,低音+旋律+和铉等等。你可以点击图3的影片剪辑进行试听。

因为每次循环播放音乐的时间是40秒,所以在不同的音轨组合中,每个阶段播放6分多钟不同的音乐组合。当然了,并不是每次播放音乐都会持续40秒。每一次游戏中的音乐组合播放发生变化时,都是由游戏设置而决定,每一首音乐都不会按照同一种方法播放两次,因为这么做才能够保证游戏音乐的多样性并减少玩家的“听觉疲劳”。

事实上,通过使用与第一阶段相同的节奏和打击乐音轨,两种以上的不同音轨还能够播放出更多不同类型的音乐。因此在整个游戏过程中,通过使用68K的压缩音乐样本和MIDI数据,玩家便能听到20多分钟各种音乐混合的多样游戏原声乐。此时你还需注意在两种不同音乐转换期间或者游戏升级期间,应该使用缓冲器让音乐进行缓冲,以听到完整流畅的音乐播放。

是的,你总是想尽可能地听到各种不同的声音,而不想让同一种声音反复“污染”耳朵。虽然《Cheese Racer》这款游戏中只有一种“喇叭”音效,但是我们通过转换音乐播放的频率,使得玩家能在每次捡起乳酪时能够听到不一样的音调(即使都是同样的喇叭声)。

因为我们会在游戏设置中随机调整音调,所以玩家不会在游戏过程中听到任何重复的音乐:

游戏中的声音能够帮助掩饰不同音乐的转换过程,使玩家的音频体验更加顺畅。

另外一种节省储存空间的方法即“调高或调低音调”技巧。首先你必须把原始的高分辨率音效调高成一个八度音阶,并将这个音阶对分。然后把它转变成一个低分辨率的压缩格式,如下:

猫叫,八度音阶以上(12K MP3)

发动汽车,八度音阶以上(16K MP3)

随之降低音阶:

猫叫,八度音阶一下(20K MP3)

发动汽车,八度音阶以下(24K MP3)

尽管游戏中的声音可能会听起来不是很连贯,但是为了使音乐更逼真,你必须尽可能地把原有的音乐文件对半压缩。显然的,如果你的音调越高,游戏中的音乐不连贯性就会越明显,但是在一些特别的工具中却不会出现这种问题,例如手机设备中的小型扬声器就能够很好地播放出这种音乐。

真正的小型扬声器的声音设计

Duke Ellington(游戏邦注:著名的作曲家)曾经说过:“我是为了所有演奏者而谱曲”。换句话说,你之所以会谱写一首喇叭乐曲,是因为你知道这首曲子用喇叭演奏出来将会很动听。所以,当你想为手机游戏创造音频时,你也必须针对你所创作的音频播放工具——拇指般大小的手机扬声器来制作音乐。

首先你必须先排除低音效果,因为你所制作的音频是针对于游戏而言,而且没有一个玩家能够忍受游戏中出现任何低沉无力的音乐。如果能够使用打击类声音更好了,因为鼓声通常都有振奋人心的效果,但是也要注意不要使用过度。

音效亦如此。当你在设计汽车碰撞的音效时,一定要尽可能地让这种声音听起来够澎湃。通过使用250 Hz的高通滤波器或者添加一个3kHz的碰撞音效将能够提高扬声器播放出的音效,并使玩家能够真实感受到汽车碰撞时的爆裂声。

波线1是一个非常有用的插件,通过使用这个插件,你能够针对手机设计出大而清晰的音频

波线1是一个非常有用的插件,通过使用这个插件,你能够针对手机设计出大而清晰的音频

你也许会想尽可能地利用波线1制作游戏中的所有声音(如上图)。但是波线1只是Waves的一个压缩插件,用于降低最高音频并提高整个音乐文件的音量。这种插件主要是用于最大限度地补充数据范围,就像以上右图所示。

这个插件有两个优点。首先通过使用这个插件,你能通过扬声器更大声且更清晰地播放游戏声音。其次这个插件还能尽可能地压缩音频样本,以帮助你能在最小的储存空间中播放最优质的音乐。如以上左图便是把高分辨率和CD品质的音效降低成11kHz,IMA 4:1的压缩文件的展示,这种模式下出现的音乐效果将会和静电干扰的效果一样。

但是这并不代表不能在手机平台上制作出好音乐,而是你必须针对扬声器,这个手机游戏音乐播放器而制定相关策略。对于手机音乐来说,旋律是制胜关键。而对于音效来说,元音则是关键。旁白,尖叫声,笑声,打喷嚏声,叹息声,咕噜声,呻吟声等各种各样通过嘴巴发出的声音都能经由手机扬声器发出,而且保真度也是相当有保证的。以下是一些典型的例子。

《Bob》这款游戏通过使用交互式音频技巧为玩家创造出了多种多样的游戏音乐。在游戏中,随着玩家游戏的进行,游戏中的不同音乐会频繁发生转换,那些频繁播放的弹跳音效完全取决于游戏中的“精灵”(游戏角色)的下降速度,而“Bob”,这个作为游戏中声音最大且最活泼的家伙会在每一次升级时发出“Yeah!”的声音,在下降太快时发出“Whoa”并在开始玩游戏时说道“All Right”。如果你们在游戏过程中觉得这些声音听起来很耳熟,那就没错啦,因为Bob的声音就是我发出来的。

注意在这款游戏中不同音乐是如何发生转换的。低音速度很快,并且在结束后会很快插入电子钢琴乐等等。但玩家在进行游戏的过程中,将会发现游戏的每一个阶段都会出现6到7种不同的乐器组合,而当玩家完成游戏时,将会听到一个完整的音乐组合播放。按照这种方法,整个游戏过程中会一直出现不同音乐,游戏最后的奖励也不例外。

蓝线表示通过tile editor所表现出的游戏某一阶段“音频Mix #5”的出现过程

蓝线表示通过tile editor所表现出的游戏某一阶段“音频Mix #5”的出现过程

如上图,通过使用tile editor在游戏级别图表中安置传输工具,使游戏能够转换不同的音乐效果。图中的每一个黑点都代表一些特殊的游戏内容,包括弹跳,遇见敌人等,特别是“音频Mix #5”更是专门用了细蓝线标示出来(因为这是一款横向卷轴游戏,所以这条蓝线会持续出现在不同的游戏点上)。如果玩家并没有掉进屏幕中间的那两个无底洞,他将能够进入下一个音频区,也就是如图8所示右边的黑点连成的线,随后游戏便会开始播放“音频Mix #6”。

接下来我想谈谈我最喜欢使用的音频制作技巧,也就是 “Secret Yanni”技巧,这种技巧其实很深奥,很难在这里详细说明。关键问题就在于:视频游戏并非电影,并不能同步播放图片,因为玩家总是不能预言什么时候会突然出现音乐等等。但是如果使用了 “Secret Yanni”技巧,你将能够发现音效的神奇之处,在游戏中合理使用音效定能为你的游戏加分不少。

为了节省储存空间,我们在Bob每次捡到“爱心”时都会播放一些MIDI注释。MIDI能够帮助你在手机环境下制作最好的音效,因为这是一种有效且灵活的技巧。但是问题就在于“该在哪里安放这些注释才最合适?”因为我们并不知道什么时候Bob能够得到积分,所以并不知道什么时候得到积分的“祝贺”音效响起才合适。虽然你希望用“Yay!”来配置得分的音效,但如果注释不相符,也许会出现“Ouch!”也不一定,如果这样的话,画面真的就太搞笑了。

解决方法便是,采用调式音乐。也就是你必须在游戏中限制音乐的使用,不要尝试太频繁的音乐变化,也不要连续播放一些奇异的和弦。很多情况下只要保持简单的音乐就好,因为这是播放背景音乐的最好方式。就像在《Bob》中,和弦的主调是C大调中的ii-IV-V-I,确定了主调后我便按照这种模式并按照五声音阶C,D,E,G和A编写得分注释。如果你想让游戏看起来更加灵活,你可以让注释和音乐同步显示或者也可以使用一些多旋律的音乐让游戏整体看起来更顺畅。按照这种方式所做的音效将不会与游戏中的音乐发生冲突,甚至会和谐到与音乐完全融为一体也说不定。

非音乐的游戏音效

如何做才能避免背景与音乐不符的情况?只要不使用音乐便可以。Danger的另外一款游戏《Meme》便是通过一些情绪类声音取代了音乐。在这款游戏的“邪恶实验室”阶段中,正是使用了“调低或调高”音调这一技巧解决了很多难题,如“如何在一个非常小的储存空间里创造出一个低沉可怕的游戏氛围?”一般来讲,游戏氛围总是需要有很大的覆盖面,因为只有这样玩家才不用总是关注到反复回荡着的一些不自然的音效。另外一个问题便是,“如何做才能用小小的手机扬声器播放出《星际迷航》这样壮观的音效?”

而答案便是,你什么都做不了。因为这是不可能的事,你只有另辟渠道。就像你可以制造出一个高声调但却低波动和低变线颤音的声音,如下:

可怕的实验室氛围(16K MP3)

然后你需要制作一个特定的RMF工具,使用这个工具你便能够播放不同的音调。尽可能地降低音量直到声音渐渐听不见,再开始慢慢提高音调。按照这种方法,你能把声音最低降至“嗡嗡声”,让它听起来若隐若现。

对于玩家来说这可以算是一个较为“吵闹”的游戏。游戏中采用的另一个技巧便是在大猩猩扔香蕉袭击机器人时,香蕉飞跃发出的“嘶嘶声”,在这里我们还特别用“声乐”突出了这种特别的音效,而这种音效将能够通过手机的扬声器大声清晰地表现出来。

同时我们也在“机器人工厂”这一阶段尝试了用“不同方法播放同一种声音样本”。即每当一个机器人被制造出来时,这个场景里便会出现一种机器摩擦声的音效。

攻击(12K MP3)

机械(16K MP3)

在游戏设置中,因为手机扬声器太小而不能播放一些设计精巧的音效,只能展现给玩家一些较大且相对粗糙的音效。也许这并不是你第一次为机器人工厂这类型的游戏氛围设计音效,但是因为是针对手机游戏所设计的,所以并不能应用那些多余的额外储存空间。

在这款游戏中我最喜欢的便是大炮的音效,即子弹落下的那个音度:

子弹(8K MP3)

但是子弹下落的频率却是不可预料。所以在游戏中玩家必须灵活移动,并需要作出合理的猜测。看图11。

我之所以能够制造出这种交互式的音效是因为:

我利用了Beatnik技术——音效设计的好工具。

我在这个行业已经打拼多年,所以很了解应该使用那些手段和技巧制作出好音频。

我很了解目标硬件和操作系统,因为我自己就会制作这些硬件和系统。但是如果你是个独立的音频制作者并想要制作大量的手机游戏音效,那么你就该好好动动脑子了。

首先,你需要注意手机操作系统是否支持你访问一些与处理转换音调或静音等情况的API。然后你还必须检查手机的音频子系统能否按照你所设想的情形进行。如果你能够使用Beatnik,那情况便再好不过,反之,你便不得不另辟蹊径了。不同手机的音频能力和音频引擎大不相同,既有最初级的也有最复杂深奥的。

然后你必须判断如何做才能在最小的储存空间里安置最大的音量,同时你还必须考虑到你所制作的游戏音效所处的平台的技术局限问题,包括这个平台支持什么样的文件类型,什么样的分辨率以及什么样的压缩格式等等。

如果你能够制作高分辨率的MP3格式,那你便能按照所有设想放手制作音效;但是如果平台限制你只能制作8k赫兹IMA WAV文件格式,你便不得不考虑使用一些较短,大声且简单的音效。比起一些复杂的“噪音”,这些简单的音效较容易转换。也就是说比起交响乐如此多类型的音乐结合体,单独一款长笛音乐更容易转换。所以针对于手机扬声器而言,最好更侧重于制作一些音量较小的音效,且避免夸张的爆炸音效。

更重要的是你必须清楚有多少储存空间能让你安置这些音乐。大部分情况下如果你想要制作音乐,就必须使用通用MIDI的内部波表(游戏邦注:MIDI的一种合成技术),因为它可以称得上是目前为止最形象的音频效果。而且你最好不要为难自己,尽可能地选择一些简单且基础的音源,如钢琴,喇叭,反冲,陷阱或者弦乐器等,最好不要使用像日本笛或者其它并不知名的音源。

而且因为你并不知道发声仪器在不同的设备中会有怎样的效果,但是不管怎样,像钢琴这类型的音乐总是能展现出最好的音效。而且我想说的是,即使我们认为单独的乐器更有利于音效的展现,但是我们并未限制巴赫,贝多芬或者肖邦等的音乐,如果适合,我们也会好好利用这些大师们的创作。

一切都是来自于过去

说实在的,我今天所提到的任何技巧都不是最新的技巧。我敢保证,任何一个创作灵感或者激进的想法都不可能只是停留在当下的一种设想。事实上,今天为手机游戏创造音频就与我们在80年代为计算机游戏创造音频以及90年代为网络游戏创造音频的情况类似,因为我们也遇到了相同的问题,如资源受限,竞争激烈等,所以我们也在尝试着用当时的解决方法来看待这些问题。

因为我们之前就接触过类似的领域,所以已经有了一定的经验,对于如何制作最适合且最有效的音频格式我们心里都有数了。因为我们知道手机游戏的音频制作需要更广阔的宽带接口,所以我们便计划着将最大限度地进行扩展。

我们也知道所有的数字版权管理(DRM)也都是预先设定好的,所以我们能做的便是按照用户的需求提供他们想要的效果,而不是操纵他们做任何事。

最重要的是,因为我们在过去制造其它音频时也应用了同样的技巧,而且这些技巧不仅对当前的手机音频制作很有帮助,我们还预测它们在未来的相关领域的功效也会非常巨大。换句话说,如果你想要制作一款有趣且高质量的游戏音频,那你最好去雇佣那些有经验的游戏音频制作者们,因为他们拥有广大新手们所不具备的制作技巧。

顺便提一下,这些“经验教训”并不是我所总结的,而是在2004年的Bar-B-Q手机音频项目会议上提出的。

过去是过去,现在才是现在

最后,所有朋友们,我想说的是当今手机产业发展如此迅速,所以我想说,其实我今天所提到的关于手机游戏音频设计的所有内容都是胡扯,因为只要我们知道如何把最新型的设备与早前设备相结合,那便没有什么是不可能做到的。

如今同时带有十亿字节移动储存卡和宽带网络连接的音乐手机已经出现了,这也标志着我之前提到的手机游戏音乐已经完全过时了。

手机游戏并不像Xbox或者PSP上的游戏,因为频带宽度和CPU的使用限制,手机游戏不得不限制于一些小型,有趣且用于消遣的游戏。它们并不是像《战神》这类型能让玩家耗费40个小时沉迷于游戏世界的游戏,反而玩家是把它们当成日常生活中随时可以用于消遣的工具。而且因为技术的限制,且价格也比较便宜,加上它们是用于手机平台而不是特定的游戏平台,手机游戏中的音频分辨率通常都比较低,不论你运用了何种技巧,这都是一个不可改变的事实。

我们必须满足于所有能够得到的东西。就像现在我们所使用的设备都能用于拍照,发邮件,上网,玩游戏,打电话以及听音乐。问问你自己,当你在打游戏消遣时最想听什么音乐?是那些不知名的MIDI音频还是从iTune中上传到手机上的格莱美奖获得者所演唱的歌曲?答案不用说大家也都知道吧!

而且我也是这么认为的。如果玩家在游戏过程中能够滚动游戏音频,选择斯密斯飞船,Greenskeepers,或者纽约爱乐乐团等自己喜欢的乐队歌手的歌曲,那这将是件非常棒的事吧!如今摩托罗拉的一些手机已经嵌入了iTune音乐播放器,同时也逐渐出现了很多音乐手机。但是从我个人来讲,我却仍在使用这台定制的黑色Hiptop,而且我还用布基胶带在Hiptop后面粘贴了iPod第四代播放器(因为我是个乐师,我认为所有的乐师都应该像我这么做)。

商店里可没有出售这台nanoHiptop,而且我这么做也未经Danger或苹果的批准,所以你们最好不要模仿我做这种尝试。然而如果仔细想想,这真的是一个很棒的理念吧,它改变了传统意义上对手机游戏音频的想法,使我们看到一个不一样的音频播放方式。(本文为游戏邦/gamerboom.com编译,如需转载请联系:游戏邦

Could Mobile Game Audio BE More Annoying?!

Peter Drescher

[Editor's Note: Peter Drescher's last article for O'Reilly Digital Media ("Could Ringtones BE More Annoying?!") was such a hit that we asked him to expand on the concept. The following article is an edited transcript of his session "Creating Audio for Mobile Games," which he presented last month at the Game Developers Conference. To simulate the live experience, we've sprinkled audio and video clips throughout the story, corresponding with the ones Drescher played from his T-Mobile Sidekick during the presentation.]

Good afternoon. My name is Peter Drescher, and I am currently sound designer at Danger, Incorporated, makers of the Hiptop mobile internet device, also known as the T-Mobile Sidekick. I used to be a road-dog bluesman piano player until about 15 years ago, when I managed to get a job doing audio for multimedia. Since then, and through no planning on my part, I’ve become something of an expert on making little, tiny itsy-bitsy teeny-weeny audio files sound like, well … anything at all.

First I did soundtracks for After Dark screen savers that shipped on floppy disks. Then it was music stamps for the General Magic device, web audio for 56k modems, and these days, ringtones and game audio for mobile devices. And that’s what I’m talking about today.

A Cell Phone by any Other Name

Now, when I say “mobile device,” I’m talking about a cell phone or “smart” phone, but not the PSP or Game Boy. In the context of this presentation, a mobile device is defined as something that contains a radio for transmitting and receiving voice and other data over a cell phone network. We are talking about devices with fairly sophisticated operating systems, like Nokia and Motorola cell phones, or devices like the Treo and the Sidekick, because we are concerned with interactive game audio produced by the device OS (as opposed to phone conversations, FM radio broadcasts, or iPod libraries).

Figure 1. Peter Drescher plays a game while listening to music on his custom black nanoHiptop.

The Point of the Operation

When making sound effects and background music for any game, you’re trying to enhance gameplay and create a fun, exciting audio experience for the user. The last thing you want to do is be annoying, irritating, painful, or trite.

You really don’t want to be repetitive.

You really don’t want to be redundant.

You really don’t want to play the same sound over and over again.

You really get the idea.

Unfortunately, this is frequently exactly what happens when your entire audio budget is less than 100 kilobytes, as is mostly the case in resource-constrained situations like the current mobile environment. Given that one second of “CD quality” audio is about 86K, you’re going to have to use some tricks of the trade if you want to have more than a single sound effect in your game.

The most important trick is to be as ruthlessly efficient as possible. You want to squeeze every last drop of variation out of each and every byte of audio data at your disposal. Repetition is the enemy, compression is your ally, and creative use of limited resources is your battle cry. If you’re making sound effects, this means low-resolution, highly compressed samples used in multiple ways. If you’re writing music, it means MIDI. Using the techniques I’ll be demonstrating here, it is possible to produce interesting, interactive, non-repetitive, evolving soundtracks for mobile games using absurdly tiny files. The real difficulty comes when trying to make these little tiny pieces of digital audio crap actually sound good. But it can be done, and here’s how we did it.

Could Mobile Game Audio BE More Annoying?!

How About a Nice Piece of Cheese, Gromit?

Cheese Racer is a game, based on Rally X, that uses a number of techniques to produce lots of sound and music in very little space, including:

Muting and unmuting multiple tracks in a MIDI file to produce various mixes.

Playing a single sample at different pitches to produce variation.

The “pitch it up, play it down” technique for saving space.

Figure 2. This MIDI sequence contains all of the tracks used for the Cheese Racer game. It plays instruments from the built-in General MIDI soundbank, along with custom samples contained in an RMF file. (Click to enlarge.)

Let’s look at the music soundtrack first. Figure 2 is a screen shot of the MIDI sequence used in the game, displaying the complete track layout. Since this game was designed to run on the Hiptop, which uses the Beatnik Audio Engine as part of the OS, we were able to use an RMF file to create the soundtrack. This means that during gameplay, the MIDI sequence pictured here is rendered on instruments from the internal General MIDI soundbank, and on custom software instruments specifically designed for the game.

For example, the top two tracks play custom samples of drum loops:

Percussion 1 (56K MP3)

Percussion 2 (68K MP3)

Next we have a group of three tracks, playing bass, melody, and chords:

Bass (92K MP3)

Melody (92K MP3)

Chords (88K MP3)

In the code for the first level, we’ve defined ten different combinations of tracks: percussion + bass, percussion + bass + melody, bass + melody + chords, etc. You can hear what it sounds like by clicking the movie clip in Figure 3.

Figure 4. Every time you play the game, the soundtrack is different. (Click image to play movie clip.)

Figure 3. Notice how the mix changes every time the mouse gets a piece of cheese. (Click image to play movie clip.)

Because the music loop is 40 seconds long and we defined ten different combinations of tracks, each level contains more than six minutes of different music mixes. Of course, not every mix is played for 40 seconds—in fact, that’s kind of the point. Because the mix changes depending on gameplay, the music will never play exactly the same way twice, thereby increasing variation and decreasing “ear fatigue.”

But wait! There’s more! In fact, there are two more sets of tracks playing two more styles of music, using the same tempo and percussion tracks as the first level.

Therefore the entire game contains almost 20 minutes of various music mixes, using only 68K of compressed sample and MIDI data. Click on Figure 4 to hear a different performance. This time, notice the use of “bumpers” to smooth out transitions between mixes, and when moving to the next level. (Special thanks to Lucas Finklestein, Danger QA engineer and game player extraordinaire, for helping me film these sequences.)

It’s the Same, Only Different

Remember, you want as much variation as possible, and you really want to avoid hearing the same sound over and over again. This game contains only a single “trumpet fall” sound effect, but we modify the playback sample rate so that each time you pick up a piece of cheese, the sound is played at a different pitch … any similarity to the 1960s Batman theme is entirely intentional:

Fall 1 (12K MP3)

Fall 2 (12K MP3)

Fall 3 (12K MP3)

Fall 4 (12K MP3)

We also randomly vary the pitch during gameplay, so there’s no repeating pattern to it:

Random Fall (48K MP3)

Figure 5. By randomly varying the pitch of the car horn beeps, we produced multiple sounds using only one file. (Click image to play movie clip.)

The sound tends to mask the transition from one mix to another, helping to create a more seamless audio experience. In Figure 5, you can hear the same kind of pitch-shifting effect applied to car horn beeps.

Another way to save space is using the “pitch it up, play it down” technique. Here’s how it works: take your original, high-resolution sound effect and transpose it up an octave, halving the length. Then convert it to a low-resolution compressed format, like this:

Cat growl, octave up (12K MP3)

Car start, octave up (16K MP3)

Then in the game, play it down an octave:

Cat growl, octave down (20K MP3)

Car start, octave down (24K MP3)

Although the game sound might be a little crunchy, you’ve just cut the size of your file in half without losing too much audio fidelity. Obviously, the higher you pitch the sound, the “crunchier” the playback will become, but the technique can be used for custom instruments as well as sound effects, and is particularly effective on the tiny speakers in mobile devices.

Sound Design for Really Small Speakers

Duke Ellington said about arranging, “Always write for your players.” In other words, if you write a horn line you know your horn section can play well, then your arrangement will be well-played and your music will sound good, kinda by definition. The same is true when creating audio for mobile games: you have to write for your players, which in this case is usually a speaker the size of your thumbnail.

First of all, this means no bass—I mean, none, nada, fuggedaboutit. Don’t write music that gets its power or groove from a deep funky bass line, because nobody’s ever going to hear it. Snap of the snare drum—yes, boom of the kick drum— nnnnnot so much.

The same thing goes for sound effects. When making a car crash, pick a sound that has a lot of high end, then EQ the bottom right out. Applying a highpass filter at around 250Hz and maybe adding a bump around 3kHz will prevent low-end rumble from distorting the speaker, and make the sound pop where the speaker is most sensitive.

Figure 6. Waves L1 is an extraordinarily useful plugin that helps audio designed for cell phone speakers play loud and clear.

Another thing you’re going to want to do is apply a liberal dollop of L1 to almost every sound you make (see Figure 6). L1 is a compressor/limiter plugin from Waves that squishes the peaks and raises the overall volume of a sound file; it is used to fill up as much of the available digital range as possible, as shown on the right side of Figure 6.

This does two good things. First, it gives you a strong signal so that your sound will play loud and clear on the tiny speaker, and second, it gives your downsampling and compression algorithms as much meat as possible to chew on while they decimate your audio data to save space. When processing a high-resolution, CD-quality sound effect down to an 11kHz, IMA 4:1 compressed sample, a soft sound like the one on the left side of Figure 6 is going to come out sounding a lot like static.

This doesn’t mean you can’t make good sounds on cell phones; you just have to write for your players. For mobile music, melody is king. For sound effects, well, there is one category of sound that cell phones are specifically designed to produce well–vocals. Voice-overs, screams, laughs, sneezes, sighs, grunts, groans, pretty much any sound a mouth can make is guaranteed to come through a cell phone speaker with some sort of fidelity. Here’s a good example.

A Bouncin’ Beachball, Lookin’ for Love

Bob is a game that uses a number of interactive audio techniques to create a varied soundtrack. The music mix changes as the player progresses through each level, the frequently played bounce sound effect is varied depending on how fast the sprite is falling, and Bob, being a loud happy guy, says things like “Yeah!” when he gets a power-up, “Whoa” when he falls too fast, and “All right” when it’s time to play. And if these vocals sound a little familiar, yes, it’s true, I admit it … I am the voice of Bob.

Figure 7. The whimsical Bob is one of Danger’s best selling games. (Click image to play movie clip.)

Notice how the music mix changed as we played the game. The bass line got faster, the electric piano came in, etc. Each level has six or seven different combinations of instruments and tracks that flow from one to the other as you continue forward, with a full mix playing when you complete the level. That way, the music is always changing, and there’s a little reward at the end.

Figure 8. This view of the tile editor shows a section of one level with the bits for “Audio Mix #5″ shown as a blue line.

This variation is accomplished by setting bits in the level map using an editor as shown in Figure 8. Each black dot represents some sort of special tile bit (bounce, enemy, heart, etc.) and the bit for “Audio Mix #5″ is highlighted in blue. Therefore, when Bob passes through that thin blue line (and he always will at some point, because the game is a side-scroller), the music will change to mix #5. Later, if you don’t fall into the bottomless pits at mid-screen, you’ll hit the next audio zone, shown in Figure 8 as a line of black dots in the sky at the right, and then Mix #6 will play.

The Secret Yanni Technique

Figure 9. Why is this soundtrack called “the Secret Yanni Technique?” Click the image to play the movie clip.

Now we get to one of my favorite tricks of the trade, which I like to call the “Secret Yanni” technique for reasons that are too arcane to explain here. Basically, the idea is this: video games are not movies; there’s no concept of “sync to picture” because you can’t always predict when a sound effect will be played. But knowing this, you can use serendipity to your advantage, as we did in Bob with the sound effects for bonus points.

In order to save space, we wanted to play a few MIDI notes every time Bob picked up a heart. MIDI is an excellent (and in many cases, the only) choice for making sound in the mobile environment because it is an extremely efficient and flexible technology. But the question becomes, “Which notes do you play?” There’s no way to know when Bob is going to pick up points, so there’s no way to know when the bonus sound will play or how it will fit with the music. You want the effect to be “Yay!” but if the notes clash with the music, the effect is going to be more like “Ouch!”

The answer? Modal music. You restrict your soundtrack to a single mode, no key changes, no fancy chord progressions. Just keep it simple, baby—and in many cases, this is a good way to write background music anyway. For Bob, the chords are mostly ii-IV-V-I in C, and then I picked notes for the bonus points that fit the mode, using a pentatonic scale consisting of C, D, E, G, and A. If you want to get really clever, you can have the notes play at the same tempo as the music, or some polyrhythmic fraction of it. Arranging things in this manner will produce sound effects that almost never clash with the music, and sometimes will even seem to be part of it. (Click on Figure 9.)

The Sound of No Music

Figure 10. Meme is a super-chimp, fighting his way past mad scientists and evil robots by hitting them with bananas. (Click image to play movie clip.)

One way to avoid the problems associated with background music is not to have any. Another Danger game, Meme, dispenses with music altogether, and uses mood-producing sounds instead. The “evil laboratory” level demonstrates the “pitch it up, play it down” technique to an extreme degree, and is used to solve a difficult problem: how do you create a low, creepy ambience using very little memory space? Traditionally, looping ambiences have to be fairly long (and therefore fairly large), otherwise you quickly hear the unnatural repetition of the same sound playing over and over. The other problem: How do you produce a Star Trek bridge-style rumble on a tiny cell phone speaker?

The answer, of course, is … you don’t. It’s impossible. Here’s what you do instead: Make a fairly high-pitched sound with a little blip in it and a little looped vibrato, like this:

Laboratory ambience (16K MP3)

Then make a custom RMF instrument out of it so you can play it at different pitches and layer it against itself. Then try playing it lower and lower until the sound starts to disappear, then bring it back up a little. That way, you get the lowest possible hum, the sound itself is very crunchy (which kinda works in this situation), and so it sounds something like Figure 10.

Man, that’s a noisy game! You don’t really miss the music much, do you? Other techniques demonstrated are the random pitching up and down of the banana whooshes, and prominent use of “vocal” sound effects (in this case, chimp noises) which punch through the tiny speaker loud and clear.

We also do the “single sample used in multiple ways” trick for the robot factories. Each time a robot is manufactured, there’s this grating, grinding noise. It’s the electrical-zap-played-down-an-octave machine:

Figure 11. Playing the zaps and pows at different pitches can produce interesting effects without using additional space. (Click image to play movie clip.)

Zap (12K MP3)

Machine (16K MP3)

During gameplay, the bottom end goes away because the speaker’s too small to produce it, leaving only the higher-end grungy sound. Maybe it’s not the sound effect you

would have first designed for a robot factory, but it sorta kinda works and it doesn’t use any extra space.

But my favorite use of that trick in this game is the cannon sound, which is the bullet played two octaves down:

Bullet (8K MP3)

This was totally unplanned; I was simply playing around with pitch-shifting and thought, “Cool, man, we got to use that!” Sometimes, happy accidents are your most valuable tool. Check out Figure 11.

Because I Can

I have a certain advantage in producing these kinds of interactive soundtracks because:

I’m working with the Beatnik technology, which is specifically designed for this kind of thing.

I’m the oldest living content provider for the system, so I know all the tricks and techniques.

I have an intimate knowledge of the target hardware and operating system, because, well, we built it. But if you’re an independent audio producer, or working on a game intended to ship on a myriad of phones, then you’ve got some special problems.

First, you’ll have to see if your phone’s operating system gives you access to the kind of APIs you’ll need to implement things like pitch shifting and track muting.

Some do, some don’t. Then you’ll need to check if the audio subsystem can even do what you want it to. If you’re running Beatnik, you’re probably golden; if not, well, good luck. The audio capabilities of cell phones and audio engines vary widely, from the completely rudimentary to the arcanely sophisticated.

To make good decisions about how to squeeze the maximum amount of sound out of the minimum amount of space, you’ll want to design your audio taking into account the technical limitations of each platform your game is intended to run on. You’ll need to know what file types are supported, at which resolutions, and what compression algorithms are available.

If you can run hi-res MP3s, you can pretty much make any sound you like, but if you’re limited to, say, 8kHz IMA WAV files, you might want to consider using short, loud, uncomplicated sounds. These will translate better than complex noise, so forget that symphonic sample and use a single flute line instead. If the audio engine’s output rate is 44k, rocking! Go for that screaming guitar solo, but if you’re constrained to 11kHz, you’re up against Nyquist, so you might be better off with a midrange piano solo. And no cell phone speaker is going to produce any bass whatsoever, so plan on using all tweet and no boom.

Most important, you’ll need to know what your audio budget is—how much space is being allocated for sound. In most cases, if you want to make music, you’re going to have to use General MIDI played on the internal wavetable, simply because it provides by far the most audio bang for your digital buck. But do yourself a favor and write for the basics—piano, horn, kick, snare, maybe some strings—and avoid things like the goblin pad, or the shakuhachi, and the other lesser-known patches. You have no idea how those instruments will sound on different devices and different soundbanks, whereas a piano is pretty much going to sound like a piano no matter where you go. In fact, solo piano music is usually a pretty safe bet in a General MIDI world, and I would suggest that being constrained to a single instrument didn’t seem to hinder Bach’s, Beethoven’s, or Chopin’s creativity.

All That’s Old Is New Again

Truth be told, none of the techniques I’ve described today are new or inventive. I am certainly not an innovative genius or a radical visionary, I’ve just been doing this stuff for a while. In fact, creating audio for mobile games these days is strikingly similar to producing audio for PC games in the ’80s and doing web audio in the ’90s. The same kind of audio problems and solutions will apparently arise in almost any resource-constrained, developing, competitive environment.

But since we’ve been here before (twice!), you think we would’ve learned a lesson or two. We’d know that closed, proprietary audio formats are bad and open standards are good.

We’d know that bandwidth bottlenecks will expand, and so we’d plan for scalability now.

We’d know that all DRM schemes are doomed, and that the best way to make a buck is to give the customer what they want, not criminalize what they’re going to do anyway.

But most important, we’d know that the same techniques that have worked for us in the past will be useful today and in the future. In other words, you want interesting, well-produced soundtracks for your mobile games? Hire old game-audio guys!

By the way, these “lessons learned” are not mine, they are just a few from the 2004 Project Bar-B-Q Mobile Audio session.

That Was Then, This Is Now

But finally, my friends, I’m going to stand here and tell you that the mobile industry moves so fast that everything I’ve said today about creating soundtracks for mobile games is already complete and utter bullshit. And I’ll tell you why: convergence.

The advent of music phones with gigabyte removable storage and broadband network connections is going to make mobile game music completely obsolete.

Really, think about it for a minute: games on your cell phones are not like games on your Xbox, or even your PSP. Given the bandwidth restrictions and the CPU usage, mobile games tend to be small, fun, time-killers. They’re not 40-hour immersive environments like God of War; they’re what you do while you’re waiting for something else to happen in your life. And because of the technical limitations, and because they’re cheap, and because, let’s face it, folks, they’re phones, not dedicated gaming platforms, mobile game soundtracks are going to be kinda low-res no matter what you do.

And this is all well and good when that’s all you’ve got. But what happens when that same device, that cool little gadget that takes pictures, does email, surfs the Web, plays games, and—oh yeah—makes phones calls also contains six hours of your favorite music? Ask yourself what you would prefer to listen to while you kill a little time playing little game—some low-res MIDI soundtrack written by who knows who (and that would be me in this case), or that funky-cool Grammy-winning groove you just uploaded to your phone from iTunes? Come on, it’s no contest.

And I know this because I’ve done it, and it’s too damn cool for school. It’s “roll your own game audio;” it’s CheeseRacer with an Aerosmith soundtrack, or the Greenskeepers, or the New York Philharmonic, or whatever you want. I’m tellin’ ya, it’s the best thing since flavored toothpaste, and pretty soon aaalll the cool kids are going to be doing it. Some Motorola phones already have iTunes built in, and the cell phone carriers are all jumping on board the “music phone” bandwagon like it was headed for Gold Country. Personally, I just took my custom black Hiptop and slapped an iPod nano on the back with duct tape (because I’m a musician, and all musicians must use duct tape) and voilà! (See Figure 12.)

Figure 12.The nanoHiptop (Duct Tape Edition) is the essence of convergence: two refined tools fused into one.

Introducing the nanoHiptop (not a real product, not available in stores, not condoned by either Danger or Apple Computer, do not try this at home, may void warranty).

Nonetheless, check it out, it’s a cool concept, it works great—and it represents a paradigm shift in the way we’re going to think about creating audio for mobile games.(source:digitalmedia


上一篇:

下一篇: