过去几年，我们已经目睹了电影中的声音从5.1/7.1声卡过渡到高通道数和基于目标的声音格式，使当下的电影制作者可以将“环绕立体声”进阶到更大大增加受众沉浸感的程度（Dolby ATMOS、DTS Multi-Dimensional Audio和Auro Technologies的Auro 3D）。这些格式在周围添加扬声器，包括上述的受众发出“elevation cue”，使声音得以在所有三个维度中环绕。
Ear Games工作室的iOS游戏《Ear Monsters》就非常有前瞻性，利用3D音效来促进玩法而不是视觉效果。使用3D声音线索来驱动玩法有助于把游戏世界的范围拓展到小小的屏幕之外，例如，玩家可以触击他听到的声音的在3D空间中来源（比如，当听到攻击的声音来自屏幕上方，就触击屏幕上方）。
Visceral Games的J White、Playdead的Martin Stig Andersen以及Freshtone Games的Thom Kellar都是音效设计师，他们在制作音效体验上经验丰富。
《地狱边境》的音效设计师及作曲人Martin Stig Andersen表示，“若你创造出音效循环及恼人内容，这种媒介就会自彰其丑。当游戏揭示机制时，这些元素背后的媒介及装置就会遭到破坏，玩家就会跳脱体验之外。”
一个典型例子就是《死亡空间》。J White（LucasArts元老，是Visceral Games《死亡空间2》的音效设计负责人）因团队能够做到这点而颇感自豪。
情感元素在更小规模游戏（甚至是iOS游戏）中的运用方式和AAA作品类似。举个例子，iOS平台的《只有一条命》赋予玩家角色一次从系列屋顶跳至另一屋顶的机会。Freshtone Games音效设计师Thom Kellar希望带给玩家这样的感觉：他们真的置身屋顶上，准备进行跳跃。
White表示，“杰出游戏机制能够自然吸引玩家眼球。《Pong》包含很棒的boops & beeps游戏机制，玩家着迷其中，因为这非常有趣。如今，作为音效设计师，我们能够获得令人满意的核心机制，但依然存在许多微妙元素有待我们深入挖掘。”
让我以Develop Awards的入围作品为例。Simogo的文本冒险游戏《Device 6》很大程度地依赖于其沉浸式音频将玩家带到游戏故事中并会在故事的重要部分出现时调动玩家的紧张感。
Develop Award的获胜者，即《Monument Valley》的声音制作人同时也是音频老手的Stafford Bawler通过提供自己曾经致力于的一个大型项目去总结了这些效果的重要性。
来自Creative Assemble同时也是《全面战争》的首席音频设计师Matt McCamley说道：“在过去，人们拥有非常有限的工具，因为这些技术上的限制导致他们很难创造各种不同的内容。而现在，随着游戏技术的发展，游戏音频也不断发展着，如此我们可以更好地向玩家传达游戏体验。”
Simogo的Flesser以及编曲家Daniel Olsen面对着一项复杂的任务，即在《Device 6》的互动故事中整合音频。该音频必须符合玩家的理解，同时还需要避免在错误的时间播放音频而失去玩家的关注。
Gina Zdanowicz是Seriallab Studios的创始人，Mini Monster Media的首席音频设计师，以及Berkleemusic的游戏音频导师。Seriallab Studios是一家全方位音频内容供应商，即面向电子游戏产业提供定制音乐和音效。Seriallab Studios已经参与了60多款游戏的音频开发。
《辐射3》有效地利用了剧情音乐和非剧情音乐。游戏中的角色配有腕带式计算机，名为Pipboy 3000，同时分配在游戏世界各地的收音机也会播放音乐和其它来自游戏内部广播电台的广播。如果玩家打开了自己的Pip-boy 3000，他们便需要小心收音机会引来NPC。当受因此被关闭时，非剧情背景音乐便会响起。
《失忆症：黑暗后裔》讲述的是手无寸铁的主角探索一个黑暗且令人产生不祥之感的城堡，在其中躲避各种怪物和其他障碍，同时游戏还包含解决谜题的内容。游戏是第一人称视角，玩家扮演Daniel随故事进展经历Brennenburg Castle的不同关卡，从Rainy Hall开始直到到达城堡的最深处，寻找城堡的所有者Alexander。
篇目1，6 Ways 3D Audio Can Expand Gaming Experiences
by Michel Henein
“Sound is that which does not simply add to, but multiplies the effect of the image.” Akira Kurosawa.
Over the past few years, we’ve witnessed sound for cinema being transformed from 5.1/7.1 to the high channel-count and object-based sound formats available for today’s film-makers to extend ‘surround sound’ to the next level of immersion for audiences (i.e. Dolby ATMOS, DTS Multi-Dimensional Audio, and Auro Technologies’ Auro 3D). These formats add speakers all around, including above the listener to emit ‘elevation cues’, allowing sound to pan around in all three dimensions.
Games can also use elevation cues to further expand audio in all three dimensions (without the need for additional speakers by using stereo headphones, for example); through the use of digital filters (HRTFs derived from dummy-head measurements or modeling of the auditory cortex), a convincing 3D ‘effect’ can be created by leveraging a sound object’s XYZ position in game space to simulate a sound being heard at a corresponding point in a 3D sound-field.
Yes, 3D audio technologies and solutions have been around for quite some time but the majority of today’s game developers do not implement true 3D positional audio into their games; instead developers tend to stick with standard stereo and 5.1 surround sound as the de facto standard for the vast majority of titles being produced today. Keep in mind that not all 3D audio solutions are alike and really compelling 3D audio typically requires quite a lot of processing resources — precious resources that aren’t always made available for audio processing. With the explosion of multi-core processors for mobile gaming, powerful next-gen consoles arriving later this year, and dedicated audio DSP resources being offered on new GPUs (i.e. AMD’s TrueAudio), the conditions are ripe for the mass adoption and standardized use of compelling and powerful 3D audio solutions by game developers for their game titles across many different platforms.
Here is what 3D sound can provide game developers:
Elevation: placement above and to some degree, below
Azimuth: placement in front, behind, and to the sides
Distance: using elevation and azimuth to define a position for a sound inside a 3D sound-field (i.e. a spherical sound-field wrapped around you) the sound can be pushed out in space with distance to allow improved depth perception (using distance cue processing, 3D room simulation, for example.)
These parameters expand the sound-field beyond what stereo, 5.1, and 7.1 can produce (which are 2D formats). For developers looking to offer a bit more realism and immersion for their players, 3D audio may be the ‘lowest-hanging fruit’ available to do so.
Here are 6 ways developers can use 3D audio to expand gaming experiences:
Enhanced immersion for mobile games:
There are tons of fun games on the small screen (for your mobile phone or tablet) however, 3D audio can be used to create the illusion of a larger game world through the use of an immersive sound-field that envelops the player. While there may be a bit of a disconnect between a small screen and an enveloping, spherical sound-field, 3d audio for mobile gaming can heighten the feeling of being “in the game.”
Ear Monsters, by Ear Games, is a forward-thinking iOS game that employs the use of 3D audio to drive gameplay, rather than visuals. The use of 3D auditory cues to drive gameplay serves to extend the boundaries of the game world beyond the small screen of the mobile device, for example, the player can tap in the general direction of the sound being heard in 3d space (for example, tapping the top of the screen if an attack is heard coming from above.)
Using 3D audio, sound-centric games can:
Help the visually impaired to enjoy gaming. Ear Monsters allows players to play without needing to see what’s happening on screen.
Create interactive stories using voice over and sound effects positioned in a 3D sound-field for an immersive story-telling experience.
Use darkness and lack of visibility to increase reliance on 3D sound cues can make more compelling and immersive horror games.
This list barely scratches the surface with all the possibilities that sound-centric games can offer players.
Improve situational awareness in 3D games (i.e. 3D FPS):
3D audio can improve situational awareness in FPS games by relaying sound cues from certain directions correctly to the player. For example, in most modern combat FPS games that rely use stereo, 5.1, or 7.1 audio delivery, when a player on the ground level is being shot at by a sniper who’s perched high in a tower, the sound of the sniper’s shot does is not heard from above. With 3D audio, the sound of the sniper’s shot would be heard from above, like it’s supposed to.
Reduction of UI graphics:
It is well known that using sound to substitute for UI (i.e. running out of ammo or player health indication using sound prompts) can help remove display clutter. 3D sound can be used to extend the sound cues by using space to indicate more information (i.e. using a sound cue with elevation to indicate a rainstorm is coming.)
Positional 3D audio for multiplayer voice chat:
By leveraging 3D audio, a ’3D radio communications’ of sorts can be created to hear team member communication based on actual position in the game (i.e. if a member of a squad is up on a hill, the radio communication from that player would be elevated.)
3D audio compliments the VR experience:
Conventional stereo, 5.1, or 7.1 audio playback limits the sound field to a two-dimensional plane creating a disconnect with the visual field offered by VR. 3D audio eliminates this problem by allowing a full 3D sound-field to perfectly compliment the 3D stereoscopic visual field; head-tracking (with the Oculus Rift, for example) coupled with 3D audio allows the player to move their head around and expect to hear sound all around them correctly, including sounds from above.
How can developers implement 3D audio in their games?
Games typically use audio middleware solutions (FMOD and Wwise, for example) for their run-time sound engines so developers can utilize 3D audio technologies made available from Dolby, DTS, GenAudio, Auro 3D, and Iosono (check with your audio middleware provider for available solutions.) Encourage your teams to explore using either of the various 3D audio solutions available today to enhance the aural experience for your title(s) beyond what stereo, 5.1, and 7.1 can offer.
篇目2，Creating Audio That Matters
by Caleb Bridge
“Immersion” has become cliché. It’s often just another buzzword when talking about how great a game is, but it’s all too infrequent that those discussing games will actually break down the finer details of what that immersion entails.
Like all pieces of the puzzle that is game design, audio must work in concert with graphics and game mechanics to help immerse the player into gameplay experiences of all shapes and sizes through its ability to convey vast amounts of the detail to the player, often without their knowing.
J White, Martin Stig Andersen, and Thom Kellar, of Visceral Games, Playdead, and Freshtone Games respectively, are three sound designers who have ample experience in creating such audio experiences.
“It can be easy for the media to reveal itself if you’ve got a sound loop or something that becomes annoying. As soon as the game reveals the mechanics, the media and the machinery behind these things is ruined and the player is thrown out of the experience,” said Martin Stig Andersen, sound designer and composer on Limbo.
Limbo was a critical success and fan favorite in 2010, putting many end-of-year awards under its belt.
It was lauded for its striking visual aesthetic, but part of its success came from its ability to make the player feel a sense of isolation and foreboding that is largely the result of top quality sound design.
Andersen and Limbo’s director, Arnt Jensen, teamed up after Andersen saw the game’s initial trailer and felt his unique area of music (electroacoustic music, a non-commercial, almost entirely funding-based form) would add to the experience.
“I watched the trailer and I was really captivated by [the boy's] expressions,” said Andersen. “It reminded me of the aesthetics of light and sound; you have something recognizable and realistic, but at the same time it’s abstract.
“It’s the same as what I love about how we use sound. We have all these slight references that focus on ambiguity, so it’s more about what the listener imagines, rather than what I want to tell them.”
Many can think back to sections of Limbo where they were struck by a certain feeling or sense for what the boy was doing in that place. But one thing Limbo never allowed the player to do was fully understand what was happening. Andersen achieved this by intentionally distorting the sounds of objects in an attempt to make the player think more about what’s going on and how they’re meant to react. This gave it a level of ambiguity that allows the player to “be there and make their own interpretation.”
“The more identity the sounds had, the more I would distort them,” Andersen said. “So I wouldn’t include sounds that gave too strong associations. If we added something that had a strong identity like a voice or an animal, then it would almost destroy the atmosphere. So with that style, Limbo offered an audio and visual atmosphere that can really get into the player’s mind, and make them feel scared, worried or on edge.”
Limbo toys with horror themes throughout, but it is in that genre that exacting sound design is essential in eliciting player emotion and getting inside the player’s head.
One series praised for its ability to do this is Dead Space. J White — a LucasArts veteran who served as sound design lead at Visceral Games on Dead Space 2 — took great pride in the team’s ability to do just that.
“During the course of production, there were meetings I would have with people — they’d come in and have never seen [the section] before. We’d be playing, and they would literally jump in their chair. It genuinely frightened people. It might be kind of mean on my part, but I took a lot of pleasure in that,” said White, when discussing the effects of the sound in Dead Space 2.
Much effort was put into thinking about how they could use the game’s audio elements to enter the player’s mind, sometimes without them even noticing it.
“A fundamental thing that people cannot help but respond to is the sound of the human voice and, even more specifically, the sound of human suffering. It’s just an unavoidable, deep reaction that people have. So one of the elements that we’ll use as part of our soundscapes are the sounds of people in misery. It may be deeply buried, but the human ear and human mind are so attuned to human vocalizations that they’ll respond to it even if it’s just a sub-audible aspect of the overall sound design.”
This sentiment applies to smaller games — even iOS games — in the same way it does to triple-A releases. One Single Life on iOS, which literally gives the player’s character a single life to jump from a series of rooftops to another, is one such example. Thom Kellar, Freshtone Games’ sound designer, wanted to give players the sense that they were actually on that roof, preparing to jump.
“I wanted to use a lot of sounds that would make you feel a bit on edge, so there was wind and a lot of noises you’d hear on a rooftop like birds fluttering and planes flying overhead,” he said. “But it kept coming back to ‘What did that make me feel?’ and it might have been the best or coolest or most wonderful sound, but if it’s not contributing to the emotion or atmosphere of the game, unfortunately it had to go into the basket.”
“Manipulation” can be a dirty word, and is generally used to call out poor attempts by developers to force emotions or reactions from the player. But the reality is, that audio designers try to use their craft to manipulate players in new and different ways as an enhancement.
Many games will use a rousing score to heighten the emotional state of a moment, and while music certainly has its place, it doesn’t have to be the crutch used for affecting sound design games.
“We experimented early on with putting dramatic music over the top, but we felt that emotionally, it was a lot more effective to only have that big score happening towards the end when players were getting to the very edge, and they’d established themselves in the world already,” said Kellar — despite one of his initial roles being to write music for the game.
As a result, One Single Life’s efficient and selective use of music gives it an extra sense of tension. Likewise, Andersen went for a similar feel with Limbo.
“[Jensen and I] rarely like music as an instrument to manipulate the emotions of the player, or manipulate anything really. We both feel that everything should be open to interpretation, and people should be allowed to project their own feelings and emotions into the experience,” he said. “When you allow for that space, and at the same time create something that’s captivating and immerses the player, it lets them let go of those feelings and emotions. So if they’re scared it will probably make them more scared when there’s no music to take them by the hand and tell them how to feel.”
Of course a game like Limbo is aesthetically such that this is possible. Conversely, Uncharted or Mass Effect would be less likely to convey emotional and energetic peaks without the use of music, as it’s such an integral part of the style of the cinematic experiences they’re trying to create.
Of course, that doesn’t mean that there’s no music at all in Limbo, it’s just more abstract, like Andersen’s own music. “It’s the personal interpretation of this boy and his journey through limbo, and instead of playing the action and emphasizing what’s already there, we’re trying to add another dimension… For me, it’s really melancholy that this boy has been subjected to all this violence. It’s just the idea that he’s been habituated to it, and there’s a kind of forgiveness in the music,” he said. But it’s also used to help juxtapose actions and emotions, such as “when you come across the Gatling guns for the first time, there’s this divine music in the background, but if you add something like action music, it becomes so one-dimensional.”
Dead Space 2 featured a more traditional mix of music and effects. Throughout, the apparition of Isaac Clarke’s ex-girlfriend, Nicole, haunts him. In the later chapters, that relationship changes and after they have a reconciliation, the audio relating to her presence transitions from being abrupt and uncomfortable as the player is unaware of what’s coming, to one which indicates to the player that her presence is actually a helping one.
According to White, one of the ways this was portrayed was by incorporating sound effects that became characteristic of her apparitions, and using them differently in ‘ambient one-shots’ — effects that elicit thoughts of a character or event – that would begin to play the closer you came to an encounter with her.
The mixture of music and effects ends up being an orchestrated experience. White gives the example of a room where the player has fought of a phalanx of Necromorphs, who were chasing off government forces, which was followed by another apparition of Nicole.
“We have these competing things we want to pay off on,” he said. “We’ve got these ambient sounds devoted to giving you the sense that you’re following this massive battle, but at the same time, you’ve got this helping presence that’s Nicole coming in. It’s really a matter of choreography to get all those kinds of elements to play together, where you’ve got this sense of the battle, the desertion, the quietness and the girlfriend.”
Sound design can also manipulate the player’s mind to give them a sense of place, for which game audio helps set the foundation of the game experience. Taking care with such base-level environmental effects is part of what can help make a game great.
“We had loud footsteps to tell the audience that the environment is really silent,” said Andersen. “Of course you couldn’t have loud footsteps through the whole game. Their level is very important, so we added different parameters like how long [they boy] has been running, and if he’s been going for a certain amount of time it starts to change. It needs constant variation, otherwise you’d go crazy hearing those footsteps all the time.”
Likewise, such meticulous care with environmental sound design can help give unique gameplay experiences. According to White, Dead Space 2 used its ambient sound to “foreshadow what’s coming next. If the player backtracks to the same place after they’ve cleared the room or beat the boss, it sounds different. In the most flowery terms, the ambient sound in Dead Space 2 is almost a character in itself. It’s such an intrinsic part of the player’s experience, and the psychological experience of playing a game.”
“A good game mechanic will naturally immerse players. Pong has a great game mechanic with boops and beeps, and people are drawn into that because it’s fun,” said White. “These days, as audio designers, we still score that core mechanic in a satisfying way, but we have a lot more subtlety available to us.”
White recognizes that audio is an important, additive element to the overall experience, and part of the role of sound designers is to help enhance those game mechanics and environments in order to immerse the player.
Andersen, who has worked on films in addition to games, gives some perspective to game audio’s immersive nature.
“When I’m playing games it’s so different from films, because there you only hear sounds when it’s important to the viewer. Often in games, you can go up and down or around a sound, you approach it and you pass it, but when you pass it you still hear it as you progress.”
Ultimately, for some sound designers, walking the tightrope of making or breaking immersion is a tough one, says Andersen. “If you create something that’s engaging but at the same time keeps the illusion alive then you’ve got something that’s very strong. But it’s so difficult to do.”
篇目3，Aural nirvana: Bringing outstanding sound to games
By Craig Chapple
As developers look for new ways to immerse players, we analyse the latest tools and techniques open to sound designers
Game audio has come a long way since the early days of Super Mario, where much of the same music was condensed to provide extra sound effects.
Enormous musical scores are now provided on the scale of film, and sound is fast becoming a near central part of all gaming.
Take our Develop Awards finalists. Simogo’s text-based adventure title Device 6 heavily relies on its immersive audio to engage the player with the story and ratchet up the tension during key parts of the narrative.
And the bigger titles like Battlefield 4 and Total War: Rome II rely on realistic sound effects to make the player actually feel like they’re in a warzone, and not just playing a game.
“To me sound is as an important factor for feedback as anything visual,” says Simogo founder Simon Flesser.
“I can’t for my life understand people that say they play games with sound turned off. To me, that’d be just as absurd as turning off the screen and playing with sound only.”
Develop Award winner, audio veteran and the man behind Monument Valley’s sounds, Stafford Bawler, sums up the importance of such effects by offering an example from a large scale project he’d previously worked on.
“I’d just delivered the first proper audio build to the game, replacing all the early tests/placeholder with first pass final audio that represented where the game was going in terms of audio,” he reminisces. “On hearing this for the first time the game’s lead designer was speaking with the audio coder and said ‘It feels like a proper game now, we’re no longer just making builds’.”
New hardware for both mobile and console, as well as a number of constantly refreshed audio technologies such as FMOD and wwise – which recently opened up new and free indie licences – are also helping studios record and implement new styles of audio.
“In previous generations, a limited set of additional tools was available, and it was more restrictive to set up multiple instances due to technological shackles,” says Creative Assembly and Total War senior audio designer Matt McCamley. “As games tech matures, so does in-game audio, chiefly in the way we can deliver an experience to the player.”
One of Creative Assembly’s hotly anticipated titles, Alien: Isolation, is a prime example of how key audio can be in games. The team has had to adopt various new and unique techniques to evoke the original film’s haunting atmosphere and turn it up to ten to keep players on the edge of their seats for its 15-hour or so duration. And from early footage, the game just wouldn’t be the same without sound.
The team’s sound designer Sam Cooper says the game’s audio has been crafted to subtly conjure up dark and scary imagery on even seemingly mundane objects, tapping into the player’s subconscious, in-built fight-or flight responses. He explains this creates an unpredictable soundscape that will keep users second-guessing what their ears are telling them, moving the experience beyond just twitch-play and how good they are at games.
And much of this has been made possible thanks to new consoles and tools.
“With additional processing power and frequently updated middleware, we’re now able to use rich convolution reverbs on new-gen consoles and experiment with emerging DSP such as in-game HRTF/binaural processing,” says Cooper. “We’re far less limited by the hardware with new consoles.”
Another CA sound designer, Total War audio manager Richard Beddow, says Rome II marked the first time the studio had used audio engine wwise, as it offered tools for sound designers in terms of how to play back assets assigned to game events, asset containers and shared resources such as DSP settings. This meant that designers could set up new items, previously the realm of coders, thanks to its visual GUI, creating what Beddow says was a more efficient workflow.
Bawler recommends other tools, such as Unity and Fabric, which he says lent themselves to the way he likes to work, creating ordered hierarchies and the careful management of data trees, allowing him to spend more time on the creative elements of audio creation.
Much like Creative Assembly is experimenting with sound implementations in its big budget console and PC outings, mobile is also home to some interesting techniques.
Simogo’s Flesser had the tough task of integrating audio, along with composer Daniel Olsén, in interactive story Device 6. The audio had to match the reader’s interpretation, while also not losing their attention by playing sounds at the wrong time.
“We wanted to keep a lot of the things up to the players’ imagination, so it was a tough balance to decide what things that are mentioned in the text should be audible or not,” he explains.
“Creating a sense of space was our biggest priority. So working with footsteps to communicate different materials, the right amount of reverb on sounds to communicate size of rooms, mixing things properly, fiddling a lot with EQ on every sound to make everything feel just right within the space that the player was in at the moment.”
Time and budget constraints can of course play a big role in the quality of audio and its variety, particularly if designers are required to go to the source, but mobile in particular throws up a unique set of challenges.
The major limitations presented by smartphones, Bawler says, is the speakers themselves, which often use mono and face away from the listener or are placed just under the player’s hands.
“The Nintendo 3DS has amazing HRTF playback going on with its speakers. It would be awesome if more phones featured this kind of thing,” he states.
“You also have to deal with the fact games on mobile are played all over the place, often crowded or noisy environments, or places where you don’t want to annoy other people. When I was traveling on the tube in London whilst working on Monument Valley, there were about ten people in the carriage all playing their phones. On the one hand I thought it was a shame they were all silent, but I was also thankful that we didn’t have an amusement arcade cacophony going on.”
Flesser adds that getting people to play mobile games with the sound turned on is actually one of the biggest challenges for developers, and suggested stating the requirement early on if it’s central to the experience. He also says designers need to make sure audio can run on older devices, which can limit effects on newew mobiles despite their more advanced capabilities.
“For some reason or another, people tend to play mobile games without sound, and quite a few puzzles in Device 6 requires the player to listen to audial clues,” he says.
“We used low pass filters here and there, and we actually had to turn that off on older devices because it was too demanding and we want to keep a steady framerate and performance to keep the experience smooth.”
In the field
While many games keep to a small size, it could be argued many more are becoming bigger and more ambitious, particularly at the top end. As such, the use of field-recording is becoming more widely used and new libraries are being created as the old sounds from past games become overused.
Bawler suggests sound designers now need to up their game if they’re to create the soundscapes consumers clamour for.
“Going out and recording your own material or investing in boutique libraries specially constructed for games is a must,” he states. “The older libraries have been our bread and butter for years, but they were all recorded for TV, film, radio etcetera – a linear environment where you’re making a ‘photograph’ of sound rather than the dynamic, living, breathing soundscapes we have to create for games.”
Beddow says the audio process isn’t necessarily becoming significantly more difficult as games get bigger, but says such scale has sparked a greater management challenge of the process.
“You have to be smart,” he says. “Largely it’s about the best way to manage the asset-volume and deploy the right resources, both internal and external, to deliver the kind of deeply immersive, cinema-grade soundscape that the modern player, quite rightly, has come to expect.”
Looking forward, many more techniques, both old and new, could become more central to the audio process, such as binaural audio in virtual reality games [See boxout Virtual hearing], for example.
Bawler says he’s excited by the prospect of working with procedural audio in future, particularly given the prominence of various uses of the procedural generation technique evident in the industry at the moment.
“Since very early days, my way of working has been built upon building complex sounds and audio systems from simple components combined in interesting ways,” says Bawler.
“Procedural audio seems like a natural fit for this way of working, and from what I can tell, you still need a sound designer to make sure it’s all sounding as it should.”
篇目4，Video Game Music: Player Immersion (Part I)
by Sande Chen
In Part I of this article, lead audio designer Gina Zdanowicz discusses how video game music enhances a player’s gameplay experience.
Music has always been an important part of entertainment media. As gaming continues to evolve, game music is more heavily relied upon to integrate with the games visuals, to set the scene, and to evoke players’ emotions. Game music should affect the gameplay, and the gameplay should affect the music. The player’s actions influence the interactivity and evolution of the music, just as the music influences the player’s decisions during game play. This combination immerses the player deeper into the gaming experience.
One of the biggest challenges in creating music for video games is in understanding the limits of the game audio engines while trying to provide a seamless interactive experience.
Techniques such as varying tempo, genre, instrumentation and musical notes can set the perfect mood for each area of the game and tell the player exactly what emotions they should feel in those areas.
A layered score is a technique that has several streams with different instruments on each. Those streams should be composed so they are strong on their own and work well with the games visuals, but also be able to be mixed together with the other streams to evolve the music as the game play changes.
Music that builds to a crescendo can signal to the player there is danger just ahead. A boss battle may require more intense music with several layers of instruments and heavy percussion. After the boss is defeated, the music slows down in tempo and the instrumentation thins out, signaling to the player that the danger is no longer imminent.
Super Mario Brothers utilized increased tempo to signal to the player that time is running out, which evokes a sense of urgency to complete the level before running out of time. Dead Space 2 uses ambient soundscapes and a large orchestra to create an eerie, yet larger than life feeling. A small string quartet was used in the game to contrast the large orchestra and to portray the vulnerability of the main character.
Both music and visuals must be well thought out and tightly integrated to create a cohesive and ambient environment. A game’s pace is just as important as the musical build up that allows the player time to feel safe in order to deliver the next tense moment with impact.
When you take a look at how far music in gaming has come, it speaks volumes to its importance in the game industry. Music is no longer just set in the background of the game. Rhythm genre game titles such as Rock Band and Guitar Hero offer a twist on standard game play and offer music as the game.
Gina Zdanowicz is the Founder of Seriallab Studios, Lead Audio Designer at Mini Monster Media, LLC and a Game Audio Instructor at Berkleemusic. Seriallab Studios is a full service audio content provider supplying custom music and sound effects to the video game industry. Seriallab Studios has been involved in the audio development of 60+ titles.
Video Game Music: Player Immersion (Part II)
In Part I of this article, lead audio designer Gina Zdanowicz discusses how video game music enhances a player’s gameplay experience. In Part II, she offers examples of diegetic and non-diegetic music in games.
A technique that is becoming more popular in games is diegetic music. Diegetic music refers to music that originates from within the game world. It’s always nice when a game score can incorporate epic music in the game world, but in real life when you are walking around in a park or on a beach, you don’t hear any music unless you have your headphones on. Diegetic music, although coming from an object within the game, can still set the mood of the environment.
Let’s take a look at some games that use diegetic music to enhance the player’s immersion into the game world.
Fallout 3 makes great use of diegetic and non-diegetic music. Characters in the game have wrist-mounted computers called the Pip-boy 3000, as well as radios scattered around the game world which play music and other broadcasts from in-game radio stations. If the player has their Pip-boy 3000 turned on, they have to be careful of the radio alerting NPC’s to their presence. When the radio function is turned off, non-diegetic background music is played through the game world.
Bioshock also uses a combination of diegetic and non-diegetic music, as well as no music, to set the mood. In the game’s opening scene, the player escapes from the plane wreckage to a lighthouse set on a small rocky island. The lack of music in this scene hints to the player the feelings of a desperate struggle to survive. After the player enters the lighthouse, music starts to fade into the scene. The music is coming from downstairs, which provokes the player to follow the music down the flight of stairs to find the radio in a bathysphere. The music plays two roles in this example: It gives the player a reason to move forward in the game, as well as sets the mood.
The use of diegetic music in Bioshock really underscores the dying city when the player enters a room with a scratchy, 60’s-era record playing. Diegetic music, which is used in place of orchestral background music, can be heard from around corners or can be muffled by doors.
Left 4 Dead allows a player to turn on a jukebox, which will attract a zombie horde. During this attack, instead of non-diegetic music playing, the jukebox music continues to play even if the jukebox is out of visual range.
Grand Theft Auto is, while cliché, a good example of diegetic music. Car radios broadcast different stations and songs that the player can choose to tune into while driving the vehicles in the game. After all, who doesn’t love riding in a car with the music pumping?
A diegetic switch is a technique which can be used to continue the diegetic music throughout the game. The music starts off as a diegetic broadcast from a radio or other source within the game, and as the scene changes, the music switches to a non-diegetic version of the same song and continues to play in that environment.
The Legend of Zelda: Ocarina of Time starts with the diegetic version of Saria’s as it directs the player through the lost woods maze. As the song grows louder, the player is aware that they are moving forward in the right direction. If they player goes off course, the song’s volume decreases, alerting the player to change direction. After the player learns the song, it becomes non-diegetic music in that environment.
As video games evolve, game music must also evolve, allowing for a cohesive integration for a seamless visual and aural experience, which will deeply immerse the player into the game world and keep them there until they press the pause button.
篇目5，How Does In-Game Audio Affect Players?
This study investigated the importance of audio in computer games. To do this an experiment was designed that compared groups of participants that played the same games with and without audio. Participants’ physical responses to the games were recorded via a bioharness that recorded participant’s breathing wave, heart rate, respiration rate and skin temperature.
Analysis of the heart rate and respiration rate of participants showed that those playing games with audio had a higher level of arousal (a combination of heart rate and respiration rate) and demonstrated the immersion capabilities of audio in games.
12 students from the University of Abertay participated in the study. All participants had experience in playing computer games but little or no experience in the games selected for the study.
Experimental trials were conducted in Abertay’s HIVE facility, which provides a six meter rear projection screen, 7.1 surround sound and adjustable lighting providing a great gaming experience.
Participants’ physical responses were recorded with a bioharness (more info available here). The bioharness records a range of physical attributes including heart rate, breathing rate, and skin temperature.
Three games were selected as stimuli for participants to play: Osmos; FlatOut Total Carnage; Amnesia: The Dark Descent.
These games provided a range of genres (racing to survival horror) and play styles (driving to first person shooter).
The aim of Osmos is to propel yourself (bright blue orb), a single-celled organism (mote), into other smaller motes (dark blue orbs) to absorb them. Colliding with a mote larger than yourself will result in being absorbed yourself, resulting in a Game Over. Changing course is done by expelling mass. Due to conservation of momentum, this results in the player’s mote moving away from the expelled mass, but also in their mote shrinking.
FlatOut is a racing game with an emphasis on demolition derby-style races, and features a sophisticated physics engine. Races take place in a range of locations from busy street to storm water drains. Players race against 11 computer-controlled opponents in races consisting of multiple laps. An additional speed boost can be gained by going off jumps and crashing into other cars.
Amnesia: the Dark Descent features an unarmed protagonist exploring a dark and foreboding castle while avoiding monsters and other obstructions, as well as solving puzzles. The player plays and sees through the eyes of Daniel, making their way through the different levels of Brennenburg Castle as the story progresses, starting off in the Rainy Hall and eventually making their way into the deepest depths of the castle in their search for Alexander, the owner of the castle.
The experiment trials began with participants signing a consent form, where they agreed to take part in the experiment, understood what the experiment would consist of, and stated they had no problem with projected images, as required by Abertay’s experiment guidelines. Following the signing of the consent form, participants were instructed how to wear the bioharness. Once the bioharness was on, recording of physical attributes began.
Participants played each of the three games in order: Osmos, FlatOut, and Amnesia. Before starting each game, participants had to be in a resting state; this was monitored with the bioharness. If participants were not in a relaxed state when they began the game, it would be challenging to determine the effects of gaming on physical responses.
Participants alternated between the audio and no-audio conditions, where one participant would play the three games with audio and the next participant would play the three games without audio. All participants were given the same levels/locations in all three games to play. For Osmos, participants were given two levels where they were tasked to become the largest. For FlatOut, all participants were given the first race (four laps) and the same car. Finally, for Amnesia, all participants were tasked with navigating through the open level of the game.
Analysis of participant responses focused on heart rate and respiration rate as variables demonstrating arousal. Participants were divided into two groups based on testing conditions: audio and no-audio.
Illustration 4 shows a comparison of the groups’ heart rate over the duration of playing the first game.
At the start of the game both groups had a heart rate around 75 beats per minute (bpm). The Audio group played the game for longer, but also demonstrates a consistently higher heart rate throughout, and had greater maximum and minimum heart rate values (audio group maximum 84bpm, minimum 68bpm; no-audio group maximum 78bpm and minimum 61bpm).
Further analysis found this difference in heart rate to be significant (Mann-Whitney, p<0.001).
Illustration 5 shows a comparison of groups’ respiration rate during game play (maximum and Minimum respiration rate of 25bpm and 7bpm for audio group, respectively and 21bpm and 2bpm for no-audio group, respectively). Analysis showed, unlike heart rate, a significant difference was not found (Mann-Whitney, p=0.182).
(Illustration 4: Heart rate comparison of groups with and without audio)
(Illustration 5: Comparison of respiration rate of audio and no-audio groups)
Illustration 6 shows a comparison of audio and no-audio groups’ heart rate while playing Game 2 (FlatOut). The graph shows a clear difference between the groups, with the audio group having a much higher heart rate throughout the game compared to the no-audio group (Audio group maximum 91bpm, minimum 57bpm and NO-audio group maximum 77bpm, minimum 64bpm). Further statistical analysis showed this to be a significant difference (Mann-Whitney, p<0.001).
Illustration 7 shows a comparison of the two groups’ respiration rate throughout the game. During game play both group show fluctuation in respiration rates with the audio group having a slightly higher maximum and minimum respiration rate (audio group maximum 24bpm, minimum 14bpm, and no-audio group maximum 22bpm, minimum 12bpm). Statistical analysis found the difference between groups to be significant (Mann-Whitney, p<0.001).
(Illustration 6: Comparison of heart rates)
(Illustration 7: Comparison of audio/no-audio groups’ respiration rate for Game 2)
Illustration 8 shows a comparison of heart rates for the audio and no-audio groups while playing Game 3 (Amnesia: The Dark Descent). The graph shows that the audio group had a consistently higher heart rate throughout the game-play session. The audio and no-audio groups obtained maximum heart rates of 90bpm and 77bpm respectively, and minimum heart rate of heart rates of 74bpm and 52bpm, respectively. Further statistical analysis found the differences in heart rate to be significant (Mann-Whitney, p<0.001).
Illustration 9 shows a comparison of respiration rates throughout game-play for the audio and no-audio groups. The graph shows overall the audio group had a greater respiration rate (maximum rate of 27bpm and minimum rate of 10bpm) compared to the no-audio group (maximum 16bpm and minimum 6bpm). Statistical analysis found that the differences in respiration rate were significant (Mann-Whitney, p<0.001).
(Illustration 8: Comparison of audio and no-audio groups’ heart rate for Game 3)
(Illustration 9: Comparison of audio and no-audio groups’ respiration rate for Game 3)
Firstly, a summary of results: During Game 1, the audio group had a significantly higher heart rate and a slightly higher respiration rate compared to the no-audio group. During Game 2, the audio group had a significantly high heart rate and respiration rate than the no-audio group.
Finally, during Game 3 the audio group had significantly higher heart rate and respiration rate compared to the no-audio group. These findings suggest that the presence of audio in games can increase in player arousal, as shown by an increase in physical responses (heart rate and respiration rate).
Focusing on the games individually, starting with Game 1 (Osmos) the results show this game produced a low heart rate (68bpm) and lowest respiration rate (7bpm) for the audio group. While these values are low both were still higher than that produced by the no-audio group.
These values are low, most likely, because Osmos is a low-stress game. The levels participants were tasked with completing were not challenging, and the audio is relaxing — therefore, participants did not express any frustration.
During Game 2 (FlatOut), the audio group produced the highest heart rate (91bpm) and a slightly higher respiration rate compared to the No-audio group. The rationale for these high values is that FlatOut is an exhilarating racing game, more so with audio (engine noise, crash sound effects, and background rock music). Furthermore, participant performance in the game may affect responses — for example, if a player is winning, they may respond with excitement (increasing both heart and respiration rate), or if a player is losing they may become frustrated, also increasing heart and respiration rate.
Game 3 (Amnesia) best demonstrates the affect of audio in games. The audio group obtained significantly higher heart and respiration rates compared to the no-audio group during game play. This is more impressive given that in the section of game all participants played through, very little happens. There are no enemies and no fighting — just exploration — and the results suggest that audio can yet increase immersion in games.
Reviewing group responses to all games it shows the audio group produced a high maximum heart rate and respiration rate for all games (heart rate Game 1: 84bpm, Game 2: 91bpm and Game 3: 90bpm; respiration rate Game 1: 25bpm, Game 2: 24 and Game 3: 27bpm). The no-audio group produced consistent maximum heart rate values over the three games (Game 1: 78bpm, Game 2: 77bpm and Game 3: 77bpm). The difference in heart rates between the groups shows the effect audio in games has on players.
To further investigate the effect of audio in games another study could be conducted that utilizes the same methodology as the above study, but instead of using commercial games as stimuli could build a bespoke game environment for testing. The advantage of a bespoke testing environment is the ability to control almost everything. Such an environment could be used to investigate aspects such as quality or realism of audio and the responses of players.