当然从Free To Play游戏模式的角度，甚至这些奖励所得在商城消费的比照下瞬间就显得黯然失色，但这并不妨碍游戏进程设定的奖励在当刻对玩家的心理触动，这些奖励是实在的归于玩家个人所有的获得，级别获得了提升，金钱数字获得了增长而一无所有的背包也开始慢慢充盈起来，更关键的是玩家只是点击了鼠标便轻易获得了为数可观的奖励，在没有比照的环境下，这种获得的满足感会被情绪刻意放大，即便奖励的次数多到数不过来和每次给以的量都无法产生实质性的效能，玩家的心态最不济也是欢迎的，只有在遭遇消费型玩家所获得的超级战力所可以比照出来的差异化中玩家才会被拉回残酷的现实，之前再多的奖励终究是抵不过一次道具消费的。
其一是完成任务或进程的奖励（完成游戏任务环节或玩家游戏关卡都伴随着特定资源的奖励，前者诸如Contract Killer完成暗杀任务，后者诸如Parking Mania完成停车关卡获得金币奖励，这部分可以用来购买剩余的游戏关卡）；
其二是游戏进程中的收集（这部分包括两个环节：第一个是设定隐藏的奖励，诸如Batman: Arkham Asylum中到处潜藏着可以收集的战利品，这种类型的奖励能够满足玩家的收集喜好更能够增加游戏的探索属性以及延长游戏的生命周期；第二个是与其他的NPC对抗获得的奖励，诸如Blade Of Darkness与鬼魅对抗获得的生命值，羊羊家园中与狼族对抗获得道具配方、卡片碎片或者在穿越三国中某项特殊技能的解锁获得）；
其三是登录奖励（一般也分为两个层面：第一个是每日登录奖励，这个更多体现在社交游戏中以激励玩家定时返回游戏中，诸如City of Wonder的奖励设定；第二个是连续登录奖励，这个主要是对不间断登录用户提供的阶梯式奖励，诸如德州扑克就以一周为循环时间点，以7天为限对不间断登录的用户进行游戏币奖励）；
其四是成就展示比照奖励（即便是单机型游戏也往往借助第三方的排行榜系统向不同的玩家做成就展示，最典型的就是App Store的Game Center，而游戏设计师Lucas Blair还专门针对游戏成就因素对玩家的激励效能做过分析，Lucas Blair在分析中认为激励玩家自我极限挑战以获得成就和给以玩家预想不到的成就奖励能够更好地提升玩家对游戏项目本身的沉浸性。因此在成就奖励层面，呈现更多的往往是精神层面的满足感，诸如玩Angry Birds某个关卡从1颗星通过晋升到3颗星通过，当然这种模式也存在着弊端，一旦玩家以3颗星的实力通过该关卡，基本上玩家就很难再有可能回到该关卡重复体验了；诸如在篮球传奇中以绝对领先的比分击败PK对手，竞技成分所隐藏了最大推动力就是驱动玩家在竞技的背后不断优化自己的相关属性以期待在竞技场能够一举击败对手）；
其五是从虚拟延伸到现实环境的奖励（这个存在多种模式：第一个是以Kiip为表征的将游戏与实物奖励进行了捆绑，玩家只要完成相应的任务就能获得对应的实物兑现；第二个是以Foursquare为表征的将LBS签到和商用环境进行了捆绑；第三个则更为直接体现为广告游戏，或者直接为实体提供介绍或者将优惠码接入其中，诸如Belly & Brain或者Burger King）
这个就是我们必须要分析的概率和不均衡选项对玩家心理的牵制：赌徒心态，其一是让奖励形态更戏剧化；其二是刺激用户不间断地回到奖励场景中来。游戏设计师Chris Birke和心理学家B.F. Skinner在研究玩家的游戏表现后都认为具有变量因素的随机概率对玩家在不想错过什么的心态中频繁回到游戏更为有利（如果是固定的，并且这个奖励的吸引力不足够大的话，对用户该有的牵制力就相应地小很多）。
完善玩家体验之后，就要考虑其核心/长期奖励。这是玩家所获得的最大奖励。这方面的例子包括主要情节开发、开启新内容的主要里程碑（例如在GTA中进入一个新岛屿），或者获得一个新游戏机制，例如《Crash Bandicoot 3》每一场boss战后出现的机制。由于这些是玩家所得到的最大有形奖励，并将影响其内在体验，因此只能让它们偶尔出现。这可以保证它们的价值，避免干扰玩家。如果人们每隔10分钟就能得到一项新技能，他们迟早会抓狂的，因为他们根本来不及学习新技能。
在20世纪30年代，哈佛大学的一位心理学家Burrhus Frederic “B. F.” Skinner创造了一个操作性条件反射室（游戏邦注：也就是著名的“斯金纳箱”）。该理念很简单：将一只老鼠放在箱子里。让老鼠拉动箱子里的杠杆。有时候提供给老鼠一些食物让它去拉动杠杆。研究怎样的条件能让老鼠更频繁地拉动杠杆。
1973年，心理学家Mark Lepper和Richard Nisbett与幼儿园教师合作组织了一次有趣的实验。他们观察到孩子们会受到内在的激励去画画—-孩子们喜欢根据自己的想法去画画而不需要获得任何额外的报酬。他们将孩子们分成三个群组。他们承诺给予第一个群组的孩子缎带作为画画的奖励。也给予第二个群组的孩子缎带奖励，但却未事先说明。而第三个群组的孩子则独自在那边画画。尽管使用了缎带奖励，但所有三个群组的孩子们都交出了基本上相同数量的图画。之后他们不再提供缎带奖励。第三个群组的孩子就像他们预期的那样继续画出同样数量的图画（毕竟对他们来说什么都未发生变化）。而第一个群组的孩子在图画数量上却出现了大幅度的下降。
篇目2，The Hierarchy of Needs for Rewards in Games
[Chelsey Webster explains how a skillfully implemented reward structure can enhance a player's experience and make a game more enjoyable throughout.]
Rewards are an important feature of any game, and as such, can be found everywhere across all genres. Rewards can come in any shape or size, and when given to the player appropriately can greatly increase their enjoyment of a game, motivating them to continue to play.
Rewards can even have a player do something that they don’t want to do, or don’t enjoy doing (which is quite opposite of the purpose of playing a game). Many players will do annoying, tedious or boring things for rewards. The ability to have a player do what they dislike of their own volition attests to their power. They are an essential tool to a designer.
Designing Reward Structures
Rewards shouldn’t be included on a whim; they must be scheduled and structured. The following diagram is a hierarchy of needs, adapted for reward structures.
Fig 1: Hierarchy of Rewards in Games
Based on Maslow’s Hierarchy of Needs, each section is a prerequisite to the one above, and can greatly enhance the one below. If a section is missing, however, those above it become ineffective.
When designing a reward structure, it is important to work from the bottom of the hierarchy upwards to achieve a solid foundation. It is therefore important that each section is linked to the next.
The higher up in the hierarchy, the more frequently the reward should be distributed; a fractal-like pattern should emerge (i.e. 10 minor rewards = 1 major reward, 10 major rewards = 1 core reward). It doesn’t have to be strictly fractal of course, but minor rewards should considerably outnumber major rewards, and so on.
As we move upwards through the hierarchy, rewards become less intrinsic/more tangible. The bottom of the hierarchy is the most fundamental and intrinsic of rewards to the player – the player experience. At the other end of the scale, the top is purely cosmetic and cannot exist without the rewards listed below – however it works to enhance those below. Think of a scope for a rifle. The scope is great, and makes the gun much better; but it is completely worthless without the gun.
The following gives an overview of each section of the hierarchy and what kind of rewards may be found therein. Examples here will often be of the kind found in adventure or RPG games, but the principles can be applied to any genre. The rewards structure should always be designed in order of importance – from bottom to top.
Rewarding Player Experience:
Before considering anything else in detail, the player experience must be perfected. Good gameplay and immersion must first be obtained through the mechanics, aesthetics and UI (including essential player feedback). It is difficult to define whether this section is fulfilled, as fun is something that can only be experienced. The player must enjoy simply playing around with the game. The Grand Theft Auto franchise achieves this very well – who hasn’t switched it on, then ignored the missions and started wreaking havoc until they’re killed? There is no set goal to this, no rewards given (in fact the player is punished by losing ammo and paying hospital bills), but people do it because the gameplay/mechanics themselves are so much fun and intrinsically rewarding. Few games are fun to the extent that a person will play them with no prospect of rewards at all. Perhaps this is why GTA is so successful.
Core and Long Term Rewards:
Once the player experience is perfected, consider the core/long term rewards. These are very big rewards that the player will have forever. Examples include major plot developments, major milestones in opening new content (reaching a new island in Grand Theft Auto for example), or receiving a new game mechanic, such as those given after every boss fight in Crash Bandicoot 3. As these are the biggest tangible rewards that the player will receive and can affect the intrinsic experience, they should be given only occasionally. This retains their value and prevents annoying the player. People would become frustrated if they earned a new ability every 10 minutes; they’d never learn how to play.
Major and Mid Term Rewards:
These may be levelling up, completing quests, gaining large quantities of XP or money, etc. These rewards are significant to the player at the time but are not end rewards that are kept forever.
They are very rewarding in themselves, but are also means to and end and will eventually be overwritten, replaced or become redundant.
Short Term and Minor Rewards:
These are small, frequent rewards which on their own do not benefit the player, but accumulate towards a bigger reward. Examples include collecting an item, small amounts of money/XP for defeating an enemy, or completing a stage in a mission. Just as Major rewards are means to a bigger end, Minor rewards are also means to an end on a smaller scale (the end being major/mid term rewards).
This is the final section to be considered. At the top of the hierarchy, they are purely cosmetic, serving only as a visual measure of the player’s progress. In some cases they are simply informative, such as numbers appearing on screen when you add to your score, or the squad selection screen in Mass Effect – the player can guess that they’re about halfway through when they’ve filled half of their squad slots.
On the other hand, visual progress elements such as unveiling a map or filling a bar can actively encourage players to do things that they would not do otherwise, as can achievements depending on design. A player might play for 10 minutes longer where they would have otherwise stopped because their XP bar shows them how close they are to levelling up. They may not attempt to round up and defeat 10 enemies with 1 grenade if there was not an achievement challenging them to do so, and may not just run around the entire area if they weren’t revealing their map.
While a game can still be enjoyable if the cosmetic rewards are removed, they will enhance the player experience as players like to see how they’re doing. Visual rewards are vital in helping the player to gauge and anticipate rewards which is an encouraging motivator. Visual rewards never actually affect progress or any other aspect of the game at all.
There is a blurred line between the player feedback (mentioned at the bottom of the hierarchy), and cosmetic rewards (at the top). Player feedback is essential communication to the player, but cosmetic rewards can exist on their own (as with achievements, etc), or tie into player feedback for visual enrichment. One big difference between the two is that omitting essential player feedback can be game breaking, or at least break immersion. An example of essential feedback is seeing your avatar animate as you move around in the world. if it simply floated across the landscape this would completely break immersion and make it unplayable to many players. A problem like this is found to an extent in Oblivion and Fallout 3 (3rd person). The avatar animates but not it doesn’t match its direction if running diagonally, so the character appears to float around. When playing in 3rd person this can completely break immersion. You can see an example of this here.
While it should be designed bottom up, the player will often experience much of this hierarchy from a top down perspective. They will experience cosmetic elements and minor rewards to begin, such as obtaining a task (this is where they are initially given the prospect of rewards), then minor rewards such as collecting items and XP, which lead to major rewards earned by completing the quest or levelling up, and eventually core rewards such as a new mechanic or unlocking a new world.
Application to a successful game:
This hierarchy includes rewards featured in World of Warcraft, a very popular (and notoriously addictive) game.
Fig 2: A Basic reward structure of World of Warcraft
(does not include all rewards)
These are just a selection of the rewards offered, but it is clear that the game is abundant in rewards on every level. The world and the gameplay are immersive and such high customisation of the player experience (classes, specialisations, PVP/PvE, RP, and so on) means there are hundreds of player experience combinations in one game.
Add to this the limitless supply of rewards of all kinds which all tie in to one another and accumulate towards something. Then, to top it off it is peppered with random chance rewards. Even the most mundane creature can drop disproportionately powerful rewards on very rare occasions. There is always a reason to keep playing; this is clearly a very strong reward structure in every aspect.
Application to an unsuccessful game:
As part of an experiment for my dissertation I created a very simple 2D space shooter game. Though it was fit for the purpose of an experiment on rewards and yielded conclusive results, the game would not be popular or successful in the real world. After developing this hierarchy I decided to apply it to my game:
Fig 3: Reward structure of a basic 2D space shooter
The game is strong at the top and gets weaker as we get to the fundamentals of the hierarchy. There is no hope for this game on the market today no matter how good the upper sections are, because the lower sections are not fulfilled. As well as gameplay being lacklustre, an entire section is missing. Tight time and resource limitations meant that the game focused on tangible rewards which could be created quickly and easily. There was no time to invest in excellent gameplay; it just had to work.
It is easy to draw a comparison between this game and popular arcade games from the 70′s. These were successful in their day, though they did include the long term reward of top slot on the leaderboards – but today their main appeal is arguably nostalgia. Todays games have far more sophisticated gameplay and reward structures, so a game like this cannot compete.
Consider the World of Warcraft hierarchy if a section is missing:
Fig 4: World of Warcraft’s reward structure with a section removed.
With this section missing, the hierarchy doesn’t make much sense. Level increments are gone, for example, so now the player must go from level 1 to level 90 (or whatever its new name would be) in one grind. The player would be consistently weak and the suddenly immensely powerful, but only after an inordinate amount of time. Needing 999,000,000,000 XP to reach the next/last level is far too discouraging when the player is getting 20XP per enemy, even more so when their XP bar never seems to budge. Bitesized chunks are essential.
The removal of this section has completely devalued those above. The minor and visual rewards contribute so little to the next goal that they don’t feel very rewarding at all. Players are much happier and more likely to play if they feel that their goal is realistically obtainable.
Quality does not equal quantity, but it is important to have at the very least one reward in each section. Aim to fulfil each section of this hierarchy to a high standard and ensure that they accumulate towards one another. Though it has not been around long enough to be tried and tested on a large scale, following this structure should achieve a strong, structured, rewarding player experience.
While many of the examples throughout this article are based on action adventure/RPG type games, they can be applied across any genre, from puzzle games to sports and racing. By applying this model when designing a reward structure, the designer can ensure that the game has solid foundations and is rewarding enough to make the player feel good. This can ultimately keep them playing the game.
篇目3，The Psychology of Rewards in Games
by Max Seidman
Over the past 2 years I’ve had the privilege of working with at Tiltfactor alongside Dr. Geoff Kaufman, who has a PhD in social psychology and is our head of research at the lab. In addition to the fantastic insights he has made running formal studies on the games we’ve been developing (publication pending on those), I’ve learned amazing things about psychology, formed many theories about how it influences game playing, and developed a list of things game designers need to keep in mind. I’ve devoted a lot of thought recently to one of game design’s most hotly debated psychological topics, reward schedules, and how they relate to what psychologists call the “overjustification effect.” But don’t worry if you’ve never heard of either of these things! I’ll explain them both before discussing what they mean for game designers.
These theories, while most obviously applicable to digital games and tabletop roleplaying games, are important for all types of game designers to understand, especially in an era where even traditional board and card games are becoming more digital.
In the 1930s, Burrhus Frederic “B. F.” Skinner, a psychologist at Harvard, invented an Operant Conditioning Chamber (better known as a Skinner Box). The concept was simple: put a rat in the box. Let the rat pull the lever in the box. Sometimes give the rat a food pellet for pulling the lever. Study what conditions cause the rat to pull the lever more or less often.
Of course Skinner Boxes also include the capacity to shock the creature because scientists are freaky like that.
The applications in game design become clear when you look at what Skinner and other psychologists found. While experimenting with pigeons, researchers found that the pigeons were more likely to push the lever more often when there was a only a chance that they would receive a reward, even more often than when they always received one. Specifically, they were most active when the chance of receiving a reward was 50%. This is an intermittent reward schedule: it gives a chance at payoff for any given action. Specifically, they found that the most effective reward schedule was a variable ratio reward schedule – inserting randomness into the equation such that there could be many pulls of the lever with no payoff, but the average payoff is set.
If this behavior can be extended to humans (and let’s be honest, it can), we can be controlled to perform an activity more often simply by giving us a chance at a reward instead of promising us a guaranteed reward. We tend to know this intuitively: it’s why we gamble. And tons of games already use these principles. For example, slot machines are basically Skinner Boxes for humans. Zynga is notorious for using variable ratio reward schedules in their social games like Farmville. Even World of Warcraft uses them by having killed mobs only drop the loot you need for quests some of the time and not all of the time.
Skinner Box for people
The use of variable ratio reward schedules in game design is often panned, however, for being “nefarious.” Detractors’ reasoning goes: if the game designers had chosen to use simple fixed reward schedules (where for each action there is a promised reward), the players would play a certain amount. With variable ratio reward schedules, the players are more active. Thus, the game designers are “tricking” the players into playing more than they really want to, and usually also spending money.
Before I go into the ethics of using variable ratio reward schedules in games, I want to talk about another crucial phenomenon that is often overlooked in discussions of morality and reward schedules: the overjustification effect.
In 1973 psychologists Mark Lepper and Richard Nisbett conducted a fascinating experiment with kindergarteners. They observed that the children were intrinsically motivated to draw —that the kids enjoyed the activity of drawing pictures for its own sake without any need for external payoff. They split the children into three groups. The students in the first group were promised ribbons as a reward for drawing. The students in the second group were given ribbons, but were not promised them beforehand. The third group was left alone to draw in peace. While the experimenters handed out ribbons, all three groups drew comparable amounts of drawings. Then they stopped giving out ribbons. The third group, as would be expected, continued to draw the same amount (after all, nothing had changed.) The first group, however, had a significant drop off in the amount of drawing they did.
Psychologists theorize (and they’re pretty certain at this point) that what happened is as the first group received ribbons, they shifted their motivations from “I’m drawing because I like drawing” (intrinsic) to “I’m drawing because I want a ribbon.” They were still motivated to draw so long as they were receiving their rewards, but once the rewards were removed the motivation did not snap back to being intrinsic.
This is an often overlooked phenomenon that occurs in many sorts of games. I only have personal anecdotes to share, but if you play many video games I’m sure you’ll be able to identify times when the overjustification effect happened to you.
For myself, after beating Diablo 3 I continued to play and sell my rare items for money on their real money auction house. I built up quite a store of rare items that I was slowly selling off. I played this way for several weeks, but then something changed: Blizzard released a patch making newly dropped rare items much more powerful. This made all of my collection worthless, and I quit after reading about the patch. I haven’t played since. What happened here was I shifted my motivation from “I like playing Diablo 3” to “I like selling items on the auction house.” This was all well and good until Blizzard took away my chance to cash in on my efforts, and then I simply quit.
In another personal example, I played League of Legends, all the while working towards unlocking a single summoner spell: the renowned “Flash” spell. As soon as I unlocked it I quit. I didn’t even play one game with it unlocked. I had shifted my motivation for playing the game to “I want to unlock Flash,” and as soon as I did I found I had no motivation left to play.
Cue 1000 screaming fan boys on how overpowered Flash is and how I should totally play with it.
In both of these cases the games presented me with a goal to work towards that undermined my intrinsic desire to play the game. From a designer’s point of view this is fine, so long as the extrinsic motivations (rewards) remain. So long as I could continue selling items on the Diablo 3 auction house, I would keep playing. Unfortunately, there are two obvious occurrences that take players’ rewards away. The simplest one, exemplified by my League of Legends story, is that the player can achieve the reward. The other problem, as shown in my Diablo 3 anecdote (and many others) is the reward becoming obsolete. Between these two issues it’s difficult for a designer to add fixed reward schedules into a game without risk of the overjustification effect coming into play.
Giving rewards in games is desirable. Designers want to give players rewards for numerous reasons, including reinforcing player behavior, increasing players’ feelings of mastery, scaling difficulty over the course of gameplay, and scaffolding mechanics and player abilities. So how can designers give rewards with the perils of triggering overjustification looming overhead and threatening to make their players lose interest in the game? Lepper and Nisbett’s second group of kindergarteners (the group I didn’t reveal the results for) give us a hint at the answer. Recall that this group was the one that were given ribbons after drawing, but were not promised them beforehand. Once the rewards were removed, this group continued to draw at the same rate as group number one.
This gives designers the solution to providing rewards while avoiding shifting players’ motivations to solely wanting rewards! Don’t let the player know for certain that she’s going to get a reward —also known as variable reward schedules. It’s pretty straightforward: when a player knows what she’s going to receive by means of a reward, she can play only for that reward. It’s much harder to make that motivational shift when the reward is uncertain. This is why games like Dota 2 give items as rewards after games fairly infrequently, at (almost) unpredictable times and with random quality: this way players can enjoy the surprise of the reward without banking on it.
Dota 2 gives payoffs when the “battle experience” bar up at the top fills up. I couldn’t for the life of me tell you how many battle points I have, which means it’s as good as random and Valve is doing a great job!
My point here is that not only are variable ratio reward schedules not inherently evil, they are actually good game design practice. Variable ratio reward schedules don’t trick players into playing more than they really want to, they trick the players’ brains to prevent them from shifting their justification for playing to external factors. This means that these reward schedules keep the players playing because they find the experience itself fun and not just to get to the next reward, AND THIS IS EXACTLY WHAT GAME DESIGN IS! Game design is the process of making systems that players find interacting with fun and worthwhile without external justification. Omitting variable ratio rewards when they could be included is just like ignoring other established design theory (say scaffolded learning or positive feed back loops) that we’ve discussed on Most Dangerous Game Design previously: it’s simply shoddy design practice.
Can designers abuse the psychology of reward schedules? Of course! Most of human psychology can be abused and often is. However, even when using psychology, “tricking” players into having more fun still counts as giving players more fun. I know this entire arguments tends to raise peoples’ hackles, so please feel free to argue with us in the comments, as always!
篇目4，Rewarding Difficulty in Game Design: Intrinsic vs. Extrinsic
by Josh Bycer
This year, two very different games released grand changing patches: Diablo 3 and Payday 2. Both were designed around the same purpose of bringing people back to playing with new challenges and rewards. However, while people enjoyed Diablo 3, there has been an outcry over Payday 2 and this brings up the challenge of rewards and difficulty when it comes to game design.
Making the Right Carrot:
Before we talk about rewards, let’s go over the recent changes. Diablo 3 released patch 2.0, bringing the PC version in line with the console in terms of loot balancing as well as getting the game ready for the upcoming expansion. I spent an entire post talking about the Diablo 3′s 2.0 patch on Game-Wisdom. The short version was completely redoing the loot tables to offer gradual improvements as opposed to polarizing ones. And a redesign of the difficulty settings to spread out the challenge instead of making the game too easy or too difficult.
Payday 2 had what Overkill called the Death Wish update. First, there was a redesign across the difficulty levels of how stealth works. Now it is a lot harder, or even impossible on some maps to take out every guard one at a time and have the level to yourself.
But the big change came in the form of the new difficulty, of course titled Death Wish. Death Wish is Payday 2′s version of the original’s infamous Overkill 145 difficulty or expert mode. Here, when you play a level on death wish, the levels feature slight variations such as unbreakable cameras, more obstacles and harder fights.
Harder fights as in enemies spawn more, move faster, aim quicker have more health and damage and there are two new enemies. Elite units with heavy armor and the affectionately titled “Skulldozer”: a heavy unit wielding a light machine gun with heavy damage.
Overkill has also introduced a new challenge: If you can beat every map on each of the difficulty levels, you’ll receive a unique mask for each. But as we mentioned earlier, people are not happy about Payday 2′s changes.
The problem is twofold, one being the intense difficulty increase that was added in the patch and the other is motivation or rewarding the player for taking on the challenge.
The difficulty increase can be a separate topic and one we’re not going to focus too much on here, but rewarding the player.
What it comes down to is that no matter the player, they want to be rewarded for playing a hard game or harder difficulty. Now I know that there are expert players out there getting ready to respond with “But playing a hard game is its own reward for me,” and they are right. However playing a challenging video game does provide an intrinsic reward.
There is a certain thrill with being able to beat something difficult, we see this in games like Ninja Gaiden Black or Dark Souls. But there is a saying that I keep coming back to: It’s easy to make a hard game, but a challenging one is a different matter.
In order for a hard game to be intrinsically rewarding there are several factors that need to be in place. First is that the game has to be fair to the player in the sense that it can’t be needlessly difficult.
Elements like just raising enemy stat values doesn’t change the game, it just means the player is going to have to spend more time grinding or fighting. Ninja Gaiden Black is a great example of making a game harder by actually changing the enemies you would fight based on your difficulty level. Harder enemies reacted quicker, had new attacks and challenged the player differently.
Another no-no is restricting options. In the sense that if you let the player have a variety of skills, power-ups etc, but then say that there is only one or two ways to beat the game on hard mode that doesn’t work. Because you’re making the game less varied at the higher levels and forces the player into a black and white experience of pass or fail unless they do X.
Moving on let’s talk about extrinsic rewards which is a lot easier to discuss. Basically anything that provides some kind of a bonus or award to the player counts as an extrinsic example. So loot, achievements, even a new avatar picture on your profile is an example. Extrinsic rewards don’t all have to be game changing, but they do need to have an impact of some kind.
The most important aspect behind extrinsic rewards compared to intrinsic, is that they need to always be within the player’s grasp. This is where that compulsion to play ARPGs comes from: That your next piece of loot is always around the corner.
You can see the difference in the right and wrong way of extrinsic rewards when comparing Crackdown to GTA in terms of collectibles. In Crackdown, every agility orb provided both a short and long term reward for collecting them.
While in the GTA series, you only received a reward for collecting certain # of special packages or killing pigeons in GTA 4.
So while the former provided constant rewards, the latter only did after the player performed enough tasks with nothing in between. With that said, let’s go back to Payday 2 and Diablo 3 to see where the differences lie.
Measuring the Stick:
Diablo 3′s biggest change from launch to patch 2.0 was how rewards are delivered. Instead of going long periods of time without finding anything good and then getting that one amazing piece, loot is now designed around a gradual and tighter increase in improvement. So you’re going to find better loot quicker and work your way up to that bad-ass level, instead of just getting there all of a sudden.
This is the right way of building an extrinsic reward system as it constantly motivates the player to move forward. Intrinsically, there is the thrill at playing on Diablo 3′s newly redesigned difficulty levels which combines both intrinsic and extrinsic for a great experience.
The problem with Payday 2 at the moment is that neither side is fully developed with the new Death Wish mode. Intrinsically, the higher difficulty doesn’t feel balanced to the skill trees and equipment currently in the game, and requires an almost robotic like method of playing to stand a chance.
Again extrinsically, the promise of a new mask but only after going through all the heists doesn’t really work. A possible alternative for example, beating each heist unlocked something in game like something for the safe house or a specialty mask in of itself, which could provide some motivation along the way towards the big prize.
For the last point of this piece, I want to touch on the question of what is better: intrinsic or extrinsic value?
This is an interesting point and I feel that this is more about the person rather than the design. For me, I could care less about achievements and more about the challenge involved.
But at the same time, I don’t want to play something that is hard just for the sake of being hard, I want to be challenged with something new and exciting that wouldn’t have been possible on the easier difficulty settings. That doesn’t mean that I don’t like rewards like hats and aesthetic items, but I want them to be attached to a meaningful action.
Understanding how people are motivated to play your game is an important consideration; not everyone wants to play a game for either one or the other. And while having both intrinsic and extrinsic options are great, they need to be paired with a tough but fair difficulty system to truly get the impact right.
篇目5，The Reward series, part 1: The basics of reward
This article is the first part of a series about reward in computer games and all that is related to it. In this article, I’ll explore the basics of rewards. First, I define what a reward is. Then, I talk about what rewards mean for the game as a whole and what happens if a game isn’t rewarding enough. After that, I’ll try to identify the different types of rewards a game designer can use and finally, I’ll discuss some ways in which we can make rewards more effective.
A reward is something you receive and feel positive about. There are three aspect of that definition I believe deserve some explanation: something, receive and feel positive about. And then there is an aspect that is not in the definition and that warrants some explanation because of it. I’ll get to that in a minute,
The term something is vague on purpose. It doesn’t really matter what you receive, as long as you feel positive about it, it’s a reward. It may be something very tangible, like a pouch of coins, or it might be less tangible like a pat on the back or a compliment. It doesn’t matter whether the game designer put it in the game explicitly, nor does it matter whether you are aware of the something. All that matters is that you feel positive about it when you receive it.
The word receive might lead you to believe that I mean that a reward is always given to you, but I don’t. Sure, it might be that someone hands you a healing potion; in that case you received a reward. Or it might be the game that gave you points or some extra time; again, you receive a reward. Or it might be that the developers of the game gave you a worthwhile experience and then that’s the reward you received. But sometimes you just encounter a reward, you just run into it. For example, you see a couple of children playing and it brings a smile to your face. You didn’t really receive that, it wasn’t actually given to you, but it’s still a reward. So, receive might not be the perfect word to use, but it’s the best I could come up with. And we do need a verb in there, because otherwise it isn’t enough; a reward isn’t just something you feel positive about. I feel positive about massages, but unless I actually receive one, it’s not a reward to me. (Giving one might be rewarding too, of course.)
The most important part of the definition is that you feel positive about what you received. I hope that doesn’t need explaining, because I really wouldn’t know how to explain it; it’s just what the word means. I do want to emphasize, though, that we are talking about a feeling here. In other words, what is or is not a reward is completely subjective. If you give me a whack on the head and I say ‘thanks, I really needed that’, then you just gave me a reward. If I repay you in kind, you might be less enthusiastic. The point is that only the one who receives the something can determine whether it’s a reward or not. As game designers we can make an educated guess whether something will be perceived as a reward by the player or not, but we can’t be sure. We might intend for something to be a reward, while it really is not or vice versa: only the player can determine that.
What is missing from the definition is the notion that a reward must be earned. When talking about rewards in (computer) games, I think it is correct to leave this out. Often a reward will be given because the player performed a certain action and yes, in that case the player earned the reward: action = reaction. But that isn’t strictly necessary. If the game is set up to give the player extra money every time a certain random time interval has expires, then the player didn’t really do anything to earn the money, but if she feels positive about receiving it, it’s a reward. (I’m assuming that time elapsed isn’t an important factor in a game, like when playing a turn-based strategy game). It’s likely that most rewards in a game must be earned, but some may not, so earning is not part of the definition. It does have an effect on how the reward might be perceived, though. I’ll discuss that later in this article.
A worthwhile experience
A game must offer what I call a worthwhile experience, otherwise I’m not going to play it. A game must provide an experience that I feel positive about. In other words, a game must be rewarding. The entire reason we decide to play a game, is because we expect to be rewarded for it, to feel that we are having a worthwhile experience.
What exactly that reward is, is of course subjective, but there must be something about the game you enjoy, otherwise you wouldn’t play it. Maybe you play Bejeweled because the sound of falling gems relaxes you. Or maybe you play Quake because killing monsters makes you feel powerful. Or maybe you play Civilization because you enjoy the fantasy of building an empire. Then again, you might prefer to play The Sims because you like the idea that you can take out the trash using nothing but your mouse. Or you might play System Shock because you like being scared shitless. Or maybe you play Paradoxion because it makes you feel smart. Whatever game you play, you play it because you want to get something out of it, something you feel positive about (even if others find it weird).
In order for a game to provide a rewarding experience as a whole, it must contain certain elements that are rewarding. You can’t create a game that constantly kicks me in the shin – something I find extremely annoying – but leaves me feeling that the game as a whole was quite enjoyable. There must be at least something in the game that I would consider a reward, otherwise the game won’t provide me with a worthwhile experience. Of course, that doesn’t mean everything should be a reward, or that nothing can be annoying. After all, I do enjoy a game of football (or soccer, if you prefer) on occasion and that includes a bit of shin kicking, which I’ll just take for granted. But still, for a game to be rewarding, it has to include rewards.
Chores and frustrations
Even a game that offers some rewards might not be rewarding as a whole. One of two things can go wrong. Either the game has too many annoyances or the game has too little rewards. In the first case, playing the game is frustrating and in the second case, playing the game is a chore.
A game is frustrating when it annoys you. Your soldiers keep walking the wrong way, or you’re faced with a puzzle that you just can’t solve, or you lose all the time, or the game is just not supposed to do that! Pound on the keyboard, yell at your friends, throw the controller across the room, it won’t help one bit: the rewards you receive cannot make up anymore for all those things that annoy you about the game. Strangely enough, sometimes we feel compelled to keep playing regardless of the frustration. Maybe it’s because we don’t like the thought of being beaten by a stupid computer game! Or maybe we still hope for that wonderful feeling we had the last time we played this game. Whatever the case may be, there’s really only one thing you should do when a game frustrates you: stop playing it.
Sometimes a game doesn’t annoy you, but you don’t get any rewards either. Such a game is a chore, it’s boring. It’s like ironing your clothes: it isn’t really a punishment, but you don’t feel positive about it either. (No, I’m not saying ironing is a game, it’s just a comparison.) Often, a game becomes a chore when rewards are spread too far apart. You spend endless minutes (or hours, doesn’t really matter since they’re endless anyway) walking around the forest looking for that Magic Chest, but you don’t find anything: no friends, no foes, no chests. Chore, I say. It might also be that what the designer intended as a reward just doesn’t fill you with that positive feeling you are after. I actually felt this way after playing Morrowind for a while. At some point I realised that running errands for some in-game character didn’t feel particularly adventurous anymore. I just did what I did, because I was supposed to do that. I wasn’t playing a hero, I was someone’s whipping boy. (Actually, I was everybody’s whipping boy.) There was too little reward left in the game for me, so I gave it up. Chore: boring.
Types of reward
As game designers, we should be aware of the rewards our games can offer. It helps us to make an educated guess about what players will feel positive about. There are many types of rewards in a game. I’m sure the following list is incomplete, but an incomplete list is better than no list at all, so here goes.
Resource rewards. In games where resources play a role, receiving those resources is often a reward. Resources can be anything: money, food, soldiers, weapons. Including resource rewards in a game is usually not hard to do, because the game requires those resources.
Skill rewards. Some games have explicit systems for letting the player improve. One example is the various skills in role-playing games like strength, stamina and speed. Another example is the technologies in Civilization. Skill rewards give the player a feeling of improvement.
Extension rewards. If a game can end because the player runs out of health or time, then there is room for extension rewards. By giving the player extra health, extra lives or extra time, you extend the time the player can spend on her current game. She’ll consider this a reward, unless of course she already considers your game a chore or a frustration; extension rewards can’t help you out of that one.
Visceral rewards. Graphics, music and sound, when well done, can be very rewarding to the player. Many people enjoy the blood and gore in games like Doom and Carmageddon or the naked ladies in a game of strip poker. A visceral reward doesn’t offer the player anything in terms of the gameplay, but it does enhance the experience.
Accomplishment rewards. When a player accomplishes something in the game that can be a reward by itself: beating an opponent, finishing a level, matching three pink bananas. Accomplishment rewards are tricky, because everyone feels differently about them and what may be an accomplishment to the player in the beginning of the game may just be routine at the end of it.
Motivational rewards. The points a player receives during a game usually have no effect on the gameplay whatsoever, but they do help to motivate the player, to encourage her to score more points. The same goes with that shiny, gold cup with the number one on it you get after winning a race. Cut scenes also fall into this category, although they might offer more than just motivation. An encouraging word from an in-game character might also do the trick.
Of the types of rewards listed above only resource rewards, skill rewards and extension rewards offer the player something in terms of the game itself. Visceral rewards, accomplishment rewards and motivational rewards have no influence on the game itself, but they can add to the experience and thus have an effect on the game as far as the player is concerned. I’ll call the first group gameplay rewards and the second group experience rewards.
Just as important to the game designer as rewards, are the various methods you can use to increase the effectiveness of your rewards. I call these methods reward intensifiers. They don’t add new rewards to the game, but they do make the rewards you receive taste even sweeter, they make you feel more positive about them. Again, the following list is not complete, but it’s better then no list.
Increased benefits. A simple way to intensify a reward is to increase the benefits the player receives from the reward. For gameplay related rewards, this can mean things like more money, more strength or more time. For experience rewards it might mean more blood and gore or more points. (I don’t think you can apply increased benefits to accomplishment rewards.) Increased benefits have only limited effectiveness; there is a point beyond which the player just won’t feel more positive for receiving more benefits. Also, with gameplay rewards increased benefits can upset the balance of the game, so be careful.
Anticipation. If all the characters a player meets during her quest keep talking about that beautiful Magic Gemstone, then it’s likely she wants to find that much sought after item. When she finally does find it, she’s probably going to be very thrilled and tell everyone she is now the owner of the Magic Gemstone. Without the anticipation, she might just pick up the gemstone, put it in her backpack and never give it another thought. Anticipation can also come from outside the game, for example, when all your friends keep talking about that cool cut scene that’s coming up after you beat the Big Annoying Boss.
Accomplishment. Accomplishment can be a reward in itself, but it can also serve to intensify other rewards. When the player just walks into the forest, walks around a bit and then finds the Magic Gemstone, it might not feel very special to her, despite the anticipation. On the other hand, if she has to beat a lot of monster for it, or if she went through great trouble to discover the location of the Magic Gemstone, then she’ll feel a lot better about herself for finding it.
Prize. If you offer a reward as a prize, then that implies that the player has earned the reward. Picking up health packs that are scattered throughout the level often doesn’t feel like receiving a prize, but getting an extra life for finishing the level might. Prizes and accomplishments complement each other nicely. The player already feels rewarded because of her accomplishment and by offering her another reward, making that reward the prize, she’ll surely be left with a positive feeling.
A reward is something you feel positive about. Rewards are essential to a game, if you don’t have enough of them your game may become a chore or even frustrating. If you use rewards just right, though, you create a worthwhile experience. There are many kinds of rewards you can use for your game. Just as important, there are ways to intensify those rewards.
篇目6，Paul Williams is undertaking a PhD in Cognitive Psychology at the University of Newcastle, under the supervision of Dr. Ami Eidels. He is interested in developing online gaming platforms suitable for the investigation of cognitive phenomena, and is currently focused on refining and implementing a novel paradigm to study the behavioral phenomenon known as the “hot hand.”
Balancing Risk and Reward to Develop an Optimal Hot-Hand Game
This paper explores the issue of player risk-taking and reward structures in a game designed to investigate the psychological phenomenon known as the ‘hot hand’. The expression ‘hot hand’ originates from the sport of basketball, and the common belief that players who are on a scoring streak are in some way more likely to score on their next shot than their long-term record would suggest. There is a widely held belief that players in many sports demonstrate such streaks in performance; however, a large body of evidence discredits this belief. One explanation for this disparity between beliefs and available data is that players on a successful run are willing to take greater risks due to their growing confidence. We are interested in investigating this possibility by developing a top-down shooter. Such a game has unique requirements, including a well-balanced risk and reward structure that provides equal rewards to players regardless of the tactics they adopt. We describe the iterative development of this top-down shooter, including quantitative analysis of how players adapt their risk taking under varying reward structures. We further discuss the implications of our findings in terms of general principles for game design.
Key Words: risk, reward, hot hand, game design, cognitive, psychology
Balancing risk and reward is an important consideration in the design of computer games. A good risk and reward structure can provide a lot of additional entertainment value. It has even been likened to the thrill of gambling (Adams, 2010, p. 23). Of course, if players gamble on a strategy, they assume some odds, some amount of risk, as they do when betting. On winning a bet, a person reasonably expects to receive a reward. As in betting, it is reasonable to expect that greater risks will be compensated by greater rewards. Adams not only states that “A risk must always be accompanied by a reward” (2010, p. 23) but also believes that this is a fundamental rule for designing computer games.
Indeed, many game design books discuss the importance of balancing risk and reward in a game:
* “The reward should match the risk” (Thompson, 2007, p.109).
* “… create dilemmas that are more complex, where the players must weigh the potential outcomes of each move in terms of risks and rewards” (Fullerton, Swain, & Hoffman, 2004, p.275).
* “Giving a player the choice to play it safe for a low reward, or to take a risk for a big reward is a great way to make your game interesting and exciting” (Schell, 2008, p.181).
Risk and reward matter in many other domains, such as stock-market trading and sport. In the stock market, risks and rewards affect choices among investment options. Some investors may favour a risky investment in, say, nano-technology stocks, since the high risk is potentially accompanied by high rewards. Others may be more conservative and invest in solid federal bonds which fluctuate less, and therefore offer less reward, but also offer less risk. In sports, basketball players sometimes take more difficult and hence riskier shots from long distance, because these shots are worth three points rather than two.
Psychologists, cognitive scientists, economists and others are interested in the factors that affect human choices among options varying in their risk-reward structure. However, stock markets and sport arenas are ‘noisy’ environments, making it difficult (for both players and researchers) to isolate the risks and rewards of any given event. Computer games provide an excellent platform for studying, in a well-controlled environment, the effects of risk and reward on players’ behaviour.
We examine risk and reward from both cognitive science and game design perspectives. We believe these two perspectives are complementary. Psychological principles can help inform game design, while appropriately designed games can provide a useful tool for studying psychological phenomena.
Specifically, in the current paper we discuss the iterative, player-centric development (Sotamma, 2007) of a top-down shooter that can be used to investigate the psychological phenomenon known as the ‘hot hand’. Although the focus of this paper is on the process of designing risk-reward structures to suit the design requirements of a hot-hand game, we begin with an overview of this phenomenon and the current state of research. In subsequent sections we describe three stages of game design and development. In our final section we relate our findings back to more general principles of game design.
The Hot Hand
The expression ‘hot hand’ originates from basketball and describes the common belief that players who are on a streak of scoring are more likely to score on their next shot. That is, they are on a hot streak or have the ‘hot hand’. In a survey of 100 basketball fans, 91% believed that players had a better chance of making a shot after hitting their previous two or three shots than after missing their previous few shots (Gilovitch, Vallone, & Tversky, 1985).
While intuitively these beliefs and predictions seem reasonable, seminal research found no evidence for the hot hand in the field-goal shooting data of the 1980-81 Philadelphia 76ers, or the free-throw shooting data of the 1980-81 and 1981-82 Boston Celtics (Gilovitch et al., 1985). With few exceptions, subsequent studies across a range of sports confirm this surprising finding (Bar-Eli, Avugos, & Raab, 2006) – suggesting that hot and cold streaks of performance could be a myth.
However, results of previous hot hand investigations reveal a more complicated picture. Specifically, previous studies suggest that a distinction can be made between tasks of ‘fixed’ difficulty and tasks of ‘variable’ difficulty. A good example of a ‘fixed’ difficulty task is free-throw shooting in basketball. In this type of shooting the distance is kept constant, so each shot has the same difficulty level. In a ‘variable’ difficulty task, such as field shooting during the course of a basketball game, players may adjust their level of risk from shot-to-shot, so the difficulty of the shot varies depending on shooting distance, the amount of defensive pressure, and the overall game situation.
Evidence suggests it is possible for players to get on hot streaks in fixed difficulty tasks such as horseshoe pitching (Smith, 2003), billiards (Adams, 1996), and ten-pin bowling (Dorsey-Palmenter & Smith, 2004). In variable difficulty tasks, however, such as baseball (Albright, 1993), basketball (Gilovitch et al., 1985), and golf (Clark, 2003a, 2003b, 2005), there is no evidence for hot or cold streaks – despite the common belief to the contrary.
The most common explanation for the disparity between popular belief (hot hand exists) and actual data (lack of support for hot hand) is that humans tend to misinterpret patterns in small runs of numbers (Gilovitch et al., 1985). That is, we tend to form patterns based on a cluster of a few events, such as a player scoring three shoots in a row. We then use these patterns to help predict the outcome of the next event, even though there is insufficient information to make this prediction (Tversky & Kahneman, 1974). In relation to basketball shooting, after a run of three successful shots, people would incorrectly believe that the next shot is more likely to be successful than the player’s long term average. This is known as the hot-hand fallacy.
A different explanation for this disparity suggests shooters tend to take greater risks during a run of success, for no loss of accuracy (Smith, 2003). Under this scenario, a player does show an increase in performance during a hot streak – as they are performing a more difficult task at the same level of accuracy. This increase in performance may in turn be reflected in hot hand predictions, however would not be detected by traditional measures of performance. While this hypothetical account receives tentative support by drawing a distinction between fixed and variable difficulty tasks (as the hot hand is more likely to appear in fixed-difficulty tasks, where players cannot engage in a more difficult shot), this hypothesis requires further study.
Unfortunately, trying to gather more data to investigate the hot hand phenomenon from sporting games and contests is fraught with problems of subjectivity. How can one assess the difficulty of a given shot over another in basketball? How can one tell if a player is adopting an approach with more risk?
An excellent way to overcome this problem is to design a computer game of ‘variable’ difficulty tasks that can accurately record changes in player strategies. Such a game can potentially answer a key question relevant to both psychology and game design – how do people (players) respond to a run of success or failure (in a game challenge)?
The development of this game, which we call a ‘hot hand game’, is the focus of this paper. Such a game requires a finely tuned risk and reward structure, and the process of tuning this structure provides a unique empirical insight into players risk taking behaviour. At each stage of development we test the game to measure how players respond to the risk and reward structure. We then analyse these results in terms of player strategy and performance and use this analysis to inform our next stage of design.
This type of design could be characterised as iterative and player-centric (Sotamaa, 2007). While the game design in this instance is simple, due to the precise requirements of the psychological investigation, player testing is more formal than might traditionally be used in game development. Consequently, changes in player strategy can be precisely evaluated. We find that even subtle changes to risk and reward structures impact on player’s risk-taking strategy.
Game Requirements and Basic Design
A hot hand game that addresses how players respond to a run of success or failure has special requirements. First and foremost, the game requires a finely-tuned risk and reward structure. The game must have several (5-7), well-balanced risk levels, so that players are both able and willing to adjust their level of risk in response to success and failure. If, for example, one risk level provides substantially more reward than any other, players will learn this reward structure over time, and be unlikely to change strategy throughout play. We would thus like each risk level to be, for the average player, equally rewarding. In other words, regardless of the level of risk adopted, the player should have about the same chance of obtaining the best score.
The second requirement for an optimal hot hand game is that it allows measurement of players’ strategy after runs of both successes and failures. If people fail most of the time, we will not record enough runs of success. If people succeed most of the time, we will not observe enough runs of failure. Thus, the core challenge needs to provide a probability of success, on average, somewhere in the range of 40-60%.
The game developed to fulfil these requirements was a top-down shooter developed in Flash using Actionscript. While any simple action game based on a physical challenge with hit-miss scoring could be suitably modified for our purposes, a top-down shooter holds several advantages. Firstly, high familiarity with the style means the learning period for players is minimal, supporting our aims of using the game for experimental data collection. Secondly, the simple coding of key difficulty parameters (i.e. target speeds and accelerations) allows the reward structure to be easily and precisely manipulated. Lastly, a ‘shot’ of a top-down shooter is analogous to a ‘shot’ in basketball, with similar outcomes of ‘hit’ and ‘miss’. This forms a clear and identifiable connection between the current experiment and the origins of the hot hand.
In the top-down shooter, the goal of the player is to shoot down as many alien spaceships as possible within some fixed amount of time. This means the number of overall shots made, as well as the number of hits, depend on player performance and strategy. The game screen shows two spaceships, representing an alien and the player-shooter (Figure 1). The simple interface provides feedback about the current number of kills and the time remaining. During the game the player’s spaceship remains stationary at the bottom centre of the screen. Only a single alien spaceship appears at any one time. It moves horizontally back-and-forth across the top of the screen, and bounces back each time it hits the right or left edges. The player shoots at the alien ship by pressing the spacebar. For each new alien ship the player has only a single shot with which to destroy it. If an alien is destroyed the player is rewarded with a kill.
Figure 1: The playing screen.
Each alien craft enters from the top of the screen and randomly moves towards either the left or right edge. It bounces off each side of the screen, moving horizontally and making a total of eight passes before flying off. Initially the alien ship moves swiftly, but it decelerates at a constant rate, moving more slowly after each pass. This game therefore represents a variable difficulty task; a player can elect a desired level of risk as the shooting task becomes less difficult with each pass of the alien.
The risk and reward equation is quite simple for the player. The score for destroying an alien is the same regardless of when the player fires. Since the goal is to destroy as many aliens as possible in the game period, the player would benefit from shooting as quickly as possible; shooting in the early passes rewards the player with both a kill and more time to shoot at subsequent aliens. However, because the alien ship decelerates during each of the eight passes, the earlier a player shoots the less likely this player will hit the target. If a shot is missed, the player incurs a 1.5 second time penalty. That is, the next alien will appear only after a 1.5 second delay which is additional to the interval experienced for an accurate shot.
Stage One–Player Fixation
After self-testing the game, we deployed it so that it could be played online. Five players were recruited via an email circulated to students, family and friends. Players were instructed to shoot down as many aliens as possible within a given time block. They first played a practice level for six minutes before playing the competitive level for 12 minutes. The number of alien ships a player encountered varied depending on the player’s strategy and accuracy. A player could expect to encounter roughly 10 alien ships for every 60 seconds of play. At the completion of the game the player’s response time and accuracy were recorded for each alien ship.
Recall that one of the game requirements was that players take shots across a range of difficulty levels, represented by passes (later passes mean less difficult shots)–this simple test provides evidence that a player is willing to explore the search space and alter her or his risk-taking behaviour throughout the game. Typical results for Players one and two are shown in Figure 2. In general players tended to be very exploratory during the practice level of the game, as indicated by a good spread of shots between alien passes one and eight. During the competitive game time however players tended to invest in a single strategy, as indicated by the large spikes seen in the competition levels of Figure 2. This suggests that players, after an exploratory period, attempted to maximise their score by firing on a single, fixed pass.
Figure 2: Results for two typical players in Stage one of game development. The upper row shows data for Player 1, and the bottom row shows data for Player 2. The left column presents the frequency (%) of shots taken on each pass in the practice level, while the right column indicates the frequency (%) of shots taken on each pass in the competition level. Note that players experimented during the practice level, as evidenced by evenly spread frequencies across passes in the left panels, but then adopted a fixed strategy during the competitive block, as evidenced by spikes at pass 4 (Player 1) and pass 5 (Player 2). For each panel, n is the overall number of shots attempted by the player in that block, m is the mean firing pass, and sd is the standard deviation of the number of attempted shots.
In experimental terms, this fixation on a single strategy is known as ‘investment’. At the end of the game the players reported that, because of the constant level of deceleration, they could always shoot when the alien was at a specific distance from the wall if they stuck to the same pass. Players thus practiced a timing strategy specific to a particular alien pass (i.e., a specific difficulty level). The number of kills per unit time (i.e., the reward) was therefore always highest for that player when shooting at the same pass. In the example graphs (Figure 2), one player ‘invested’ in learning to shoot on pass four, the other, on pass five. This type of investment runs counter to one requirement of a hot-hand game, creating a major design flaw that needed to be fixed in the next iteration.
Stage Two–Encouraging Exploratory Play
The aim of the second stage of design was to overcome the problem of player investment in a single strategy. The proposed solution was to vary the position of the player’s ship so that it no longer appeared in the same location at the centre of the screen but rather was randomly shifted left and right of centre each time a new alien appeared (Figure 3). Thus, on each trial, the shooter’s location was sampled from a uniform distribution of 100 pixels to the left or to the right of the centre. This manipulation was intended to prevent the player from learning a single timing-sequence that was always successful on a single pass (such as always shooting on pass four when the alien was a certain distance from the side of the screen).
Figure 3: The screen in Stage two of game development. The blue rectangle appears here for illustration purposes and indicates the potential range of locations used to randomly position the player’s ship. It did not appear on the actual game screen.
Once again we deployed an online version of the game and recorded data from six players. Players once again played a practice level for six minutes before they played the competitive level for 12 minutes.
The results for all individual players in the competitive game level are shown in Figure 4. Introducing random variation into the players firing position significantly decreased players’ tendency to invest in and fixate on a single pass. This decrease in investment is highlighted by the increase in the variance seen in Figure 4 when compared to Figure 2. Thus, the slight change in gameplay had a significant effect on players’ behaviour, encouraging them to alter their risk-taking strategy throughout the game. Furthermore, this change helps to meet the requirements necessary for hot hand investigation.
Figure 4: Individual player results for the competition level in Stage two testing. Player’s tendency to fire on a single pass in the competition level has been significantly reduced compared to Stage One, as evidenced by the reduction in spikes and, in most cases, increase in variance. For each panel, n is the overall number of shots attempted by the player in that block, m is the mean firing pass, and sd is the standard deviation of the number of attempted shots.
In Figure 5 we present data averaged across all players for both the practice and competition levels. This summary highlights how the game’s reward structure influenced player strategy throughout play. The left column corresponds to the practice level (not shown in Figure 4), while the right column corresponds to the competition level.
Figure 5: Average player results for Stage two. The left column presents the frequency (%) of shots taken on each pass in the practice level, while the right column indicates the frequency (%) of shots taken on each pass in the competition level. For each panel, m is the mean firing pass and n is the overall number of shots attempted by all players in that block. A comparison of mean firing pass for practice and competition levels highlights that as the game progressed, players fired later.
An inspection of Figure 5 highlights the fact that players’ shooting strategy altered in a predictable manner as the game progressed. For example, the mean firing pass for the practice level (m = 5.8) was smaller than that seen in the competition level (m = 6.21). Thus players tended to shoot later in the competition level. This suggests that the reward structure of the game was biased towards firing at later passes, and that as players became familiar with this reward structure they altered their gameplay accordingly.
Given the need to minimise such bias for hot hand investigation, we examined the risk and reward structure on the basis of average player performance. We were particularly interested in the probability of success for each pass, and how this probability translated into our reward system. Recall that firing on later passes takes more time but is also accompanied by a higher likelihood for success. As the aim of the hot hand game is to kill as many aliens as possible within a 12 minute period, both the probability of hits as well as the time taken to achieve these hits are important when considering the reward structure.
We therefore analysed how many kills per 12-minute block the average, hypothetical player would make if he or she were to consistently fire on a specific pass for each and every alien that appeared. For example, given the observed likelihood of success on pass one, how many kills would a player make by shooting only on pass one? How many kills on pass two, and so on. Results of this examination are reported in Figure 6. Figure 6A shows the average number of shots taken by players on each pass of the alien (overall height of bar) along with the average number of hits at each pass (height of yellow part of the bar). Figure 6B uses this data to plot the observed probability of success and shows that the probability for success is higher for later passes. This empirically validates that later passes are in fact ‘easier’ in a psychological sense.
Figure 6: Averaged results and some modelling predictions from Stage two of game development. In Panel A, the frequency (%) of shots attempted on each pass is indicated by the overall height of each bar. The proportion of hits and misses are indicated in yellow and blue. Panel B depicts the average probability of a hit for each pass, given by the number of hits out of overall shot attempts. Based on the empirical results, Panels C and D show the predicted number of successful shots if players were to consistently shoot on only one pass for the entire game (see text for details).
These probabilities allow empirical estimation of the number of total kills likely to be attained by the hypothetical average player if they were to shoot on only one pass for an entire 12 minute block. By plotting the number of total kills expected for each pass number, we produce an optimal strategy curve for the current game, as shown in Figure 6C. The curve is monotonically increasing, indicating that the total number of kills expected of an average player increases as the pass number increases. In other words, players taking less difficult shots are expected to make more hits within each game. The reward structure is clearly biased toward later passes, which validates the change in player strategy (i.e. firing on later passes) as the game progressed. As the players became accustomed to the reward structure, their strategy shifted accordingly to favour later, easier shots.
In game terms it might be considered an exploit to shoot on pass eight. Figure 6C indicates that consistently firing on pass 8 would clearly result in the greatest number of kills, making it the ‘optimal’ strategy for the average player. Given that an exploit of this kind reduces the likelihood of players to fire earlier in response to a run of successful shots, the current design still failed to meet the requirements for our hot hand game.
One simple adjustment to overcome this issue was to reduce the penalty period after an unsuccessful shot. While the current time penalty for a missed shot was set to 1.5 seconds, the ability to vary this penalty allows a deal of flexibility within the reward structure. Given that players make many more shots, and thus many more misses, if they choose to fire on early passes – decreasing the time penalty for a miss substantially increases the relative reward for firing on early passes.
In line with this thinking, Figure 6D shows the predicted number of kills in 12 minutes for the average player if the penalty for missing is reduced from 1.5 seconds to 0.25 seconds. This seemingly small change balances the reward structure so that players are more evenly rewarded, at least for passes three to eight. Estimation of accuracy rate on passes one and two were based on a small number of trials, which makes them problematic for modelling; participants avoided taking early shots, perhaps because the alien was moving too fast for them to intercept. Allowing for players to fire on passes three to eight still provided us with sufficient number of possible strategies for a hot hand investigation.
Stage Three–Balancing Risk and Reward
In stage two of our design we uncovered an exploitation strategy in the risk and reward structure of the game where players could perform optimally by shooting on pass eight of the alien. We suspect this influenced players to fire at later passes of the alien, particularly as the game progressed. Using empirical data to model player performance suggested that reducing the time penalty for a miss to 0.25 seconds would overcome this problem.
A modified version of the game, with a 0.25 seconds penalty after a miss, was made available online and data were recorded from five players. Averaged results show that players shot at roughly the same mean pass of the alien in the practice level and the competitive level (Figure 7). This pattern is in contrast with Figure 4, which highlighted a tendency for players to fire at later passes in the 12 minute competitive level. This data confirms the empirical choice of a 0.25 second penalty, and provides yet another striking example of how subtle changes in reward structure may influence players’ behaviour.
Figure 7: Average player results for Stage three of game development. The left plot presents the frequency (%) of shots attempted on each pass in the practice level, while the right plot indicates the frequency (%) of shots attempted on each pass in the competition level. For each panel, m is the mean firing pass and n is the overall number of shots taken by all players in that block. As indicated by the mean firing pass, under a balanced reward structure players no longer attempted to shoot on later passes as the game progressed.
Recall that we began the development of a hot-hand game with the requirement that for each level of assumed risk the game should be equally rewarding (total number of kills) for the average player. By balancing the reward structure, the design from stage three is now consistent with this requirement for investigating the hot hand.
Finally, we required the game to have an overall level of difficulty such that players would succeed on about 40-60 percent of attempts. Performance within this range would allow us to compare player strategy in response to runs of both success and failure. That is, testing for both hot and cold streaks. As highlighted by Figure 8, the overall probability of success does indeed meet this criteria; the overall probability of success (hits) was 43%. Thus, the game now meets the essential criteria required to investigate the hot hand phenomenon.
Figure 8: Averaged results from the competition level of Stage three of game development. In Panel A, the frequency (%) of shots attempted on each pass is indicated by the overall height of each bar. The proportion of hits and misses are indicated in yellow and blue. Panel B depicts the average probability of success for each pass, given by the number of hits out of overall shot attempts. In Panel B, ps is the overall probability of success (hits).
We set out to design a computer game as a tool for studying a fascinating and widely studied psychological phenomenon called the ‘hot hand’ (e.g., Gilovitch, Valone, & Tversky, 1985). For this we needed a game that allowed us to investigate player risk-taking in response to a string of successful or unsuccessful challenges.
We designed a simple top-down shooter game where players had a single shot at an alien spacecraft as it made eight passes across the screen. During the game the player faced this same challenge a number of times. The goal of the game was to kill as many aliens as possible in a set amount of time. The risk in the gameplay reduced on each pass as the alien ship slowed down. Shooting successfully on earlier passes rewarded the player with a kill and made a new alien appear immediately. Missing a shot penalised the player with an additional wait time before the next alien appeared.
As a hot hand game it was required to meet specific risk and reward criteria. Players should explore a range of risk-taking strategies in the game and they should be rewarded in a balanced way commensurate with this risk. We also wanted the game challenge to have an average success rate roughly equal to the failure rate, between 40 and 60 percent so that we could use the game to gather data about player’s behaviour in response to both success and failure.
To achieve our objective we developed the game in an iterative fashion over three stages. At each stage we tested an online version of the game, gathering empirical data and analysing the players’ strategy and performance. In each successive stage of design we then altered the game mechanics so they were balanced in a way that met our specific hot hand requirements. The design changes and their effects are summarised in Table 1.
Table 1. A summary of changes to design in each of the stages and the effect of these changes on meeting the hot hand requirements.
Books on game design tend to prescribe an iterative design process. Iterative processes allow unforseen problems to be addressed in successive stages of design. This is especially important in games where the requirements for the game mechanics are typically only partially known and tend to emerge as the game is built and played. Salen and Zimmerman describe this iterative process as “play-based” design and also emphasise the importance of “playtesting and prototyping” (2004, p. 4). For this purpose successive prototypes of the game are required. Indeed we began with only high-level requirements and used this same iterative, prototyping approach to refine our gameplay.
The main difference in our approach is that we more formally measured player’s strategies and exploration behaviours in each stage of design. Given that our game requirements are rather unique, it is unlikely that subjective feedback alone would have allowed us to make the required subtle changes to game mechanics. For example, during the initial testing of the game we found that players tended to invest in a single playing strategy. Further analysis also revealed a potential exploit in the game as players could easily optimise their total number of kills by shooting on the last pass of each alien ship.
The issue of exploits in games is often debated in gaming circles and is also well studied in psychology. Indeed trade-offs between exploitation and exploration exist in many domains (e.g., Hills, Todd, & Goldstone, 2008; Walsh, 1996). External and internal conditions determine which strategy the organism, or the player, will take in order to maximise gains and minimise loses. For example, when foraging for food, the distribution of resources matters. Clumped resources lead to a focused search in the nearby vicinity where they are abundant (exploitation), whereas diffused resources lead to broader exploration of the search space.
Hills et al. showed that exploration and exploitation strategies compete in mental spaces as well, depending on the reward for desired information and the toll incurred by search time for exploration. In the context of our game, a shooting strategy of consistently attempting the easiest shooting level produced the highest reward. This encouraged players to drift toward later firing as the game progressed, and in turn inhibited players from exploring alternate (earlier firing) strategies. It is unlikely we could have predicted this without collecting empirical data from players.
A further advantage of gathering empirical data was that it allowed us to remodel our reward structure based on precise measures of player performance. In stages one and two players lost 1.5 seconds each time they missed an alien. In stage three we reduced this penalty to 0.25 seconds based on our analysis and modelling of player behaviour. This relatively minor change was enough to change players’ behaviour and encourage them to risk earlier shots at the alien. The fact that our game is quite simple in nature reinforces both the difficulty and importance of designing a well-balanced risk and reward structure.
Another common principle referred to in game literature is player-centred design which is defined by Adams as “a philosophy of design in which the designer envisions a representative player of a game the designer wants to create.” (2010, p. 30). Although player-centred design is often a common principle referred to in game-design texts there is some suggestion that design is often based purely on designer experience (Sotamaa, 2007). Involving players in the design process typically involve more subjective feedback from approaches such as focus groups and interviews which have been generally used in usability design. In our study, when designing even a simple game challenge it is clear that the use of empirical data to measure how players approach the game and how they perform can be another vital element in balancing the gameplay.
We also recognise some dangers with this approach, as averaging player performance can hide important differences between players. It would be nice to have a model of an ideal player but it is unlikely such a player exists. In fact there are many different opinions about who the ‘player’ is (Sotamaa, 2007). The empirical data therefore need to be gathered from the available players’ population. If there are broad differences among these players then it may require the designer to sample different groups, for example, a group of casual players and a group of hard-core gamers.
Importantly for future research, the game design at which we arrived is now suitable to investigate the hot hand phenomena. Such a game can potentially answer a number of questions:
1. How do players respond to a run of success or failure in a game challenge?
2. Will a player take on more difficult challenges if they are on a hot streak?
3. Will they lower their risk if they are on a cold streak?
4. How will this variable risk level impact on their overall measure of performance?
5. How can the hot hand principle be used in the design of game mechanics?
Answers to such questions will not only be of interest to psychologists, but could also further inform game design. For example, it might allow the designer to engineer a hot streak so that players would take more risks or be more explorative in their strategies. Of course in a game it might even be appropriate to use a cold streak to discourage a player’s current strategy. The game mechanics could help engineer these streaks in a very transparent way without breaking player immersion. Further investigations of the hot hand hold significant promise for both psychology and game design.