1973年，心理学家Mark Lepper和Richard Nisbett与幼儿园教师合作组织了一次有趣的实验。他们观察到孩子们会受到内在的激励去画画—-孩子们喜欢根据自己的想法去画画而不需要获得任何额外的报酬。他们将孩子们分成三个群组。他们承诺给予第一个群组的孩子缎带作为画画的奖励。也给予第二个群组的孩子缎带奖励，但却未事先说明。而第三个群组的孩子则独自在那边画画。尽管使用了缎带奖励，但所有三个群组的孩子们都交出了基本上相同数量的图画。之后他们不再提供缎带奖励。第三个群组的孩子就像他们预期的那样继续画出同样数量的图画（毕竟对他们来说什么都未发生变化）。而第一个群组的孩子在图画数量上却出现了大幅度的下降。
The Psychology of Rewards in Games
by Max Seidman
Over the past 2 years I’ve had the privilege of working with at Tiltfactor alongside Dr. Geoff Kaufman, who has a PhD in social psychology and is our head of research at the lab. In addition to the fantastic insights he has made running formal studies on the games we’ve been developing (publication pending on those), I’ve learned amazing things about psychology, formed many theories about how it influences game playing, and developed a list of things game designers need to keep in mind. I’ve devoted a lot of thought recently to one of game design’s most hotly debated psychological topics, reward schedules, and how they relate to what psychologists call the “overjustification effect.” But don’t worry if you’ve never heard of either of these things! I’ll explain them both before discussing what they mean for game designers.
These theories, while most obviously applicable to digital games and tabletop roleplaying games, are important for all types of game designers to understand, especially in an era where even traditional board and card games are becoming more digital.
In the 1930s, Burrhus Frederic “B. F.” Skinner, a psychologist at Harvard, invented an Operant Conditioning Chamber (better known as a Skinner Box). The concept was simple: put a rat in the box. Let the rat pull the lever in the box. Sometimes give the rat a food pellet for pulling the lever. Study what conditions cause the rat to pull the lever more or less often.
Of course Skinner Boxes also include the capacity to shock the creature because scientists are freaky like that.
The applications in game design become clear when you look at what Skinner and other psychologists found. While experimenting with pigeons, researchers found that the pigeons were more likely to push the lever more often when there was a only a chance that they would receive a reward, even more often than when they always received one. Specifically, they were most active when the chance of receiving a reward was 50%. This is an intermittent reward schedule: it gives a chance at payoff for any given action. Specifically, they found that the most effective reward schedule was a variable ratio reward schedule – inserting randomness into the equation such that there could be many pulls of the lever with no payoff, but the average payoff is set.
If this behavior can be extended to humans (and let’s be honest, it can), we can be controlled to perform an activity more often simply by giving us a chance at a reward instead of promising us a guaranteed reward. We tend to know this intuitively: it’s why we gamble. And tons of games already use these principles. For example, slot machines are basically Skinner Boxes for humans. Zynga is notorious for using variable ratio reward schedules in their social games like Farmville. Even World of Warcraft uses them by having killed mobs only drop the loot you need for quests some of the time and not all of the time.
Skinner Box for people
The use of variable ratio reward schedules in game design is often panned, however, for being “nefarious.” Detractors’ reasoning goes: if the game designers had chosen to use simple fixed reward schedules (where for each action there is a promised reward), the players would play a certain amount. With variable ratio reward schedules, the players are more active. Thus, the game designers are “tricking” the players into playing more than they really want to, and usually also spending money.
Before I go into the ethics of using variable ratio reward schedules in games, I want to talk about another crucial phenomenon that is often overlooked in discussions of morality and reward schedules: the overjustification effect.
In 1973 psychologists Mark Lepper and Richard Nisbett conducted a fascinating experiment with kindergarteners. They observed that the children were intrinsically motivated to draw —that the kids enjoyed the activity of drawing pictures for its own sake without any need for external payoff. They split the children into three groups. The students in the first group were promised ribbons as a reward for drawing. The students in the second group were given ribbons, but were not promised them beforehand. The third group was left alone to draw in peace. While the experimenters handed out ribbons, all three groups drew comparable amounts of drawings. Then they stopped giving out ribbons. The third group, as would be expected, continued to draw the same amount (after all, nothing had changed.) The first group, however, had a significant drop off in the amount of drawing they did.
Psychologists theorize (and they’re pretty certain at this point) that what happened is as the first group received ribbons, they shifted their motivations from “I’m drawing because I like drawing” (intrinsic) to “I’m drawing because I want a ribbon.” They were still motivated to draw so long as they were receiving their rewards, but once the rewards were removed the motivation did not snap back to being intrinsic.
This is an often overlooked phenomenon that occurs in many sorts of games. I only have personal anecdotes to share, but if you play many video games I’m sure you’ll be able to identify times when the overjustification effect happened to you.
For myself, after beating Diablo 3 I continued to play and sell my rare items for money on their real money auction house. I built up quite a store of rare items that I was slowly selling off. I played this way for several weeks, but then something changed: Blizzard released a patch making newly dropped rare items much more powerful. This made all of my collection worthless, and I quit after reading about the patch. I haven’t played since. What happened here was I shifted my motivation from “I like playing Diablo 3” to “I like selling items on the auction house.” This was all well and good until Blizzard took away my chance to cash in on my efforts, and then I simply quit.
In another personal example, I played League of Legends, all the while working towards unlocking a single summoner spell: the renowned “Flash” spell. As soon as I unlocked it I quit. I didn’t even play one game with it unlocked. I had shifted my motivation for playing the game to “I want to unlock Flash,” and as soon as I did I found I had no motivation left to play.
Cue 1000 screaming fan boys on how overpowered Flash is and how I should totally play with it.
In both of these cases the games presented me with a goal to work towards that undermined my intrinsic desire to play the game. From a designer’s point of view this is fine, so long as the extrinsic motivations (rewards) remain. So long as I could continue selling items on the Diablo 3 auction house, I would keep playing. Unfortunately, there are two obvious occurrences that take players’ rewards away. The simplest one, exemplified by my League of Legends story, is that the player can achieve the reward. The other problem, as shown in my Diablo 3 anecdote (and many others) is the reward becoming obsolete. Between these two issues it’s difficult for a designer to add fixed reward schedules into a game without risk of the overjustification effect coming into play.
Giving rewards in games is desirable. Designers want to give players rewards for numerous reasons, including reinforcing player behavior, increasing players’ feelings of mastery, scaling difficulty over the course of gameplay, and scaffolding mechanics and player abilities. So how can designers give rewards with the perils of triggering overjustification looming overhead and threatening to make their players lose interest in the game? Lepper and Nisbett’s second group of kindergarteners (the group I didn’t reveal the results for) give us a hint at the answer. Recall that this group was the one that were given ribbons after drawing, but were not promised them beforehand. Once the rewards were removed, this group continued to draw at the same rate as group number one.
This gives designers the solution to providing rewards while avoiding shifting players’ motivations to solely wanting rewards! Don’t let the player know for certain that she’s going to get a reward —also known as variable reward schedules. It’s pretty straightforward: when a player knows what she’s going to receive by means of a reward, she can play only for that reward. It’s much harder to make that motivational shift when the reward is uncertain. This is why games like Dota 2 give items as rewards after games fairly infrequently, at (almost) unpredictable times and with random quality: this way players can enjoy the surprise of the reward without banking on it.
Dota 2 gives payoffs when the “battle experience” bar up at the top fills up. I couldn’t for the life of me tell you how many battle points I have, which means it’s as good as random and Valve is doing a great job!
My point here is that not only are variable ratio reward schedules not inherently evil, they are actually good game design practice. Variable ratio reward schedules don’t trick players into playing more than they really want to, they trick the players’ brains to prevent them from shifting their justification for playing to external factors. This means that these reward schedules keep the players playing because they find the experience itself fun and not just to get to the next reward, AND THIS IS EXACTLY WHAT GAME DESIGN IS! Game design is the process of making systems that players find interacting with fun and worthwhile without external justification. Omitting variable ratio rewards when they could be included is just like ignoring other established design theory (say scaffolded learning or positive feed back loops) that we’ve discussed on Most Dangerous Game Design previously: it’s simply shoddy design practice.
Can designers abuse the psychology of reward schedules? Of course! Most of human psychology can be abused and often is. However, even when using psychology, “tricking” players into having more fun still counts as giving players more fun. I know this entire arguments tends to raise peoples’ hackles, so please feel free to argue with us in the comments, as always!(source:mostdangerousgamedesign)