作者：Ian Schreiber

当游戏在你自己（游戏邦注：或其他游戏设计师）不在场观察的情况下进行测试，这就是所谓的盲测。这便是我们今天要探讨的话题。（请点击此处阅读本系列第1、第2、第3、第4、第5、第6、第7、第8、第9、第10、第11、第12、第13、第14、第16、第17、第18课程内容）

盲测的挑战

你可能也会想象到，与你亲自进行的游戏测试相比，盲测会面临众多问题：

1、因为你不在场，所以玩家碰到的任何问题都是他们完成游戏测试的阻碍。

2、可能出现更糟糕的情况是，玩家或许会错误地理解规则和玩游戏。如果他们不知道玩游戏的方向错误，那么你的测试结果就可能存在偏差，而测试者甚至都不知道这点。

3、反馈结果会有很大的差异。现在你或许已经知晓，测试群体的质量参差不齐。当你不在场时，无法询问目标问题，这个麻烦会更加严重。

4、你无法进行实时审查。你所获信息的唯一来源是测试群体在回忆游戏过程时的反馈。其信息的真实性可能比你在场的情况更差。

5、进行盲测需要花费更多的时间。你不能简单地将游戏原型带到好友的家中，然后马上让他体验。你需要找到方法，让测试者得到游戏的副本，然后你必须得离开。然后你必须等待测试者根据自己的时间安排体验游戏后向你反馈结果。盲测的结果要1到2天才能够收到，这是最好的结果，但比亲自测试所需的数分钟或数小时要长得多。

因而，盲测所能提供的信息类型有限。

game testing(from maximumpc.com)

盲测的原因

既然盲测有如此多缺点的话，为什么还要使用呢？它有什么用处？

这是项我们从科学领域借鉴的技术。之前，某些研究人员注意到，调查者是否同测试者在同一个地方会影响实验结果。有时，测试者所说的内容是调查者想听的，而并不是他们自己心中所想的。有时，调查者会提供微妙的非语言线索，而他们自己和测试者都没有意识到这一点，但是这会使测试结果产生偏差。比如，假设进行的是两种软饮料的口感测试实验，如果调查者知道两份饮料的对应品牌而且他们希望测试者选择哪个品牌，他们可能会发现所有的测试者都选择了“正确的”饮品，原因只是调查者不经意间向他们传递了要做出哪个选择的信息！

应对这种情况的解决方案便是所谓的双盲实验，也就是调查者和测试主题都不知道哪个是“正确的”选择。这消除了许多意外产生的偏见，使得测试结果更加准确。

游戏测试同科学实验有许多共同之处。二者都有假设前题（“我觉得这个游戏很有趣”）、设计实验（构建原型）、运行实验（游戏测试）和结果分析。二者的目标都是发现所研究系统内在运转情况的更多信息。

盲测是获得玩家对游戏真实想法的唯一途径，也就是说，如果玩家是刚刚从商店中购买游戏并首次体验游戏，在设计师不在场的情况下玩家会对游戏做出何种反应。

盲测的时机

因为盲测所能提供的信息量很少，所以在游戏状态粗糙的早期并不适合进行盲测。测试者很可能在碰到问题之后无法进行下去，你也需要花费大量的时间来等待结果，而亲自在现场测试得到结果的速度要快得多。盲测最适合开发末期，也就是在你已经自信游戏几乎没有问题的时候。

盲测的目标是找到你在亲自测试中可能无法发现的潜在问题。在亲自测试中，即便游戏测试者无需询问你任何问题，你对游戏的了解有时也会让人们轻松地通过某些规则（而这些规则在盲测时或许会让他们感到困惑甚至放弃游戏）。或者，你可能会在向玩家介绍游戏时说出某些并没有在游戏中提供的关键信息片段（游戏邦注：比如“这是款拍卖/竞标游戏”），这种情况可能会让测试者脑中产生某些先入为主的观念。如果你不在场，你就会发现亲自测试结果的准确性有问题，你或许可以找到之前没有注意到的某些漏洞。

盲测的对象

多数桌游需要多个玩家。出于实践原因，游戏的多数盲测对象是固定的游戏群体。某些公司有个自愿者群体，他们会在联系测试者之后向他们发送游戏的升级版本，由此来获得他们的反馈。对我们来说，这种方法并不适用，因为多数开发者并没有如此大的游戏公司，也没有可供征募的志愿者数据库。

理想情况下，你的盲测对象应当是游戏的目标群体用户。如果你制作的是儿童游戏，你的盲测对象应当包含目标年龄段的儿童（游戏邦注：有时候还要考虑到孩子的家长）。如果你的游戏针对的是那些已经对德式桌上游戏狂热的玩家，那么你最好找个经常玩此类游戏的玩家群体。其他类型游戏也是如此。

在这个阶段，以下方法可以帮助你的设计项目找到盲测对象：

1、你的社交圈，包括好友、家人和同事。这些人可以是当地的，也可以居住在其他城市甚至其他国家。因为你无需待在测试现场，所以在此地理问题将不再是障碍。

2、其他处在这个阶段的人。如果你在当地已经拥有了游戏测试群体，而且之前与他们合作过，你可以在Twitter或论坛上向他们发出援助请求。可以提供盲测交换：如果他们接纳你的请求，那么你也可以作为对象来盲测他们的游戏。

3、最后，你可以通过其他的讨论论坛或网络媒体寻找桌游设计师或游戏测试者，看看是否能找到招募盲测测试者的方法。

盲测的方法

在亲自测试中，如果出现某些纰漏，你可以应付得过来。或许你意外将某些游戏组件忘在家中，或者某些规则有些不清晰，需要澄清。尽管会出现这些不理想的情况，但是当你在场的时候，这些游戏测试问题可以得到适当的挽救。这在盲测中是无法实现的，因而你必须特别谨慎，确保你的测试者拥有为你的游戏进行测试的一切所需东西。这包括：

1、一套完整的游戏组件。再三审查清楚，确保你提供的游戏包内有玩家所需的全套东西。

2、你的游戏包内的项目清单（包含测试游戏所需的一切组件），这样测试者就可以亲自确认没有东西被遗漏。

3、一套描述如何玩游戏的完整规则。

4、一套独立的指导，阐述如何开展盲测。是否存在你特别希望玩家去做的某些测试？你只是想知道他们是否能够打通游戏？还是想知道他们觉得游戏是否有趣？你是否想让他们找出规则的漏洞和不平衡性？

5、一套指导规则，阐述在总结游戏测试时应当做什么。测试者要如何联系你（游戏邦注：例如提供电话号码、邮箱地址等）？当他们联系你的时候应当告诉你什么内容？如果你不提供他们反馈所需回答的问题清单，他们可能就会天马行空地“畅所欲言”，这样你得到的结果就会很混乱。要考虑下对你来说获得何种信息最有价值，何种反馈对你来说最为重要，然后要求测试者提供这些信息！

你要如何将游戏包提供给游戏测试者呢？如果他们与你住在同一个地区，你可以直接将用于测试的游戏原型交给他们。如果盲测者在外地，这就会带来额外的挑战。有两个选择可供你考虑：

首先，你可以通过邮政将游戏原型邮寄给他们。有些测试者可能较远，仅这个步骤就会花费大量的时间（游戏邦注：也会增加开发的成本），因而要制定相对妥善的时间计划。如果你希望玩家将原型交回给你，那么可以考虑使用往返邮件。

其次，你可以通过电子邮件来递交所有的东西。发送可以打印和组合成可玩游戏原型的文件，包括如何打印这些东西的指导文件。尽你所能地减少游戏测试者的工作量。毕竟，你已经让他们花费时间来帮你测试游戏，不可再让他们花上几个小时的时间来打印和组建游戏原型。同时，如果你期望盲测者能够提供最真实的反馈，那么就要确保材料打印的成本较低，同时可用性很高。比如，如果用普通纸张打印即可，就无需要求使用硬卡片纸（游戏邦注：因为这可能会让测试者再花时间跑趟印刷店）。

意外情况应对措施

在现实情况中，游戏公司绝不会只有单个盲测群体，通常都会准备数个后备军。除了更多的测试者可以提供更多数据这个显而易见的原因外，还有个原因是盲测群体并不可靠。因为你不在场，他们可能只花一点点时间来做测试，而反馈时间可能会拖延得更久。有些群体可能会遗失部分游戏组件，或者忘记了他们的任务，或者很可能因为生活中的其他事情而将你的游戏放在一边。你邮寄的游戏原型也有可能遗失而无法被测试者拿到。在盲测的这个阶段，你可能会发现自己处在非常令人懊恼的境地，测试结果迟迟不来，或者没有在计划内的时间内获得反馈信息。

你有3个选择来处理这个潜在的问题。何种方法最合适要取决于你自身的情况。

1、你可以设立多个盲测群体。通常来说，设置3个较为安全。如果1个或2个群体没有反应，你至少可以获得部分结果（游戏邦注：应当注意的是，如果你与其他参与者进行“盲测交易”，这就意味着你需要测试3个项目，因而要确保自己有足够的时间）。

2、你可以选择你认识、信任和可靠的盲测者。如果你将游戏发送给某些总是信守诺言的好友，你就不用担心他们不给你提供反馈。

3、你可以抱侥幸心理，祈祷这些糟糕的事情不发生在你身上。但在此不推荐专业游戏项目使用这种方法。

游戏邦注：本文发稿于2009年8月17日，所涉时间、事件和数据均以此为准。（本文为游戏邦/gamerboom.com编译，拒绝任何不保留版权的转载，如需转载请联系：游戏邦）

Level 15: Blindtesting

Ian Schreiber

When your game is playtested without you (or another of the game’s designers) being personally present to observe, it is called blindtesting. This is the subject we cover today.

Readings

No readings for today. As before, if you know of any relevant readings you have encountered, post it as comments to this blog post, or on Twitter with the #GDCU tag.

The Challenges with Blindtesting

As you might imagine, blindtesting has a lot of problems compared with a playtest that you run in person:

Without you there, any issue the players run into is a show-stopper that prevents them from finishing.

Perhaps worse, the players may interpret the rules incorrectly and play anyway. If they are unaware they are playing “wrong” your test results may be skewed, and the testers won’t even know it.

Feedback is highly variable; as you have no doubt seen by now, some playtest groups are better than others. This problem is made worse when you are not present and cannot ask targeted questions.

You cannot observe in real-time. The only information you get is what the group can report back to you in retrospect. The fidelity of information is much lower than when you are present.

Setting up a blindtest takes a bit more time. You cannot simply bring your prototype with you to a friend’s house and play right then. You have to find a way to get a copy of your game into the hands of your testers, and then you have to leave. Then you have to wait while your testers play your game, on their schedule, and then finally get back to you with some results. Even in the best case, blindtest results may take a day or two to get back to you, which is far more than an in-person playtest which you can conduct in a matter of minutes or hours.

Blindtesting is therefore limited in the type of information it can provide, and the schedule on which it can provide it.

Why Blindtesting?

If blindtesting is so limited, why bother with it at all? What use is it?

This is a technique we borrow from science. Awhile ago, some researchers noticed that the results of an experiment would be different if the researchers are in the room. Sometimes, test subjects would say what they thought the researchers wanted to hear rather than what they actually thought. Sometimes, the researchers would give subtle non-verbal cues that neither they nor the subjects would consciously notice, but that would bias the results of the test. For example, if the experiment is a taste test between two leading soft drinks, if the researchers know which drink is which and they know which one they want the test subjects to pick, they might suddenly find that all of the test subjects are choosing the “right” drink… but only because the researchers are (accidentally) telling them what to choose!

The solution to this is the so-called double-blind experiment, where neither the researchers nor the test subjects know what the “right” answer is. This eliminates many sources of accidental bias, making the results of an experiment more valid.

Playtests share a lot in common with science experiments. In both cases there is a hypothesis (“I think this game is fun”), an experiment is designed (building a prototype), the experiment is run (the playtest), results are analyzed. The purpose of both is to find out more information about the inner workings of the system that you are studying.

Blindtests, then, are the only way to get a true idea of what your game is like “in the wild” – that is, how players will react to it if they have just purchased it from a store and are playing it for the first time, without a designer present at their table to answer questions.

When to Blindtest?

Because you can get so little information from blindtesting, it is not suitable for an early test when your game is in a rough state. The testers would likely run into problems, be unable to continue, and you would have spent a lot of time waiting for something that you could have figured out much faster with an in-person playtest. Blindtesting is most suitable near the end of development, when you already have a high degree of confidence in your game.

The purpose of blindtesting is to catch the non-obvious problems that you may not be catching in your in-person playtests, because you accidentally bias the results of the test by helpfully being available to answer questions. Even if your playtesters never have to ask you anything, the very knowledge that you could can sometimes make people relaxed enough to get through the rules, where they might otherwise get flustered and give up. Or, you might say some key piece of information when introducing the players to the game (“this is an auction/bidding game”) that is not written anywhere and clarifies a lot by creating some preconceived notions in the minds of your testers. Without you present, you’ll see just how accurate your in-person playtests are, and you may catch some surprising errors that you did not notice before.

Who to Blindtest With?

Most board games require multiple players. For practical reasons, most blindtests done with professional games are done with regular game groups. Some companies keep a list of volunteer groups and put out calls to their private list for playtesters; they will then send out advance copies of their game in exchange for feedback. For our purposes, this is unhelpful, because most of you do not work at such a company and do not have a database of volunteers to call at a moment’s notice.

Ideally, your blindtest should be with people who are in the target audience for your game. If you’re making a children’s game, your blindtest should involve kids in your target age range (and perhaps their parents). If your game is made for people who are already avid Eurogame players, you’d do well to find a group that plays these kinds of games regularly. And so on.

There are a few ways to find blindtesters for your Design Project in this course:

Your social network of friends, family, and colleagues. These people may be local, or they may live in another city or even another country from you. Since you don’t have to be present, at least this time, geography is not a barrier.

Other people who are taking this course. If you already have a local playtest group that you’ve used before, you can put out a call for help on Twitter or the forums. Offer a blindtest exchange: you’ll blindtest their game if they will return the favor with yours.

As a last resort, you can look outside of this course for other discussion forums or other online hangouts for board game designers and/or playtesters, and see if there is a method for recruiting blindtesters.

How to Blindtest?

With in-person tests you can afford to be a little bit sloppy. Maybe you accidentally left some of your game components at home, or some of your rules are a little unclear and need clarification. It’s not ideal, but a playtest session that runs into problems can be salvaged when you are there. With blindtesting, you do not have this luxury, so you must be extra careful to make sure that your testers have everything they need to give your game a proper playtest. This includes:

A complete set of game components. Double and triple check to make sure that you have everything the players need, together, in one package.

A list of everything that your package should contain, so that the testers can verify for themselves that nothing was left out (and if it was, it will at least give them a clue of what they need to supply as replacements).

A complete set of rules describing how to play. (The components list can simply be part of the rules.)

A separate set of instructions on how to conduct the blindtest. Is there anything in particular you are looking for the players to do? Do you just want them to see if they can play through the game? Do you want to know if they find the game enjoyable? Do you want them to try to find rules exploits and imbalances?

A final set of instructions on what should be done on conclusion of the playtest. How should the testers contact you (phone, email, etc.)? What should they tell you when they contact you – if you don’t give a list of questions for them to answer, you will just get whatever they happen to feel like telling you, making your results a lot less focused than they could be. Give some thought to what information would be the most useful to have, what kinds of feedback are most important to you… and ask for it!

How do you get your package into the hands of the playtesters? If they are local to you, it is as simple as taking your playable prototype and handing it off. If your blindtesters are not local, this presents additional challenges. You have two options here.

First, you can mail your prototype to them through the postal service. Depending on where they are, this step alone can slow things down considerably (not to mention making it more expensive), so plan your schedule accordingly. If you want your prototype back, consider including a return package inside the main one, already addressed to you and with postage already paid.

Second, you can handle everything over email. Send documents that can be printed out and assembled to make a playable prototype, and include instructions on how to print everything. Do everything you can to reduce the amount of work that must be done at the playtesters’ end of things. After all, you are already asking for their time in the form of a playtest; asking for another hour or two to print and cut sheets of cards is adding insult to injury. Along the same lines, try to keep the cost of materials down and availability high if you’re expecting your blindtesters to provide their own. For example, do not insist on printing on heavy card stock (which may require a special trip to a print shop) when printing on plain paper will do.

Contingency Planning

In the field, game companies do not rely on a single blindtest group, but several. Aside from the obvious reason that more tests give more data, there is also the problem that blindtest groups are unreliable. Without you in the same room, they may take awhile to organize a playtest, and they may take even longer to get back to you. Some groups may lose the components, or forget about their obligation, or they may simply be so busy with other things in their life that your game takes lower priority. Or maybe you’ll send your game through the post and it will get lost. In this course, when you set up a blindtest, you may find yourself in the frustrating situation of waiting for test results that are simply not coming… or at least, they may not arrive within the schedule you set for yourself.

You have three options for dealing with this potential hazard. It is up to you which method best fits your personal situation.

You can set up multiple blindtest groups. As a rule of thumb, three is a fairly safe number. That way, if one or two groups don’t show, you’ll at least have some results. (Note that if you are doing a “blindtest exchange” with other participants of this course, that means you’ll be testing three other projects, so make sure you have time for this.)

You can choose a blindtester that you know and trust to be reliable. If you are sending your game to a friend who is highly organized and always keeps their promises, you may have confidence that they will get back to you when they say they will.

You can cross your fingers and hope that something bad won’t happen to you. This third method is not recommended for professional projects. (Source: Game Design Concepts)