作者：Pascal Luban

在游戏开发过程中询问测试者的反馈并非新鲜事。但是，通过接近科学的协议执行的玩法测试操作，并将它们整合到早期开发周期中却是最近才出现的趋势。

真正玩法测试在游戏开发周期中的扩展可能就是这种低调革命的一部分，而这个革命却深刻影响了开发环境。

如何影响？玩法测试迫使游戏开发围绕玩家而非开发团队的意志进行。让我们看看这种关注点转移的效果：

*玩法测试可以鉴别出那些令一般测试者转移视线的玩法或关卡设计瑕疵。

毕竟，测试者总是富有经验的玩家，他们未必能够代表目标用户。还有谁会比休闲玩家更合适指出与难度曲线或对游戏的整体理解有关的问题？

*玩法测试扮演了设计团队意见不一致或产生争论时的缓和剂。

一系列玩法测试几乎可以解决任何争论或争端，因此以避免从争论不断转向僵局的意见不一致。玩法测试也是一个管理工具。

*玩法测试和设计之间的伙伴关系可能极具建设性。例如，它可能对游戏极具指导意性，让关卡设计师可以在测试过程中观察玩法，以便他们快速判断自己的设计的特定环节是否像 预期一样可行。

*在实物模型上执行的玩法测试可以尽早预测到问题，并及时更正所谓的问题（在开发周期中越早更正问题，成本就越小）。游戏开发也因此变得真正“以玩家为中心”。

*根据玩法测试协议和测试者的选择（硬核、休闲等类型），玩法测试可以用较高的准确性来检查特定游戏的层面：如游戏平衡性，导航、对游戏目标的理解等。

我们都可能玩过那些具有较高产品价值但却具有一些明显瑕疵的游戏，它们存在如游戏早期不稳定的难度曲线，导航问题，过于复杂的界面等问题。

如果早点发现，就有可能避免这些问题的出现了。

行业中的主流公司相当了解这一点，比如育碧这家拥有过硬团队的并在这个游戏开发层面投入许多资源的公司就是如此。

我们通过玩法测试可以解决或避免什么问题呢？这些例子包括：

*通俗性与易用性（界面、游戏中的导航等）

*鉴别出压倒性的战略，例如那些允许玩家轻易克服由设计师创造的任何挑战，并因此导致他们对游戏失去兴趣的战略。这个问题在多人模式地图中表现最为显著。

*微调游戏系统：根据我的经验，玩家对游戏功能（包括武器、装备、操作等）的使用强度会因一系列因素而产生变化。

这些包括玩家资料，玩家用于让自己熟悉游戏的投入时间，当然还有游戏本身的协调性

只有通过具有关联性玩家样本的长期测试，我们才能确定游戏在多个小时后还能维持其平衡性和关联性。

*在早期会话阶段分析不同玩家类型的早期反应。这将指明他们对游戏的初次印象以及初期受挫感。有些游戏样本可能会对他们打算推广的游戏营销产生一定消极影响，因为它存在 易用性和协调性问题，这些都是很容易在测试时发现的情况。

*对于多人模式游戏来说，游戏系统的健全性和地图潜力很重要。

我曾有数次机会深入研究玩法测试管理。我在育碧Annecy工作室（游戏邦注：其开发作品包括《Splinter Cell：Pandora Tomorrow》和《混沌理论》）创建了玩法测试框架。

我设置了招募方法，玩法测试协议，以及这个程序所采用的询问方法。我还在育碧Bucarest办公室设置了一个测试格子间，并在那里主导了玩法测试。玩法测试改变了我作为创意 总监的工作方式，所以我感觉有必要在此与各位分享自己的经验。

让我们先从定义入手。玩法测试包括分析富有代表性的玩家群体对玩家的反应，以便提升最终游戏，并确保成品与自己的预期一致。

有些人会说游戏测试并没有什么新鲜的。诚然，但真正的玩法测试与开发周期末尾执行的调试测试毫无关系。

传统上，游戏设计师会询问测试者的意见。测试者通常是出色的玩家，并不总能代表由主流玩家组成的目标用户群体。

另外，测试者通常太了解游戏了，他们对游戏长处和弱点的了解会深刻影响他们的玩法。因此，他们并不能代表那种首次发现游戏的群体。

执行完善的玩法测试允许我们以极佳的准确性来评估玩法长处和弱点，它们有赖于两个核心原则：

*仔细筛选玩法测试者

*使用正规的协议

筛选玩法测试者

就像农民需要肥沃的土地才能获得好收成一样，优秀的玩法测试也需要群精挑细先出来的测试者。我认为招募和评估测试候选人的重要性这一点已经无需再强调。

招募标准是什么？当然，这要取决于我们寻找的是哪类测试者。我们可能需要硬核玩家，初学者，只玩主机游戏的用户，多人模式粉丝等。

这些候选者的游戏精通度和整体游戏文化代表了首个标准。其次就是候选者从自己的游戏体验进行分析和总结的能力。

注意，这并不是说玩法测试者在这两个标准上一定要兼具较高水平。此外，测试者类型将决定需求条件。

我很尊重自己曾经共事过的那些测试者。他们极为善良和热情，许多人是从Lyon、Grenoble或Belfort等偏远城市来到Annecy，只是为了参与这种无报酬的半天测试！

这种慷慨和激情正是我们行业的特征，让我们以感激和尊重来对待这些测试者，继续培养这种行业特征。

使用正规协议

协议是玩法测试过程中的一个统一思路，定义了目标，资源分配，尤其是特定测试的收集和分析信息的方法。玩法测试协义必须根据手头上的挑战进行特别调整（游戏系统调整， 导航，地图概念等）。

在我所主导的玩法测试过程中，我会针对每次测试准备不同的协议。的确，这些测试的一个重要部分包括框架之下的多人模式地图或游戏系统调整。每次测试都会透露出一些在之 后测试将进行分析的特定问题。

我想在此重复一次，玩法测试过程必须以真正科学的准确性来指导，没有人可以简单地带来一些好友享受数小时的乐趣和通过一轮简单的问题而执行玩法测试。

测试的每个层面都要仔细根据手头的目标来执行以便实现预期效果。

管理测试本身也需要持续的注意力，这不但是因为我们可以通过观察测试者的操作过程而掌握经验，还因为有些事情总是计划赶不上变化！

在这样一个游戏发行商财政风险不断增长的行业中，玩法测试可以作为高质量玩法的一个强大保障。我将在此与各位分享自己准备和执行玩法测试的方法和经验。

留心客户需求：设计团队

最根本的一点是，你必须清楚一个基本说法：玩没测试的角色并非代替设计团队重做设计，而是用于帮助后者改进设计。

首先，我们必须尊重设计团队的劳动成果。鉴于我自己在游戏和关卡设计方面的职责，我知道制作一款“好游戏”究竟有多难。我们必须尊重这些全身心投入开发好游戏的人，我 们绝不可以蔑视或低估他们的成果。

其次，玩法测试必须适应设计团队的需求。地图或玩法机制的良好调整通常是一个试错的结果。设计师应该认识到这一点并要求进行试验。玩法测试有助于他们检测自己的假设， 并因此适应其中出现的特殊需求。

最后，玩法测试必须尽早让相关团体参与，因为它在游戏开发中所分配的时间总是很少。

准备玩法测试活动

玩法测试通常需要一个月左右的准备时间。我们必须确定它的目标，因为它们将决定我们应该招募哪种类型的测试者，以及测试的规模（比如1、2、4、8或12名玩家参与），以及 持续时长（从半天到一整周）。

我们将还参与后勤和法律框架（非公开协议，以及测试持续超过半天时给予测试者的金钱补偿等方面）。我们当然还必须准备好让设计团队有效利用测试。

没有人可以在干涸的土地上种出好庄稼来，玩法测试的有效性扎根于测试者本身。有效进行玩法测试活动有一半应归结于明智地选择测试者，这需要时间、精力以及一点金钱和耐 心的投入。

招聘测试者需要时间：我们绝不可只是引进更多候选者（以便获得较可靠的测试样本）。我们还必须对其进行评估。评估的目的显然就是判断候选者的游戏能力，以及他们分格和 自我表达的能力。

评估可能会采取数种形式。最初的筛选可以是发放调查问卷让候选者完成，但真正的评估则必须是在测试阶段完成，而我们在这一阶段要观察候选者的游戏情况。

我们必须确立一个尽量保留最为一致性结果的协议。这里并不存在所谓的“用于所有目的”的评估协议，我们必须能够情况变化适应特定环境。

当我在Bucarest育碧办公室创建一个玩法测试框架时，我遇到了一个有趣的问题：我们需要为主机游戏进行玩法测试，但我们所找到的本地玩家都是纯PC游戏玩家。我不得不设置 了一个特定协议来评估我们这些罗马尼亚候选者适应主机游戏的可能性。

该协议包括简要地解释复杂游戏的玩法控制系统（《Splinter Cell：Chaos Theory》多人模式），之后任由他们体验游戏，以便检测他们适应玩法的速度。结果证明这种筛选方法 极为有效。

候选者筛选必须根据特定玩法测试活动的目标来完成。我们可能需要那些已经精通这种游戏题材的高技能玩家，但如果目标是测试游戏易用性的话，也可能只需要新手。

与玩法测试相关的沟通也需要时间。在候选者执行测试之前，必须确保他们清楚你的需求。从我的经验来看，通过一般分类广告途径可以招聘到较高数量的候选者，其中许多人年 纪都太小了（这时候就要注意劳动法），多数只是休闲游戏玩家。

招聘经验丰富玩家的一个好方法就是通过论坛，游戏公会或专门商店。它需要更多时间，但我总能通过这个方法找到出色的测试者。在玩法测试中，质量总比数量更重要！

组织测试

我应该解释一下组织玩法测试的三个方面：团队构成，测试协议准备，以及后勤。

招募必须至少在测试前四五天就开始。在这个阶段，测试管理人员已经掌握了已被评估过的候选者数据库。他可以根据测试主题来挑选测试者，并通过电子邮件向对方发送邀请。

此时我们已经认识到拥有相当数量候选者的重要性，因为多数人未必能够到场。因此我们必须投放大量邮件以确保当天会有足够的测试者到场。

最好邀请比实际所需更多的测试者，因为也有不少人会在最后关头临时取消计划。另外也最好要求测试者通过邮件确认自己能否出席。

协议设置是这个准备过程中的重要环节。有些测试者会被安排在将近开发周期末尾，以便调整地图或游戏系统。这种类型的测试协议通常都很直接：我们允许测试者玩游戏的最长 时间，注明游戏参数，并组织公开的问答环节。

测试最管用的时候是在开发早期阶段，当时的游戏系统和地图仍然处于萌芽状态。不要忘了我们越早发现问题，就越易于纠正问题。

在多人模式版本的《Splinter Cell：Chaos Theory》的地图开发过程中，我组织了玩法测试来评估这个当时仍处于早期阶段的地图结构。

我对Aquarius地图印象尤其深刻：通过经验丰富的玩家测试，我们（包括创建了地图的关卡设计师在内）很快意识到这个地图太大了。

注意到这个问题后，他立即重建了地图，这花不了多少时间，因为当时的地图还只是一个原型。他经过数次迭代将地图缩小到理想大小，最终Aquarius才能成游戏最受欢迎的地图 之一。

玩法测试有助于我们发现许多问题，并确认（或作废）由设计团队提出的假设。在《Splinter Cell: Pandora Tomorrow》多人模式版本的开发过程中，我们也出于调整特定装备（ 如烟雾弹）的目的而进行了一些特定的测试。

烟雾弹是间谍最常用的配件之一，因为它的烟雾可以拖延竞争对手（雇佣兵），如果时间够长的话甚至还会导致后者晕睡。

调整烟雾弹参数并不容易——它的射程太广泛了，对于攻击者来说它可能成为一种势不可挡的武器（他们只需要在走廊中使用一颗烟、雾弹就能阻止任何对手靠近）。

另一方面，如果烟雾弹的影响面区域太小，这项武器就会毫无用处（防御者拥有可让他们在烟雾中具有部分可视性的视觉模式）。所以寻找正确值耗费了我们大量时间。

最后，协议必须根据之前测试所遇到的问题，以及设计团队所提出的测试需求作出调整。另外，开发团队的需求也是成功测试的标志之一。我会在之后说明这一点。

现在来谈谈后勤。良好的玩法测试要有一个稳定的游戏构造，其中不可有太多漏洞。在开发周期的中间阶段执行测试时，这可能是说得容易做得难。尽管如此，游戏还是必须尽可 能满足这一条件，地图也不能有太致命的漏洞（游戏邦注：例如无法攀爬梯子）。

游戏交付协议必须与开发团队一起设置。后者必须向内部调试团队交付一个已经可用于测试的游戏版本，调试团队会快速审核游戏以确保该版本具有可测试性。

当问题出现时，调试和开发团队的合作有助于快速更正问题，并为测试提供一个稳定的版本。

这种组织性技巧需要所有团队多个学科人员的参与。另一个好方法就是为关卡和图像设计师准备一个清单，以便他们确保自己的地图已经没有障碍性的漏洞。最后，测试管理人员 本人必须确保该版本的确具有可玩性。

测试过程

当设计团队人员参与时，玩法测试尤其具有指导性。的确，游戏或关卡设计师会根据自己观察到的玩家行为所得的想法来执行工作。

但是，玩家并不总会出现预期的反应，我们必须考虑到他们的多样性。

设计师通过亲眼看到真正玩家如何使用装备或导航地图布局，通过询问后者某种行为的理由，就可以快速进行优化调整——事实总是胜于雄辩！因此我强烈推荐团队鼓励开发团队 参与玩法测试。

这正是我为何强烈建议在开发工作室本身的假设上执行测试的原因。远程测试对于调整地图和系统设置来说很有价值，但对于尚于处萌芽期的游戏来说却不甚管用。

显然，玩法测试观察者必须遵从一定的规则：他们不可发表自己的评论，或者进行询问，直到自己获得测试管理人员的授权为止，以免影响游戏测试或测试者的判断。

原因如下：早期测试的测试者数量通常很有限，并且可能出现大量问题。事实上这可能会影响到所收获反馈的关联性，最好的情况就是结果不一致，最糟的情况就结果自相矛盾。 管理人员必须考虑到这一切，自己评估反馈的关联性。

但要注意，测试管理人员的参与可能就是这一争论点的成因。在某些情况下，管理人员只能作为一个纯粹的观察者。事实上，这是开发过程末期，即准备微调游戏系统设置时执行 测试的最佳态度。

这个要点的目标是从大量测试人员收集最大化的统计数据。

相反，在评估初期地图或游戏系统的强弱这一早期测试过程中，相对低质量和更多差异性的数据则需要管理人员更为积极而直接的参与。

此时，他必须“埋头苦干”与不完整的数据打交道。虽然这里存在错误风险，但我的经验表明这一阶段的测试结果实际上更准确，也更有用处。

我在法国最棒的开发工作室之一的经历告诉我，玩法测试管理人员必须完整投入游戏的最终质量，不可满足于仅仅充当一名观察者。

这个结论再次表明了测试与开发团队之间的亲密关系。

询问

我们因此可得到测试的最终结果。一般看法是将测试结论尽快交付到最需要的人手中——通常是指设计师和项目领导。询问可以有多种形式。

首先，观察测试的设计团队成员可能向测试者提出自己最紧迫或直接的问题。他们通常会带着一些强烈的想法离开测试间。

然后就是出报告，它必须与事实（如统计资料）、测试者的观点，以及管理人员自己的观察和结论具有明显区别。必须提供原始数据以便设计师了解管理人员的结论来源。

将全部数据摊牌是一个让报告读者产生信任感的好方法。不要忘了测试的目的是提升游戏，而不是拿学分。

完整的报告要花上一定的编写时间，所以如果需要获得重要反馈时就有必要进行一些更为简短而中立的询问。

最后值得一提的是，我曾经在米兰育碧工作室用一个允许远程办公室（在另一座城市或另一个国家）的协议进行试验，以便获得关于一个地图测试的报告。

这个协议简称D3（远程动态询问），包括快速确立一个主要公开问题的列表，组织相关设计师（在开发办公室）和测试管理人员（在测试办公室）可登录的在线会议。

他们之后就可以探索地图，而测试团队则更准确地说明问题，大家可以一起研究解决方案。而测试者也可能参与其中，提供自己的观点促进交流。

相关拓展阅读：篇目1，篇目2（本文由游戏邦编译，转载请注明来源，或咨询微信zhengjintiao）

The Silent Revolution Of Playtests, Part 1

by Pascal Luban

[Starting a new series, former Ubisoft designer Luban looks at why regular and detailed playtests are vital to center a game's development around the player.]

There is nothing new about asking testers for their feedback on a game in development. However, the practice of managing playtests by following near- scientific protocols, and of integrating them very early in the development cycle, is a more recent trend.

The spread of real playtests in the game development cycle is probably part of this silent revolution; a revolution profoundly affecting the development environment.

How? Playtests force game development to center around the players instead of the hopes of the development team. Let’s look at the effects of this shifted focus:

- Playtests allow the identification of gameplay or level design flaws that could elude the grasp of normal testers.

After all, testers are always seasoned gamers who are not necessarily representative of the target audience. Who better than a casual gamer to pinpoint issues related to the difficulty curve or the overall understanding of the game?

- Playtests fulfill a moderator role in situations of disagreement or controversy within the design team.

A series of playtests can quickly settle a contested issue by resolving almost any counter-argument or dispute, thereby preventing the disagreement from spiralling into an impasse. Playtesting is also a management tool.

- The partnership between playtesting and design can be very constructive. Fo example, it can be quite instructive for game and level designers to observe gameplay during playtesting, allowing them to immediately determine whether or not particular aspects of their design work as planned.

- Playtests executed on pre-prod mock-ups allow the anticipation of problems very early on, as well as timely corrections of said problems (the faster a problem is corrected in the development cycle, the less expensive it is). Game development can therefore become truly “player-centric”.

- According to the playtest protocol and the selection of playtesters (hardcore, casual, etc.), playtests allow the examination of a specific aspect of the game with heightened acuity: game balance, navigation, understanding of the game objectives, etc.

We all have the opportunity to play games that display high production values but nonetheless suffer from obvious flaws: erratic difficulty curve early in the game, navigation issues, overly complex interface, and so on.

Such flaws could often have been easily avoided if they had been identified early enough.

Major names in the industry understand this quite well, such as Ubisoft, which possesses qualified teams and invest lot of resources in this aspect of game development.

What kind of problems might we fix or prevent with playtests? Some examples include:

- Accessibility and ease of use (interface, navigation within the game, etc.).

- Identification of sure-fire-wins, i.e. strategies allowing a player to easily overcome any challenge created by the designers and therefore remove any interest in the game or the current mission. This issue is especially sensitive for multiplayer maps.

- Fine-tuning of the game system: experience has shown me that the intensity of use of game features (weapons, equipment, actions, etc.) tends to vary considerably according to a number of factors.

These include player profiles, the time a given player spends on familiarizing himself with the game, and of course the game tuning itself.

Only through long-term playtests with relevant samples of players can we ensure that the game tuning maintains its balance and relevance even after long hours of gaming.

- Analysis of the early reactions of different categories of players during their first session. This will highlight their first impressions and initial frustrations. Some game demos have probably had a negative effect on the marketing of games they were meant to promote because of accessibility and tuning issues that could have easily been spotted during playtesting.

- For multiplayer games, the robustness of the game system and the potential of maps.

I have had several opportunities to delve deeply into playtest management. I built the playtest structure from scratch at the Ubisoft Annecy studio, where the successful multi-player “versus” modes of Splinter Cell: Pandora Tomorrow and Chaos Theory were developed.

I set up the recruiting methods, playtest protocols, and the debriefing methods employed in this program. I also set up a playtest cell at the Ubisoft Bucarest office and led playtests there myself. Playtests have changed the way I perceive my job as creative director, so I feel the need to share my experience with everyone.

Let us start with a definition. Playtests consist in analyzing the reactions of a representative pool of players toward gameplay in order to improve the final game and to make sure it matches their expectations.

Some will argue that game testing is nothing new. True, but real playtests have nothing to do with the debug testing executed at the end of the development cycle.

Traditionally, game designers ask testers for their opinions. Testers are often excellent players and are therefore not always representative of the targeted demographic which is often made up of mainstream gamers.

Moreover, testers generally get to know a game so deeply that their knowledge of it strengths and weaknesses profoundly influences the way they play. Therefore, they do not play as someone who discovers the game for the first time.

Well-executed playtests allow us to evaluate gameplay strengths and weaknesses with great accuracy since they rely on two solid principles:

- The careful selection of playtesters.

- The use of ad-hoc protocols.

The Selection of Playtesters

Just as a peasant needs fertile ground in order to ultimately obtain the best yields, good playtests require a group of carefully-selected playtesters. I could never insist hard enough on the importance of the recruitment and evaluation of the playtest candidates.

What are the recruiting criteria? This depends, of course, on what kind of playtests we are planning. We may need hardened gamers, beginners, console-only gamers, multiplayer fans, and so on.

The candidate’s gaming proficiency and overall game culture represent the first criteria. The second is the candidate’s ability for analyzing and drawing conclusions from their gaming experience.

Note, however, that it is not mandatory that a playtester should possess a high level of competence on both criteria. Again, the type of playtests will determine the requirements.

I have the utmost respect for the playtesters I have worked with. Their good will and enthusiasm are boundless. Many came to Annecy from distant cities like Lyon, Grenoble, or Belfort simply for an unpaid half-day session!

This generosity and enthusiasm are characteristics of our industry; let us nurture these characteristics by treating playtesters with the gratitude and respect that they deserve.

The Use of Ad-hoc Protocols

The protocol is the unifying thread of the playtest session, defining the objectives, allocation of resources, and especially the methods of collecting and parsing information for a given playtest. The playtest protocol needs to adapt to the specifics of the challenge at hand (game system tuning, navigation, map concept, etc.).

During the playtest campaigns that I led, I would prepare a different protocol for each session. Indeed, an important part of those playtests involved multiplayer maps under construction or game system tuning. Each session revealed specific problems to be analyzed in the subsequent session.

I shall conclude this first part by repeating that a playtest campaign must be directed with a true scientific rigor if it is to be of any use; one does not conduct playtests simply by bringing over one’s buddies for a few hours of fun followed by a session of easygoing Q&As.

Each aspect of the session must be carefully tailored in order to best realize the objectives at hand.

Managing the session itself requires constant attention, not only because one can learn much by watching the playtesters in action, but also because things do not always go as planned!

I shall address concrete aspects of playtests in the second part of this article.

[Continuing his series on playtesting, ex-Ubisoft veteran Pascal Luban (Splinter Cell series) examines the practicalities of getting consumer feedback on your game.]

Proximity, responsiveness, relevance… these are the watchwords of efficient playtests.

In the previous installment of this article, I had explored the reasons for the rising importance of playtests in game development.

In an industry where games represent increasingly high financial risks for publishers, playtests have come to function as a strong guarantee for quality gameplay. I will share with you today my experience regarding the methodology employed in preparing and conducting them.

Heeding the Clients: The Design Teams

Foremost, one must be aware of a fundamental say: the role of playtests is not to redo the design in place of the design teams — for either game or level design. They are instead conducted to help them. This observation is crucial, because it drives the entire approach to playtests.

Firstly, we must respect the hard work of the design teams. Having had my own responsibilities in game and level design, I know how difficult it is to make “a good game”. We must respect those who put their whole hearts into building the best game possible; we must not scorn or undervalue their work.

Secondly, playtests must adapt to the needs of the design teams. Good tuning for maps or gameplay mechanics is often the result of trial and error. Knowing this, designers should require experimentation; playtests can afford them the opportunity to test out their hypotheses regarding design issues, and must therefore adapt to particular needs as they arise.

Lastly, playtest results must be made available to the concerned parties as soon as possible, as time allotted for game development is always short.Preparing a Playtest Campaign

A playtest campaign generally requires around one month of preparation. We must first define its objectives, because they will determine what types of playtesters we shall have to recruit, the scale of the sessions (1, 2, 4, 8, 12 players), and their duration (from half a day to a full week).

We will also have to attend to the logistics as well as the legal framework (non-disclosure agreement, eventual monetary compensation for playtesters when sessions last over a half-day, etc.) And we must, of course, prepare the design teams to effectively utilize the playtests.

One does not grow the best crops in dry land; a playtest’s effectiveness is rooted in the playtesters themselves. Half the battle in running an effective playtest campaign lies in wisely choosing playtesters, which requires investment of time, energy, and perhaps a bit of money and patience.

Recruiting takes time: we must not only hire as many candidates as possible (in order to have a solid pool of playtesters). We must also evaluate them. The purpose of evaluation is obviously to judge the candidate’s gaming competence, but also his ability for analysis and self-expression.

Evaluation may take several forms. An initial selection can be done through a more or less thorough questionnaire, to be completed by the candidate. The true evaluation, however, must be done during the sessions themselves, where we can observe the candidates at play.

We must establish a protocol for obtaining the most consistent results possible. There is no “all-purpose” evaluation protocol; we must also be able to adapt to specific circumstances as the situation mandates.

When I built a playtest structure at the Bucarest Ubisoft office, I encountered an interesting problem: we needed playtests for console games, but all the players we could find locally were exclusively PC gamers. I had to set up a specific protocol to evaluate the ease with which our Romanian candidates could adapt to console gaming.

Ubisoft’s Splinter Cell: Chaos Theory

The protocol consisted of briefly explaining the gameplay controls of a complex game (the multi-player mode in Splinter Cell: Chaos Theory), and then setting them loose in the game in order to gauge the speed at which they adapted to the gameplay. This selection method proved to be quite efficient.

Candidate selection must therefore be done according to a given playtest campaign’s objectives. We may have need of only extremely skilled players who have already mastered the genre, or we may require novices, if the objective is to playtest the accessibility of the game.

Communication regarding playtests also takes time. Before candidates can turn up on your doorstep, they must first be made aware of your need. In my experience, while recruiting through generic classified ads will yield a high number of candidates, many will be too young (careful of those labor laws!), and most will be only casual gamers.

A good way to recruit experienced players is to make use of forums, gaming clans or specialized stores. It takes much more time but I always got great playtesters this way. In playtesting, quality matters more than quantity!

Organizing the Sessions

I shall address three aspects of playtest organization: the composition of the team, the preparation of the playtest protocol, and its logistics.

Recruiting must start at least four or five days before the session itself. At this stage, the playtest manager already has access to a database of candidates that have already been evaluated or, at least, identified. He can thereby choose his playtesters according to the session’s theme. Invites are sent by e-mail.

At this point, we realize the importance of having a great number of candidates, since most are not available at will. We must therefore engage in mass- mailing to ensure sufficient availability of playtesters come session day.

It is also best to invite at least one more playtester than necessary, since last minute withdrawals are commonplace. It is also usually a good idea to ask playtesters to confirm their presence via e-mail.

Protocol setup is an important part of session preparation. Some playtests are organized near the end of the development cycle, to tune up maps or the game system. The protocol for this type of playtest is often straightforward: we must allow the playtesters to play for a maximum of time, note game statistics, and organize open Q&A sessions.

The time when playtests are most useful, however, is during earlier stages of the development cycle, when the game system and maps are still in gestation. Let us not forget that the earlier we detect any issues, the easier and cheaper it will be to correct them.

During the development of maps for the multiplayer version of Splinter Cell: Chaos Theory, I had organized playtests to evaluate the structure of the then still-embryonic maps.

I specifically remember the Aquarius map: By having it tested by highly experienced playtesters, we — including the level designer who had built the map — quickly realized that the map was far too large.

Having noticed this problem, he immediately rebuilt his map, which took little time as the map was still just a prototype. It took him a few iterations to downsize his map to the optimal size. In the end, Aquarius became one of the game’s most popular maps.

Ubisoft’s Splinter Cell: Pandora Tomorrow

Playtests allow us to shed light on many problems and to validate (or invalidate) hypotheses set by the design team. During the development of the multiplayer version of Splinter Cell: Pandora Tomorrow, specific playtests were undertaken with the purpose of tweaking the characteristics of certain pieces of equipment, such as the smoke grenade.

The latter is one of the most-used accessories by the spies, since its cloud slows down the spy’s opponents (the mercenaries), and it can even put them to sleep if they stay too long in its area of effect.

Tuning the smoke grenade’s parameters was not so simple — if its range was too wide, it would be an unstoppable weapon for the attackers (they would simply need to employ a single grenade in a corridor to block any access by their opponents).

On the other hand, if the grenade’s effect zone were too small, the weapon would be completely useless (defenders have vision modes allowing them partial visibility through the cloud). Finding the right values took us a lot of time.

Lastly, to be relevant, protocols must adapt to problems encountered in previous sessions as well as to the test requests put forth by the design team. This commensurability with the development team’s needs is one of the hallmarks of a successful playtest. I shall address this point later on.

Let us now talk about logistics. Good playtests require a stable build of the game without too many bugs. When directing playtests in the middle of the development cycle, this may be easier said than done. Regardless, the game must be sufficiently stable, and maps must be rid of the most detrimental bugs (such as the inability to climb a ladder, for example).

A game delivery protocol must be set up with the development team. The latter must deliver a playtest-ready version of the game to the internal debug team, which will rapidly review the game to ensure that the version is playtestable.

When issues arise, cooperation between the debug and development teams will allow for swift corrections of issues, and subsequently the production of a stable version suitable for playtests.

Such organizational finesse requires a lot of discipline from all of the teams involved. Another good practice is to prepare a checklist for the level and graphic designers, so that they can make sure that their own maps are free of blocker bugs. Finally, the playtest session manager himself must make sure that the version is indeed playable.

Playtest Sessions

Playtests are especially instructive when design team personnel attend the sessions; indeed, a game or level designer will base his work on ideas he will formulate upon observing the behavior of the players.

However, players do not always react as expected, and we must take their diversity into account.

By seeing with his own eyes how real players use equipment or navigate a map’s topology, and by asking them the reasons for their behavior at the end of the session, the designer can rapidly make optimizing adjustments — a demonstration is always more efficient than a long speech! It is thus highly recommended to encourage the designers to attend the playtests.

That’s why I strongly recommend that playtests should be conducted on the premises of the development studio itself. Remote playtests are valuable for tweaking map and system settings, but less so for playtests on an embryonic game.

Obviously, playtest observers must follow certain rules: they must not voice their comments or ask any questions until they are authorized by the playtest session manager, in order to preclude influencing the game session or the playtesters’ judgement.

If it is desirable for designers to attend the playtests, it is simply essential that the playtest session manager does so. He must not simply organize the session and ask his questions at the end; he must actually watch the playtesters at play.

The reason is as follows: early playtests often have a limited number of playtesters, and the problems found are liable to be numerous. This fact is likely to affect the relevancy of feedback received, rendering it inconsistent at best and flat-out contradictory at worst. The manager must take all of this into account, evaluating the relevance of the feedback himself.

Note, however, that the involvement of the playtest manager can be cause for controversy. In some cases, a playtest manager must simply behave as a mere observer; in fact, this is generally the best attitude to have during playtests occurring later in the game development, when it is time to fine-tune game system settings.

The objective at this point is to collect a maximum of statistical data from a high number of playtesters.

By contrast, during early playtesting meant to evaluate the strengths and weaknesses of embryonic maps or game systems, the comparatively low quantity and greater heterogeneity of the collected data require a more aggressive, reactive, and direct involvement on the part of the manager.

At this point, he must necessarily “get his hands dirty”, as he’ll be working with incomplete data. While there is a risk of error here, my experience has shown me that playtest results are actually more concrete at this stage, and thus more useful.

My experience amidst one of the best development studios in France has taught me that the playtest manager must be wholly invested in the final quality of the game, and must not be content with being a mere observer.

This conclusion once again indicates the need for a close relationship between the playtest and the development teams.

Debriefing

We thus arrive at the final result of a playtest session. The general idea is to bring the playtest conclusions as quickly as possible to those who most need it — generally the designers and project leaders. Debriefing may take several forms.

First, design team members who observed the playtests may put their most pressing or immediate questions to the playtesters. They often leave the playtesting room with some strong ideas burning in their mind.

Then comes the report, which must make a clear distinction between the facts (statistics etc.), opinions from the playtesters, and the manager’s own observations and conclusions. Raw data must be provided so that the designers know on which bases the manager drew his conclusions.

Putting all the cards on the table is a good way to establish trust with the ones who will read the report. Let us not forget that the purpose of playtests is to improve the game, and not to settle scores.

A full-fledged report takes time to compile and to write so a shorter, intermediary debriefing might be needed if the needs for crucial feedbacks is urgent.

As a final note, I’ll mention that I had begun to experiment at the Milan Ubisoft studio with a protocol allowing a remote office (in another city or even another country) to obtain a hot report on a map playtest.

Named D3 for “Debrief Dynamique à Distance” (Remote Dynamic Debrief), this protocol consists in quickly establishing a list of the main open issues, and organizing an online session where the concerned designers (at the development office) and the playtest session managers (at the playtest office) can log on.

They can then explore the maps while the playtest team explains the issues with much precision, and all can work together in developing possible solutions. A playtester may even join them, contributing further to the dialogue.