作者：Maurice Tan

如今，游戏开发的各个阶段都少不了某种形式的系统测试过程。测试能够有效地帮助游戏剔除各种漏洞（QA），能够找出游戏的最佳卖点（市场研究），并基于玩家体验确保游戏能够完整地呈现出设计师的观点（用户研究）。

不管是在游戏产业还是学术界，用户研究都还是一块尚未成熟的领域，但是多年来人们一直围绕着各个学科，包括工效学，可用性，心理学，通讯以及人类学等支持着该领域的研究并努力回答一些重要问题，包括：对于实现最佳玩家体验是否存在何种可预见的阻碍因素，以及我们该如何完善这种体验？

一般情况下人们都不会对测试的好处产生怀疑，但是同时人们却会常常疑惑到底是谁在进行着何种测试？用户研究拥有一些专业术语，从而导致那些不了解该领域的人常常会对此感到困惑；加之这里涵括了各种学术研究，使得用户研究不仅会让外行人感到混淆，甚至有时候连专业人士也不免费解。

例如“游戏测试”等术语同时适用于QA和用户研究中，但是在不同环境下它们却代表不同技术。即使是那些大众非常熟悉的市场研究术语，如“焦点小组”也会让人产生疑惑；也难怪很多人会将“焦点测试”等同于“可用性测试”。而我正想通过本篇文章帮助读者搞清楚这些内容。

如何理解用户研究

让我们先从“游戏用户研究”这一术语说起：这是指游戏产业中的游戏用户研究这一总学科。该学科也包括了一些职称，如可用性专家，游戏测试管理者以及玩家体验研究员。所以“游戏用户研究”是一种涵盖性术语。

就其本身而论，游戏用户研究也是众多游戏产业学科中的一员（游戏邦注：其它学科还包括动画，市场营销，编程，QA以及其它重要的开发和发行游戏过程）；而游戏用户研究在功能，研究目标以及方法方面有别于这些学科。

游戏用户研究的主要功能是测试并评估游戏设计的可行性，并最终帮助开发者创造出一款真正有趣的游戏。

但是仅此一点却不足以区分游戏用户研究，因为测试和评估游戏设计同样也是QA和市场研究的重要功能。而能够突出游戏用户研究的重要因素是，它是专门研究特定类型的评估，也就是基于“用户测试”的评估，并且比起QA或市场研究它拥有更多不同的研究目标，因此也需要使用不同的方法。

Crystal Dynamics和IO Interactive用户研究经理Janus Rau Sorensen以汽车研发为例来形容这种区别：“如果我们正在制作一辆小汽车而不是游戏，那么QA的任务将是寻找垫圈爆破，轮胎破裂，电路连接松动或者A/C故障的原因。他们需要努力回答‘怎么做才能让这辆车正常运行？’所以他们在此的主要研究目标是‘汽车’。”

“市场研究则将为潜在用户群体呈现汽车的相关理念，或研究消费者的习惯和喜好，尝试着回答‘怎么做才能更好地销售这辆汽车？’所以他们的主要研究目标是‘消费者与营销之间的关系’。”

“而汽车的用户研究则将进行‘用户测试’，即让潜在用户与汽车待在同一个房间，以观察该用户是否能够顺利进入车中，找到A/C按钮，打开引擎，且顺利地启动汽车从A点行驶到B点，并研究该用户是否觉得驾驶这辆汽车有趣。”

“而用户研究应该回答的问题是‘汽车与用户体验，满意度以及娱乐性之间的关系。’因此用户研究的目标是‘消费者与产品内容之间的关系和互动’，而基于这种互动去收集数据和信息的活动则被称为‘用户测试’。在游戏用户研究中亦是如此。”

所以简而言之，游戏用户研究是：

*强调特定测试的一门学科，以寻找阻碍最佳享受和可用性体验的因素

*能够帮助你尽早从潜在玩家身上获取可以付诸行动的重要反馈

*通常是由那些经过学术培训的专业人士执行，而非制作人

*拥有不同研究目标而有别于市场研究

*如果说QA侧重编程，那用户研究也就侧重于设计

游戏用户研究的方法，工具和技巧

根据Sorensen所称，总的来说，用户测试是指用户（如潜在用户）与产品内容之间的互动，并且测试者将基于“系统方法”追踪，分析，合成并传达这种互动。由此而论，“系统方法”也就是测试中所试用的方法，工具和技巧。

“方法”是指在特定环境下解决问题或应对挑战的严谨有序的方法，如“通过可用性测试能够察觉出角色在游戏世界移动中所出现的各种问题。”方法总是拥有一定的“观点”，或者说是帮助我们理解特定现象的理论基础。

最后，对于方法的应用还有一定的指导方针，即“技巧”，“工具”和“组织原则”。技巧和工具是关于一些实用性方法（游戏邦注：包括调查，观测技巧，访问技巧，数据挖掘，目测等）以及如何实践这些方法，而组织原则则是关于劳动分工，项目管理模式以及资源分配等内容。

注：不同工具和技巧可以用于多种方法中，而关于方法或工具的构成之间的区别还有待学术界做出进一步研究。

尽管所有游戏用户研究人员都在进行用户测试（因为开发者或用户研究员对于“游戏测试一词都不是很明确，所以作者便在此选择试用用户测试这个术语），但是不同工作室以及不同发行商之间所采取的用户测试活动也存在着很大的差别。这主要也是因为使用不同的方法，工具和技巧而带来的不同。

到底该使用何种方法，工具和技巧主要是受到时间和预算的影响，同时研究人员的背景以及当前用户测试的目标也直接作用于这种选择。

有些研究人员偏向于使用观察，视频录像，问卷调查等方法，通过面向小群用户（游戏邦注：一般8至10位）而快速收集关于主要游戏玩法和UI的相关数据。也有些研究人员更青睐于展开小组会议讨论各种游戏或者一款游戏中的多个元素。而关于采用何种方法都是基于用户研究人员致力于解决何种问题而定。

焦点小组和焦点测试

作为游戏开发者你也许会说“我并不关心你将它称为什么或者怎样做的。我只关心结果！”，所以我们应该采取一种真正有效的方法。通常当开发者询问以下问题时便会衍生出更多问题，即“我们是否能够成立一个焦点小组去测试我们单人游戏的部分内容？”或者“我了解了一些关于生物统计学的内容，你是否能够将其应用于我们的开放RPG世界中？同时，是否能够在任何特定时间提供给玩家多达20种的故事和支线任务？”很不幸，这些要求往往都会导致更多误解，并且只会更浪费时间而不可能创造出真正有成效的结果。

当我们在进行用户研究时经常会提及“焦点小组”和“焦点测试”这两个术语，并且原因也很简单。这两个术语的出现已经有段时间了，并且在过去几十年里它们一直作为市场研究中产品评估的重要方法。所以，既然你的游戏也是一种产品，为什么你不使用焦点小组呢？

用户研究中存在的问题是，焦点小组只是众多可行方法中的一种，研究人员并不会频繁使用它，如果有的话也是许多游戏用户研究小组一起使用。而对于外行来说，他们对于这个术语本身的理解就已经很模糊了。

很明显，焦点小组自然包含小组人员，那焦点又是指什么呢？是关注于衡量用户对于IP的定性态度，关于多少用户喜欢游戏故事或游戏玩法，或者用户觉得游戏有趣还是无聊，还是有其它看法呢？而对于这些内容，还存在其它更有效的方法和工具能够帮助研究人员进一步明确用户的真实想法。

《Focus Groups: Getting More Useful Feedback from Consumers》便是游戏用户研究中一位重要人物Bill Fulton写的一篇文章，他在Microsoft Game Studios（MGS，也就是现在的Microsoft Studios）创建了用户研究小组，而MGS的资深用户研究员Michael Medlock更是阐述了如何更好地利用焦点小组这种方法。

本质上来看，焦点小组的优点在于能够在早期阶段便形成各种理念，并衡量人们对于这种理念的普遍反应，但是这种方法同时也要求必须是受过专业培训的专业人士才能创造主观数据，并确保它们不会受到团体动态的影响。

但是如果你所追求的是关于玩家行为的主观和客观数据，那么选择少量的参与者去进行游戏的可用性测试将更加合适。

这使用户研究员能够观察并评估玩家的行动并将其与他们的陈述进行比较；从中经过专业培训的观察员便能够将其有效地结合在一起而找出玩家做出特别行为的原因。

并不能说焦点小组没用，它只是用于实现某特定目标的一种简单方法；而这种特定目标又是区别于用户研究员所瞄准的一般可估价目标。

对于大多数人混淆了用户测试与焦点小组这一问题，Fulton指出：“‘焦点测试’这一术语经常被等同于‘用户研究’，即并不是指一种特殊的方法，而是一种帮助‘外行’理解的涵盖性术语。”

同样地，Fulton也指出“大多数人在进行‘焦点测试’时也并未将自己定义为是‘焦点小组’（尽管他们真正在执行一些‘焦点小组’的任务），”从而导致更多人对这两个术语产生更多疑惑。

EA资深游戏用户研究员Rich Ridlen向我们阐述了与不知如何定义用户测试的开发者共事的经验：“我在EA遇到的许多开发团队都只是将‘焦点小组’或‘焦点测试’当成是将用户带进游戏测试中的方法。”

“他们理解并看到不同方法论（或工具）所存在的价值，如一对一方法，建立小团体，或者施行个案研究测试等。他们甚至知道如何与我们一起进行测试。但是不管我提醒他们多少次，他们都不会将自己定义为‘焦点小组’；我不会常常提及这个术语，除非我需要以此阐述一些特定的内容。”

解决方法

我们的建议是：如果你希望使用某种方法进行用户测试，只要直接告知你的用户研究员便可。

而如果你提出一些特殊的要求，如“建立一个焦点小组”，但是开发团队却会以这么做没有意义拒绝你——纵然你可能只是希望进行用户测试而与焦点小组无关。

你的用户研究小组可能会提出另外一种方法去测试你所提议的内容，而也许这种反建议会让你感到抓狂，你可能会说“这是我所提出来的，不是吗？！”这时候你将会不断衡量这两种方法，而浪费更多时间于目标的考究中。

所以，不论你扮演的是何种角色（是负责开发还是管理），你都应该为自己省下许多不必要的麻烦，并牢记用户测试是用户研究员的工作，他们有责任向你解释为什么要这么做，并解答你所布置的任务。如此，你便可以只是描绘出你的想法，而让游戏用户研究员去设计一次测试以检验你的想法是否有效（就像用户研究员的报告只会描写问题所在而不会阐述特定的解决方法一样）。

最后，同样需要注意的是，用户研究员能够帮助你更好地理解设计师的真正想法，并且他们所收集的数据也不会对你产生任何约束，反而将提供更多额外信息以帮助你创造出一款更加优秀的游戏。

（本文为游戏邦/gamerboom.com编译，拒绝任何不保留版权的转载，如需转载请联系：游戏邦）

Understanding User Research: It’s Not QA or Marketing!

by Maurice Tan

In this day and age, it is nearly impossible to think about developing a game without some form of systematic testing being performed at different stages of production. Testing has proven itself to be instrumental in ridding the game of bugs (QA), figuring out how to best sell the game (market research), and getting the product closer to the designer’s vision in terms of the player experience (user research).

The field of user research may be relatively young within both the games industry and the world of academia, but decades of multidisciplinary research in fields like ergonomics, usability, psychology, communications, and anthropology form the pillars that support researchers to answer that all-important question: Are there any unforeseen impediments to an optimal player experience, and how can the experience be improved?

While there is little doubt about the merits of testing in general, there can be some confusion about who does what kinds of testing. User research has its own lingo that can sometimes confound those who aren’t familiar with it. Especially with its diverse academic baggage, user research can produce jargon that means little to outsiders, or leads to misunderstandings (even amongst insiders).

For example, some terms like “playtesting” are used both in QA and user research, but in the two contexts can refer to very different techniques. General familiarity with decades of market research jargon such as “focus groups” has created further opportunities for confusion. As such, it’s no wonder that the term “focus testing” is sometimes mistaken as a synonym for “usability testing”. This article seeks to clear up some of these misunderstandings so we can hopefully all understand each other a little better.

How to Understand Your User Researcher

Let’s start with the term “games user research”: This is the name for the overall discipline that Games User Researchers occupy within the game industry. The discipline also covers job titles such as usability specialists, playtest managers and player experience researchers. “Games user research” is the umbrella term.

As such, games user research is a game industry discipline on par with other disciplines like animation, marketing, programming, QA, and other vital parts of the development and publishing process; it distinguishes itself from these other disciplines primarily via a) its function, b) its object of inquiry, and c) its method.

The primary function of games user research is, simply put, to test and evaluate the viability of the design with the ultimate goal of helping to make more enjoyable games.

However, this alone is not enough to distinguish games user research, as the testing and evaluation of design is also a function of QA and market research. The factors that do distinguish games user research are that it specializes in a specific kind of evaluation, namely “user test”-based evaluations, and that it has a different object of inquiry than QA or market research, and therefore uses different methods.

Janus Rau Sorensen, user research manager at Crystal Dynamics & , offers building a car as an analogy to explain this difference: “If we were making cars instead of games (in which case this site would be Carasutra), QA would be looking for blown gaskets, flat tires, loose connections in the electrical circuitry or a malfunctioning A/C. They would try to answer the question ‘To what extent does the machine work?’ and the main object of inquiry is ‘the machine’.

“Market research would present the concept of the car to a group of potential consumers, or explore consumer habits and preferences, and try to answer the question ‘To what extent can we sell this machine to these people?’ the main object of inquiry being ‘the relationship between the consumer and the sales pitch.’”

“A car user researcher, on the other hand, would run a ‘user test’ by putting a potential buyer in the same room as the car and see if this person can make it into the car, can find the A/C button, turn on the engine, drive from A to B without crashing the car, and perhaps investigate whether or not this person feels the car is fun to drive.

“The question the user researcher would be trying to answer is ‘To what extent is the interaction between the car and the user usable, satisfying and enjoyable?’ Therefore, the object of inquiry for user research is ‘the relationship and interaction between the consumer and the product content,’ and the principal activity in which data and information about this interaction is acquired is called ‘user testing’. This is also the case for games user research.”

In short, games user research is:

A discipline which stresses specialized testing to find impediments to optimal enjoyment and usability

A way to receive early, actionable feedback through data from the type of players who will end up playing your game

Usually conducted by professionals with academic training, not by producers

Different from market research because it has a different object of inquiry

To design what QA is to programming

The Methods, Tools and Techniques of Games User Research

According to Sorensen, a user test is, broadly speaking, a user (e.g., an actual representative from the potential target audience) interacting with product content while this interaction is tracked, analyzed, synthesized, and communicated in a “systematic way”. In this context, “systematic way” refers to the method, tools and techniques used.

A “method” is a disciplined and structured approach to solve a problem or a challenge within a given context, for instance “usability testing to detect errors in how a game character moves in the world”. A method has a certain “perspective”, or in other words a theoretical foundation for understanding a phenomenon.

Finally, a method has practical guidelines for its application: “techniques”, “tools”, and “principles of organization”. The techniques and tools are the practical instruments (surveys, observation techniques, interview techniques, data mining, eye tracking, etc) and how to apply them, and the principles of organization concern the division of labor, the project management model, the allocation of resources, and so on.

Note: different tools and techniques can be used in more than one method, and the difference between what constitutes a method or tool can be up for academic debate.

Even though all games user researchers do user testing (which is the recommended term for this activity, since neither developers nor user researchers are clear about what “playtesting” means exactly) there can be a huge difference in how the activity of user testing is carried out from studio to studio and publisher to publisher. This is caused by differences in the method, tools, and techniques used.

In turn, which methods, tools, and techniques are used depends on the time and budget available, as well as the background of the researcher, and the goals of the current user testing.

One group of researchers may prefer observations, video recordings, and questionnaires, using a small group of users (eight to 10) to get data on the key gameplay and UI issues very fast. Others may prefer group sessions for party games or for the multiplayer aspects of a game. It all depends on what problem the user researcher is trying to solve, but it is all games user research.

But What About Those Focus Groups and Focus Testing?

As a game developer you might think, “Ah, but I don’t really care what you call it and how you do it. Just get me the results!” and that would be a great approach to take. The problems instead arise when developers ask, “Can we do some focus group testing on this part of our single-player game?” or “I’ve read some stuff about biometrics; can you do that for our open world RPG? By the way, the player can have up to 20 story and side-quests at any given time.” Unfortunately such requests more often than not create opportunities for misunderstanding and wasted time rather than productive results.

The terms “focus group” or “focus tests” in particular seem to be often mentioned when user research is brought up, and it’s not hard to understand why. They are terms with a long history and are methods of product evaluation that have been used in market research for decades. Therefore, since games are still products, why wouldn’t you use such a focus group?

The problem with this is that in user research, a focus group is just one among a wealth of methods available, and not one that is often used, if ever, by a lot of games user research teams. To the outsider, the meaning of the term itself is as muddled as it can be.

Sure, it involves a group, but what is the focus? Is the focus on measuring qualitative attitudes towards the IP, on how much the users like the story or gameplay, if the game is experienced as fun or frustrating, or on something else? For all these aspects, there are other methods and tools that can offer better and more precise insights into the minds of users.

The article Beyond Focus Groups: Getting More Useful Feedback from Consumers by Bill Fulton, an important figure in games user research who started the user research group at Microsoft Game Studios (MGS, now Microsoft Studios), and Michael Medlock, Senior User Researcher at MGS, offers an excellent perspective on the use of focus groups.

Essentially, what focus groups are good for is generating ideas at a very early stage, and gauging a general response to a concept, but they require a highly trained moderator to yield good and usable subjective data that isn’t overly tainted by group dynamics.

However, if what you are after is both subjective and objective data on player behavior using a playable build of your game, a usability test with, usually, a small number of participants, is more appropriate.

This allows the user researcher to observe and evaluate the player’s actions and compare them to the player’s statements; a trained observer can combine the two to pinpoint the reason for a player’s unexpected behavior.

Focus groups aren’t necessarily useless or evil — they’re simply one method used for a specific goal. This goal differs from the generally evaluative goals that user researchers try to attain.

On the subject of the confusion between user testing and focus groups that exists today, Fulton points out that “the term ‘focus testing’ is often used in the same way as ‘user research’ — not to denote a specific method, but as an umbrella term for ‘bringing in outsiders’.”

As such, Fulton suggests that “most people are not limiting themselves to what we’d call ‘focus groups’ when they say ‘focus testing’ (although they do sometimes mean ‘focus group’),” which creates opportunities for confusion.

The experience of working with developers who don’t know what to call user testing is shared by Rich Ridlen, senior games user researcher at EA: “Many of the dev teams that I work with at EA use the term ‘focus group’ or ‘focus test’ to mean any method wherein consumers are brought in to test,” Ridlen notes.

“They understand, have observed, and see value in different methodologies (or tools), like one-on-ones, small group, high N survey-based tests, etc. They have even correctly ascribed methods to tests from working with us. However, it’s all ‘focus groups’ and it doesn’t matter how many times I correct them or that I never use that word unless that’s specifically what I’m describing.”

Okay, Just Tell Me How to Get the Solution to My Problem

Here is what we suggest: If you want something to be user tested, perhaps it is best to simply tell your user researcher that you want them to perform user testing on a certain aspect of the game.

When you ask for something specific like “do a focus group”, then that team’s response might be that there’s no point in doing so, even though perhaps you just wanted something user tested that has nothing to do with running focus groups.

Your user research team will likely propose another method for testing what you want, and perhaps that counter proposal will make you go, “Yes, well, that’s what I asked for, isn’t it?” and eyes will be rolled on both sides, with ten minutes having been wasted on just communicating the goals.

So, whatever your role is (development, management, etc.), save yourself some trouble and remember that user testing is what user researchers do, and it is their job to explain to you how they will do it and for what reason given the problem area that you lay out for them. As such, it is best for you to describe what your concerns are, and the games user researcher can then design a test that can examine the validity of those concerns (just like the report from the user researcher only describes the problem areas and does not prescribe specific solutions).

As a final point, it should be noted that user researchers are ultimately here to help you realize the designer’s vision, and as such the data they collect is not intended to dominate or limit you. Rather user research aims to simply provide additional information to help you in building a great game.

As an aside, if you are a user researcher (or just interested in the discipline) and haven’t joined the GUR-SIG LinkedIn group yet, you can do so here. Joining allows you the opportunity to participate in discussions such as the one that led to this article, connect with others in the area, and share your insights into methodologies and novel approaches.(source:GAMASUTRA)