作者：Lewis Pulsipher

我经常对学生说：“游戏测试至关重要。”但是往往有很多电子游戏开发公司经常忽略了这一步骤，同时也有不少游戏公司即使做了游戏测试也遭受了失败。

去年发行的《Final Fantasy XIV》便是一个典例。《Final Fantasy》游戏经常受到赞扬，但是这款新发行的游戏却遭到了评论者的批评。它在Metacritic（游戏邦注：Metacritic是专门收集对于电影、电视节目、音乐专辑、游戏的评论的网站）网站上的得分是49，这个分数对于一款视频游戏来说真的是糟糕透了（请点击此处链接查看其评论内容）以下是从Metacritic上摘录的一些评论：

“当看到官方公布的《Final Fantasy》这一章节后，我很难相信也不想去接受这款游戏，因为它不仅成为了这系列游戏中最差的一部，也可以说是我们玩了这么长时间的MMORPG游戏中最糟糕的一部。显然Square-Enix已经失去昔日的魄力了。”

“《Final Fantasy XIV》决不算是一款完全成型的商业化游戏。我们建议开发者进一步完善游戏体验，调整游戏中的缺陷，但是即使这样也不能够完全挽回残局。”

GameSpy（给出了40分的评价）（游戏邦注：也被称做 GameSpy Industries, 是IGN Entertainment下属的一个部门）：“在整款游戏中，我们总是带着一些疑惑感进行游戏，且会时不时中断游戏，甚至会因为愤怒而终止游戏。是的，对于我来说这款游戏就像是一间很脏的厕所！”

PC Gamer（给出了30分的评价）：“这是一款肤浅的，节奏缓慢且基调沉重的大型多人网游，它活生生地被一个糟糕的游戏界面，以及一些列没有意义的限制因素而毁了。”

难道Square-Enix就没有对《Final Fantasy XIV》进行游戏测试？不，他们做了。我找到了他们所做的公开测试的内容。“每个人希望抢鲜体验这款将于2010年在PlayStation 3和PC上同步发行的MMORPG游戏的玩家，都可以在官方测试网站注册。参与测试的玩家不需要任何MMO游戏经验，那些被选中的玩家将在游戏的Windows版本中进行测试。”

final-fantasy(from getps3games.com)

所以到底发生了什么事？难道他们只挑选了一些该游戏狂热粉丝参与测试？还是他们完全忽视了游戏的测试结果？但是似乎Square Enix不像会浪费大把时间而推出一款还未成熟的游戏，因为这家公司毕竟已是业界里极有威望且表现出众的电子游戏发行商。

所以，为什么会在游戏测试中得到“错误的肯定答案”？

首先，你可能是让那些很了解游戏但是却不能明显指出毛病的人去测试游戏。而这只可能是发生在让内部员工进行测试的情况之下，但是在这个例子中，《FFT 14》采用的是公测。

其次，即使进行的是公开测试，但吸引来也只可能只是一些狂热的粉丝，无论你推出什么产品，他们都只会鼓掌叫好，只要涉及自己拥护的游戏公司，他们就失去了批判能力。这确实是一种常见情况，也正是为何你得找到更多不了解你的产品的用户进行测试，以免陷入这种局面的原因。

第三，你可能忽略了测试结果，而说道：“哇，我就知道这是一款好游戏。”我确信这是频繁发生在微软的情况，他们时常确信无疑：“我们知道什么才是最好的。”我可以列举出微软的很多例子，但是就视频游戏而言，最典型的一个案例便是微软最初发行的XBOX控制器。很显然那是一款专门为那些身高2米且手大的人而设计的游戏机。那时候微软还一直宣称这款游戏机很好没有什么缺陷，但是最后还不是自己将其控制器调整到最适合的大小。

第四，也许你问错了问题，而那些测试者也并未完全投入于游戏测试状态。经常会发生这种情况，就是你向别人询问看法，但实际上他们的所言与所行并不一致，你应该静静观察他们的操作行为，不要对其进行干涉，然后以此得出更准确的结论。网站可用性测试专家Jakob Nielsen总是告诫我们，如果想得到更准确的数据，就不要一味地区询问别人将做什么或者已经做了什么。

第五，有些主事者出于节省预算的考虑，会按照项目进度安排所制定的发行计划，在原定时间将未完善的游戏推向市场。但我认为Square Enix这种大型游戏公司并不需要这种财务上的顾虑。

早前我把这篇文章发表一个博客里时，一名曾经担任《Final Fantasy XIV》评测员的评论家说道，当他看到这篇文章的标题时立马便想到了《FFT 14》这款游戏。“每个人都说游戏还不能发行。评论者一直在向Square Enix汇报游戏缺陷。实际上我甚至想不起关于这款游戏有多少正面的评述。有些人说道‘这款游戏的理念不错，但是需要做进一步的完善。’据我所知，在测试到正式发行这段期间，除了多开放了一些额外场景以及提高硬件效率外，这款游戏并未做出任何改变。”

Square Enix并不想改变原有的游戏设计而只是想通过测试寻找游戏漏洞，我想这也是游戏测试失败的另一大原因。比起改善游戏设计，他们更想通过测试对游戏发布进行预热宣传。

我曾在罗利（美国北卡罗来纳州首府）举办的电子游戏大会（现命名为East Coast Game Conference）上见到Jason Schklar，他们公司的经营方向是帮助那些不能自行测试以改进游戏设计（与旨在寻找游戏漏洞的测试截然相反）的电子游戏公司测试游戏。很多电子游戏公司只是让内部成员对游戏进行测试，而这种做法是不对的。因为那些内部人员有一部分是属于游戏开发团队成员，而他们对游戏都很熟悉，如果能够发现游戏缺陷早就发现了。你需要在目标市场里寻找一些普通玩家来帮助你测试游戏，以发现游戏设计是否适合游戏玩家。

此外，即使电子游戏是一个不寻常且奇特的商业市场，但是却有很多电子游戏开发者总是想当然地认为自己喜欢的东西就是大众喜欢的。

Jason说道：“我们工作的真正目标便是确保玩家的游戏体验与设计者想要玩家感受到的游戏体验相一致。而这有可能造成：

玩家体验=设计愿景+“玩家喜欢游戏”

玩家体验=设计愿景+“玩家不喜欢游戏”

玩家体验优于或逊于设计愿景”

在《FF XIV》中出现了以上的第二，三种可能，但是Square Enix却并未意识到这种情况，或者说不以为然地忽视了这两个结果。（本文为游戏邦/gamerboom.com编译，如需转载请联系：游戏邦）

A Really Big Playtest Failure

by Lewis Pulsipher

“Playtesting is sovereign”, I like to say to students.  But it has taken a long time for videogame companies to recognize this, and there’re still examples of complete failures to do playtesting right even when they play test.

An example from last year is Final Fantasy XIV.  Final Fantasy games are normally praised but this one has been trashed by reviewers.  It’s score on Metacritic (an aggregator of reviews) is 49, which is really bad for a videogame.  http://www.metacritic.com/game/pc/final-fantasy-xiv-online/critic-reviews

Here are some excerpts:

“It may be hard to believe, and painful to accept, but the last ‘official’ chapter of Final Fantasy is a huge disappointment, and it not only becomes the worst chapter of the saga, but also one of the worst MMORPG we have played in a long time. It’s obvious that Square-Enix is no longer what it used to be.”

“Final Fantasy XIV is in no way, shape, or form ready for commercial release. We suppose there is potential down the line for the experience to improve with patches and tweaks, but that doesn’t justify why the game is in such shambles at launch. ”

GameSpy (gave a 40).  “Extended stretches of confused incredulity, punctuated now and then by bursts of intense anger. And yeah — I just equated FF XIV with a filthy bathroom.”

PC Gamer (gave a 30).  “A shallow, slow, grind-heavy MMO crippled by a horrible interface and nonsensical player limitations.”

So didn’t they playtest it?  Yes, they did!  I found evidence of an open beta: (http://www.ffxivvault.com/ffxiv-beta-information.php and http://www.wired.com/gamelife/2009/12/final-fantasy-xiv-beta-test-now-recruiting/).  “Anyone that wishes to be among the first to play the new MMORPG, which will launch on PlayStation 3 and PC in 2010, can sign up at the official beta site. Square Enix says that no prior MMO experience is required, and players selected to be in the beta will test the Windows version of the game.”

So what happened?  Did they test only with fanboys and girls?  Did they ignore the results?  It seems unlikely that they ran out of time and put the game out before was ready, because Square Enix is one of the more respected and consistently good videogame publishers.

So what can happen to give a “false positive” from playtesting?

First, you could be playtesting with people who know the game so well that the problems just don’t register.  But this would be likely only if the people within the studio were the playtesters, and in this case it was an open beta.

Second, you can have open playtesting but somehow you only get fanboys and fangirls, people who will say good things about your product no matter what.  The have lost their critical faculties where you are concerned.  Perhaps this is what actually happened.  This is why you want to try to get many playtesters who don’t know your products and you aren’t prone to this kind of thing.

Third, you could just ignore the results, saying “oh well we know it’s good”.  I’m convinced that Microsoft has done this quite frequently, adding smugly “we know what’s best”.  There can be many examples but in the video game category the most famous one is the huge controller that Microsoft included with the original Xbox.  Apparently it was sized for people my height (2 meters) with correspondingly large hands.  According to what I’ve read Microsoft continued to maintain that it was fine but finally changed to a much more normal sized controller.

Fourth, perhaps they asked the wrong questions and yet didn’t pay attention to the actual play.  Frequently if you ask people what they think you’ll get a different story than if you simply watch them and listen to them without interfering.  Jakob Nielsen, the guru of web usability, always cautions you to watch people, not ask them what they will do or what they have done, because you get better data.

Fifth, though it seems unlikely in this case, the bean counters can win out in the battle over schedule and the game can be released because it’s scheduled to be released even though it’s not right.  I supposed Square Enix is big enough to avoid the schedule trap. Maybe not.

When I originally posted this in one of my blogs one commentor who had been a Final Fantasy XIV playtester said that when he saw the title of the post he immediately thought of FF14.

“Everyone was saying it wasn’t ready.  All of the flaws that the critics mention were reported to SE over and over again.  In-fact I recall very few truly positive comments on any of the forums. A few people saying “such-and-such is a neat idea, but needs a bit more development.

As far as I can tell, absolutely nothing was changed between the beta and going live, aside from extra areas being opened up and some souped-up hardware.”

This brings up another possible reason for playtest failure, that there was no intention to improve the design of the game but only to check for bugs.  They may also be much more interested in building up prerelease hype than it actually improving the design of the game.

I met Jason Schklar at the video game conference in Raleigh (now called East Coast Game Conference), whose company arranges playtesting-the-design (as opposed to playtesting-for-bugs) for video game companies that cannot, or do not have the time to, do it themselves.   Many video game companies only playtest for design internally, which is surely a mistake.  The people who are part of the production team are often too close to what they’re doing to recognize when there are design flaws.  You need ordinary players to try the game, people who are your actual target market, to find out whether the design is suitable for the intended audience.

Furthermore, many video game people seem to make the mistake of thinking that what they like is what everybody likes, even though videogame production people are by definition strange and unusual, and not typical of any commercial market.

Jason said “The real goal of my work is to ensure that players experience the game as the designers intend the game to be experienced. This can result in:

* Player Experience = Design Vision + “Players enjoy it”

* Player Experience = Design Vision + “Players do NOT enjoy it”

* Player Experience <> Design Vision”

In FF XIV some combination of the second and the third happened, but Square Enix didn’t recognize it, or wishfully ignored it.（source:gamasutra）