作者：Aaron Lee

所有游戏都要接受测试。但看看游戏零售网站或应用商店的用户评价，显然不是这么回事。

测试工作室努力执行他们的完善游戏的任务。他们找出游戏中的有害漏洞，为游戏工作室提供修复漏洞和指导方案。

继我们的游戏本土化报告后，我们把注意力转向QA。因为新游戏平台层出不穷，游戏行业仍将继续风云变幻。虽然下一代主机即将诞生，但外部QA工作室为他们能够最先接触到新主机和AAA项目感到兴奋吗？还是他们准备把手机当作未来的收益来源？

质量

过去，QA总是被当作游戏开发中“单调乏味的”最后一个环节。有些发行商武断甚至愚蠢地认为，只要游戏不受太大影响，就不值得花时间做测试。

Testronic工作室的游戏运营副总裁兼英国区经理的Alastair Harsant指出，现在情况有改变了：“QA已经是一种成熟的服务了，在开发周期中越来越受重视和认可。随着越来越多社交、在线和手机游戏诞生，用户留存率作为评估成功的指标越来越受关注。现在，发行商比过去更加强调游戏的完美和无漏洞。”

在过去的18个月里，QA工作室本身也受到新考验。不出意料，也许正是手机游戏开发使得这么多改变渗透到这些工作室中。

José Ramón Sagarna是Triple A Testing工作室的主管，他确认道：“开发者更加关注手机游戏的质量保障，不只体现在关注测试结果，还体现在通常为大主机游戏项目保留完整的质量保障计划。”

同时，Babel首席执行官Richard Leinfellner认为，正是社交游戏的迅速普及影响了现在的QA公司的商业前景。

他表示，“主机游戏和休闲游戏已经一决胜负了，结果是，主机选择少数成功的AAA游戏，推出大量续篇，因为续篇比较没风险。这些游戏的QA也一样，但规模比以前大了，特别是考虑到多人模式的重要性。”

“社交游戏几乎完全相反；无论是小预算还是大投资，社交领域中出现了许多新游戏、使用各种语言和各种设备，简直是开发西大荒。与动辄持续两个小时的主机游戏相比，游戏时间零零碎碎的社交游戏更像肥皂剧，每周加入一点新内容。

“在许多方面，游戏机游戏好比电影，休闲游戏好比电视剧，所以我们在这些媒体资源中还有很多东西要学习，而许多电影人也必须好好研究电视行业。”

适应

Leinfellner的评估也得到了一些QA从业者的响应。根据几个工作室最近的数据，申请外部测试的主机游戏项目显著减少了。但因为手机和社交游戏的野心越来越大，客户对平台的期望和对测试的要求也随之提高，数量的减少被质量的提高弥补了。

这种变化也体现在《无尽之剑》系列、《银河风云2》和《极速赛车》等游戏中，这些都是针对智能手机和平板、具有精良的3D图像的游戏。随着衍生类型的增加，这些游戏的特征和功能也增加了。

在线多人模式（无论是异步的还是即时的）和其他模式出现在手机游戏中，已非罕见。在游戏开发者大会上，Havok等公司宣布发行针对3D手机游戏开发的免费工具，意味着有甚至更多开发者选择将游戏做成3D的。

而对于QA，这意味着与开发者和各种规模的团队建立一种新的关系。面对没有大预算的新客户，QA外包商将改变他们的服务以满足特定的要求，如AAA测试的Sagarna所说的：“现在，小开发者不需要大预算就能开发游戏。通过不断的升级和改造某种做法或流程，新平台催生了新的开发和测试方法。灵活和以开放的心态寻找项目要求和项目成本之间的平衡点，是非常重要的。”

QA工作室VMC Game的主管Ben Wibberley也同意以上观点。

Wibberley指出，“旧模式正面临挑战，这模糊了传统的开发、发行和推广这间的界线。然而，我们并不同意发行商已死的说法，游戏商业的发展方向正在从‘商业对商业’模式向‘商业对消费者’模式转移。”

“几乎每一周都会出现新游戏公司和设备，商业模式不断发展，所以关键是要保持灵活和始终站在理解客户需求的前沿。

“例如，我们成功地制作了一款针对独立游戏开发者的客户端程序，我们确保为我们的客户提供高层次的服务。”

项目经理Harrison Baker也赞同。“当我们开始考虑手机平台的机遇时，我们很难无视手机开发者的要求。”

“我们希望根据客户要求制定相应的服务——无论是什么平台或领域或团队规模。我们希望能够为各种预算的项目提供最好的QA服务，这样这些公司就能发行高品质的应用了。”

显然，QA不再“以不变应万变”，而是应不同客户的需求制定方案。

自动化测试

就他们提供的服务和使用的技术来看，QA公司的工作方式也在不断发展。

与VWC的独立程序一起，Testronic工作室已经招募玩家进行研究和支持它的UX测试，并在它的Burbank办公处开设了特殊的UX实验室。同时，Babel已经在TestMyApps推出专门的应用测试和本土化服务。这些服务旨在提高产品的适应性和专门化。

Testronic工作室的Harsant表示其项目有更长的周期，并且他们已经在处理更复杂的项目了，这通常必须调整时间安排。他表示，公司对自动化测试能力更有兴趣了。

虽然他并不认为自动化技术已经可以取代人为测试，但他确实认为：“对自动化的兴趣也在增加。虽然我们在这个领域有专业知识，并且自动化非常有助于某些测试，但不能解决所有程序的问题。人为测试仍是主流，特别是为某些大客户提供在线测试操作。”

自动测试通常是指在无用户互动的情况下，用于验证游戏的部分功能的代码或数据。Wibberley表示，VMC正在为某些大客户展开大量自动化测试工作。

“测试自动化通常被推销为解决任何类型的软件开发问题的‘高招良方’。然而，我们把测试自动化作为有才能、创意的工作人员所使用的工具，支持高品质游戏的开发。借此，有价值的开发者开放或测试员将不承担重复性工作，而把时间腾给更重要的任务。

“高效外包现在已经标准化了且不会消失，但人力并非总是越多越好。反应灵敏的测试工作室正在迅速向工具和测试自动化迈进。”

下一代主机

自动化当然会与下一代主机的要求同步发展。但随着手机游戏的普及，QA工作室得到新的客户来源，主机游戏开发对其商业的重要性就不那么确定了。

Testology的Baker表示，“多亏了手机平台，我们的商业有了显著增长。”

“我们希望工作室能更加投入于下一代主机测试中，至于多少程度，这取决于需求。在过去两年，我们已经看到外包需求的显著转变，主机市场对外部QA服务的需求减少了。”

越来越多大发行公司内部自组QA团队，印证了Baker的观点。

Babel的Leinfellner有所保留：“现在，很难说下一代主机的走向。硬件的发展也大有潜力，令人兴奋。然而，开发成本也高得令开发者和发行商望而却步。

“个人认为，我有一点儿担心只有游戏大作才能供得起这么高的开发成本，它们没什么风险，所以可以大量推出续作。另一方面，索尼和微软选择PC平台。这显然有助于开发者充分利用他们的技术投入，因为我们已经知道如何测试网络和社交PC游戏，所以我们的作用会很大。”

Universally Speaking的运营经理Anna Wojewodzka表示：“因为主机现在的情势不佳，项目必然会减少，这个阶段的游戏机周期可以预料。我们是所有平台的测试专家，希望业务继续涵盖手机、在线和游戏机，但保持比例平衡可能有困难。

相反地，其他人的观点更务实。Triple A Testing工作室的Sagarna表示主机是“主心骨”，所以训练和熟悉新游戏机将会是新一轮挑战。Harsant指出下一代主机对Testronic的商业“非常重要”。而VWC的Wibberley的观点是“只要还有市场，无论我们的客户选择什么设备，我们就会提供支持那种设备的服务。”

Rupert Young是EC-Interactive的总经理，他评估了当前游戏的状况后觉得“很难设想有什么硬件能完全改变我们QA的方式或根本地动摇我们作为QA供应商的期望。”

传统的主机发行商和开发者是否会大量求助于外部QA工作室，尚不得而知。

但Young的最后的想法确实揭示了这个行业的真实情况：“到目前为止，当我们专注于我们现有的平台，我们面临的最大挑战是确保我们不会错过或损失下一个大平台或QA服务机遇。不断训练我们的员工和决定以什么设备测试作为收益来源，是永远的挑战，但我们喜欢迎接挑战。”

内部QA的优势

外部QA工作室面临的挑战，部分来自发行商自组内部QA团队。

Reflections的QA主管Joseph Rogers表示，内部QA团队具有多种优势。这近20年以来，内部QA一直是他的工作室的一部分，所以论经验，该团队与育碧开发团队接近。并且还具有早期从事项目的经验，在质量和成本上，它可以保持竞争优势。

“尽管一定程度上所有公司都可以自组QA团队，但大公司显然更容易做到。有了大QA团队，意味着它将为你提供持续性的反馈和支持”，但他也承认内部QA存在缺点：“最大的挑战是管理这个团队的内部变更。来自经验和反复使用某种方法的优势可能阻碍变革——毫无疑问，我们的行业总是风云变幻。”（本文为游戏邦/gamerboom.com编译，拒绝任何不保留版权的转载，如需转载请联系：游戏邦）

Games QA: A testing road ahead

by Aaron Lee

Testing is something that every games should receive. But look on any retail website or app store and it’s all too clear from customer reviews that this isn’t the case.

Quality assurance firms strive to be thorough in their mission to give games the polish they need. They rid games of hurtful bugs and provide game studios with guidance to fix the holes that are holding their game back. They are the discreet lab coat-wearing assistants who ‘test ’em to destruction’ – only, without the lab coats.

Following our report on games localisation (Develop #137), which many of the same firms in this article also handle, we turn our attention to QA. With so many new games platforms, the sector is continuing to experience rapid change. The next generation is just starting to awaken, but are external QA firms excited by the new consoles and the triple-A projects they will herald, or do they see mobile as the source of their fortunes going forward?

QUALITY MATTERS

First things first, however. In the past, QA has typically been viewed as the ‘unglamorous’ end of games development. Some publishers have boldly – or perhaps foolishly – decided that its not worthy of devoting time to, only for their product to feel the impact.

Testronic Labs VP of games operations and UK site manager Alastair Harsant says that things are different now: “QA has certainly matured as a service and has become an increasingly important and recognised part of the development cycle. With more social, online and mobile titles, there is an increased focus on customer retention rates as a measure of success. It is more important than ever to launch a perfect and bug-free title.”

QA firms have themselves been put to the test in new ways over the last 18 months. Unsurprisingly perhaps, it is mobile development that has proliferated many of the changes these firms face.

“Developers are focusing more on mobile quality assurance,” confirms José Ramón Sagarna, head of testing at Triple A Testing. “Not only with end of development testing, but also with a full quality assurance plan usually reserved for main console titles.”

Meanwhile, it’s the rapid uptake of social games that Babel CEO Richard Leinfellner feels has affected the business most in terms of what is now expected of QA companies.

“Console and casual have been slugging it out, with console going for very few triple-A megahits, and lots of sequels as few are taking risks. QA for these is the same, but bigger than before, especially due to multiplayer being so important,” he says.

“Social is almost the total opposite; we see everything from tiny to big budget, lots of new IP, lots of languages, tons of handsets, it’s really like the wild west. It’s really fragmented with some jobs lasting literally two hours and others more like TV soaps, which just go on and on as new content comes in on a weekly basis.

“It’s really like movies – or console – versus TV – or casual – in many ways, so we have a lot to learn from these media sources and many movie people now need to learn the cut throat world of TV.”

TAILOR-FITTED

Leinfellner’s comments echo that of some of his fellow QA practitioners. Based on what several firms have said, there seems to have been a marked reduction in the number of console projects that have been submitted for external testing. But this reduction has been countered by higher expectations from clients across alternative platforms and a need for more rigours testing as mobile and social titles become ever more ambitious.

This progression can be seen in the likes of the Infinity Blade series, Galaxy on Fire 2 and CSR Racing, all of them graphically-accomplished 3D games that have been created for smartphones and tablet computers. And as the number of relevant genres expands, so too has the feature set for such games.

It’s not uncommon to see online multiplayer, be it asynchronous or real-time, and other additional modes appearing in mobile titles from the get-go. And announcements by companies like Havok at GDC last month to release free tools for 3D mobile development means that we may see even more developers choosing to realise their visions in the third dimension.

What this has meant for QA is new relationships with developers and teams of varying sizes. Dealing with new clients that don’t have huge budgets to spend has led QA outfits to adapt their own services to suit specific needs, as Triple A Testing’s Sagarna explains: “Now, small developers don’t need a huge budget to develop. New platforms brought new developing and testing ways, with constant updates and the need to reinvent certain methods or procedures. It is important to be flexible and have an open mind to find a common ground between the project needs and the project costs.”

This is something that chimes with Ben Wibberley, director of games at QA specialist VMC Game Labs.

“Established models are being challenged, which is blurring the lines between the traditional roles of development, publishing and distribution. Whilst we don’t subscribe to the idea of the death of the publisher, the games business is moving from a business-to-business to a business-to-consumer model,” Wibberley tells Develop.

“New games companies and devices appear almost every week, and the business models are constantly evolving, so the key is being agile and continuing to stay on the leading edge of understanding our clients’ needs.

“For example, we’ve successfully created an independent games client programme to specifically support indie games developers and ensure the high level of service that we expect ourselves to provide to all of our games clients.”

The same is true for Testology, says project manager Harrison Baker.

“When we began to consider the opportunities with mobile platforms, it was difficult to ignore the requirements of the mobile developers,” he says.

“We wanted to ensure that our services were tailored to all client requirements – no matter the platform, or industry or team size. We wanted to be able to provide the very best QA services with every type of budget so that these companies could release applications with impressive stability and functionality.”
Evidently, QA is not longer about ‘one size fits all’, but a tailored approach to clients’ needs, whatever their situation.

AUTOMATED TESTING

Just how QA firms are performing their task is also evolving in terms of services they have and the technology they use.
Along with VMC’s indie programme, Testronic Labs has enlisted Player Research to bolster its UX testing, and has opened a special UX lab at its Burbank office, while Babel has launched a dedicated app testing and localisation service in TestMyApps. These services are intended to improve the flexibility and specialism on offer.

Testronic’s Harsant says its projects have longer tails, and over time they have been handling more complex projects, which has often necessitated scheduling changes. In conjunction with this, and likely as a result, he says there has been more interest in the company’s automated testing capabilities.

“Interest in automation is also increasing,” he says, though he doesn’t feel the technology is ready to replace human testers just yet. “While we have expertise in this area, automation remains a valuable aid to certain testing, but not a solution to all procedures. The human element remains prevalent, and where appropriate we provide on-site testing operatives for some major clients.”

Automated testing generally refers to code or data that is used to verify the functionality of portions of a game without any user interaction. It’s an area that Wibberley says VMC is doing a lot of work in with some
high-profile clients.

“Test automation is often sold as a ‘silver bullet’ to solve all manner of software development problems. However, we view test automation as a tool used by talented, creative people to support the development of quality games. Repetitive tasks are automated to free up valuable developer or tester time for more critical tasks.

“Ideas about effective offshoring or outsourcing are standard practice now and that will not go away, but more bodies isn’t always better. The most agile organisations are moving towards tools and test automation at pace.”

ON TO THE NEXT ONE

Automation is certainly something that will continue in step with the demands of the coming generation of games consoles. But with the rise of mobile giving QA firms a new stream of clients, the significance of console development to their businesses is in dispute.

“Our business has seen a significant percentage increase in favour of mobile platforms,” offers Testology’s Baker.

“We will anticipate Testology being heavily involved in next generation games console testing, but to what extent will be determined by demand. In the past two years we have seen a significant transition in outsourcing requirements, and the console market seems less in need of external QA services.”

A gradual move toward in-house QA teams by major publishers (See ‘Insider advantage’ below) would certainly corroborate Baker’s view.

Babel’s Leinfellner also has reservations: “Right now, it’s hard to tell where next-gen will go. The hardware is exciting and the potential for content is too. However, the development costs are also scaring developers and publishers.

“Personally, I am a little concerned that only the big guys will be able to afford it and none of them will take any risks, so expect sequels galore. That aside, it appears both Sony and Microsoft are choosing a PC architecture. This clearly will help developers make the most of their technology investment, and since we already know how to test in the connected and social PC world, it will put us in a great position.”

“With the current disruption in console, there are definitely less projects around and, of course, at this stage of the console cycle this is to be expected. We are experts in testing on all platforms and want to maintain a mixed portfolio of mobile, online and console, but maintaining that balance can be a challenge,” admits Anna Wojewodzka, operations manager of Universally Speaking.

By contrast, others have a more pragmatic opinion. Triple A Testing’s Sagarna says its a fact that consoles are “here to stay”, so training and familiarising themselves with the new architectures will be a recurring challenge. Harsant says the next-gen consoles are “very important” to Testronic’s business, while VMC’s Wibberley takes the view that “as long as there is a market and our clients are making games for any device, we will provide test support on that device”.

Rupert Young, managing director of EC-Interactive has taken stock from the current generation and feels that is “hard to envision any hardware completely reinventing the way we do QA or fundamentally changing what is expected from us as a QA provider”.

It remains to be seen whether traditional console publishers and developers will return in significant numbers to external QA firms.

But its Young’s final thought that holds true for this sector: “By far the greatest challenge we face, when we’re focusing so closely on our existing set of platforms, is to make sure we don’t lose sight or get left behind when the next big platform or QA service opportunity surfaces. Making sure our staff are continually retrained and deciding on what equipment to source is a never-ending challenge, but one that we relish.”

INSIDER ADVANTAGE

Part of the challenge for external QA firms comes from publishers own internal teams.

Reflections lead QA Joseph Rogers says there are many advantages to internal QA teams. The internal QA structure has been part of his studio for almost two decade now, so as well as experience, the team’s proximity to Ubisoft’s development team and its earlier involvement in projects means it can maintain a competitive advantage in terms of quality delivered and overall cost.

“Although in-house QA on some scale can work for all companies, it’s obviously easier for larger companies. Having a large QA structure means you have constant and continuous production feeding and supporting this structure,” he suggests, but admits there are weaknesses: “The biggest challenge is to manage change inside the structure. The benefits coming from experience and the multiple iterations of certain approaches can cause some bumps on the way to change – and there’s no doubt our industry changes continually.”(source:develop-online)