作者：Ryan Keable

我发现，许多小型团队已经意识到QA能够对制作产生的影响已经超出任何人的想象。你能够更有效地管理QA，游戏的开发就会越快，而且它们也能够得到更高程度的润色。在我们最新游戏的开发中，我们采用了某个QA程序，将通过下文与你分享某些技巧和建议（游戏邦注：作者是澳大利亚独立游戏工作室Anomalous Interactive联合创始人）。

漏洞跟踪和标签管理

这是QA程序中最重要和最显而易见的部分。当你写下漏洞时，就是在识别和分类游戏中需要被修正的问题。正确地报告漏洞信息并清楚地将其分类会让相关问题负责人的工作更加简单。

需确定的要素包括：何人、何物、何时、何地和如何。

何人：指当玩家可以从多个角色、模式和场景中做出选择时那些与玩家相关的选择。这是确定如何修补漏洞的首个步骤。

何物：发生什么问题？使用你所掌握的知识，用最简单的术语写下所发生的问题（游戏邦注：细节越多越好）。

何时：问题在何时发生？在游戏过程中的何时发生等问题。

何地：指的是漏洞发生的地点，即漏洞发生的关卡和场景等。

如何：在适当的地方写下重新制作的步骤。这很重要，因为会帮助程序员重新制作错误之处，并在代码中进行恰当的跟踪，看看究竟发生的是什么情况。

注意：漏洞的原因并非总是显而易见的，应当尽可能对程序员给予帮助。

在某些情况下，列出相同类别的问题也是有好处的。这对UI之类的游戏内容有所帮助。单个UI屏幕中找到的漏洞可以分组到相同的标签中，以任务的形式来呈现。这可以帮助你防止标签系统中充斥过多的标签，以及寻找相同领域标签的麻烦。

类别：

在你写下漏洞信息后，各种跟踪系统让你可以创造出诸如“代码”、“艺术”和“设计”等开发类别。用这些关键词来分类漏洞可以帮助你确定跟踪系统。

优先顺序：

确定漏洞的主次顺序对于工作流程来说非常重要。当你的系统中包含上百个漏洞时，以优先顺序等级来组织标签可以帮助制作人和负责解决问题的标签所有者决定要从何处开始。组织优先顺序还可以帮助确定长期和短期的工作流程。

提交测试：

当标签所有者已经确定他们的漏洞被解决时，就要对这个解决方案进行测试。为标签创建“测试”，将所有需要同时测试的标签分组。这可以让制作和QA负责人了解重新测试需要的工作量。分配任务也应当使用此类程序，这样QA就会知道何时新功能已被执行并为测试做好准备。

交流和评论：

你的标签系统和工作流程应当支持报告者和所有者进行充分的沟通。从分配到测试再到最终的解决问题，标签在报告者、所有者和测试者间的所有传递流程都应该有评论内容。这对项目的重复制作也有一定的好处。

我在QA岗位上最棒的经历是，在我和UI程序员配合解决具体游戏漏洞或可用性问题时，我们二人创建起基于评论的对话。这些简单的对话使得我们无需依次解决问题，也可以避免系统中出现同种类别的多个标签。

以下以《Wii AFL》这款游戏为例：

我的漏洞报告：

1、伤害键悬浮状态缩放比例不当

2、“玩家姓名”出错

3、没有通知信息显示玩家何时从伤害中康复

注：能否在伤害领域中添加文本信息来呈现康复情况？

程序员的回应是：

1、伤害键悬浮状态缩放比例不当——已修复

2、“玩家姓名”出错——已修复

3、没有通知显示玩家何时从伤害中康复

注：能否在伤害领域中添加文本信息来呈现康复情况？

我们该呈现何种信息？你想要如何呈现这个信息？

以这种方式，我们解决漏洞并保持设计和制作部门之间的对话，同时避免了任何不必要的会面。

Google电子表

或许你已经知晓，Google文件很棒。它们在测试母体、QA测试和资产跟踪方面也很好用。

测试母体：

测试母体应当用作最终的综合清单，确定产品是否功能正常。测试母体是在特定系统中包含所有功能性的清单。你可以用其作为漏洞的资源领域或作为编写漏洞的清单来源。

数据输入和分析

电子表还可以用来追踪游戏中产生的数据和信息。随后，可以对收集到的信息进行分析和比较，确定相关功能是否存在漏洞和设计瑕疵。

这在跟踪游戏中的统计数据时很好用，包括分数、用户成长系统、经验值和能力提升等。

纵览这些已经记录的数据，你可以对比并积累数据，或者用来寻找漏洞。它还可以帮助你评估用户进展和用户体验，但是这取决于功能是否适当。

比如，在我的《Wii AFL》中，足球队员可以在每场比赛后获得经验值。获得XP和耗费（游戏邦注：游戏中有某些耗费经验值的途径）比率都用电子表来跟踪。

从这些数据中我们发现了许多漏洞，而用传统的审查方法却难以发现这些问题。它还让我们得以纵览这些系统的用户体验，以便对功能进行适当的调整。

资产跟踪：

Google电子表还作为资产跟踪系统，包括任务开发跟踪和普通资产清单。仍以《Wii AFL》为例，我们使用这个过程来跟踪上千个动画效果的执行、导出和开发状况。

Google文件的共享功能可以让我们的动画师同时对文件进行更新，从而避免产生版本控制所有权的问题。

动画漏洞测试也被整合至这个Google文件系统中，可以创建用来校验某些动画具体层面的栏目，包括技术上和美学上的内容。所有与动画相关的漏洞最终都将在这个系统中处理，以相关栏目内评论的形式编写同时添加正确的颜色以便引起动画师注意。

Google其他工具

虽然我只阐述了电子表这个部分，但别忘了还有Google的文件处理器和绘图工具。你可以使用这些工具来快速创建并动态更新设计文件和数据，使用简单的图表来支持设计文件。

将它们制作得漂亮些！使用标题、栏目和分割线让文件更易于阅读。电子表的颜色规则创建很容易，而且可以动态运转，因而应当用来呈现不同的开发状态。你可以使用以下这个简单的颜色标注系统：红色代表未执行，黄色代表执行中，蓝色代表已执行，绿色代表已通过！

Google文件是可以分享的。这个工具的重要性不言而喻，完全突破了版本控制所有权和锁定带来的制作瓶颈，这两个原因总是将每个文件编辑人数限制在一个以内。

Google文件还可以自动保存修改！

最后要提醒的是，将Google文件转化成Excel文件会存在导出问题，所以请不要试图在Google文件中创建游戏数据然后导出到你的引擎中。

测试游戏版本

拥有行之有效的游戏版本对开发和制作总是很重要的事情。不要让出问题的结构影响到美术人员和设计师的工作。尽量在每周拿出1到2个清晰的游戏版本进行测试。在编写漏洞时，每个标签上都应当报告结构日期，这是非常重要的！版本日期应当以某种调试文本的形式显示在界面上。

在测试过程中应该经常使用调试版本，将解锁数据所需的游戏时间最小化。如果你想测试游戏末期的功能，那就不要让测试者玩1个小时后才让其解锁和测试该内容。在调试版本中呈现内存和帧率对于寻找内存漏洞来说很有帮助。

QA报告和测试的可行与禁忌之事：

最后说下某些有关QA报告和测试的注意点：

1、不要无视问题——包括报告所有的漏洞，甚至是那些细小的问题。如果可以的话，放下手上的工作优先做这件事，否则随后你可能会将其遗忘。

2、综合广泛——不要假设一切情况都正常，你需要去确认某功能的每个方面确实能够有效地运转。

3、小团队中或许没有专门负责QA的人，那么每个人都要承担起这份责任。如果你看到问题，就要报告。如果这是个你自己可以解决的问题，就将其添加到未完成工作的列表中。

4、将未完成工作列表简化到跟踪系统中。你应当可以在任何阶段访问所有的漏洞和任务状况以及正在修补的人员。

5、需要记住的是，QA和制作人员相辅相成！

6、不要在QA上花过多的时间！找到平衡点，不要让QA影响到游戏开发的重要工作。

7、不要只为自己编写标签，你编写的标签应当能够为所有人理解。这不仅对其他负责解决问题的人来说很重要，而且有助于你进行回复测试时清楚地回想起问题的细节！

8、将前半周的时间用于添加功能，然后周三测试新功能，努力在周四完成漏洞的修复，周五重新测试漏洞并添加其他润色内容！

9、记住，在制定短期和长期工作计划时优先考虑QA，努力实现每周至少分配1次QA任务。

10、分享任务。每周每个人都应当花费半天的时间来测试近期添加的功能。

11、你添加新功能或更新功能时，可能会对当前系统造成副作用，所以要记得对系统进行母体测试。

如果你采用以上所述的实践方法和过程，你就能够以周为单位来进行QA，无需担心在濒临游戏发布之时才临时抱佛脚地进行大量的QA工作，也可以提高发布无漏洞游戏的机率。

游戏邦注：本文发稿于2011年7月20日，所涉时间、事件和数据均以此为准。（本文为游戏邦/gamerboom.com编译，拒绝任何不保留版权的转载，如需转载请联系：游戏邦）

An Indie Guide to Quality Assurance Testing and Procedures

Ryan Keable

My name is Ryan Keable and I am a co-founder of Anomalous Interactive, a new indie development studio out of Melbourne, Australia. Before I began running my own studio I was many different roles in the Australian Games industry. One of those roles was lead Quality Assurance manager on Wicked Witches latest title, Wii AFL.

After this role I began QA consultancy for The Voxel Agents, another indie dev team in Melbourne who are responsible for the successful Train Conductor iPhone series. Working with these guys I realized that many indie dev teams don’t know enough about correct QA procedures and operations!

I found that many small teams fully realize is that QA affects production more than you think. The more efficiently you can manage your QA the faster your games will be developed and they will release with a higher level of polish. We followed this QA process in development of our latest game, Slingshot Justice and I would like to share a few hints and tips with you.

Bug Tracking and Ticket Management

This is the most important and most obvious part of the QA process. When you are writing down bugs, you are identifying and categorizing an issue in your game that needs to be fixed. Correctly reporting bug information and categorizing it clearly will make the job easier for those who will be required to solve it.

Identification:

The first step is to identify Who, What, when, Where and How (why is the same thing as how respectively).

Who: refers to player based choices when the user has the option to select from multiple characters, modes or scenarios to play as. This is the first step for identifying how to resolve a bug

What: what occurred? In the most simplest terms write down what occurred to the best of your knowledge (you may not always be sure but the more detail the better)

When: When did the issue occur? about what time during the game play? etc…

Where: is purely location based. what level were you playing on? What screen were you on? etc…

How: Write down Steps to reproduce where applicable. This is important as it will help your programmers to reproduce the error and apply the appropriate traces in the code to see what is actually occurring.

Remember: The cause of bugs are not always obvious; programmers ARE human, assist them as much as possible.

It is also beneficial under certain circumstances to list out an array of issues that are covered by the same category. This is helpful with areas of the game such as UI. A list of bugs found in a single UI screen can be grouped into the one ticket and displayed as tasks. This helps to prevent bloating your ticket system with too many tickets and the annoyance of having to look at multiple tickets that all cover the one area you are currently working in.

Categorization:

Now that you have an informative bug written up, many tracking systems will allow you to create development categories such as ‘code’, ‘art’ and ‘design’. Categorizing your bugs with these kinds of keywords will help identification and sorting in your tracking system.

Priorities:

Identifying the priority of the bug is very important for work flow. When you are working with a system that involves upwards of 100+ bugs, organizing tickets into levels of priority will help both production and the ticket owners responsible for fixing the issues determine where to start. Organizing priorities also helps establish the overall work flow for Milestones and Sprints.

Submitting for Testing:

This was a system I loved to work with. When the owner of a ticket has decided that their bug is resolved on their end, that bug is now ready for testing. Create a ‘testing’ resolution for tickets to group all the tickets that need to be tested together. This allows production and QA to see just how much work is required on the re-testing end. Allocated tasks should also be put through this process so QA is aware when a new feature is implemented and ready to be tested.

Communication and Commenting:

Your ticket system and work flow should allow for a good deal of back and forth dialogue between the reporter and the owner. All transfers of tickets between reports, owners and testers from allocation to testing and to final resolution should be accompanied by comments. This does not just signify ticket sign off but is also great for iteration.

One of the best experiences I had on QA was creating comment based dialogues back and forth between myself and the UI programmers as we worked through specific bug or usability issues. This simple dialogue freed up our time from having to do specific 1 on 1 sessions over simple issues or creating multiple tickets for the same category and bloating the system.

Here is an example from Wii AFL:

My bug report:

- Injuries’ button hover state is scaling incorrectly

- ‘Player name’ is injured and is out for 1 weeks

- There is no notification when an player has recovered from injury

* can we add textual information in the injury field to display recoveries?

This was returned by the Programmer with:

- ‘Injuries’ button hover state is scaling incorrectly – fixed

- ‘Player name’ is injured and is out for 1 weeks – fixed

- There is no notification when an player has recovered from injury

* can we add textual information in the injury field to display recoveries?

What information should we display and how do you want this information to be displayed?

In this fashion we polished the bug and maintained a design and production dialogue without any unnecessary meetings.

Google Spreadsheets are Awesome

As you probably know already, Google docs are awesome. They are also amazing for Test Matrices, on the fly QA testing and Asset Tracking.

Test Matrices:

Test Matrices should be used as final comprehensive checklists to ascertain whether your product is functioning correctly. Test Matrices are checklists that cover all functionality within a given system. You can use as a resource domain for bugs or as a checklist to write bugs from.

Data Entry and Analysis:

Spreadsheets can also be used to record data and information generated in your game overtime. The collected data can then be analysed and compared in order to determine if there are bugs or design flaws with the related feature.

This is fantastic for tracking statistical data in your game such as: scores; user growth systems; experience, power ups, etc…

With a recorded overview of data you have the capacity to compare and contrast the data accumulated data and find any errors or idiosyncrasies. It will also help you to evaluate user progress and user experience, depending on the feature.

For example, in my Wii AFL, football players gained experience after every match. The rate at which XP was gained and spent (amongst other things) was tracked in this Spreadsheet.

From this data we reported numerous bugs that would of been difficult to spot through general observation. It also provided us with an overview of the user experience of these systems and allowed us to adjust the features accordingly.

Asset Tracking:

Google spreadsheets can also be used as an asset tracking system, either for task development tracking or as a general asset checklist. Again on Wii AFL we used this process to track the current level of implementation, exportation and development of over 1000 animations.

The shared access to google docs disabled any bottle necks that are normally created by version control ownership by allowing all our animators to post their updates simultaneously into the document.

Bug testing for the animations was also integrated into this google doc system by creating columns dedicated to checking specific aspects of the animation; both technical and aesthetic. All bugs related to an animation were eventually handled by this system and written as comments inside the related column and coloured correctly for the animators to interpret.

Final Notes on Google Docs:

While I have really only covered spreadsheets, don’t forget about google’s word processor and drawing tools! Use these tools to create on the fly, dynamically updating design docs and specs. You can support design docs with simple illustrations that can all be worked on collaboratively.

Make them pretty! Use headers, columns and row breakers to make your docs easy to read. The colouring rules for spreadsheets are really easy to set up and work dynamically and should be used to display the different states of development. A simple colour system is red for not done, yellow for in progress, blue for done, green for approved!

Google docs are shareable. I cannot stress the importance of this, it completely removes production bottlenecks created by version control ownership and lockouts that limits only one person to work on a document at any given time.

Google Docs are automatically saved, revisions included!

Finally there are exporting issues when converting from google docs to excel so please do not attempt to create game data in google docs then export it into your engines!

Builds for Testing

It is always important for development and production to have working builds. Do not let broken builds stop your artists and designers from continuing their work. Try to get one or two clean builds out each week for testing. When writing bugs, the build date should be reported on every ticket – this is very important! Build dates should be displayed using some form of debug text on the interface.

Debug builds should also be used rigorously during the testing process to minimize time spent playing the game to unlock data. If you want to test a feature at the end of the game, don’t force your tester to play through an hour of content before they can unlock it and test it. Displaying memory and the frames per second rate in a debug build is very useful when looking for memory leaks (derr).

The Do’s and Don’ts of QA Reporting and Testing

A final few notes on QA reporting and testing:

Never ignore an issue – always report everything, even trivial issues. If possible, interrupt what you are working on to do so otherwise if you put it in the back of your mind you’re bound to forget.

Be comprehensive – do not assume something will work, know that it does work, every single facet of it.

When working in a small team where there is no dedicated QA then QA becomes the responsibility of everybody. If you see an issue, report it. If its an issue for yourself to fix add it to your to-do list.

Streamline your to-do lists into your tracking system. You should all be able to assess where all the bugs and tasks are at any given stage and who’s working on them.

Remember QA and Production go hand in hand!

Don’t spend too much time getting caught up with QA! Find a balance where minute QA detailing does not impede your more important work of getting the game done!

Do NOT just write tickets for yourself, write the ticket with an adequate description so any layman can understand it. Not only is this important if someone else is responsible for the ticket you’ve created but when you are doing regression testing you will be able to remember exactly what the issue was!

Spend the first half of the week adding features Test new features on Wednesdays Aim to have bug fixes out of the pipeline by the end of day Thursday Re-test fixes and add any additional polish on Friday!

Keep in mind prioritize your QA around your sprint and milestone production schedules but try to allocate at least one QA task a week.

Share the tasks. One person per week should really be spending half a day testing the features that have been added recently.

When you add a new feature or update a feature that you know will have a compounding effect on a current system, allocate a Test Matrix check of that system after it has been implemented.

If you follow the recommended practices and process detailed in this report you should be staying on top of your QA process on a week to week basis. You will have no fear of massive amounts of QA grind work to be done just before release and you are more likely to release bug free. (Source: Gamasutra)