游戏邦在:
杂志专栏:
gamerboom.com订阅到鲜果订阅到抓虾google reader订阅到有道订阅到QQ邮箱订阅到帮看

分享游戏工作室管理QA团队的正确方法

发布时间:2012-08-20 15:25:06 Tags:,,,

作者:Gabriel Swan

QA已经在游戏产业中迅猛发展。在最初阶段是由程序员自己动手做测试,接着发展到80年代出现了游戏测试评估中心。在20世纪90年代,我们引入了专业的游戏测试员,现在21世纪,我们以逐渐将此职业发展成了“质量保证分析师”。

也就是在最近几年,我们才意识到QA在改进游戏中的重要性,目前,我们已经建立了一系列“最佳方法”,试图将这个正在成长中的游戏领域提高到专业阶段。现在让我们来谈谈什么是QA中的“最佳方法”吧。

专业的分析师会怎么做呢?检查,试验结束?还是检查……等待,没有结果。在整个QA行业中,最不合理的做法就是要求每个测试员在每次任务结束后都得写一份报告。不管你相信与否,这已经成为这一行业的惯用做法。其实,专家或者主管助理的单份EOS(结束转变)报告就可以解决所有事情,因为这份报告包括了当天所有的任务和所有的bugs。既然这样,为何还要花大量的时间和费用写所谓的报告呢?

qa-analyst-performance-testing(from tucowsinc.com)

qa-analyst-performance-testing(from tucowsinc.com)

在启用bug回归池时要避免的一个做法就是,让测试者要回归其它人的bugs。其实50%的工作时间里,测试员都是在不断地追问RP(报告方):“嗨,兄弟,你现在能停下几秒钟,快点过来帮我复现一个bug吗?我实在不知道如何按你的步骤开始。”【暂停】停下你手头的工作,等个2-3分钟,他们就会教你如何复现一个bug了。

由于难以区分技术报告上的细微之处,所以即使是直接从RP那得到即时信息答复,二次回归量也需要花些时间分析和推断原先bug报告中的小错误。这使我妄自下此结论——测试员们需要回归他们自己的bug。因为他们知道如何处理,不会胡编乱造。

要建立一个让制作人直接让报告方重新启动已确认的项目,QA每天早晨做的第一件事就是回归他们自己的bug。显然,这个实践不仅浪费了测试者的劳动力,也浪费了QA工程师的时间,因为他们得创建原本并不存在的回归池工具。而这些工作人员原本应该专注于自动化操作方式。

鼓励办公室间的通信是最好的办法吗?可以不限制员工在任何时间内联系开发人员吗?一些工作室是鼓励此做法的,而在一些地方是禁止的。其实这是最好的办法,因为我们知道程序员、设计师或美工每天都会忙于某个项目,所以有时测试员在发现一些可能影响最近完成的工作的相关信息时,可以同他们联系和商榷。

所以,别让你的分析师只是简单地把一些信息输入数据库,或者等着专家/助手同开发人员交流后给你一个明确的答复,因为有些关键信息可能会在开发人员启动他认为合理的设置后才被发现。你可以按照惯用的方法操作数据库,但也应该允许办公室之间的即时信息沟通。某天我为一位深陷困境数小时的高级职员帮了个大忙,但却无法直接与他沟通。这样还有意思吗?

一般情况下,过去不鼓励QA与开发者交流的原因包括:开发公司担心游戏测试结果会让设计师大伤脑筋,但如果此类事情一再发生,那就是QA主管/经理在雇佣测试员上的失误了。而这也反映了管理层或者制作人在雇佣QA主管时,没有仔细筛选求职者。揭露出自己的错误可能让你很痛苦,但正因为这样,你更应该勇敢地直面自己的错误,这样才能提升自我以及你所领导的团队。

除此以外,在QA部门技术设备上的预算不能太少,也能发生内部人员借用大量资金开发技术的情况。微软游戏工作室是我所知的游戏产业中,在这方面做得最好的,他们毫无保留地向员工提供桌面支持、专有硬件、DevKits、网络安全协议以及防火墙。暴雪在行业中突出的技术是绝对兼容支持。

可是,有一家我不提名称的AAA工作室甚至没有为QA或者文职人员提供一套内部工具。他们只提供纸质材料!你能相信有这种事吗?特别是在21世纪,这完全是无法接受的商业操作。也让我想起了当911调度员一次经历,那时停电了,而我们正用纸记录着电话内容,因为没有CAD(计算机辅助调度),我们就带着这几张纸跑到停车场旁的救护车,让管理人员通过卡车上的无线电将呼叫传送出去。当时一片混乱,就像那家AAA公司现在所做的那样。现在是时候改变这种做法了。

现在我们讨论测试员所需具备的特质。他们不需要经受过技能训练,但需要有可靠的职业操守,热爱游戏,一点点专业知识。而成为一个专业的分析师最重要的就是别看轻数据库。但也有一些例外。我曾经在一家我钟爱的公司就职的时候,有个不懂英文语法的助理,直接取消了我完成的bug报告。我去追问那个没有从大学毕业的主管,可他们拒绝就此事给出调解。

我认为在QA部门中拥有可靠的领导能力和个性非常重要。充分利用你的一线测试员的关键在于,引导专家和助理来管理和领导测试人员。你的助手不能总坐在自己的办公室,他们应该四处走动,无私地帮助测试员,或者在他们办公处分析EOS报告。如果专家/助理在上班期间都呆在自己的办公处,那么他们就会是以下两种情况:A)他们是YouTube中严格守纪律的人(很快就摆脱了领导);B)过于专心自己的工作。他们会向每个走过来向其提问的分析人员发火,因为他们担心这些人会打断自己的思路,从而影响自己的工作。而事实上,给予分析人员答复正是他们的职责所在。

在办公室里被人训斥并不是一件好事,除非你是John Carmack或者Mike Morhaime,如果因为我向你提了个问题就冲我发火,那我以后是不会再跟随你了。而且这种行为只会削弱你在公司的地位,从而降低员工留存率。重新招聘、雇佣和培训上的成本,远比你想象中的还要多。

经理或主管应采用禅宗式的,菲尔式的方法来激发测试人员的职业道德以及创造力,而不是过多关注他在QA上的技能。这也就是说,你在面试管理人员时,首要考查他们的理解能力,不一定是组织管理能力。

甚至,你可能会发现出色的求职者却在游戏测试人员这一岗位上默默无闻地工作。你可以让这些未经打磨的石头变成钻石的。这样,你的团队就能保持灵活的适应能力以及优势,因为你的公司拥有大批优秀的人才。

现在,大多数游戏行业的专业人员极少花时间学习领导科学之类的知识,但这就是我在此为帮助各位而给出这些建议的原因。一个优秀的QA管理者应该是:随时走访,可以数小时在自己的办公位置上独自分析bug报告,并且会不时到外面走动走动。现在,我会透露一个天大的秘密,这应该是你最想知道的,即如何成为拥有极高忠诚度和敬仰度的指挥官或管理者的办法。在每个上班的第一天,你应该把每个人面前的垃圾倒掉。这样做的话,你的员工会从内心知道,他们没有权利去怀疑那个最先给自己派出任务的人的命令了。

现在,我要引入另一个经常在我们领域会遇到的问题,就是如何与资质过高的测试员打交道呢?只要给他们一个提示,他们就明白了自己拥有晋升的机会,然后他们就会和其他合适自己部门(游戏邦注:比如设计部和开发部)的主管,讨论人员内部调动的问题。

否则他们可能会给你一些比你高明的想法和建议,这将会一发不可收拾,就像是“打地鼠”一样让你疲于奔命。可悲的是,我们极少发现有才华的管理者能真正理解到,与其担心那些测试员的能力威胁你的领导地位,不如利用这些杰出人才让整个部门变得更加强大。

如果你抛给员工的不是好吃的萝卜,而是坚硬的大棒,凡是有一点自尊心的员工,就会立马对你不满并迅速辞职,到其他地方寻求更大的机会。你应该不想发生这样的事情吧,特别是这种以较高人才流动率著称的专业领域(游戏测试)。在IT行业,降低人才流动率非常重要,因为在我们的行业有大量关于专业工具和原则等令人厌恶的学习内容,一般而言新来的员工需要花数周才掌握这些内容,一个聪慧的工程师甚至可能需要花数个月才会熟悉他所使用的工具。

难道你认为QA预算不可能拖垮整公司的利润吗?这当然可能发生。是否采取提高自动化能力,稳定员工留存率,减少EOT等浪费资源的做法,正是工作室究竟是取得收支平衡,还是实现利润增长的关键区别。(本文为游戏邦/gamerboom.com编译,拒绝任何不保留版权的转载,如需转载请联系:游戏邦

Managing QA The Right Way

by Gabriel Swan

The evolution of Quality Assurance in the gaming industry has grown in leaps and bounds. Initially, programmers did all their own testing themselves. Next in the 80’s we had playtest evaluation centers, leading to professional game testers in the 1990’s, and now in the new millennium we call them Quality Assurance Analysts.

Just in the last few years we’ve even raised the bar a notch to realize the importance of QA in polishing games and we’re now establishing a series of what’s termed “best-practices”, in an attempt to standardize this growing career field. Now let’s talk about just what best-practices are in QA.

Professionalizing analysts? Check. EOT’s? Check. Errr…. Wait. No. Requiring End Of Task Reports from every tester for every single task is the single worst practice in the entire QA industry. Believe it or not, this is actually standard practice at some major developers. This can all be handled by a single EOS (End of Shift) report from a Specialist or Assistant Lead, comprised of a report that involves all the tasks and bugs for the day. Why waste massive amounts of time and money writing reports just about reports?

Another worst-practice to avoid is the implementation of a bug-regression pool in which testers are regressing other peoples’ bugs. Literally 50% of the time, the tester finds themselves pinging the RP (Reporting Party) going “Hey bud, you got a second to put down what you are doing right now and help me repro your bug real quick? I’m not exactly sure how to trigger your steps.” [PAUSE] Stop all your work, and wait 2-3 minutes for a reply on how to repro it.

Due to the difficulties in learning the nuances of technical writing, even the direct real-time office messenger reply from the RP may take some time for a Secondary Regressor to additionally analyze and deduce whatever half-step was missing from the original bug report. This leads me to the happily pompous conclusion: TESTERS NEED TO REGRESS THEIR OWN BUGS. Because they know just how to do it, without fudging around.

A good convention to establish is to have the producer sling implementations for verification right back to the RP, and first thing that QA does when they come in the morning AM is regress their own bugs. In addition to being wasteful of tester labor hours, this practice is also wasting the time of your QA Engineers who are now building and supporting regression pool tools that need never have existed in the first place! You know those guys should be focused on automation, anyways.

Do encourage interoffice communication as best-practice, and do not restrict any employee at any time from contacting a dev. At some studios this best-practice is encouraged, and at others, this is directly prohibited. This is best-practice because when a programmer, designer, or artist is known to be addressing an issue that very day, sometimes the tester finds pertinent information that might affect the work being done presently down the hall.

So, don’t have your analyst simply throw relevant information into the database or wait for an approved Specialist/Assistant Lead to contact the dev, because the critical information might not make it down the line until after the dev has already implemented what he thinks is a legit fix. Now you have just wasted tomorrows build! Conventionalize notation in the database, but also be OK with a brief ping on the office instant messenger. One day I saved a senior staffer several hours on a fix, but got busted for messaging him directly. Does that make any sense?

Generally in the past reasons for discouragement of QA > Dev communication include the fear that fanbois in Game Test will pester your designers with ideas, but if this is going on then this is a failure on the part of the QA Supervisor/Manager who hired that unprofessional tester. That in itself shows a failure on your part as the Sr. Manager or Producer for hiring a lieutenant who doesn’t screen employment candidates carefully. It may be painful to autopsy your own failures, but as with all bad habits, taking a fearless moral self-inventory is critical to improving yourself and the organization you lead.

Additionally, do not go cheap on your budget for technological infrastructure in your QA department, nor allow internally created tools to acquire excessive technical debt. Microsoft Game Studios does the best job I have ever seen in the industry in providing unbelievable help desk support and proprietary hardware, DevKits, and Network Security Protocols and defense. Blizzard stands out in absolutely amazing compat support- best in the industry there.

However, there is a AAA studio out there who I will not name that does not even offer a single internal tool for either QA or CS. They do everything on paper! Can you believe that? That is a fully unacceptable business practice especially in the 21st century. It reminds me of an experience I once had as a 911 dispatcher when the power went out and we were taking calls on paper, not CAD (Computer Aided Dispatch) and running slips of paper out to an ambulance in the parking lot for a supervisor to dispatch calls from the mobile radio in the truck. It was total chaos, just like AAA (blank) is doing today. Time to change that, right now guys.

Now let’s talk about what you need to look for in a tester. These guys don’t need a lot of technical training, just a solid work ethic (no “Offscreen Youtube Kings and Queens”), a passion for games, and a shred of professionality. One already well known example of being a professional analyst, is never to trade barbs over the database. However, there is one key exception to this. I was once working at a company I loved dearly wherein an Assistant Lead without a grasp of English grammar canceled my bug report on a typo I had caught. I pinged my Supervisor, who hadn’t graduated from college either, and they declined to intercede.

I decided that I cared too much about the product and the team, and consciously decided to fall on a grenade to protect everyone in my squad, for the Greater Good of the company’s reputation. The only thing left for me to do was to force this issue by calling someone out (for all time and the whole world to see) in the database. I did this despite knowing the industry standard response to this kind of behavior was a citation. Would you have done the same? I thought about this carefully, and remembered a valuable lifelong lesson I had learned while reading Richard Ambrose’s seminal book: “Band of Brothers”. Major Richard “Dick” Winters, one of the most influential leaders in the United States Army during World War II, once said:

“Always do what’s right… because it’s right”.

So this brings up the criticality of having both solid leadership AND character in your QA department. The true key here to getting the most out of your line testers is the manner in which you deploy your Assistant Leads and Specialists to supervise and lead them. Your sergeants should not be chained to their desk, they should be walking down the aisles selflessly helping testers, or at their station cutting the EOS report. If the Specs/AL’s are chained to their desk for the entire shift, then they are either A: YouTube Martinets themselves (get rid of these leaders fast) or B: Too immersed in the work on their screen. They are going to bark at any analyst who walks up to them to ask a question, daring to interrupting their chain of thought by breaking it with what in fact, their duty to respond to.

Getting barked at is never cool in the office, and unless you are John Carmack or Mike Morhaime, if you yell at me for approaching you with a question, which is your job to answer… then I will not follow your lead. Not into a firefight. Not into a parking lot. Not even into a (slightly) uncomfortably cold swimming pool, OK? Plus this kind of behavior is going to decrease pride in your workplace and thereby lower retention rates. Recruiting, hiring, and training, cost much more than you think. I’ll touch more on this later.

The Manager or Supervisor should take a zen-like, Phil Jacksonish approach to inspiring both work ethic and creativity (to help with edge-case testing), without too much concern for technical instruction in QA. What that means is that you need to select a Manager based on intelligence, not necessarily organizational seniority. A good way to gauge managerial candidates is to ask them, in interview, who Sun Tzu is.

You may even find an exceptional candidate grinding it out obscurely as a game tester. Do not be afraid to spot promote these diamonds in the rough. Field commission based upon merit is one habit often found in the Green Beret model of leadership. It will keep your organization adaptive and successful, for the greatest benefit of everyone in the company.

Now, most game industry professionals have not spent one iota of time studying Leadership Science, but that is why I am here to help you out with some advice. A good QA manager should: drop in from time to time (surprise!) right into an empty cube on the line for a few hours and cut bug reports themselves, right out in the open, once in a while. Do not be a Foxhole Norman. And now, I’m going to let you in on a priceless secret. You want to know the very best tactic you can possibly do as a Commander or Manager to earn loyalty and respect? Take the office trash can out right in front of everyone on the very first day. I mean it. Then your employees know in their hearts that they never have the right to question the orders of the one who got their hands dirty first.

Now, I am going to address a topic that occasionally arises in our field. What do you do with an overqualified tester? Dangle carrots in front of them so that they understand they do have a strong chance of upward mobility, and be quick to discuss internal mobility moves with other department heads such as design or production where these testers may be better suited.

Otherwise they may have ideas and suggestions for processes superior to your own, which will open up an entire can of rotten worms, and you will typically end up playing “Whack-A-Mole” on your most promising prospects to protect your own position. Sadly, it is rare to find a gifted manager who understands it is better to let talented workers make the entire department look good- as opposed to the fear that enabling strong performance will make your own shaky leadership look bad.

If you use the stick, not the carrot, intelligent employees with a shred of self-respect will become quickly disgruntled and cut out fast, leaving for better opportunities elsewhere. You don’t want to let this happen, especially in a specialty (game testing) with notoriously high turnover rates to begin with. Lowering your turnover rate in IT careers is critical because in our industry there is clearly a nasty learning curve for proprietary tools & conventions, wherein it generally takes several weeks for a new employee to get effective at testing, and possibly even months for a very smart engineer to get the hang of his tools.

Don’t you think it’s possible for the QA budget to drag down an entire company’s profits? It is. Increasing automation, stabilizing retention, and decreasing wasteful practices such as EOTs can make the difference between a studio that is breaking even and one that is showing a profit. After all, isn’t that what this is all about?Running a business?(source:gamasutra


上一篇:

下一篇: