游戏邦在:
杂志专栏:
gamerboom.com订阅到鲜果订阅到抓虾google reader订阅到有道订阅到QQ邮箱订阅到帮看

如何通过封闭测试进一步完善你的游戏

发布时间:2015-04-28 10:16:13 Tags:,,,,

作者:Lukasz Deszczulka

在过去2个月时间里有一组超过200人且来自世界各地的玩家尝试了iOS游戏《地核:破碎元素》的早前开发架构。我们从这些测试者身上学到了许多,并且我认为将我们的经验和发现与其他开发者进行分享是非常有益的。

Earthcore_Sage(from gamasutra)

Earthcore_Sage(from gamasutra)

在那些参加我们的封闭测试的玩家的帮助下,我们识别出了一些游戏玩法问题并在开发过程中努力去解决这些问题。我们同时也决定优化美术,用户界面和用户流等方面。与此同时我们也未停止致力于其它功能和漏洞,所有的一切还是按照开发计划那样进行着。我们同样也会为测试者和在加拿大与荷兰进行软发行的发家定期更新游戏内容。虽然我们的团队付出了许多努力与牺牲,但幸好都得到了巨大的回报。

我希望以下发现能够带给像Tequila Games等正在考虑通过TestFlight Beta Testing推广游戏(官方发行游戏前)的独立开发商帮助。

招募与甄选

我们决定面向全球招募测试者,我们同时也请求了许多在线杂志和博客的帮助。对此我们使用了MailChimp准备了一个注册表单并将其刊登在一个特殊的登录页面。

测试项目的独特性便足以吸引编辑们的关注,特别是他们还能够为我们带来读者。不过我们只会将这一权利给予拥有我们的目标用户的网站—-桌面游戏和数字纸牌游戏玩家。我们同样也使用了Touch Arcade的论坛广告宣传了这一测试项目,并因此吸引了一些硬核玩家。

我们已经拒绝了那些使用不兼容设备的人,并选择了那些拥有iPad 3或更新设备的用户。最终我们选择了来自世界各地大概360名的测试用户基础。

earthcore_infographics_part(from gamasutra)

earthcore_infographics_part(from gamasutra)

这些人会收到通过TestFlight Beta Testing下载游戏的邀请函。我们同时也会通过MailChimp提供给他们关于如何安装于游戏的指示说明。之后我们还会使用MailChimp给他们发送关于更新的时事通讯以及在线调查的链接。

根据我们最初的调查,我们最终的测试群组是由休闲玩家,硬核手机玩家,桌面游戏粉丝,不怎么在手机上玩游戏的硬核PC玩家以及主机玩家所构成。他们的年龄范围非常广,最年轻的的玩家只有12岁,而最年长的玩家已经将近50岁了。我们的测试群组也拥有一个较广的地理位置和国家范围。如此我们便能够收集到更具多样性的数据,这对于像《地核》这样的免费游戏来说非常重要。

收集反馈和发现

在封闭测试项目期间我们使用了两组工具去收集反馈,这也提供给我们不同的信息。

第一个工具便是在背景中运行的我们的游戏内部分析软件。这与我们整合到在加拿大和荷兰软发行的版本中的软件一样,我们也将其用于游戏的最终版本中。这一中间件让我们能够追踪玩家的进程并观看他们对于游戏中特定元素的反应。这同时也让我们能够比较所选择的的测试群组与下载了软发行版本的玩家们。

我们的一大目标便是理解哪个玩家任务太难或太让人受挫—-未能完成任务会导致玩家退出应用。我们将交叉引用信息与来自TestFlight报道的技术记录(包括所有因为过多内存和电力消耗导致的崩溃—-也能是在较低端的设备上)。我们会使用这一原生的数据去解决有关游戏玩法进程,平衡与优化的问题。

01_Earthcore_iPad_Gameplay(from gamasutra)

01_Earthcore_iPad_Gameplay(from gamasutra)

我们所使用的第二个工具便是能够收集包括在线调查,论坛发帖以及其它我们的测试者所发给我们的直接信息等反馈。我们很高兴发现大多数测试者都喜欢我们的战斗系统,这是基于一种三个元素的机制,并且具有《地核》的策略性深度。

我们从一些较容易理解的内容开始(只是为了提醒你:《地核》的机制是基于石头剪刀布般的游戏方式所运行),并突出这是一款将通过纸牌锻造系统和许多技能结合而提供许多策略可行性的游戏。在某种程度上我们害怕这款游戏对于硬核玩家来说也太过复杂。不过幸好我们的测试者们都喜欢较高难度的单人玩家比赛。

如果我们想要自夸的话,我们便会引用那些“喜爱”我们的“华丽”视觉效果的测试者的评价。这些反馈认可了我们在纸牌绘制,动画以及整体风格的选择。

最重要的发现

事实上,最有价值的教训是来自具有建设性的批评:你必须学会接受消极反馈并听取改善建议!

在封闭测试期间,我们发现游戏一开始拥有较高的玩家丢失率。大概只有30%的玩家(有时候甚至更低)会完成教程任务并继续前进。多亏了测试者的建议,我们决定不仅重新创造游戏中的教程任务,同时还将使用一些关于规则和战术的较短视频去支撑它们:在游戏桌中该寻找什么,在选择下一张纸牌时该考虑什么等等。基于这种方式我们可以更详细地解释游戏机制并从策略选择中呈现出更多乐趣。

现在,即在封闭测试结束后,我们在加拿大和荷兰的软发型版本在教程方面已经拥有80%的完成率。这意味着我们在15至20分钟的教程游戏阶段能够更有效地留住玩家了。我们相信最初的体验对于之后的玩家留存非常重要,因为那时候我们刚刚开始于玩家建立起一种连接。我们在过去几周的经历以及游戏内部数据也都证实了这一方法的正确性。

在封闭测试后我们同时也致力于平衡冒险模式中的挑战(游戏邦注:不是将其变得更容易挑战,而是呈现给玩家更多技能和组合)。我们同时也完善了打胜战的奖励,提供给玩家完成任务的更厉害的纸牌,特别是在他们获得3颗星级别(最大)时。另外我们还完善了助推器包—-增加了纸牌以及稀有品种的数量。

我们的测试者所要求的另外一个重要功能便是对于第一把纸牌的再调度—-从桥牌到你的手上。玩家想要在开始战斗时对自己的纸牌拥有更多控制权,我们也觉得这是一个适当的要求。

02_Earthcore_iPad_Mulligan(from gamasutra)

02_Earthcore_iPad_Mulligan(from gamasutra)

有时候使用全新TestFlight Beta Testing系统的经历也不是那么让人愉快:为早前游戏版本接收并激活下载链接对于玩家来说是一个非常复杂的过程。甚至连一些较有经验的游戏记者也不知道如何避免重定向循环,即当你打开电子邮件并使用Safari以外的网页浏览器时。这一过程导致我们的玩家丢失率为40%,这是非常让人惊讶的结果。就像之前所提到的,这还是一种新工具,而我们也期待这一问题能够尽早得到解决。

还有一点并未让我们感到太打惊讶的是,不到一半的玩家在玩了游戏后并未填写反馈表格或给我们发送信息。幸好我们还可以对这些玩家进行适当的分析。

结论

来自测试者的反馈,漏洞和完善建议列表远长于我们真正执行的内容。拥有一支大型测试团队便意味着你能够获得较大的样本,并以此去理解大多数人想要的内容以及你必须解决的问题。

总之,我们的封闭测试项目帮助我们明确了最新的问题并证实了我们已经在团队中讨论过并需要基于一种全新看法去看待的改变列表。这是一次非常有帮助且带有启发性的经历,我们希望所有想要进一步了解玩家的独立开发者也可以这么做。

询问玩家对你和游戏的期待总是非常有帮助的,但你需要为此做好吃惊的准备。

本文为游戏邦/gamerboom.com编译,拒绝任何不保留版权的转发,如需转载请联系:游戏邦

How a Closed Beta helped improving Earthcore – findings and tips

by Lukasz Deszczulka

For over two months a group of over 200 players from all over the world tried out early development builds of Earthcore: Shattered Elements for iOS. We have learned a lot from our beta testers and I thought it would be useful to share our experience and findings with other devs.

With the help of people who joined our Closed Beta program we identified quite a few gameplay issues and managed to solve them during development. We also decided to optimize art, user interface and user flow at the same time. We managed it without stopping the work on other features and bugs, all of it planned in our development schedule. We also kept updating the game regularly both for the beta testers and Soft Launch players in Canada and the Netherlands (in fact, the local SL started at the same time as the global Closed Beta). It took a lot of effort and sacrifice from our team but the result is a great payoff.

I hope the following findings will be useful for other indie developers, who just like Tequila Games, are thinking about distributing their game through TestFlight Beta Testing before it’s officially released.

Recruitment and selection

We decided to recruit beta testers worldwide, we also asked for help a number of online magazines and blogs. For that we prepared a signup form with MailChimp and posted it on a special landing page.

The exclusiveness of a beta program is enough to get the attention of editors, especially if they can offer their readers a guaranteed access. We only gave this right to the websites with an audience similar to our target – tabletop and digital card gamers. We also advertised the beta program with a banner on Touch Arcade’s forum, which brought us some additional core players.

We had to reject people with incompatible devices and select only those who possessed iPads 3 or newer, with iPad 2 and iPhone not supported back then. This left us with a beta tester base of around 360 people from around the world.

Those people received an invitation to download the game through TestFlight Beta Testing. We also provided them with a short document with easy-to-follow instructions on how to install and how to play sent through MailChimp. Later we used MailChimp for sending newsletters with updates and links to online surveys.

According to our first survey, our final group of beta testers consisted of a mix of casual players, core mobile gamers, fans of tabletop games as well as hardcore PC and console players who don’t play a lot on mobile devices. Age-wise, we had a big cross-section with the youngest player at 12 years old while the oldest ones were in their late 40’s. We managed to keep a broad variety of geographical locations and nationalities as well. It allowed us to gather a lot of varied data which is very important for a free-to-play title like Earthcore.

Gathering feedback and findings

During the closed Beta program we used two sets of tools for gathering feedback, which gave us two different sets of information.

The first tool was our in-game analytics software, running in the background. This is the same software we’ve integrated in the versions soft-launched in Canada and the Netherlands and that we’ll use in the final version of the game. This middleware allows us to track players’ progression and see how they react to with certain elements of the game. It also allowed us to compare our selected group of testers with regular players who downloaded the Soft Launch version.

One of our objectives was to understand which of the single-player missions were too hard or too frustrating – failing them could cause in players quitting the app altogether. We cross-referenced the information with our technical logs from TestFlight reporting that included all the crashes happening due to excessive memory or power usage – especially on lower-end devices. We used this raw data to work on gameplay progress, balancing and optimization issues.

The second set of tools we used to gather feedback included online surveys, forum posts and all other channels for our testers to send us direct messages. We were very pleased to find out that most of our beta testers enjoyed our combat system, which is based on a three-element mechanics, and the strategic depth of Earthcore.

We started with something easy to understand (just to remind you: the mechanics of Earthcore operates on the Rock-Paper-Scissors game but with elements) and built on top of it is a game that offers a lot of tactical possibilities via the Card Crafting system and a large variety of skill combinations. At one point we were afraid it will be too difficult even for core players. However, our beta testers enjoyed (at least most of them) the rather high difficulty level of the single player campaign.

If we were to brag a little, we would quote testers that “loved” our “gorgeous” visuals. This feedback ratified the choices we made in terms of card illustrations, animations as well as overall style.

Top findings

As it is, the most valuable lessons come always from constructive criticism: you have to embrace negative feedback and listen to the ideas for improvements!

During the Closed Beta we faced a high dropout rate at the beginning of the game. Only about 30% of players (sometime even less!) finished the tutorial mission and advanced in the game. Thanks to the suggestions from our beta testers we decided not only to rebuild the tutorial missions in the game, but also to enhance them with some additional short (and skippable) videos about basic rules and tactics: what to look for on the game table, what to take into account when choosing your next card, and so on. This way we could explain our gameplay mechanics in more details and squeeze more fun from strategic choices.

Right now, after the Closed Beta, our Soft Launch version in Canada and the Netherlands have over 80% completion rate on the tutorial. This means we’re getting better and better at retaining players for longer than the first 15-20 minutes required to finish this stage of gameplay. We believe that this first experience is also crucial for further player retention, as it’s when you build a connection with the gamer. Our experience and in-game data from the last few months proves this approach right.

After our Closed Beta we also worked on balancing the challenges in the adventure mode (not making them easier to win, but showing players more skills and combos). We also improved the rewards for winning a battle, giving gamers better cards for completing missions, especially when they achieve three stars (maximum) rating. On top of that, we improved the booster packs – increasing both the number of cards and their rarity.

Another important feature that was requested by our amazing testers was a mulligan for the first draw of cards – from your deck to your hand. Players wanted to have more control over the cards they start their battle with, and we felt it was a fair request.

The experience with the new TestFlight Beta Testing system has been a little painful at times: receiving and activating the download link for an early version of the game is still a rather complicated process for players. Even some more experienced game journalists didn’t know how to avoid a redirection loop that happens if you open your emails with anything but iMail and use other web browsers than Safari. We had a dropout rate of 40% caused by this process, which was surprising. As said before, it’s a new tool and as they are working on it, we expect this issue will be resolved soon.

Slightly less surprising was the number of testers who didn’t provide us with any active feedback, less than half of the people who actually played the game didn’t bother to fill the feedback forms or send us a message. Good we at least had our analytics for them.

Conclusions

The list of feedback, bugs and suggestions for improvements that we received from the testers was of course far longer than what we actually implemented. Having a large beta group though, gives you a sample that’s big enough to understand what the majority wants and what are the issues that you absolutely have to fix.

Overall, our Closed Beta Program helped us to identify new issues and confirm the list of changes that we already discussed within the team but needed a fresh perspective for. It was a very helpful and illuminating experience which we can recommend to all indie developers that want/need to understand their players.

It’s always good to ask what do players expect from you and your game (and hopefully confirm your initial assumptions), but one needs to be ready for surprises.

With only a few weeks before Earthcore: Shattered Elements launches globally on the App Store, me and my team really can’t wait to share it with you in its improved form!(source:gamasutra)

 


上一篇:

下一篇: