游戏邦在:
杂志专栏:
gamerboom.com订阅到鲜果订阅到抓虾google reader订阅到有道订阅到QQ邮箱订阅到帮看

如何通过A/B测试去检测游戏图像和图标

发布时间:2016-03-09 17:51:30 Tags:,,,,

作者:Simon Newstead

当提到免费游戏设计时,我们的工作室便会从自由创造开始,然后通过定量反馈做出最终决定。

在实践中这意味着什么?

需要进行许多实验和A/B测试。不只是关于功能,UI和经济,还关于游戏图像。以下便是我们所使用的一些策略:

应用图标

当提到如何提高游戏下载量时,应用图标便是最基本的元素。

我们使用了Store Listing Experiments,并将其带到 Google Play Developer Console上。

以我们的3D模拟游戏为例:

3d game(from gamasutra)

3d game(from gamasutra)

为了战胜那些领先的对手,我们使用了不同的图标组合。包含一个角色,两个角色,还是没有角色?是来自主题,游戏玩法的元素还是只是关于角色?

在观察了几天的数据后我们发现一个角色和游戏背景表现得最好,甚至超越之前我们最有信心的图标。

在我们最早设计的游戏中,我们发现一个有点“流行歌手感觉”的图标,相比最初图标它甚至提高了10%的安装结果。

除了图标,我们同时还使用了Google Play Store的清单实验对截图,描述等等内容进行A/B测试。这是一种非常有帮助的工具!

测试游戏图像

我们在两个主要开发阶段测试了游戏图像:

首先我们测试了概念并观察它是否可行。

然后我们创造了图像去引导平衡和内容分布

测试概念

在设计循环早期阶段我们的设计师想出了一些美术概念,我们也努力去收集有关风格/主题的反馈。

例如在一个农场原型中,我们想要添加更多锻造内容,并通过测试以了解玩家想要创造什么样的内容。在这种情况中,我们知道比起其它食物和饮料玩家更想要蛋糕:

prefer cakes(from gamasutra)

prefer cakes(from gamasutra)

我们可以通过不同属性,即付费/免费玩家,关卡等去过滤不同的反应。

在另外一个例子中我们想了解玩家在一款精品店模拟游戏中想要出售什么样的商品:

boutique simulation game(from gamasutra)

boutique simulation game(from gamasutra)

为了Unity我们使用了polljoy插件并为了应用内部图像和文本调查使用了仪表盘。

弹出窗口总是会让人觉得是垃圾邮件,所以我们通常会这么做:

设置调查为选择性内容

不去打扰位于较低关卡的玩家

只收集能够帮助我们提高统计数值的少部分回应

在完成调查后提供给调查对象虚拟货币作为奖励

通过仪表盘加载定制图像从而让调查显得更加自然

有时候我们也会想问那些不是我们应用当前用户的人问题。

对此我们使用了Google Consumer Surveys,虽然它不是免费的,但在探索全新用户类型的时候真的很有帮助。你可以设置一些问题,如你是否玩社交博彩游戏,并只调查那些符合你的标准的对象。

除了调查外我们还会使用Facebook的硬核玩家群组去获取一些更传统的反馈和讨论:

response(from gamasutra)

response(from gamasutra)

测试制作好的图像

一旦我们完成了图像创造,我们便对完成的内容进行测试。例如一款时尚游戏中的鞋子:

fashion game(from gamasutra)

fashion game(from gamasutra)

为什么呢?这主要是平衡和内容分布的输入方式。

例如让用户对于NUX流具有很好的第一印象能够提高隔天游戏的用户留存,所以我们便会让一些评价较高的道具出现在游戏前面。

我们同样也希望确保付费捆绑内容和道具能够有效提高游戏转换率。

就像用户的兴奋指数会在电影或游戏中上下起伏一样,我认为在一款手机3D游戏中分配内容时也可以使用同样的方法。

A/B测试架构

一旦所有内容都准备好了,我们便会使用Google Play的阶段式展示功能对全新架构本身进行A/B测试。基于这种方法我们便能够了解到全新版本或控制获得多少安装量。

如果主要图像或功能发生了变化(游戏邦注:例如在假期期间出现的圣诞主题更新),我们便会先提取20%的流量并检查一周多时间的情况。这让我们在测试所有内容前可以先面向较早前的用户留存和盈利参数。

结论

测试图像能够为我们工作室带来许多好处。

首先它让我们能够测试各种类型的理念,即在概念阶段我们将能够获得更多可能性(比起只是拥有一个计划A)。

测试同时也让我们在真正投入成本去制作游戏并发行游戏前更有信心。

所有的这些反馈循环都能够帮助我们更加清楚我们最重要的利益相关者,也就是玩家的看法。

本文为游戏邦/gamerboom.com编译,拒绝任何不保留版权的转发,如需转载请联系:游戏邦

How we A/B test game art and icons

by Simon Newstead

When it comes to free to play design, our studio starts with creative freedom and then decides through quantitative feedback.

In practice what does that mean?

Lots of experiments and A/B tests. Not just for features, UI and economy – but also game art. Here are some of the tactics we use:

App Icon

An app icon is always lowest hanging fruit when it comes to increasing downloads.

We use Store Listing Experiments, built into the Google Play Developer Console for this purpose.

An example with one of our 3d simulation games:

Here we tested different icon compositions trying to “beat” the prior champion. One, two or no avatars? Elements from the theme, the gameplay, or just focus on characters?

After a couple of days of data we found a single avatar and gameplay background performed best, outperforming the original icon with a high degree of confidence (it’s handy to be able to see the expected performance interval directly in the results)

In one of our older dressup games we found an icon with a “popstar feel” improved install performance over 10% relative to the original – more than we would have guessed.

Apart from icons, we also use Google Play Store Listing Experiments to A/B test other aspects – screenshots, description etc. A handy tool!

Testing Game Art

We also test game art in two main phases of development:

First to experiment with concepts and see what works

Then with produced art to guide balancing and content distribution

Testing Concepts

Early in the design cycle our designers come up with art concepts and we get feedback on style / subject matter.

For example, in a farm prototype we wanted to add more crafting, and tested to see what players preferred to make. In this case there was a strong preference for cakes over other types of food and drinks:

We can view and filter the responses by different attributes – pay/free user, level etc

In another example we wanted to see what type of items players would want to sell next in a boutique simulation game:

We use the polljoy plugin for Unity and web dashboard for in-app image and text surveys.

The feeling of spamming is always a concern with popups so typically we:

configure surveys to be opt-in

don’t ask players at low levels

only gather minimum responses needed to gain statistical confidence

configure research polls to give virtual currency as thanks

upload custom graphics via the dashboard to make the poll feel native

Other times we want to ask questions of people outside of our apps’ current audience.

For this we use Google Consumer Surveys which, while not free, can be helpful especially when exploring a new audience type. You can setup qualifying questions, for example – do you play social casino games – and only survey those who meet your criteria.

Apart from surveys we also use Facebook groups with hardcore fans for more traditional feedback and discussions:

Testing Produced Art

Once we have art created, we then poll the finished items. For example shoes in a fashion game:

Why? Mainly as input for balancing and content distribution.

For example an excellent first impression in NUX flow helps next day retention, so we plan for some of the highest rated items to appear up front.

We also want to make sure premium bundles and items have the most desirable content to improve conversion etc.

Just like the level of excitement tends to cycle up and down through a movie or game, I think a similar approach works when it comes to arranging content in a mobile 3d game.

A/B Testing Builds

Once everything is ready, we’ll often A/B test the new build itself using Google Play’s staged rollout capabilities. Using this, we can decide how much of install traffic will be on a new version or the control (existing one).

If it’s a major art or feature change (for example a recent Christmas themed update during the holidays), we usually roll out 20% of our traffic and monitor over a week or more. This allows us to check at least the early retention and monetization metrics before we commit to roll it out 100%.

Conclusion

Being able to test art has had a number of benefits for our studio.

First it lets us try lots of ideas both mild and wild, at the concept phase which can open up more possibilities than if we just set out with a plan A.

Testing also lets us have confidence before we commit to production and launch, where the costs are vastly greater.

Most of all this feedback loop helps us get closer to the most important stakeholder of all – our players!(source:Gamasutra

 


上一篇:

下一篇: