作者：Peter Castle

过去几周我们三个Whale Hammer Games创始人都非常非常的忙碌。在8月31日我们面向Steam和GOG.com发行了自己的第一款游戏《Tahira：Echoes of the Astral Empire》，这是我们花了3年时间所创造出来的游戏。

游戏发行后的接受度

我们都对游戏发行后的接受度感到满意。在GOG上，我们在发行后的33个评级中获得了3.9分（总共5分）。在Steam上我们获得了将近81%的正面且专业的评价。

不过最让我们开心的还是我们的Kickstarter支持者们，他们最清楚我们过去三年里的投入，并且他们也都对最终的游戏感到满意。

Valve改变了评价政策

在继续前让我们先澄清一件事，即我们理解为什么会发生改变。系统设置方式很容易让人给予正面的评价。而全新系统也并未完全解决这一问题，即人们还是很容易购买游戏并留下正面评价，但是经济上的不利诱因却能够有效阻止大多数人去这么做。Valve的Greenlight便是如此。

除了有效解决问题外，Valve还创造了这样的情况：

这从根本上扼杀了Kickstarter游戏了。现在Kickstarter既能帮助你去获取粉丝也可能压制你的粉丝。https://t.co/vZSgbLAmV3

—-Tyrus Peace，2016年9月13日

也许这听起来有点奇怪，那就让我们的程序员Tom来解释清楚吧，他将通过评价数据进行解释并会提供一个帮助解决Steam评价系统的方法。

分解数值

着眼于我们的评价数值，我们可以将其分成4个类别：

来自Steam购买的正面评价：11

来自Steam购买的负面评价：8

来自game keys的正面评价（主要是Kickstarter的支持者）：@3

来自game keys的负面评价：0

（这部分的所有数据都是来自2016年9月20日我们的Steam页面）

基于之前的Steam评价系统，我们所获得的正面评价的比例是34/42，即81%。而基于全新评价系统，我们的评价分数则有所下降，即正面评价为11/19（58%）。

再继续前让我们先看看一些观察结果：首先，比起Steam上的购买者评价，我们拥有更多Kickstarter评价，这是合理的，因为比起Kickstarter支持者，我们的商业销售更少些。其次便是我们的Kickstarter评价全部都是正面的，即我们所有的Kickstarter支持者都表示喜欢我们的游戏。鉴于此，我便清楚为什么Steam认为来自keys的评价可以不予以考虑了，因为它们往往是来自玩家的偏见，并不具有真正的代表性。虽然我不是很同意这一观点，但是现在我可以看出其中的原因了。

Steam在他们的媒体发布会上表示，全新系统还带有缺陷，他们是这么说道的：

steampolicy(from gamasutra)

可行的解决方法？

问题3很奇怪，所以我并不打算在这里解决这一问题，不过我想提出解决问题1和问题2的方法。为此我将专门讨论Steam的整体评价分数，即面向那些没有大量市场营销预算的游戏（也就是大多数游戏），对他们来说这是非常重要的数字。我同样也会采取一些容易执行的方式（不需要全新数据或棘手的算式），去抵抗突然的改变（没有极限变量或其它突然会让评价变得比之前更加重要的元素），并抵抗元游戏（限制一个或多个“治安”评论者的可能影响）。

我认为最简单的解决方法便是基于它对于其它用户的效能去衡量每个评价。我通过假设一个写下评价的人在发现评价是有帮助后便会对现有的系统产生一些细微的改变。这能够让我们更有效地去衡量全新的评价，我将在之后进一步解释这点。在这一系统中，基于正面评价，3/4认为有帮助的人将为游戏的正面评价贡献0.75分。而在负面评价中，6/10认为有帮助的人将为游戏的负面评价贡献0分。从中我们便可以看出为什么《Tahira》在Steam上的用户评价为什么那么低了：

table(from gamasutra）

在这个表格中，“分数”指的是那些有帮助的票数，即和整体票数相比较之后。如果评价并未推荐游戏，那么“正面分数”便为0.

我们最终在10.09的总数值中获得了7.47分（正面分数栏中的总值）。

这一系统的最大优势便在于它能够简单地解决Steam所面临的第1个问题，即关于游戏无用的评价会显著降低对游戏分数的影响。这同时也是一个非常简单的系统，即执行该系统并不需要额外的数据，并且也不需要复杂的算式。

不过也有一个有效的批评指向了该系统，即评价并不是基于多少投票的人进行衡量的，例如基于3/4有帮助的投票的正面评价与基于300/400有帮助的投票的正面评价的衡量方式是一样的。如此的结果便是，一旦评价被发表，那么即使只有1/1有帮助的评价，它也会马上对游戏分数产生最重要的影响力。不过考虑到Steam想要创造一个游戏只是基于用户评价去获取有意义分数的自立系统，或许这便能够创造Steam想要的行为的系统。而就像我之前所提到的，这意味着用户对于游戏分数的最重要的影响便是留下评价，而一旦他们做出了评价，这便意味着用户对游戏的第二大影响便是对还没有多少票数的评价做出投票（游戏邦注：用户总是更愿意对那些没有太多票数的对象进行投票，即意味着评价将随着时间的发展累积一定数量的投票）。

这能够进一步缓解之前改变所带来的冲击。基于全新系统，Kickstarter的支持者和捆绑消费者将不能对游戏分数产生直接影响，他们将通过对其它评价的投票而创造间接影响。尽管他们仍然属于“二等公民”，但就评价系统而言，他们仍然可以真正表达出自己的想法。

（本文为游戏邦/gamerboom.com编译，拒绝任何不保留版权的转发，如需转载请联系：游戏邦）

A Case Study of Steam Reviews for an Indie Kickstarter Game

by Peter Castle

The last couple of weeks have been busy for the three of us who make up Whale Hammer Games, really busy. On August 31, we released our debut game, Tahira: Echoes of the Astral Empire, which we’ve been working on for the last three years (and a little change) to Steam and on GOG.com.

Reception on Release

We were very happy with the reception to the game when we launched. On GOG we’ve been sitting around 3.9/5 (google ‘tahira gog’ to see) since launch from 33 ratings. On Steam we were sitting comfortably around 81% positive and professional reviews were generally averaging out to around there too, with some reviewers going lower and others going higher.

The most pleasing thing for us though, was that our Kickstarter backers, the people who knew best what we’d been working on for the last three years, were overwhelmingly happy with the game that we had made for them.

Valve Changes Their Review Policy

Boom, just like that we’re down to about 50% positive on Steam.

Now, let’s get one thing clear before I continue – We understand why the change was made. The way the system was set up very clearly had the issue that it would be easy to trade keys for positive reviews. The new system doesn’t fix this completely – It is still possible to game it by paying people to buy the game and leave positive reviews, but a financial disincentive is usually a very a good way of stopping the vast majority of people. Valve did exactly the same thing with Greenlight.

So that’s all well and good, except that in fixing that problem, Valve have created a situation that does this:

Wow, yeah. This basically kills Kickstarter games. Now KS is just a way to find and silence your biggest fans.https://t.co/vZSgbLAmV3

— Tyrus Peace (@TyrusPeace) September 13, 2016

Maybe that sounds a bit dramatic, but let me throw it over to our programmer Tom, who will take you through our review numbers and will suggest a solution to help fix Steam’s review system.

Breaking Down the Numbers

Looking at the stats for our reviews, we can break them down into four categories:

Positive reviews from Steam purchases: 11

Negative reviews from Steam purchases: 8

Positive reviews from game keys (almost entirely Kickstarter backers): 23

Negative reviews from game keys: 0

(All data in this section was pulled from our Steam page on 20 September 2016)

Under the old Steam review system, this gives us a review ratio of 34/42 positive reviews, or 81%. Under the new system, our review score drops considerably, to 11/19 positive reviews (58%).

Before I continue, a couple of observations: first, we have more Kickstarter reviews than Steam purchaser reviews – this makes sense as we have fewer commercial sales at this point than we had Kickstarter backers. The second major point is that our Kickstarter reviews are unanimously positive – every Kickstarter backer who decided to review the game enjoyed it. Given this, I can see why Steam is making the argument that reviews from keys should not be considered – they appear to come from a biased, non-representative section of our players. I don’t agree with this argument, but I can see why it’s being made.

Steam also allows users to vote on whether they found a review helpful or not, and the average ‘helpfulness’ of our reviews is also interesting to look at:

Average helpfulness of positive reviews from Steam Purchases: 65%

Average helpfulness of negative reviews from Steam Purchases: 29%

Average helpfulness of positive reviews from backers: 77%

(note that I have excluded two reviews made by Kickstarter backers from this analysis, because they had no votes on their helpfulness)

Here’s where you can most clearly see the problem with the changes Steam has made: by cutting out our backer reviews, they have silenced the group of reviews which users find most helpful (positive backer reviews), and amplified the value given to the reviews which users find least helpful (negative purchaser reviews).

Steam acknowledge that the new system is flawed in their press release, saying:

A Possible Solution?

Problem #3 is a strange one, which I’m not going to attempt to solve here, but I’d like to put forward a way to solve problems #1 and #2. In doing so I’ll be exclusively talking about what goes into Steam’s overall review score, as for any game without an extensive marketing budget (i.e. most games), this is the singular number which matters the most. I’ll also be doing so in a way which would be easy to implement (no new data or tricky algorithms needed), resilient to sudden changes (no threshold variables or other factors which can suddenly make one or more reviews significantly more or less significant than they were previously), and resistant to meta-gaming (restrict the possible impact of one or several ‘vigilante’ reviewers).

The simplest solution, I believe, would be to weight each review based on how useful it’s deemed by other users. I’m making one slight change to the existing system by assuming that the person who wrote the review finds it helpful (so a review which 4 out of 6 people found helpful would now have a score of 5 out of 7, by taking the reviewer themselves into account). This is done to give us a way to meaningfully weight new reviews, which is important, as I’ll explain later. In this system, a positive review which 3 out of 4 people found useful would provide 0.75 points towards that game’s positive score, and 0.75 points towards its total score. A negative review which 6 out of 10 people found helpful would provide 0 points towards a game’s positive score, and 0.6 towards its total score. A visualization of how this would work for Tahira’s steam user reviews is below:

In this table, “Score” refers to the number of helpful votes, as compared to the total number of votes. “Positive Score” is zero if the review does not recommend the game, and the same as “Score” otherwise.

This gives our game a weighted score of 7.47 (the sum of all values in the ‘positive score’ column) out of 10.08 (the sum of all values in the ‘score’ column), or 74%.

The single largest advantage of this system is that it neatly solves Steam’s problem #1 – a game’s unhelpful reviews have a significantly reduced impact on that game’s score. It’s also a very simple system, with no additional data required to implement it, and no convoluted maths required to make it work.

There is a valid criticism which could be levelled at this system, which is that reviews aren’t weighted based on how many people have voted – i.e. a positive review with 3/4 helpful votes is weighted the same as a positive review with 300/400 helpful votes. As a corollary to this, a review is at its most impactful to a game’s score immediately after it is posted, with 1/1 helpful votes. However, given that Steam appears to be trying to create a self-sustaining system where games receive meaningful scores based only on their user reviews, I would actually argue that this incentivizes the behaviour that Steam wants. As I mentioned above, it means that the most impactful thing a user can ever do for a game’s score is to leave a review (encouraging more people to leave reviews), and once a review has been left, it means that the next most impactful thing a user can do is vote on a review which doesn’t yet have many votes (making the system self-correcting, as users become more likely to vote on things with few votes, meaning reviews are more likely to accrue a meaningful number of votes over time).

This has the added bonus of softening the blow of the previous changes. Under the new system, while Kickstarter backers and bundle purchasers can’t directly influence a game’s score, they can do so indirectly by voting on other reviews. While they’d still be second-class citizens, as far as the review system is concerned, they’d still be able to have some say, whereas currently they have none.（source：gamasutra）