我利用MobyGames的数据库来进行此项分析，结合RA团队的作品来获取游戏荣誉和成功公司的数据。所调查的游戏包括畅销游戏（游戏邦注：如《模拟人生》、《过山车大亨》和《星际争霸》）和小型却更专业化的项目（游戏邦注：如《不朽之都》、《波斯王子：王者无双》和《Battlecruiser 3000AD》）。我使用NPD Funworld中的盈利数据来辨别游戏的成功度。我也尝试过使用GameRankings和MetaCritic的排名，行业内的人士应该不会对此有异意，尽管它们的数据并不适合用来衡量游戏的成功与否，但显示出的结果与上文相同——制作人的作用至少是设计师的两倍。统计中，我还考虑到制作团队的大小、游戏的题材、发布年份、游戏是否申请专利、是否是续作以及游戏的发行商等诸多要素。将这些不同之处纳入统计中，我探究游戏盈利差异与设计师、项目制作人和公司的关联。
Why Producers Matter (Statistically) More than Designers
Is it the company or the people on the team that determine the success of video games? And, to the extent that it is the people that determine success, is it the designer or producer role that has the largest impact? A recent research paper of mine attempts to answer these questions, by studying 854 computer games, 537 individual producers, 739 individual designers, and 395 companies between 1994 and 2006 (representing $4 billion in revenue), using a sophisticated econometric technique called Multiple Membership Cross-Classified Multilevel Modeling. Controlling for a wide variety of factors, I found that the difference among individual producers accounted for over 22% of the financial success of the game, the differences among designers around 7%, and the rest of the game developer’s people and organization, about 21%. These results, while surprising, turned out to be extremely robust over many types of analysis – the choice of producer is key to determining a game’s revenue, while the choice of designer matters very little. Further, these skills are portable, even when producers move from one developer to another
For this analysis, I used the MobyGames database, combined with the work of a team of RAs, to get data on game credits and company success. The list of games included everything from the top selling games (The Sims, Roller Coaster Tycoon, Starcraft) to smaller and more specialized projects (Immortal Cities, Two Thrones, Battlecruiser 3000AD). I used NPD Funworld’s revenue figures to track success. (It may be unsurprising to people in the industry that I tried to use GameRankings and MetaCritic ratings as well, but they turned out to be bad statistical measures of game success, though they showed similar results – that the role of producers were at least twice that of designers). I statistically controlled for the size of the team, the genre of the game, year of release, whether titles were based on licensed property, whether they were sequels, and the publisher of the game. After accounting for these many differences, I looked at how much of the revenue differences between games could be explained by the designers and producers on the project, and how much by the company.
Compared to the impact of individual designers and producers, the importance of the game developer as a whole was surprisingly small, accounting for 21% of the difference between games. Especially given that I do not include other important team members in my analysis (such as programmers and artists), it is unclear how significant top managers at game developers actually are in explaining performance, but they are, at most, on the same scale as the role played by just one individual within the product team. Far from being interchangeable, the individuals involved uniquely contribute to the success or failure of a game.
Additionally, the relative importance of producers over designers in explaining a game’s success is surprising. Even in an industry that rewards creative and innovative products, the difference between top and bottom designers seems to have much less impact than the difference between top and bottom producers. Rather than acting as cogs in the machine, dwarfed by the importance of top managers, the effect of producers on performance was actually larger than that of organizational factors, implying that differences between producers play an outsized role, even over the $4 billion in revenue generated by games in the sample. Though this finding might seem surprising, it is supported by research on the role of project middle managers in the innovation process.
Recent research on the role of individuals and groups in industries as diverse as consulting and comic books supports a longer literature on project management that has demonstrated the complex interaction between individuals and teams in successful innovation. The finding that producers have significantly more impact on firm performance than individual designers aligns with this tradition. It suggests that high-performing designers alone are not enough to generate performance variation; rather, it is the role of individual producers to integrate and coordinate the innovative work of others.
It is worth noting that even though producers seem vitally important to the success of a game, they are not compensated in a way that acknowledges the vast difference between the best and worst producers. Based on a survey conducted with the attendees of the Game Developers Conference in 2008, I found that individuals reported relatively weak connections between performance and reward in the game industry. Producers, for example, reported bonuses that accounted for less than 9% of their total salary on average, with only 10% of producers reporting either a share of royalties or profits. It may be the case that many producers are not truly aware their own significance, or at least cannot prove it, without the kind of detailed longitudinal data I used in my analysis. Successes are claimed by many, while the blame for failures is often blurred, or, in the words of the old proverb, “victory has many fathers, defeat is an orphan.”
It is worth returning to the data one last time and noting that the difference in individual ability among producers, and, to a much lesser extent, designers, is one of the most important drivers of a game’s revenue. A game that would normally generate $30M in revenue could see their sales change by as much as $7M, depending on the quality of the producers involved. The difference between selecting either the top and bottom designers, however, accounts for a less significant $2M. And, keeping the same producer and designers, but changing the developer, could result in a difference in revenue of about $6.5M. You can read more detail in the full paper, People and Process, Suits and Innovators: Individuals and Firm Performance.
This does not mean that designers are not important, or that they do not play a vital role, but it suggests that getting the “best” versus the “worst” design talent has far less impact on the game than getting the best possible producer.
Some additional information (updated, again): Based on the comments I have received, a few additional points. First off, this is basically the result of a statistical, economic analysis – the results surprised me as well, I was certainly expecting to find a bigger impact on revenue from designers than I did. The results are not intended to show a lack of respect to designers, nor was the title of the post, and I have attempted to clarify the language in hopes that I do not cause offense. It is merely the results that I found, and that I am reporting here. The analysis shows that having the best versus worst designers on a team only changes project performance slightly, while having the best versus worst producers on a team has a tremendous impact on the performance of a game. I can only offer theories about why this is the case, based on other academic work. It might be the case that producers have a wider range of talent than designers (the best producers are really good, and the worst producers are really bad, while most designers fall somewhere in the middle), for example. Or, as is the case with many project managers in creative industries, the producer’s role in bridging the gap between all of the stakeholders during development might be far more important than is realized. Any ideas you have to explain the gap would also be appreciated.
As to the source of credits, I used Moby Games, with some spot-checking of the data to ensure accuracy (which seemed high). I realize credits do not always reflect contributions, but unless there is a reason to suspect that there is severe bias that would cause either designer or producer credits to be particularly inaccurate over 12 years and over 850 games, this shouldn’t have a large effect. There is much more discussion of Moby Games and data collection in the comments below. (Source: Gamasutra)