Tabula Rasa has been quietly garnering middling reviews after its early November release. Gamasutra recently interviewed Richard Garriott, the game's famous producer, who had some surprisingly candid comments about TR's open beta testing: "We burned out some quantity of our beta-testers when the game wasn't yet fun. As we've begun to sell the game, the people who hadn't participated in the beta became our fast early-adopters. And the people who did participate in the beta, we've had to go back to and say 'look, look, we promise: we know it wasn't fun two months ago, but we fixed all that. Really, come try it again.' We've had to go out and develop free programs to invite those people back for free before they go buy it. So the beta process, which we used to think of as a QA process, is really a marketing process."
I find this a fascinating realization on the part of Garriott, mostly because it begs the question of how increasingly more complex MMOs will QA their final product. If the beta should be morphed into a marketing process, then how will game producers successfully stress test their gameplay and servers with large numbers of players?
I didn't beta test WoW, but I did participate in its stress test. By then, the game was largely complete and the gaming experience was nearly identical to its retail equivalent. Looking back now, the WoW stress test was as fun to play as the retail version. What Garriott realizes, perhaps too late, is that their open beta wasn't fun to play which ultimately translated into those beta testers not picking up the game on its release.
I think Garriott makes a good case for the shortcomings of TR's open beta, but I think it's more likely that gamers beta tested TR, got a pretty good idea of what it was like, and then went back to WoW or LoTRO or whatever floats their boat. I do think he's on to something about shifting the open beta concept to more of a stress test: get the game polished, practically to retail, and then hold an open beta to whet gamers' appetites, generate some buzz, and test your servers.
But if gamers are shunning TR in large numbers, I think it's because WoW has a strangle hold on the MMO crowd and TR didn't have the mojo to pull away some market share. It isn't the first MMO to suffer this indignity, and it won't be the last.