Chris McElligott Park
May 30, 2013
Hello there! Looks like, as usual with a game launch, we have a lot of new players to our games. Welcome! I think you’ll find Arcen is a bit… unusual… when it comes to our development process, though, and this may come as a bit of a surprise.
Two Different Software Development Philosophies
I used to have a boss who liked to say something to the effect of “a piece of software stops changing only when it’s dead.” This was in the business software services arena, so that aphorism was pretty universally practiced in that industry. How many of you use Gmail? They’re always fiddling with that thing and adding improvements and changes, right?
In the games industry, there’s more of a “dump em” sort of model that most game developers use. A new Mario game comes out, and it is what it is. That game isn’t going to see any patches, period. But… the game isn’t “dead,” either. It’s only just come to life in fact, as far as the public is concerned, and it will live on in the annals of gaming history, with some players fondly revisiting it, forevermore. I still crack open Mario 2 from the original NES every year or so.
The Benefits of Constant Iteration
Needless to say, given my background, I am not overly thrilled with the “here this game is, bye” sort of model. I mean for games that I make, to clarify — it doesn’t bother me at all that Nintendo does it.
I think a lot of other indies feel the same way about the games that they make. Terraria and Minecraft see tons of updates all the time (well, Terraria used to), and Don’t Starve and others update almost weekly as I understand it. How many updates has Team Fortress had again? And they aren’t even indie. Etc.
The constant-iteration approach is really cool because it allows for a constant dialogue back and forth between players and the developers. Once players have their hands on a game, they inevitably have opinions. Things they would like to see improved, things they would like to see added just for the fun of it, and bugs they find that none of the testers found prior to release.
None of that is possible if the developer isn’t listening to you, and doesn’t keep doing updates.
The Downsides of Constant Iteration
I’m not going to pretend that it’s all glory on the iteration side, of course. There’s something to be said for having a perfectly-polished game that never changes. It’s an icon, something immutable that players can enjoy and remember forever. I come back to Mario 2 all these years later, and it’s the same. That’s actually pretty cool.
But more than that, sometimes during an iteration, we make a misstep — we change something that people freak out about (like the mountains/lakes thing in Skyward Collapse, and the bandit keeps against mythologicals thing in the same). Sometimes new bugs get added. Sometimes Herobrine keeps getting added and removed, over and over. ;)
What mitigates the downside of fixes like that is the developer actually listening to players when they freak out about a change or find a new bug. Players talk, the developer listens, the developer makes a change, and people are happy — and the game has still evolved despite a temporary speed bump.
Update Frequency
I think the biggest thing that freaks out some people new to Arcen is just how frequently we update our games. Most workdays in the week, we put out at least one update to at least one of our games. It is extremely rare that an entire week goes by without an update, and that’s generally just happening when we are all knee-deep in working on a new game behind closed doors. And even then, if we’re in a private alpha with players, the updates are pretty much daily.
Why all the rushing around? Well, it’s not really that we’re rushing, honestly; we work every day of the week, same as any other game developer. However, thanks both to our own internal updaters for our games and to the update model in Steam, updates take very little time. Actually pushing out the code and assets for a new update takes me literally 5-8 minutes. I’ve timed it. It takes me longer to write up the post about the release than it does to actually push out the release.
In other words, the barrier to us actually doing releases at this frequency is basically nil. So just because we can, does that mean we should? I obviously think the answer is yes, since that’s how we do things. My reasoning is that there are fewer risks of big incidents (huge bugs or players really hating something) if we do our updates frequently. And with this frequency, if players do find a new critical bug or really have a giant backlash against a change, we can have a fix out often within hours.
How many times have you been frustrated by a developer who knows about a bug, and takes 6+ months to issue a patch for it, if ever? Man, I start getting heartburn when it goes 24 hours, no joke. Maybe I have a tiny bit of OCD or something, but I just don’t like things hanging out there like that if I know about them.
But What’s With All The Fixes Right After Release?
Day One Patches are a Bad Thing, right? Well… I don’t know about that.
AAA Developers
First off, let me talk about AAA developers (which we are not) just to set the stage here. Those guys have to get a build ready and “go gold” with it a month or months in advance of when you actually get the game. It has to go through the process of manufacturing and all that good stuff, right? That takes time, and once the discs are printed that’s it for the 1.0 version.
But what are those same developers doing during the months between when they finish the gold master and when you first get your hands on it? Twiddling their thumbs? I don’t think so. They’re still working, testing, tweaking, etc. I’m sure the dev leads are often slapping their heads and going “oh my god, how did we miss this one with 200 testers looking at it?” But that sort of thing is inevitable in a game of sufficient complexity.
So what they do is dutifully fix, then test, all the things they find during this time period. They’re doing a good job! They’d call back the discs and apply said fixes to them if they could, but they can’t. So day one when you get your disc, there’s a patch waiting for you, and the AAA dev team is feeling happy about that. You’re feeling mad at them about releasing something buggy in the first place. I understand the sentiment, and I’ve felt that from the player end of things as well, but I don’t think that’s really justified.
Indie Developers
As with AAA developers, we never stop working and just sit around twiddling our thumbs. However, every indie developer has a massive disadvantage compared to their AAA counterparts: a tiny staff and no giant QA staff. Most new indie developers are going to do all their QA testing themselves, plus roping in friends and family as much as they can. An indie developer new to the market is going to have some launch bugs, and so long as they fix those pronto they should be cut some slack in my opinion.
But that’s not talking about Arcen — at this point we’ve been around for four years, and this last release is our sixth full game on Steam. So none of what I just said about indie developers who are new to the market applies to us. We have the benefit of an existing fanbase, and we use them to do QA testing for us in exchange for a free copy of the game. And we also keep using ourselves and friends and family, of course.
In the private alpha for Skyward Collapse, which lasted almost a month, we had around 35ish testers banging on it as frequently as they could. We had around 10ish testers who went just crazy above and beyond and were banging on it daily. Holy cow are we indebted to those folks.
By the time any of our games launch, we’ve fixed all materially-significant bugs in them, and the testers are happy. Same as with the AAA developers when they “go gold” for manufacturing. Yay celebrations!
What Happens Right After Launch
This is the same for both indies and AAA developers: suddenly there are massively more people looking at your game. Skyward has sold more than 15,000 copies in less than a week. A large AAA game might sell hundreds of thousands, or even a million copies in its first week.
How can thirty testers, or two hundred, or pick your number, ever compete with that many players in terms of finding bugs, exploits, and things that are confusing? So as soon as that many new eyeballs are on a game, there’s this whirlwind of sudden feedback. This is great, and I love how engaged players get — and how clever they are. The important thing is that whatever they find that is a problem is addressed quickly, because I think it’s pretty impossible that they will find nothing unless the game has been through an insanely high-budget QA process that only a few companies (like Nintendo) can afford to do.
There’s also the issues of complexity: Nintendo does not make games with procedural generation, for instance. They make closed game worlds where they control your experience minutely. For a game with any procedural component to it, there’s always going to be new things that crop up unexpectedly. You put a bucket on the head of a guy, and then that lets you rob him blind. Oops, how did nobody find that? Well, the game was just that complex, and none of the testers happened to think of taking that specific action. Like I said, players are clever and will think of all sorts of things, particularly in a game with a high degree of complexity and randomization.
There’s also the issue of platform: Nintendo makes their own hardware, and don’t have to support a variety of operating systems and an even wider variety of hardware. Holy heck is the PC the wild west compared to any console. I don’t think even Nintendo could avoid some patches if they released PC games.
Finally, there’s the issue of player cleverness in certain genres. Specifically, the kinds of genres that Arcen serves. For a Mario game, this problem doesn’t exist: you can only be so clever, because the levels are controlled very tightly and the abilities all have very specific functions. There is not more than one way to solve a Mario level unless the developers specifically designed it to have multiple ways to solve it. This is just fine, but this very different from the sort of games we make.
With any strategy game, really, or with any game where there is crafting or procedural loot or a vast world with lots of interlocking rules… players are going to figure out some funky stuff. Some of them on day one. Some players are going to find a crazy shotgun-reloading exploit, and then the normal game balance falls to pieces when they use it. Why wasn’t it found? Complexity and open-endedness of the game world.
What’s the next step when an exploit is found? For Arcen: trying to work out a fix with the community that stops the exploit without stepping on normal gameplay, and then implementing that as swiftly as possible. If you complain about how AAA developers don’t fix the exploits, or just resort to banning people who use the exploits, then this should be a welcome thing. But you can’t have it both ways: in the process of fixing the exploit, sometimes we’re going to make a misstep and change things that pisses off other players. At that point we roll back that change and discuss it more with players.
Conclusion (TLDR)
The Arcen way of doing things is unusual, to be sure. Possibly singular in the sheer frequency of how we do updates (weekly is not uncommon with some other developers, but I don’t know about daily). But the way that we do things lets us better connect with our player community, lets us be more proactive on changes and fixes, and lets players see the fruits of said changes and fixes faster.
I think it’s a good system, but like everything it has its pros and cons. We’re specifically making some changes to our process lately in order to try to minimize the cons: rather than immediately implementing a fix to an exploit that we think is a good solution, we’re instead taking it to our forums and inviting feedback and discussion. Players are experts at poking holes in our ideas, and so we now wind up going through some lengthy discussion before we actually make a change.
What that leads to is slower fixes to exploits or certain tricky balance issues, but less “flailing” in the sense of “let’s try this and see what the players think” and then “oops, revert.” That process of flailing is something that is a byproduct of being prototypers by nature (“let’s try this thing and see how it feels in action” works great in an alpha or beta — it’s a fast, effective, and visceral way to form opinions on a possible change — but it’s not a good idea in a released game).
Overall I think our new process of talking out substantial changes before implementing them is the way to go; and that doesn’t impact our ability to make small tweaks and fixes and improvements on a daily or near-daily basis. I think that’s a win for everyone, frankly.
Thanks for reading, and for your support of our company in general!
Arcen Games Forums
Arcen Games Idea Tracker (Mantis)
