All Advice You Hear (Even From Me) Is Wrong

Bold statement, no? But this is a pretty simple post, actually. It shouldn’t come as surprise to anyone that we live in a complex world, but we tend to rely on mental models that oversimplify things. Even without discussing heuristics and the problems those can create, we can simply think of this in terms of the “perceptual screen.” In other words, we all form preconceptions about all manner of things in the world, and we tend to discount or even completely ignore data that doesn’t fit our model.

Yeah, there are exceptions, and yes, many people try to make a point of working around their perceptual screens (myself included), but the fact is that people tend to do so much subconscious filtering that there is only so much your conscious mind can override that. I drive down the same streets every day, and miss a thousand details that are there because I’m distracted, watching the road, or otherwise. Only if something really violates my expectations of what I should see on that road am I likely to notice it, and I might not even notice it then.

Perhaps that’s getting overly technical, but my point is simply this: none of us knows what we are doing, really. If someone asks me why I think AI War sold as well as it did, or even why it didn’t sell better than it did, I can give my theories, but those theories are wrong and incomplete. If you ask a developer, an executive, and a marketer that were involved with a AAA game the same thing, they’ll all give you slightly (or majorly) different answers, too. None of them know, really. To some extent it’s unknowable: if you ask the average consumer why they’re so excited about some game or another, often their response is going to be pretty vague and unhelpful.

This isn’t related to games, either. Nobody knows why one movie is more successful than another, or why one author is suddenly a bestseller overnight when another similar author is not. Sometimes there are obvious things: quality differences, timing differences, the amount of marketing support, etc. Having a quality product that is well marketed — or an established brand that people are already loyal to — is a well known way to ensure some degree of success in a project.

Until you start looking at all the exceptions: Rock Band Beatles, anyone? That was way less successful than anyone expected, and it had absolutely everything going for it. In retrospect you can point to this or that factor, but the fact is that no one can be really sure why the game didn’t do as well as expected. The problem: there is a market filled with hundreds of millions of consumers, and they have thousands of reasons why they may or not buy something. And not only that, individually they all have various timing issues (being too busy, being out of a job, whatever) that are impossible to collectively predict.

Statisticians may be scoffing at me around now, as many seem quite confident of their ability to predict what large groups of people will do. To that, I say: meh. The best statisticians are pretty accurate in the main, but it’s those pesky outliers that confound all the regular models. They’re using heuristics or mental (or mathematical) models like everyone else. Their models might well be far better than the norm, but they’re still so far from perfect it’s not funny. Not that I’m trying to insult statisticians or their work, which is still important, but I’m making a point here: the future is unknowable, and the reasons for past events when those events involve millions of people are also pretty well unknowable.

Here’s a fun example: if I were releasing AI War now, instead of last year, how would that have affect things? Would the proximity to Starcraft hurt it? Would I have had an easier or a harder time forging partnerships with digital distributors? Would less or more people have bought it do to economic factors with the recession? Would competition with other AAA and indie games have been better or worse for AI War in 2010 compared to 2009? Even the weather might have had some effect, indirectly, in terms of how much people were inclined to stay inside and play games last year versus this year.

And those are just the major factors. There’s also all the hundreds of personal-life factors for every person who bought AI War, or who didn’t-but-might-have-bought AI War. It’s a mess of probabilities, of emergent patterns based on multiple overlapping systems. In short, it’s that whole Chaos Theory thing.

So if you ever ask anyone advice about, well, anything, bear that in mind. We’re all walking around behind our own perceptual screens, making wrong and/or incomplete assumptions and models of the world, and acting based on an accumulated multitude of beliefs. And even if your beliefs, history, etc, closely line up with those another person, if they’re advising you about something like (say) how to be a successful indie developer, bear in mind that there are still so many different factors that can’t be helped: the timing of their release versus yours, the fact that you both developed games that are probably quite different, and so on.

Perhaps this post sounds pessimistic, but I don’t mean for it to. Advice, wrong and/or inapplicable though it might be, is still better than no data at all. Seeking advice is a great thing to do, just like the study of statistics is extremely valuable. You just have to understand what it is you’re getting out of either process: a model, or a way of looking at some small slice of the world. All models are incomplete, so you can’t just rely on any single one. That’s why you want to listen to the advice of others, then make your own decisions.

In the end, the only way to find out how AI War would do was to release it and see. Same with Rock Band Beatles. I made my predictions and I’m sure Harmonix made theirs, but in the end all predictions tend to be at least somewhat wrong. If you’re an aspiring indie developer, you need to worry about two things:

1. Making the best game you can.
2. Doing the best job you can in promoting it and getting people interested in it.

Then, based on a thousand factors — many of which you can’t control or even really even perceive — your game will do well, or it won’t. Most likely it will fall somewhere in the statistical norm for the type of game you are making and the market in which you release it. There’s a chance you might be an outlier in a positive or negative sense, but planning to be an outlier is like planning to win the lottery. Awesome games flop all the time, while their inferior counterparts get accolades piled on top of them. More often than not quality rises to the top, though, so by making a good game you’re certainly playing the odds.

I’m sure that I’ll continue giving out advice for indie developers, just like other indie developers or industry veterans do, but in the future when you read advice from anyone (me included) keep this post in mind. Too many commentators get hung up on extolling the virtues (or imminent demise) of this or that model — casual or not, social or not, sandbox or not, are the current hot topics — and what you need to remember is to do what makes sense for you, and to not try to trend follow. Forge your own path, and see what happens. That’s all any of us are doing, really, if you get right down to it.