Test Driven Indie Game Development

It’s been now half a year since I first started working on Hypnagonia and slightly over a year since I’ve started the card game framework. As the features and complexity of such a project expands, it’s becoming more more and more fragile and cumbersome as well.

Some classic examples:

• Every time I add new content (Torment, card, journal entry, curio etc), I have to actually test it’s working. This means adding the new content in debug code to spawn it in the game, run the game, and then manually interact with the new content to make sure it’s working. This can take from 5 minutes to half an hour, depending on the complexity of what I’m adding.
• Every time I do a refactoring of the code, to enable to me to add more complex features, there is a risk I’ll break multiple elements of the game without realizing. The only way to ensure that the refactoring didn’t break something unexpected, is to go back and check each piece of content one by one. This is naturally unfeasible. I would still go back and test a few things I imagine the refactoring might affect, but there’s no certainty I checked thoroughly. Companies employ armies of beta testers precisely for this reason.

These two means that as the features and content of the game expand, there’s high likelihood that bugs sneak in that I will only catch when someone reports them. And while that’s usually acceptable, it has to caveats.

• It’s stressful. I hate that people have a bad experience with my games, so if a bug report comes in, I feel the need to drop everything and fix it. Even if it’s just one random person for an obscure indie game like Hypnagonia. I just gotta! And if I can’t, it will disturb me until I do. The stress becomes even worse when I do big refactor jobs, as it’s there, that game-breaking bugs can be introduced. Things like the game soft-hanging and not allowing the player to proceed at all.

  This naturally leads to many late nights (especially for tricky stuff) and general psychology impact. And if the game were to become popular? hoo-boy!

• It might not be reported at all. For a game like Hypnagonia, where I count the amount of people trying it out in the low digits, one of the most important things I want to provide is a good initial impression. If someone starts the game and three encounters in they get stuck in the shop, there’s a high likelihood the will close the browser tab and never come back. Not only will I never see the bug report, but I’ll have lost a potential player.

Fortunately, there is a help against these issues, and that is automated testing, AKA Unit and Integration Tests.

The way it works is:

• Every time I add new content, I also have to write one or more tests for it. For example, I add a new card which does some interpretation and draws a card. I will then also write a test which loads a fresh board, loads that card, plays it against a torment and then check:

  • Did the torment take that amount of interpretation? Check.
  • Did the player’s hand size remain the same (-1 played card, +1 drawn card)? Check.

  If all those checks were successful, the test for that card passes. (These kind of tests are typically called “Integration Tests” as they test multiple components of the game working in tandem)

• Every time I add a new feature, such as a new class of content (say, memories), I also write some generic tests to test its methods. For example, if I call the progress() method, does the memory’s progress increase by 1? Check.

  If all these checks were successful, the test for that class passes. (These kind of tests are typically called “Unit Tests” as they test one component in isolation.

Now the process for adding new content requires more upfront effort, but it brings a massive advantage: The testing I will do though automated tests will remain forever and will be re-executed every time I push a new commit to my repository. All those cards I’ve added in the past? If a new refactor I did breaks the interaction of one hundreds of cards, I will know.

If a small, seemingly innocent change completely breaks the game (you’d be surprised how easy this can happen), I will know before my changes go live. I can literally push a massive refactoring live before going to bed to thousands of players, and be assured that not a single one will be affected because I broke the whole game.

The piece of mind this sort of thing can bring is hard to state.

This whole approach is called Test Driven Development* and it the current paradigm for a lot of software companies, but it has not yet been embraced by a lot of game devs. Especially indie ones who might be learning coding as they go.

*EDIT: Many people have pointed out that i misrepresented Test Driven Development and they’re actually right. In TDD, you write the tests first, and then the code to implement those features, whereas I write the features first and then the tests to ensure they’re working. But whatever you call my process, or if you use actual TDD, the point is that it’s quite beneficial.

And that is usually fine for small projects, but anything even slightly more complex than a game jam, can only benefit from this approach. It’s all fun and games when your codebase is small and every day you’re adding more content, but a year-in your development, you’ll find you spend 20% adding features, and 80% of the time fixing bugs.

That way lies madness.

Are there detriments to test driven development? Sure. It is not glamorous at all. It increases development of every new piece of content and feature by a significant percentage. And if you hadn’t started your game design with it in mind, you might spend weeks adding tests and have literally nothing to show to your audience for that period. It’s always more sexy to add a fancy new enemy, or weapon or what have you. The short-term benefits are difficult to see.

But, trust me on this, you are doing your future self a massive favor. I cannot even tell you how many regression bugs I’ve caught in the card game framework through the tests I introduced early on. And every time I do it, especially when the bug is very obscure to track down. I profusely thank my past self!

Unfortunately for Hypnagonia, I hadn’t started development with tests in mind, as I wasn’t sure how the core mechanics were going to shape up. So now I have to play catch up, and add tests for all mechanics and content I have. Which is why I’ve basically been adding nothing new for the past few releases. It’s all backend testing work instead.

It’s not sexy, but my future self will love me!