Creating card illustrations using the midjourney Neural Network

2 weeks ago, my plan for the next update was to finally design and release the second injustice archetype, but then randomly, I was introduced to midjourney, a new Neural Network and my mind was blown.

I had already experience with GANs, particularly artbreeder which I was already using extensively for background shots and for Torment illustrations, but midjourney is using a diffusion model, which results in both way better quality as well as easier way to generate images based on a text prompt rather than explicitly mashing specific images together. This made it impressively fast to go through iterations of images to find what I matched my vision, and the result was also very close to an actual human drawing instead of an obvious AI mashup.

So I basically dropped everything else I was doing at the moment, and for the past 2 weeks I’ve been non-stop creating images for Hypnagonia. With the small help of another contributor, I have now more than a 100 new illustrations, where before there were none!

The process of using midjourney is also a bit novel. Rather than having a web interfact like most would expect. It instead is doing everything via Discord. You have a bot (or a channel) where you send prompts to generate, then get the results back as messages. I am guessing it is done this way to promote social aspects of generating images with others, but since it provides a private channel for paying users, I used that.

My process to generate a card goes generally like this:

• I open my card library in grid mode and then filter to a specific archetype.

• For that archetype, I pick a colour theme the images should have. For example, Laser Cannon I decided would have a “Neon Red” colour theme, while Vindictive would have an “Ochre” theme.

• I Pick a card which does not have a card illustration, and based on the title (and sometimes the effect), I try to conceptualize what would represent the action the card is referring to.

  With a lot of card, like the Laser Cannon stuff, it was fairly easy, as it pertains to still life images. For example the below two:

Fusion Grenade: Nano-machines:

The AI starts to run into more trouble when you’re trying to generate humans in action, or motion in general. Fortunately with enough attempts you can get something halfway close, such as:

Cheeky Approach: Sidestep:

However if you’ve played Hypnagonia, you know the game has a lot of psychological topics, and a lot of the cards I’ve designed express explicit psychological states. And this is where things get real tricky.

Fortunately, my game is also explicitly surrealistic, which means the uncanny valley of the AI generations, kinda work in my favour, as the cards illustrate things you’re doing in dreams, which are always a bit “wobbly”.

Some concepts tend to be easier to figure out than others, but do not be mistaken, there were dozens and dozens of attempts behind each of these:

Shadenfreude: Stewing: Vexing Concept:

But then there’s some which really gave me trouble to think of. With some of them, I even had to turn to the midjourney community for ideas! These are the cards which are impossible to make the AI understand what you’re trying to do, no matter how well you describe it. And even if you do, the random nature of generation will mean the audience won’t understand it.

So instead, I leaned into the abstract and the metaphor. It will not “click” for everyone, but I hope they will make sense for at least some.

It’s the small things: Excuses:

And even with all these, I’m barely halfway done! I still want to finish all archetypes (only Flyer left) and then will start replacing all the existing Artbreeder art other than backgrounds, with midjourney, as the kind of disturbing torments I can craft is amazing.

All in all, this process has plugged a massive hole in Hypnagonia, which was is non-existent or bad card illustrations. There’s just a different feeling during gameplay, when each of your cards in hand, speaks to you.

If you’re making a card game, and like me, you have basically 0 budget, check out NNs like midjourney. They even started their open beta yesterday.

If you’re a member, you can also check my profile and see (and vote on) my new creations ;)