Integrating AI into storyline

As I quickly mentioned before, I have started integrating KoboldAI into Hypnagonia and I am using it for dynamically generating introductions to the various encounters in your dreams.

KoboldAI is a brilliant little software that allows one to run their own GPT text-generating instance either on their own hardware, or on the cloud, but completely under their control. Also unlike the generalist GPT-3, it is focusing mostly on novels and adventures which is perfect for my needs.

From what I’ve seen, it’s the only free options available to the consumer and it’s AGPL to boot! Perfect! This allows me to do whatever I need with it, even up to modifying its source for my purposes (and it’s written in python!).

I had initially started trying to write stories using OpenAI’s GPT-3 model, but i was quickly starting to consume most of my “free credits” and it was also nothing I could distribute to everyone. GPT-3 is also quite generic and not well suited for writing first-person stories. I had to give it a lot of guidance to make it do what I wanted.

So I joined the KoboldAI (“KAI” from now on) community in order to discover how I integrate it directly into Godot, as opposed to crafting the prompts manually on my KAI instance and then hard-coding it in. Unfortunately, it seems that initially this was not possible at all. KAI provided only a socket.io integration and Godot has no way to use that protocol. After trying for a day to make it work with websockets instead, I have up and asked for a possible REST API integration, expecting I would have to continue with manual prompts until then.

To my surprise, one of the core devs (VE FORBRYDERNE#6568 in discord) immediately stepped up and started working on a REST API implementation. Within a few days they had already a working branch which allowed me to start working on a Godot-KAI integration within a few days after discovering it! I was overjoyed because I had so many cool ideas to follow!

Now that I had a REST API, it was trivial to connect Hypnagonia to it through basic HTTP requests which Godot can handle easily. But there were some dynamics to take care. First, Story generation, especially on local hardware can be slow. About 5 seconds per 60-word intro. That can be fine if you’re using KAI directly, but when the player is playing H, this can add a frustrating amount of time waiting.

My solution was to do this asynchronously in the background. Every time the player sees a story, Hypnagonia automatically sends a request to generate the next story for that encounter, which will be seen the next time the player encounters it. The player will have no delays, and I still get plenty of content.

But not everyone has a powerful enough GPU to run KoboldAI, nor might they want to go to the trouble to install it, or run it on google collab. As fun as it is for me, it’s still quite a technical task, outside the skills of non-IT people. I needed the way to have the people with the technical know-how transparently help everyone else.

So I wrote a simple server in python which can receive freshly generated stories from people and then redistribute it to everyone else. So now people who do not have the possibility to run their own KAI instance, can simply receive generated text from others and then rate it, to help train the future versions of the model.

The approach I’m using is that a story generated in Hypnagonia is first only shown to the person who generated it. If that person rates is positively, it is then sent to my server as an “evaluating story”. Evaluated stories are sent to others who play the game and they can rate them as well. Once someone rates an evaluating story, they never see it again until it’s finalized.

A story is finalized once it’s rated at least 5 times. If the majority of the ratings are positive then it is saved into the “finalized stories” bucket at which point it starts rotating as a “standard” story, along with the hard-coded ones. These are seen once there’s no more stories for the players to rate only (or if they’ve disabled ratings). If a story does not have a majority positive ratings, it is instead deleted forever.

Furthermore, each story can be rated in one of 3 ways. Surrealism, Comedy or Horror. At this moment I am storing this metadata but not using it, but once I have a good amount of finalized stories of each type, I’ll have a setting which will allow someone to specify the kind of story-line experience they want to have in Hypnagonia. Do they want to play in Surrealistic dreams, or in Nightmares?

As I was working on it, I also realized that the generated text from the KAI model I was using (Nerys) was a bit weak at the genre and prose direction I wanted to go, no matter how much help I provided. So I needed some way to train it further. Fortunately KAI provides something called a “soft prompt” which is a way to train the model of a smaller datasets (a couple dozen Mb of sample text) that one wants to simulate.

As I primarily want to simulate someone recounting the previous night’s dreams, I needed a good amount of content that is written in such a manner. The best source I discovered was the Dreams subreddit, where people constantly post their previous night’s dreams and request interpretations. So I wrote yet another python script, this time a small scraper, which would suck up all self posts in /r/Dreams, figure out the worthwhile ones and then convert it into a dataset format.

After I gathered the content and with quite a bit of tweaking (there is a surprising amount of typos and grammar shortcuts in those posts) and pruning (need to get rid of all the sexual abuse and advertisement stuff) I then passed it (and some other surrealistic datasets) through the Easy Softprompt Tuner provided by the KAI peeps and I got my first softprompt. The output of KAI immediately improved considerably in the direction I wanted!

But while I have now the potential to write good AI-text, I didn’t yet have the right input. And the quality of the stories is very dependent on how well your prompt will guide it. Unfortunately as I am planning on writing short isolated 50-60 word stories instead of consistent adventures, novels or dialogues, I could not really find any good examples of how to do that. It seems what I’m trying to do is quite unique :-/

So I needed to discover what kind of prompt would generate a surreal story in the first person, past tense. Unfortunately I have not had amazing success with that, as trying to get it to generate the whole thing from scratch is proving impossible to move to the right direction. Instead, the best option I’ve seen is to provide a short 10-20 word sentence to start it with, and then it will mostly follow the right approach. About 6 of the 10 stores I generate using this approach are good enough to keep and add to the game.

However my master plan is that once I have a good amount of finalized stories, I can then create a new and very adapted softprompt which will be trained only on generated stories people actually liked from Hypnagonia instead of scrapped /r/Dreams. Each such story will have a very specialized text in front of it, which will hopefully train the AI to recognize what I want to write without a short sentence. So for example, instead of giving the AI something like “I found myself in the middle of an argument with a clown” in order for it to generate me a Clown intro, I would give it something ”[ Keywords: Clown ]”. In turn the text in brackets would not be displayed, but only the subsequent generation, which would start in any direction the AI wanted, while sticking to the style of all the other stories we already have. The benefit of this approach is that we avoid all Clown stories starting with one of the same few sentences.

All in all, this process took me about a week, at which point the KAI REST API was also finalized so I could use directly the main dev instance of KAI on google collab. So yesterday I took it for a drive using the more powerful Google collab instance, and within 2 hours, I generated 50Kb of introduction stories for torments. This amount to about 3 stories of 50-60 words per torment, or around 5500 words in total.

So the next time you fire up the local version of Hypnagonia, you should automatically see a fresh stories on all torments. As I keep playing the game, this content will only increase to the point where you might never see the same story twice!

I also have a lot more plans to utilize the AI going forward. For example I am thinking of adding taunts to each torment, which would take into account the names of the cards you played last round, and comment on them, in the style of that torment. AI allows for crazy stuff like that!

But, still quite a lot more work to do. I still need to finish all the prompts for the non-combat encounters, and then also add a few more prompts to all torments, so that you don’t always see everything start with the same sentence.

Btw, if you have an interest in AI/gamedev interaction, and this all sounds exciting, I could definitely use more help!