Week 10: Chasing [Fireflies] amidst an Undreamt [Sky]

Hello everyone, and welcome back to Constructing [daybreak]! As you may or may not have derived from this week’s title, I will be addressing the bugs in the AI portion that I discovered upon distributing a prototype game to my beta testers.

01. Local [Supernova]

Last week, I mentioned that I introduced a larger LLM to the game in an attempt to generate higher quality content. However, an 8 billion parameter AI model came with its own drawbacks—mainly, the amount of CPU power that it requires to run. 

While I did not experience much inconvenience running the AI model on my own computer, a rather new M4 Mac, running the same AI model on one of my beta tester’s older M1 Mac resulted in a catastrophic freeze and subsequent crash. As it turns out, running an 8 billion parameter LLM model on an M1 Mac was pushing the upper limits of the hardware, naturally leading to my beta tester’s inability to play the game.

As a tiny model (1 billion parameters) produced repetitive output but a significantly larger model (8 billion parameters) rendered the game unplayable on a number of slightly older computers, I had to settle for a middle ground—a small LLM model at 3 billion parameters. 

02. [Laser] point

However, as you may expect at this point, a lower parameter count typically corresponds to lower quality generation. To make up for the drop in quality from using the smaller 3 billion parameter model, I had to narrow down my prompt. Specifically, I had to remove character expressions from the prompt, as the LLM tended to swap out the character’s pose with their lines, leading to sudden gaps in the AI-generated narrative. 

In addition, the 3 billion parameter model displayed a tendency to forget the ending header that tells my binary tree script when the scene is over, resulting in no binary tree being fully created. As a result, no dialogue was actually displayed despite the LLM generating a whole scene. Unfortunately, no amount of prompt engineering could resolve this problem; I simply had to add a check at the end of my original binary tree script to create the tree regardless whether the ending header was there or not.

And that’s a wrap for this week! Next week, I will be discussing the final result of this project as well as the beta testers’ opinions and feedback. See you next week!