an image

AI Game Jam

Hugging Face Open Source Game Jam

I've been looking for ways to jump headfirst into AI and machine learning. A game jam is just the thing!

Hugging Face is hosting their first Open Source AI Game Jam, and I'm excited to participate.

The theme is "EXPANDING", which immediately makes me think of games like:

StarCraft

StarCraft

Factorio

Factorio

Satisfactory

Satisfactory

What to build

I don't know what kind of game I want to build yet, but I am opinionated about some tools I want to use along the way.

In particular, since I've been spending a lot of time learning rust, I want to do the bulk of the programming in rust.

Since the end result needs to be a game that runs in the browser (or on Windows). I'll choose the browser. I think that means that I'll need to figure out how to compile rust to web assembly and then how to use browser apis from my rust application.

I know that there are existing game engines for doing this (like [bevy](https://bevyengine.org/)), but since I've never written an application that compiles to wasm and I haven't used wasm-bindgen yet, I think I'll get more out of it if I ignore the engine piece.

Heck, maybe I'll avoid doing a graphical game at all and instead aim to make a text-based game inspired by a dark room.

Getting Started

Ok, first things first. I want:

Hmm. It seems that the Hugging Face Hub has something called Spaces that lets people host their content. I suspect that my submission will need to live there to be judge. I better read the docs for Spaces real quick...

Oh wait, no. It seems that the game needs to be submitted to itch.io. So if I want to make a web game (I do), I'll follow these instructions.

SO, the path to ship the game seems to be:

That seems easy enough.

I guess I'll do that first with a "hello world" application before starting with the rust-to-web-assembly path.

Hello, world!

I went through the steps to publish a "game" on itch.io. It was very simple.

index.html:

<!DOCTYPE html>
<html>
  <head>
    <meta charset="utf-8" />
    <meta http-equiv="x-ua-compatible" content="ie=edge" />
    <title>Hello, world!</title>
    <meta name="viewport" content="width=device-width, initial-scale=1" />
    <link rel="apple-touch-icon" href="/apple-touch-icon.png" />
    <!-- Place favicon.ico in the root directory -->
    <link rel="stylesheet" type="text/css" href="style.css" />
  </head>
  <body>
    <!--[if lt IE 8]>
      <p class="browserupgrade">
        You are using an <strong>outdated</strong> browser. Please
        <a href="http://browsehappy.com/">upgrade your browser</a> to improve
        your experience.
      </p>
    <![endif]-->

    <h1>Hello, world!</h1>
    <script src="main.js" type="text/javascript"></script>
  </body>
</html>

main.js:

console.log("Hello, world!");

function main() {
  console.log("Hello, main!");
  let node = document.createElement("p");
  node.innerText = "This element is added from javascript.";
  document.body.appendChild(node);
}

document.addEventListener("DOMContentLoaded", main);

style.css:

h1 {
  color: blue;
}

p {
  color: red;
}

build.sh:

#!/usr/bin/env sh

zip -r game.zip src/

And that's it!

Hello World

Random idea

Another "expansion" theme game I've played was pandemic, where you're an illness of some kind an your goal is to infect all of humanity (or as much of humanity as you can).

Perhaps it's not in good taste after the global pandemic we just went through, but was definitely a fun game to play at the time. Apparently there's a sequel, pandemic 2.

Pandemic

Next, rust and web assembly

Ok, so I know that I need to build a zip file with my game in it, and otherwise I can just pack whatever assets I need. How do I build a wasm module in rust and include that?

That will be the next step.

Another random idea

Here's an idea for an "expanding" story game.

It's a text adventure. You tell the ai what to do or what happens next, just like you do in AI Dungeon.

The difference is that each interaction you have with the AI creates a node, and you can travel to any previous player interactions from that node or make a new (different) choice.

For example, imagine this interaction:

Game: You're in a room with three doors. You see three doors. One to the left, one in the middle, one on the right.

[ ] Open the door on the left.

[ ] Knock on the door to the left.

[ ] Break down the door in the middle.

[ ] Use x-ray vision to look behind the door.

[ ] Do something else... (Type what happens next.)

If you choose any of the first 4 options, you would visit a node that already exists (and get the output from the previous run of the ai). Then for any subsequent action you take, I'd need to prime the AI with information about the stuff that has happened along the way.

To create something like this, I'd need a data store that persists between users and sessions. I don't really want to do that, but I wouldn't mind knowing how (from itch.io).

Anyway, back to setting up a rust-to-wasm project.

The rustwasm book

Ok, so the rust and web assembly book suggests I get started with cargo-generate, which lets me use a pre-existing git repo as a template. This kind of template approach tends to annoy me (e.g. create-react-app because I feel like I fumble around for a long time before I understand how to put a project together "from scratch" when I start with a template). But maybe I need to have more of an open mind. After all it seems pretty useful for how I'd want to spin up things once I know how they're put together.

Let's do what it says and cargo generate a new project based on their template.

Ah, there's more. The way they're setting up their project is in two separate stages:

The cargo project turns rust into wasm. The npm project takes care of everything else.

That makes sense! Building a wasm library ought to be just like building any other library, so it shouldn't necessarily be coupled into "one" (although I'm very curious to know how things like yew are set up. Maybe I'll find out later.)

Ok! So now I have a rust -> wasm and an npm project that uses the wasm blob. Let's see if I can pack it up in a zip and upload it to itch.io.

npm run build

zip -r wasmgame.zip dist/

Sure enough, it works!

wasmgame

This is just the "hello-world" application from the game-of-life tutorial in the rust + wasm book, running on itch.io.

It's been two and a half hours since the start of the game jam and I'd say we're off to a good start!

Implementing life

The tutorial I'm following is about Conway's Game of Life. I followed along, got it working, and made sure everything worked as expected when I uploaded it to itch.io:

life

Restarting

Ok, so that's all fine and good, but I really don't like the feeling of starting with the template and not knowing how to put something together. I expect to want to do this several times over the next few months, so I want to make sure I understand the process.

Let's do it again. Only this time, with no prebuilt templates.

Getting webby

It's Saturday now. I have not done any game development per se, but I've added yew and tailwindcss to my application and learned enough about each to be writing all of my UI in rust now. That's pretty cool.

So, what does my fancy new UI do (on Saturday at 5pm, more than half way through this gamejam)?

It... Has a button that toggles a dark mode theme! (Queue applause).

light mode dark mode

Haha... Not very impressive but very satisfying for me.

Colors

Websites (and games) typically have whole color systems to work with. More than a simple 5-color pallete, this is a collection of 50-100 colors that include every color a button or modal might be when hovered, disabled, etc. It's an essential part of the application's design system, which I learned about by reading Design System Foundations.

I don't have any desire to go find or create such a color collection, especially because from the looks of things I won't have the time to use it anyway. But I do want a visual guide to the colors that are available by default in tailwind. Annoyingly, I couldn't find their layout in a simple text file, so I created one by copying their website:

screenshot

If you're wondering why a file that looks like this is helpful, check out what it looks like when I have rainbow-mode enabled:

screenshot

With those ready, I'll give my buttons a deliciously purple look:

screenshot

While I'm at it, I'll also move all my theming into rust as simple strings, rather than style.css.

This is actually kind of annoying, because it means to reload my themes I need to rebuild my wasm bundle and then reload webpack-dev-server. Since the wasm bundle is a dependency of the npm project, webpack-dev-server won't auto reload when it changes.

The best I've come up with so far is this shell script:

#!/usr/bin/env sh

dir=$(dirname $(dirname $(readlink -f "$0")))
expansion_game_wasm="$dir/expansion_game_wasm"
expansion_game_client="$dir/expansion_game_client"

build="cd $expansion_game_wasm && wasm-pack build &&"
webserver="cd $expansion_game_client && npm run start"
onchange="$build $webserver"

alacritty -e zsh -c "cd $expansion_game_wasm; cargo watch -s '$onchange'" &

Unfortunately, this means it takes several seconds to see changes to css. Certainly not ideal considering "live edit / live reload" has been a thing in the front-end world for over a decade.

A useful checkpoint

Since I was learning along the way, it took me a long time to get the tooling set up and configured how I like. I might end up wanting to start from this point for a future project, so I went ahead and create a repo with the work done so far: rust-wasm-webapp-template.

Some actual AI

One of the main points of the game jam was to play around with open source AI models. I've been using openai's chatgpt quite a lot, and I've been using midjourney, but there are no open source ai models that have made it into my regular rotation. If Stable Diffusion weren't so slow when I set it up, I might use it more. Perhaps I need to be more patient with it, or find a way to run it on more powerful machines than my own.

I found an open source ai project that caught my eye called coqui. Started by some former Mozillians, it is an app / service / ai / model that makes it easy to create voice / audio data. I played around with it for a while. I thought a fun voice to try to get it to replicate would be Bojack Horseman's saying, "What are you doing here?". But then I got distracted by a Bojack Horseman soundboard app, watched a few clips from Bojack Horseman, and somehow 30 minutes had gone by and I hadn't actually gone and done anything interesting with coqui.

The final hours

I had about 2 hours left in the game jam without any kind of game in sight. I recently played the web "clicker" game called a dark room. It has these buttons that timeout after you click them:

screenshot screenshot screenshot screenshot

I spent the remaining time of the game jam (2 hours) bouncing between the docs of yew, gloo, web-sys, js-sys, and css animations to understand how to write a yew Component, how to use the browser's setTimeout api from rust, and how to animate the background color of a button.

Finally, I got it working:

screenshot screenshot screenshot screenshot screenshot

Conclusion

With that, it was the end of the game jam. I had embarrasingly little to show for it:

screenshot

The "game" is live on itch, I did not bother entering a submission.

I didn't end up creating a game worth submitting but I learned a lot this weekend...

That was fun! Hopefully I'll get farther next time.

I sent the same message in the Discord (because I'm trying to participate more online) and received a wholesome response from a stranger.

screenshot

And that felt like a nice way to wrap things up.