Game Engine with AI✨ Part 2 - DALL E Sprites

Hey, devs! 🖐🎮
Hope you're all having a nice day! I have some cool updates on helicity.ai

For reference, this was part 1 (nobody saw it🙃)
I'm making an AI JS Browser Game Engine

Inbuilt DALL E Support

I've integrated DALL-E support! 🎨🖌️ Yes, so from now on, you can define your sprite URL as "lookup: some description", and Helicity.ai will generate a unique sprite for you on the fly using DALL-E. Create unique, custom images without having to leave the IDE.

How I made it

1. So it all happens in the Gameobject.js module. Whenever you make a new instance of the GameObject, this code runs
2. When you put "lookup:" string in your image source, it takes this as a request and calls a firebase function made in Express
3. The firebase function calls the DALL E API looking up the string after "lookup:" and asynchronously returns the image-url, this is sent back to the game object.
4. The game object image is drawn every frame using *render() *(automatically in the game loop) or else, its just a black box
5. The savedImages makes sure that if 2 game objects are created with the same lookup description, multiple called aren't made to dall e. For example pipes in flappy bird.

Gameobject.js

var savedImages = {};

export class GameObject {
  constructor(x, y, width, height, type, imageSrc) {
    this.x = x;
    this.y = y;
    this.width = width;
    this.height = height;
    this.type = type; // can be Used for collision detection
    this.image = new Image();
    this.image.loaded = false;

    //check if imagesrc contains the string lookup:
    if (imageSrc.includes("lookup:")) {
      //check if the image is already saved
      if (savedImages[imageSrc]) {
        this.image = savedImages[imageSrc];
        this.image.loaded = true;
      } else {
        //make a post request. this is in html to use XMLHttpRequest
        var xhr = new XMLHttpRequest();
        xhr.open(
          "POST",
          "https://us-central1-myaigameengine.cloudfunctions.net/api/Image",
          true
        );
        xhr.setRequestHeader("Content-Type", "application/json");
        xhr.send(
          JSON.stringify({
            userPrompt: imageSrc.split("lookup:")[1],
          })
        );
        xhr.onreadystatechange = () => {
          if (xhr.readyState == 4 && xhr.status == 200) {
            //console.log(xhr.responseText);

            //get the imageURL from the response
            var imageURL = JSON.parse(xhr.responseText).image_url;
            this.image = new Image();
            this.image.src = imageURL;
            this.image.onload = () => {
              this.image.loaded = true;
              //save the image
              savedImages[imageSrc] = this.image;
            };
            this.image.onerror = () => {
              console.error(`Error loading image ${this.imageSrc}`);
            };
          }
        };
      }
    } else {
      this.imageSrc = imageSrc;
      this.image = new Image();
      this.image.src = imageSrc;
      this.image.onerror = () => {
        console.error(`Error loading image ${this.imageSrc}`);
      };
      this.image.onload = () => {
        this.image.loaded = true;
      };
    }

    Game.addGameObject(this);
  }

  update(deltaTime) {
    // Overridden by child classes
  }

  render() {
    //draw a black rectangle if there isnt an image else draw it
    if (this.image.loaded) {
      Renderer.drawImage(this.image, this.x, this.y, this.width, this.height);
    } else {

      Renderer.context.fillStyle = "black";
      Renderer.context.fillRect(this.x, this.y, this.width, this.height);
    }
  }
}

Firebase function's Express Image Endpoint

app.post("/Image", async (req, res) => {
  //var uuid = req.body.uuid;
  var userPrompt = req.body.userPrompt;
  //console.log("uuid is " + uuid);
  console.log("user prompt is ");
  openai
    .createImage({
      prompt: userPrompt,
      n: 1,
      size: "256x256",
    })
    .then((result) => {
      image_url = result.data.data[0].url;
      console.log(image_url);
      res.send({
        success: true,
        image_url: image_url,
      });
    });
});

Thats it!

Think about it - need a sprite of a "purple dragon with a top hat?" 🐉🎩 Just type lookup: purple dragon with a top hat and voila, your personalized sprite is ready to be used in your game! It's that simple! 🎲🚀

Multiplayer support is coming soon!

If you'd like to talk about and participate in the development, hit me up on discord!

https://discord.gg/RkH98RgVDr

Stay tuned for more updates! Happy coding 🎮💖