Jeffrey Ip for Confident AI

Posted on Nov 29, 2023

⤴️How to build a Midjourney API with Nest.js 🚀

#javascript #midjourney #api #nestjs

TL;DR

In this post I will show you the architecture of building an unofficial Midjourney API with Typescript and Nest.js.

DeepEval - open-source evaluation framework for LLM applications

DeepEval evaluates performance based on metrics such as factual consistency, accuracy, answer relevancy

We are just starting out.
Can you help us with a star, please? 😽

https://github.com/confident-ai/deepeval

➡️ Please Like, Heart and star this article

What are we going to build?

To start off, let's understand how Mindjourney works on Discord. People use simple commands to talk to an AI bot. The bot then takes these commands and creates pictures that match the descriptions given by the users.

In order to mimic this type of behavior, we will need to create an API that interacts with a Discord Bot. This bot has the ability to interact with Discord, hence send commands to Midjourney. Here is a high level design of the command flow:

Important: This tutorial is only for educational purpose and understanding the nature of Discord and Midjourney interaction. It is not recommended to use it as a production service or in any official project.

The reason I chose Nest.js is because there are plenty of Python examples of how to build API for Midjourney, but not a decent one that shows the process using Javascript or Node.js. I prefer using Nest.js as it also well organized and easy for bootstrapping a project.

Connecting a Midjourney's Discord bot

In the absence of a formal API, a connection to Midjourney is facilitated via a Discord bot. The process includes the following steps.

Important : In order for this to work, you must have Midjourney subscription.

Step 1: Create a Discord bot.

Take a moment to help me please. I am working very hard to create the best open source for LLM evaluation.

Please give me a star - I will truly appreciate it.

🌟 DeepEval on GitHub

The first step towards a complete Midjourney API is to create our Discord bot. Discord has an interface for creating bots for different purposes. Go ahead and create your MJ bot.

Here is a great article for creating a Discord bot.

Once you've created the bot, you'll receive an invite link. Use it to invite the bot to your Discord server - we'll use this later to generate and receive images.

Step 2: Implementing /Imagine command

Once creating a Nest.js app, go ahead and create your discord module. This module will interact with our Discord server and MidJourney.”

Let's begin with our controller that should look something like that:

@Controller('discord')
export class DiscordController {
  constructor(private discordService: DiscordService) {}

  @Post('imagine')
  async imagine(@Body('prompt') prompt: string): Promise<any> {
    return this.discordService.sendImagineCommand(prompt);
  }
}

As you can see, I have created a discord module with a single POST request. We will pass a prompt to our discord/imagine request.

Next, let's create our discord service:

@Injectable()
export class DiscordService {

constructor(private readonly httpService: HttpService) {}


  async sendImagineCommand(prompt: string): Promise<any> {
    const postUrl = "https://discord.com/api/v9/interactions";
    const uniqueId = this.generateId();

    const postPayload = {
      type: 2,
      application_id: <APPLICATION_ID>,
      guild_id: <GUILD_ID>,
      channel_id: <CHANNEL_ID>,
      session_id: <SESSION_ID>,
      data: {
        version: <COMMAND_VERSION>,
        id: <IMAGINE_COMMAND_ID>,
        name: "imagine",
        type: 1,
        options: [
          {
            type: 3,
            name: "prompt",
            value: `${prompt} --no ${uniqueId}`
          }
        ],
        application_command: {
          id: <IMAGINE_COMMAND_ID>,
          application_id: <APPLICATION_ID>,
          version: <COMMAND_VERSION>,
          default_member_permissions: null,
          type: 1,
          nsfw: false,
          name: "imagine",
          description: "Create images with Midjourney",
          dm_permission: true,
          contexts: [0, 1, 2],
          options: [
            {
              type: 3,
              name: "prompt",
              description: "The prompt to imagine",
              required: true
            }
          ]
        },
        attachments: []
      }
    };


    const postHeaders = {
      authorization: <your auth token>,
      "Content-Type": "application/json"
    };

    this.httpService
      .post(postUrl, postPayload, { headers: postHeaders })
      .toPromise()
      .then(console.log);


    return uniqueId;
  }


  generateId(): number {
    return Math.floor(Math.random() * 1000);
  }
}

You will notice a few things here:

We are using https://discord.com/api/v9/interactions discord endpoint to interact with Discord server and send commands. This is the main entry point to deal with requests to Midjourney.
We mimic a web-browser request to Discord, and here is the real "magic" - go ahead and send /imagine command from your Discord web interface to Midjourney, after signing in to Midjourney web.
Once sending a request , you will notice the imagine command sent in Network tab as well, which is very similar to the above.
Copy the relevant fields : IMAGINE_COMMAND_ID , COMMAND_VERSION, SESSION_ID, GUILD_ID, CHANNEL_ID and APPLICATION_ID. This will be used on our service. We also need to copy MIDJOURNEY_TOKEN which is sent as part of the request.
Copy BOT_TOKEN from the bot application page we created earlier. This is important in order to communicate with our bot.
You will also notice the uniqueId that we generate using our generateId() function. This is using Midjourney's --no command so we can later track back the unique request sent to Discord and get the generated images.

Once completing this step, you are now able to call Discord with /imagine command and generate images with Midjourney.

Reminder : This is only a technical post describing how this flow works, and is not recommended for use for any project.

Step 3: Fetching generated images.

Let's create a new controller to fetch images:

@Get('mj/results/:id')
  async getMidjourneyResults(@Param('id') id: string) {
    const image = await this.discordService.getResultFromMidjourney(id);
    const attachmentUrl = get(image[0].attachments[0], 'url');

    if (attachmentUrl) {
      const urls = await this.discordService.processAndUpload(attachmentUrl);
      return urls;
    }
    return image;
  }

We are going to use the unique id generated when creating our /imagine request, in order to fetch results from Discord.


async getResultFromMidjourney(id: string): Promise<any> {
    const headers = {
      "Authorization": MIDJOURNEY_TOKEN,
      "Content-Type": "application/json"
    };
    const channelUrl = `https://discord.com/api/v9/channels/${CHANNEL_ID}/messages?limit=50`;

    try {
      const response = await this.httpService.get(channelUrl, { headers: headers }).toPromise();
      const data = response.data;


      const matchingMessage = data.filter(message =>
        message.content.includes(id) &&
        message
          .components
          .some(component => component.components.some(c => c.label === "U1") ) // means that we can upscale results
      ) || [];

      if (!matchingMessage.length) {
        return null;
      }

      if (matchingMessage.attachments && matchingMessage.attachments.length > 0) {
        for (const attachment of matchingMessage.attachments) {
          attachment.url = await this.fetchAndEncodeImage(attachment.url);
        }
      }

      return matchingMessage;

    } catch (error) {
     // do something 
    }
  }

  async fetchAndEncodeImage(url: string): Promise<string> {
    const response: AxiosResponse<any> = await this.httpService.get(url, {
      responseType: 'arraybuffer',
    }).toPromise();

    const base64 = Buffer.from(response.data, 'binary').toString('base64');
    return `data:${response.headers['content-type']};base64,${base64}`;
  }

https://discord.com/api/v9/channels/${CHANNEL_ID}/messages?limit=50 endpoint is being used to fetch our Discord channel and get the response in order to retrieve our images.

Since Midjourney generation takes about 60 seconds or more, we will need to poll this channel every x seconds to check for results.

Let's give it a try with { prompt: "a cat" } :

That's it! You should now have a fully working Midjourney API for testing and fun and you've learned how Discord bot architecture works.

Final thoughts

You now have a bootstrap project that demonstrates how Discord communicates with MidJourney to generate the most amazing AI images.
You can build a nice UI and have your own generative AI platform. Good luck!