Generate text-to-image right in your Frontend.

Maxim — Wed, 06 Mar 2024 06:30:14 +0000

Intro

As a frontend developer, when I want to play with some AI trends they usually require some additional setup, like backend/ml/etc and I just give up and find some service. But what I actually want is to play with some AI tool right now in my frontend application, for example create some react app locally and use API. Fortunately enough some AI tools have API which can be accessed right from the browser and today I want to demonstrate how to work with Kandkinsky API.

Let's code!

First of all you need to get an auth key. For it you can just sign up on the https://fusionbrain.ai and generate a new key/secret pair.

Once you have keys, you can use it to generate images. First of all let's add a class to work with API. Before you do it, you should install axios library.

The following code uses typescript, but if you have plain JS you can just remove typescript parts.

import axios from 'axios';

type GenerationRequestAPIResponse = {
  data: {
    status: string,
    uuid: string,
  }
};

class Kandinsky {
  private apiKey;

  private apiSecret;

  private baseUrl = 'https://api-key.fusionbrain.ai/key/api/v1';

  public modelId: string | null;

  constructor({ apiKey, apiSecret }: { apiKey?: string, apiSecret?: string }) {
    if (!apiKey || !apiSecret) {
      throw new Error('Kandinsky must be initialised with api key and secret');
    }
    this.apiKey = apiKey;
    this.apiSecret = apiSecret;
    this.modelId = null;
  }

  async init() {
    this.modelId = await this.getModelId();
  }

  generateAuthHeaders() {
    const headers = new Headers();
    headers.append('X-Key', `Key ${this.apiKey}`);
    headers.append('X-Secret', `Secret ${this.apiSecret}`);
    return headers;
  }

  async getModelId() {
    if (this.modelId) {
      return this.modelId;
    }
    const headers = this.generateAuthHeaders();

    const requestOptions = {
      method: 'GET',
      headers,
    };

    const response = await fetch(`${this.baseUrl}/models`, requestOptions).then((res) => res.json());
    return response[0].id;
  }

  async requestImageGeneration(prompt: string): Promise<GenerationRequestAPIResponse> {
    // here we're using axios lib because it doesn't work with native fetch API
    // native fetch API has issues with constructing blobs for multipart/form-data
    const modelId = await this.getModelId();
    const headers = Object.fromEntries(this.generateAuthHeaders());

    const params = {
      type: 'GENERATE',
      width: 256,
      style: 'KANDINSKY',
      height: 256,
      generateParams: {
        query: prompt,
      },
    };
    const paramsAsBlob = new Blob([JSON.stringify(params)], { type: 'application/json' });
    const formdata = new FormData();
    formdata.append('model_id', modelId);
    formdata.append('params', paramsAsBlob);

    return axios.post(`${this.baseUrl}/text2image/run`, formdata, {
      headers: {
        ...headers,
        'Content-Type': 'multipart/form-data',
      },
    });
  }

  async getImageById(id: string): Promise<string> {
    const headers = this.generateAuthHeaders();
    const requestOptions = {
      method: 'GET',
      headers,
    };
    const response = await fetch(`${this.baseUrl}/text2image/status/${id}`, requestOptions).then((res) => res.json());
    if (response.status === 'DONE') {
      return Promise.resolve(`data:image/png;base64, ${response.images[0]}`);
    }
    await new Promise((resolve) => {
      setTimeout(resolve, 3000);
    });
    return this.getImageById(id);
  }

  async generateImage(prompt: string): Promise<string> {
    const imageMetaData = await this.requestImageGeneration(prompt);
    const imageId = imageMetaData.data.uuid;
    return this.getImageById(imageId);
  }
}

export default Kandinsky;

Since API requires auth keys, I don't recommend publishing it somewhere without backend proxy, but if you just want to play around locally, feel free to use it like I describe and add a proxy layer later.

const imageApi = new Kandinsky({
  apiKey: '<api key>',
  apiSecret: '<api secret>',
});

imageApi.init();
const image = await imageApi.generateImage(text); // base64 image

Then once you have base64 string returned from generateImage method, you can insert to the image tag like this:

<img src={generatedImageString} />

And it should render the image correctly

What is next?

You can read more about API params in the docs, and just change params variable to play around with different styles, sizes and other options. Build your own interesting app based on images generation!

DEV Community: Maxim

Generate text-to-image right in your Frontend.

Intro

Let's code!

What is next?