Bradston Henry for Developers @ Asurion

Posted on Oct 3, 2023 • Edited on Feb 25

Using React to Store Azure OpenAI DALL-E images using Base64

Author's Note: If you are interested in jumping right into the code, please skip down to Getting our DALL-E images. The paragraphs prior to that section are intended to give context on how and why I chose this particular conversion/storage method.

In Mid-September 2023, I helped to host and participated in a 48-hour Generative AI hackathon at my company, Asurion. The Hackathon was called the Code Jam: AI Hackathon and while hacking with my team, I stumbled into a slightly novel way to store DALL-E images, using React, for later use.

The Magic of Base64

If you have worked in software development for some amount of time you may have encountered the use base64 to convert JPEG and/or PNG images. Base64 is a data format that allows you to store certain types information in, essentially, a plain text format.

For our Hackathon project, we were in need of using DALL-E generated images but we realized that continuously generating new DALL-E images for our application was not sustainable (or cost effective) for the long term. And because image reuse wasn't an issue within our project (and we stored quite a bit of our application data in JSON-based database), base64 was the logical approach for image storage.

Though Base64 text strings for images can be pretty long, storing our images as text was preferable to using an image storage service to preserve images in their original format.

Now instead of needing to store an image like this in its original PNG format:

We could now store images as text like this:

data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAAgAAAAIA....

With that decision made, I began, what seemed to me, the trivial task of converting DALL-E images to Base64.

IMPORTANT NOTE:

This blog is best suited for those with some knowledge of React and Javascript and with a basic understanding of APIs and network calls. I've tried my best to explain my process as much as possible but I do not explain everything about the code that I've shared.

The Azure OpenAI Caveat

So one thing that made me excited about pursing the base64 conversion approach, was that I was aware that OpenAI's DALL-E API offered an option to request DALL-E images in base64 format instead of image format. That essentially meant that OpenAI would have done most of my work for me, and all I had to do was configure our application to save those base64 text strings into our database.

Well, sadly I was not so lucky..

As some of you may be aware, Microsoft recently created an enterprise ready version of many of OpenAI's service offerings after some corporations ran into trouble when their proprietary information inadvertently was exposed using its tools. This version of OpenAI, Azure OpenAI, has been touted as a "closed system" that aims to protect corporations from the accidental leaking of crucial intellectual property feed into their systems.

Asurion, my employer, has been very interested in leveraging Generative AI technology and has had the opportunity to be one of the first companies working with the Azure OpenAI environment.

The only caveat is that Azure OpenAI's API varies in slight ways from the public version on OpenAI.

One of those ways it varies is the DALL-E API's lack of a base64 option.

Note: If you are interested in learning about the ImageGenerationOptions offered by Azure OpenAI's DALL-E API, you can check it out here

So that left me with no choice but to do the conversion manually within our React application.

Keeping the Conversion Native

When I started this task, one key thing that I wanted was to keep the conversion process "simple" within our code base.

If at all possible, I wanted to avoid using any React or Javascript frameworks that were not default libraries within a standard React project.

There are many reasons for why that was important to me, but the main reason was that I wanted to keep the code lightweight and easily transferable to other projects.

With that in mind, I went on the hunt for native Javascript and React library/framework options.

Getting our DALL-E images

Once I knew my desired approach to the problem, I first wanted to make sure I knew what type of data I'd be getting from the OpenAI DALL-E API so I could be aware of how I would need to approach this conversion process.

Because this blog is not about how to use the Azure OpenAI DALL-E, I am going to skip some steps here and share an example of an output you'd receive when requesting an image via the Azure OpenAI API:

https://dalleproduse.blob.core.windows.net/private/images/8d5392fb-34a9-43aa-93f8-d05da65b5b29/generated_00.png?se=2023-09-30T03%3A49%3A45Z&sig=lPn2mhiURJxgxM04wFj46U0AB%2B%2BM5X%2FV0mEVSyhb0zU%3D&ske=2023-10-05T05%3A34%3A21Z&skoid=09ba021e-c417-441c-b203-c81e5dcd7b7f&sks=b&skt=2023-09-28T05%3A34%3A21Z&sktid=33e01921-4d64-4f8c-a055-5bdaffd5e33d

Looking at the above URL you might notice a few things.

Though it does have a somewhat standard URL structure, you will notice that after the ...b29/generated_00.png portion of the URL, that there is an interesting URL parameter.

Note: From my understanding, this parameter denotes the lifespan of the URL. DALL-E images are only available for a limited time after generation.

The reason I point out this URL parameter is that if you were to think to yourself that you would be able to access a generated image from the URL without the URL parameter (aka removing everything after ...b29/generated_00.png in the URL) you would be wrong.

It is crucial when accessing the images generated by DALL-E to use the entire URL as a shortened URL will not work.

For whatever reason, OpenAI has created a URL structure that only allows access to their generated images if you use the entire provided by the API.

Locally saving your image for conversion

Once I knew how images were presented and had retrieved the corresponding URL from the Azure OpenAI DALL-E endpoint, I needed to store any images I retrieved locally so I could encode them into Base64.

The easiest way to do this in React was to use the Fetch API which is one of the most basic protocols for fetching resources from network resources.

Within a async function, it has a pretty straightforward syntax:

export async function getDatafromURL(url:string) {
    //Fetch data from url 
    const data = await fetch(url);
}

As the code reads within the async function, you pass the desired url with the resource you'd like to retrieve into the fetch function, using await to return the data from the url.

Because of the nature of base64 conversion in React, the easiest way to convert image data from a URL into base64 is to save that image as a blob.

This is actually very simple process when using the standard javascript library within React.

Here what that would look like:

export async function getDatafromURL(url:string) {
    //Fetch data from url 
    const data = await fetch(url);
   //Create blob of data
    const dataBlob = await data.blob();
}

Once I had my data from the retrieved URL, all I needed do was use the .blob() method of the data object to quickly convert it into a blob that could be used later.

So let me get more specific to the actual DALL-E resource and what that would all look like .

Here is the same functionality described above while using an example DALL-E URL:

const dalleImageURL= 'https://dalleproduse.blob.core.windows.net/private/images/8d5392fb-34a9-43aa-93f8-d05da65b5b29/generated_00.png?se=2023-09-30T03%3A49%3A45Z&sig=lPn2mhiURJxgxM04wFj46U0AB%2B%2BM5X%2FV0mEVSyhb0zU%3D&ske=2023-10-05T05%3A34%3A21Z&skoid=09ba021e-c417-441c-b203-c81e5dcd7b7f&sks=b&skt=2023-09-28T05%3A34%3A21Z&sktid=33e01921-4d64-4f8c-a055-5bdaffd5e33d';

 async function getImageBlobFromURL(url:string) {
    //Fetch image data from url 
    const imageData = await fetch(url);
    //Create blob of image data
    const imageBlob = await imageData.blob();
    return imageBlob
};

const dalleImageBlob = await getImageBlobFromURL(dalleImageURL);

Now that I had locally saved the DALL-E image URL data as a blob, it was time to make the base64 string of from the DALL-E image data.

Converting our image blob data into base64

Now came the most important step; converting the image blob that I had created from the DALL-E image URL into Base64.

To do this, I used the FileReader library within a async function using the FileReader's readAsDataURL method to transform the blob into a base64 string.

Here is the async function I used:

//Use File Reader to decode Blob as a Base64 string
async function blobToBase64(blob: Blob) {
    return new Promise((resolve, _) => {
      const reader = new FileReader();
      reader.onloadend = () => resolve(reader.result);
      reader.readAsDataURL(blob);
    });
}

Using FileReader within a Promise closure, I created a reader object that used the .onloadend listener (which fires when a file read event has completed) to return the base64 string after the reader.readAsDataURL(blob) completed converting the blob object.

Note: If you are not familiar with Promise syntax or FileReader object functionality, I encourage you to look into the documentation to see how they work (links in this note)

If I were to use this method using this syntax with a set image blob, imageBlob,

const dalleBase64String = await blobToBase64(imageBlob);

and were to console.log(dalleBase64String) I would get something that would look like this:

data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAAQAAAA....

The above is the exact base64 text data I had been working to get.

Note that the base64 output has a three parts:

The data: tag at the beginning.
The designator marking what type of data it is: image/png;base64,
and the actual base64 data of the image which is represented by the extremely long string of characters at the end.

Here is the full code that could be used to convert a DALL-E URL to base64 data:

const dalleImageURL= 'https://dalleproduse.blob.core.windows.net/private/images/8d5392fb-34a9-43aa-93f8-d05da65b5b29/generated_00.png?se=2023-09-30T03%3A49%3A45Z&sig=lPn2mhiURJxgxM04wFj46U0AB%2B%2BM5X%2FV0mEVSyhb0zU%3D&ske=2023-10-05T05%3A34%3A21Z&skoid=09ba021e-c417-441c-b203-c81e5dcd7b7f&sks=b&skt=2023-09-28T05%3A34%3A21Z&sktid=33e01921-4d64-4f8c-a055-5bdaffd5e33d';

async function getImageBlobFromURL(url:string) {
    //Fetch image data from url 
    const imageData = await fetch(url);
    //Create blob of image data
    const imageBlob = await imageData.blob();
    return imageBlob
};

const dalleImageBlob = await getImageBlobFromURL(dalleImageURL);

//Use File Reader to decode Blob as a Base64 string
async function blobToBase64(blob: Blob) {
    return new Promise((resolve, _) => {
      const reader = new FileReader();
      reader.onloadend = () => resolve(reader.result);
      reader.readAsDataURL(blob);
    });
}

const dalleBase64String = await blobToBase64(dalleImageBlob);

Be wary of 'no-cors' mode with fetch

Something I wanted to note, because it took me down a non-productive rabbit hole, is to be careful when using the 'no-cors' mode with Fetch API calls.

When I first began this process of building the functionality of converting DALL-E to base64 my original getImageBlobFromURL function looked like this:

async function getImageBlobFromURL(url:string) {
    //Fetch image data from url 
    const imageData = await fetch(url, {mode: 'no-cors'});
    //Create blob of image data
    const imageBlob = await imageData.blob();
    return imageBlob
};

I originally took this approach of adding the optional parameter of {mode: 'no-cors'} because I was concerned about Cross-Origin issues that arise when downloading image data using fetch using the browser.

I tested the above method on other image resources hosted at other URLs before trying with DALL-E and, at the time, the function seemed to work fine. But when I attempted to implement it with the DALL-E generated URLs I ran into issues.

The no-cors parameter was problematic when trying to trying to download the image data from the DALL-E image URL.

Because of how the DALL-E images are generated at their designated URL (wrapped in HTML by OpenAI), when using the no-cors parameter my base64 data that I received through the blob conversion process could not be converted into an image.

When I used the no-cors parameter, I would get something like this:

data: text/html; base64,V2VsY29tZSB0byA8...

This data type, text/html, can not be directly converted back into an image but holds html page data, which is a bit more complicated to deal with.

If you are interested in the nature of base64 conversion and the nuances of how conversion works, then the no-cors issue might not bother you. But if you are looking for the conversion to remain relatively simple, I would be wary of that no-cors parameter.

Conclusion

So just like that, I easily took a Azure OpenAI DALL-E generated image, hosted at a OpenAI URL, and converted it into base64 string for storage.

In this blog, I did not cover how you could store these base64 strings for retrieval at a later date in some external filing system, but now you have the tools you need to take on that challenge yourself.

In all honesty, this task should have been a relatively straightforward process but the minor nuances of how OpenAI manages image generation made this task a bit more complex.

I hope this helps any developer who is using Azure OpenAI DALL-E services with the goal of storing DALL-E images in base64 format.

Thank you for reading this blog and I wish you all the best!

Bradston Henry

Cover Photo generated by DALL-E

Cover Photo
Image Prompt: A person taking a headshot with face visible, with the right half of their face being photorealistic and the left half of their face being drawn in matrix style

Doggo Photo
Image Prompt: A happy doggo in ukiyo-e style