Deploy ComfyUI with RunPod Serverless

What is ComfyUI? ComfyUI, according to author, "lets you design and execute advanced stable diffusion pipelines using a graph/nodes/flowchart based interface". Basically, we can play with stable diffusion pipelines by just moving around some nodes.

So what is stable diffusion? Its a GenAI (Generative Artificial Intelligence) which can generate images from text prompts, images or both.

Before moving on, let's understand what is Runpod. Execution of an AI model need to take place in GPU (Graphic Processing Unit) as it allows millions of tiny processing unit, working at the same time. And GPUs are costly. So rather than buying GPU and running the models locally, we can rent out the GPU service from services like RunPod. RunPod provides us the cloud service for developing and executing AI models.

RunPod allows us 2 types of services, Pods and Serverless. Pods are the services that keep on, no matter whether the required service / API is being used or not. Where as in serverless provide workers, which only works when needed, and hence charged less (only for the time worker is).

Now let's dive into deploying ComfyUI with RunPod.

Runpod Worker Comfy

We are going to use runpod-worker-comfy for the deployment.
Although the git repo has detailed explanation for Comfy deployment, I had to face some issues which may stuck you if you doing this for the first time.

We do not need to clone this repo, rather we are going to use the docker images provided by runpod worker comfy.

Runpod worker comfy provides us 5 images as follow:

• timpietruskyblibla/runpod-worker-comfy:3.1.0-base: doesn't contain anything, just a clean ComfyUI
• timpietruskyblibla/runpod-worker-comfy:3.1.0-flux1-schnell: contains the checkpoint, text encoders and VAE for FLUX.1 schnell
• timpietruskyblibla/runpod-worker-comfy:3.1.0-flux1-dev: contains the checkpoint, text encoders and VAE for FLUX.1 dev
• timpietruskyblibla/runpod-worker-comfy:3.1.0-sdxl: contains the checkpoint and VAE for Stable Diffusion XL
• timpietruskyblibla/runpod-worker-comfy:3.1.0-sd3: contains the checkpoint for Stable Diffusion 3 medium

Dockerfile

Based on your requirement, select one of the image.
We can use the selected image as raw, or we can add some customised layers, like adding packages or custom nodes (custom nodes can be added in the network volume, discussed later). In my case, I had to use a package rembg[gpu], need to remove the background from a given image. So here is my updated Dockerfile:

FROM timpietruskyblibla/runpod-worker-comfy:3.1.0-sd3
RUN pip install rembg[gpu]
CMD /start.sh

For adding custom nodes, you can do:

...
RUN git clone <custom_node_repo.git> /comfyui/custom_nodes
...

Now build an image by docker build <dockerhub-username>/<your-image-name>. Once done, push the image to docker hub by docker image push <dockerhub-username>/<your-image-name>. You can use Docker GUI to push your image to docker hub.

You may skip the above steps if you do not need any customisation.

Runpod

Now we need to configure our runpod.

Storage

Let's make a Network Volume, which will have our output images, custom nodes (if we haven't added those using Dockerfile) and custom models.

Go to Storage in sidebar. Click on New Network Volume, select the data center, volume name and the storage volume it holds.

You can select any data center, usually the one which supports the cheapest (and medium - highly available) GPU, as GPU would only be required to deploy the instance. Won't affect the image processing.

For storage volume, it depends upon your requirements. How much space is required by custom models and custom nodes. If you are not using any custom models/nodes, I think 10GB is fine.

Once the network volume is created, we need to deploy it temporarily to make it accessible so we can add the custom things.

Select any of the GPU, select GPU count as 1, and select instance pricing as Spot

Once network volume is deployed, go to Pods in side bar and you will a see a newly listed Pod. Click on it and connect the terminal through web.

Now you will see the terminal interface in web. We need to place everything in workspace directory because its the directory of volume and contains persistent data. Anything not in this directory will be removed as soon as we delete the deployed instance of Network Volume.
So go to /workspace by cd workspace and make respective directories i.e for custom nodes, make custom_nodes by command mkdir custom_nodes. Move into custom_nodes by cd custom_nodes and clone your desired nodes from git by git clone <repo url>.
Once you are done with adding custom nodes, exit the terminal by exit and close your web terminal tab.

Serverless Deployment of Comfy

Now head over to Serverless in sidebar and click on New Endpoint. The below screen will shown

We have to provide our configurations.

1. Select GPU
2. Enter you Endpoint Name, this could be anything
3. For Worker Configuration section, you should go through these docs
4. For container configuration, leave Template and Container Registry Credentials as empty. Enter the container image as the image you chose from the runpod-worker-comfy timpietruskyblibla/runpod-worker-comfy:3.1.0-<your model> like timpietruskyblibla/runpod-worker-comfy:3.1.0-sd3 For Container Start Command, write: sh -c "ln -sf /runpod-volume/custom_nodes/* /comfyui/custom_nodes && /start.sh". This command makes a symlink between comfyui directory and network volume workspace. You may also link other directories too. Note: You can skip the above container start command if you haven't installed any custom nodes, models or any other custom stuff. If so, leave this field empty.
5. For Container Disk, it should be more than 20GB.
6. Leave HTTP and TCP ports empty
7. For Environment Variables, enter these:

BUCKET_ENDPOINT_URL --- for endpoint of s3 i.e https://bucket.s3.region.amazonaws.com

BUCKET_ACCESS_KEY_ID --- for access key of s3
BUCKET_SECRET_ACCESS_KEY --- for secret key of s3

1. Keep the advance sections as itself.
2. Finally deploy your serverless instance.

Creating API Key

Now to access your deployed instance, you would require an API key. Which you can get by:

1. Go to Settings in sidebar
2. Create new API Key, enter any desired name, select its access i.e only read or read & write.
3. Copy the generated API Key and store it somewhere safe.

Using comfy

Let's test if our comfy works as expected. But before moving on, we need the API endpoint where we are going to request.
Go to Serverless in sidebar and click on the instance you just deployed. It will be like this:

You can access the logs in case of debugging. For now, we need API endpoint which you will see at the bottom of image. We got 5 different routes. You may read the guide here.

You can either go with run or runsync to get result by accessing status or getting result immediately, respectively. In most cases, comfy takes some time so its better to go with run and check the status after some interval.

API Request

You may read the docs of API Request Format here
Once workflow is created you can request the above mentioned API Endpoint with API Key in header by Bearer <API Key>

API Response for /run

It will return id of the job, at the root of response body. You will use this id to check the status of of job by /status/

API Response for /status

It will return the status of the job, at the root of response body, along with output if status is COMPLETED

All Done, now you can make another layer of API which will may only take desired inputs and return the output of comfy by checking the status at some intervals.

That's all. Happy learning❤️