DEV Community: Netanel Ben

Creating a Custom Node for ComfyUI Using React + TS

Netanel Ben — Wed, 09 Oct 2024 15:01:23 +0000

Introduction & Code

Hello dev.to community, In this article, we’ll walk through the development of a custom node for ComfyUI built with React + TS that utilizes Stable Diffusion model to generate images from text prompts. This tool helps developers and artists produce creative visuals with a simple UI. The implementation showcases WebSocket communication, workflow.json integration, and real-time image generation feedback—all key aspects for a powerful and intuitive user experience.

Source code

You can jump right in and clone this repo into custom nodes folder and run React build from /web folder

Prerequisites

ComfyUI Installation

You need to have ComfyUI set up. ComfyUI is a powerful and modular UI that allows you to create workflows and pipelines for image generation using Stable Diffusion. You can follow the installation instructions on their GitHub page to get it up and running.

The model im using for this example is DreamShaper 8

ComfyUI Custom Nodes

Custom nodes are user-defined nodes that extend the functionality of ComfyUI. They allow developers to create new types of operations or integrate different models and algorithms to achieve specific goals that aren’t possible with the built-in nodes. Custom nodes are highly useful for:
• Extending Capabilities: Integrate new models, image filters, or custom transformations.
• Personalized Workflows: Customize workflows to manipulate prompts or generate unique image variations.
• Enhancing User Experience: Add nodes for user control, feedback, or dynamic parameter adjustments.

Running the React App Server with ComfyUI API

Upon startup when running python main.py from ComfyUI folder, Comfy scans the custom_nodes directory for Python modules and attempts to load them. Modules that export NODE_CLASS_MAPPINGS are identified as custom nodes.

The __init__.py Python module integrates a React frontend with the ComfyUI backend. It sets up routes to serve the React application, allowing users to interact with the custom nodes via a web interface.

• Web Root Setup: Defines the directory for serving the built React assets (web/dist)

WEBROOT = os.path.join(os.path.dirname(os.path.realpath(__file__)), "web/dist")

• Routes: Adds server routes to serve the React app (/Text2Image) and static assets (/assets).

@server.PromptServer.instance.routes.get("/Text2Image")
def init(request):
    return web.FileResponse(os.path.join(WEBROOT, "index.html"))

• Node Mappings: Exposes custom node class mappings for use in the workflow.

React client side code walkthrough

set up WebSocket connection with ComfyUI backend
listen to specific incoming messages
handle workflow api json file
generate image based on a text prompt

ComfyUI WebSocket connection

• hostname and protocol: Constructs the server address using the current window’s hostname and port. Chooses wss: for secure connections (https:) and ws: for others.
• wsClient: Initializes a WebSocket connection to /ws with a unique client ID.
• wsClient.onopen: Logs a message once the connection to the server is successfully established.
• must provide a unique ID via clientId param, i'm using uuidv4 for it.

const hostname = window.location.hostname + ":" + window.location.port;
    const protocol = window.location.protocol === "https:" ? "wss:" : "ws:";
    const wsClient = new WebSocket(
      `${protocol}//${hostname}/ws?clientId=${clientUniqueId}`
    );

Listen to a response from ComfyUI

wsClient.addEventListener("message", (event) => {
if (data.type === "executed") {
        if ("images" in data.data.output) {
          const image = data.data.output.images[0];
          const { filename, type, subfolder } = image;
          const rando = Math.floor(Math.random() * 1000);
          const imageSrc = `/view?filename=${filename}&type=${type}&subfolder=${subfolder}&rand=${rando}`;
        }
      }
});

Modifying ComfyUI workflow api json

The workflow api json is the set of nodes configuration for ComfyUI, in the following code I have searched via the JSON file for the relevant node ID (since node IDs are randomly generate when adding new nodes)

In this example the seed is randomly generated and the prompt text is applied too;

// Find the key of KSampler node
    const samplerNodeNumber = Object.entries(workflow).find(
      ([key, value]) => value.class_type === "KSampler"
    )[0] as keyof typeof workflow;

workflow[samplerNodeNumber].inputs.seed = Math.floor(
      Math.random() * 9999999999
    );

workflow[inputNodeNumber].inputs.text = prompt.replaceAll(
      /\r\n|\n|\r/gm,
      " "
    );

Generate an image based on prompt

ComfyUI is exposing api routes such as /prompt and /interrupt to stop the process

async function queuePrompt(workflow = {}) {
    const data = { prompt: workflow, client_id: clientUniqueId };

    const response = await fetch("/prompt", {
      method: "POST",
      cache: "no-cache",
      headers: {
        "Content-Type": "application/json",
      },
      body: JSON.stringify(data),
    });

    return await response.json();
  }

Check out the full source code for more tweaks such as progress tracking for the queue task.

Source code: https://github.com/netanelben/comfyui-text2image-customnode

Creating a Smooth Animated Menu with React and Framer Motion

Netanel Ben — Mon, 16 Sep 2024 07:54:58 +0000

Creating a Smooth Animated Menu with React and Framer Motion

Introduction

Building an intuitive and engaging user experience often involves adding motion to UI elements, making the interface feel more dynamic. In this post, I’ll guide you through how I built a menu with smooth, staggered animations using React and Framer Motion. By leveraging these technologies, I achieved a polished interaction flow, enhancing the menu’s functionality and visual appeal.

Here’s how I approached the task.

Tools and Libraries

For this project, I used:

React: To structure and manage the UI components using custom hooks paradigm.
Framer Motion: A powerful library for animations in React, enabling declarative and complex animations with ease.
TypeScript: For adding type safety to the components.

Setting up the Component

The main component, AnimatedMenuButtons is responsible for rendering the menu items and handling their animation states. Here’s a breakdown of the key parts of the code:

1. Component Props

The component takes in three main props:

menuItems: An array of menu items.
MenuButton: A custom React component for rendering the menu buttons.

Code snippet:

interface Props {
  menuItems: any[];
  MenuButton: React.FC<any>;
}

2. Animation Setup

Using Framer Motion’s motion and stagger features, I set up a staggered animation to display the menu items sequentially, creating a smooth transition. The staggerMenuItems function delays the appearance of each list item by 0.24 seconds.

Code snippet:

const staggerMenuItems = stagger(0.24, { startDelay: 0.05 });

This creates a clean, staggered animation where each button slides into view after a short delay and disappear the same way upon click.

Adding Animations with Framer Motion

3. Handling Animations

I used the useAnimate hook from Framer Motion to control the animation. The animate function handles the transitions for the menu items. Initially, each menu item (li) starts with zero opacity and is translated 200px to the right, which then animates to full opacity and its original position.

animate(
  "li",
  { opacity: 1, translateX: 0 },
  {
    duration: 0.8,
    delay: staggerMenuItems,
  }
);

4. Click Animation

When a menu item is clicked, the clickHandler triggers an animation where the item fades out and slides back to the right. This animation runs before triggering the onClick event provided by the menu item.

const clickHandler = (onClick: () => void) => {
  animate(
    "li",
    { opacity: 0, translateX: 200 },
    {
      duration: 0.8,
      delay: staggerMenuItems,
      onComplete: () => {
        // Important note: mind here that this callback will invoke for each of the <li/> items and if this is not intended, use the then chain callback to run only once.
      },
    }
  ).then( onCompleteFunc );
};

This approach ensures the button clicks are animated smoothly, providing a more immersive experience for the user.

Rendering the Menu Items

Each menu item is rendered inside a motion.li component, where the initial animation state is defined. The onClick event is passed down to trigger the clickHandler, which runs the animation before executing the item's click logic.

Code snippet:

<motion.li
  key={index}
  initial={{
    opacity: 0,
    translateX: 200,
  }}
  onClick={() => clickHandler(item.onClick)}
>
  <MenuButton {...item} />
</motion.li>

Conclusion

Using Framer Motion in React made it straightforward to build an interactive menu with smooth animations. The ability to easily control animation sequences, such as staggered entrances and exits, adds a layer of sophistication to the UI.

If you’re looking to enhance your next project with modern animations, I highly recommend giving Framer Motion a try. It’s incredibly versatile, and with just a few lines of code, you can transform a static interface into something dynamic and engaging.

The full code can be found here:
https://codesandbox.io/p/sandbox/animated-menu-65zz88

Stay tuned for more animation tips!

Let me know in the comments how it worked for you!