DEV Community: Ytalo

Iterating Over a Visual Editor Compiler

Ytalo — Wed, 10 Jul 2024 20:15:25 +0000

The Visual Editor

When it comes to creating visual editors for workflows, React Flow stands out as an ideal choice. It offers robust performance, is highly customizable, and facilitates document export. With it, you can build everything from chatbots to complex backends.

Compiling the Visual

Directly exporting the visual format is not efficient for execution. Therefore, we convert this structure into something more executable. A basic example of the structure would be:

[{prev: null, data: {}, id: id, next: <some id>}]

This makes it easy to navigate and filter the objects.

Data Maintenance

To reverse the "compilation" and maintain node metadata, we use:

const step: any = {
    data: node.data,
    height: node.height,
    id: node.id,
    position: node.position,
    positionAbsolute: node.positionAbsolute,
    selected: node.selected,
    type: node.type,
    width: node.width,
    prev: another step,
    next: another step
};

[Format Conversion

Below is the function that converts the React Flow format to a custom format:

function convertToCustomFormat(reactFlowObject: any) {
    const customFormatObject: any = {
        steps: [],
        viewport: reactFlowObject.flow.viewport,
    };

    reactFlowObject.flow.nodes.forEach((node: any) => {
        const step: any = {
            data: node.data,
            height: node.height,
            id: node.id,
            position: node.position,
            positionAbsolute: node.positionAbsolute,
            selected: node.selected,
            type: node.type,
            width: node.width,
        };

        if (node.data.nodeType === "conditionNode" || node.data.nodeType === "pixNode") {
            const trueEdge = reactFlowObject.flow.edges.find(
                (edge: any) => edge.source === node.id && edge.sourceHandle == "a"
            );
            const falseEdge = reactFlowObject.flow.edges.find(
                (edge: any) => edge.source === node.id && edge.sourceHandle == "b"
            );

            step.true = trueEdge ? trueEdge : null;
            step.false = falseEdge ? falseEdge : null;
            step.prev = reactFlowObject.flow.edges.find(
                (edge: any) => edge.target === node.id
            );
        } else {
            step.prev = reactFlowObject.flow.edges.find(
                (edge: any) => edge.target === node.id
            );
            step.next = reactFlowObject.flow.edges.find(
                (edge: any) => edge.source === node.id
            );
        }

        customFormatObject.steps.push(step);
    });

    return customFormatObject;
}

Reconversion to React Flow

To reconvert the custom format back to React Flow, we use the following function:

function convertToReactFlowFormat(customFormatObject: any) {
    const reactFlowObject: any = {
        flow: {
            nodes: [],
            edges: [],
            viewport: customFormatObject.viewport,
        },
    };

    customFormatObject.steps.forEach((step: any) => {
        const node = {
            data: step.data,
            height: step.height,
            id: step.id,
            position: step.position,
            positionAbsolute: step.positionAbsolute,
            selected: step.selected,
            type: step.type,
            width: step.width,
        };

        reactFlowObject.flow.nodes.push(node);

        if (step.type === "nodeContainer") {
            if (step.prev) {
                reactFlowObject.flow.edges.push({
                    ...step.prev,
                    target: step.id,
                });
            }

            if (step.next) {
                reactFlowObject.flow.edges.push({
                    ...step.next,
                    source: step.id,
                });
            }
        }
    });

    return reactFlowObject;
}

Execution

To execute from the metadata, especially when user inputs are needed, we can use variables that change continuously and can be stored in memory or in a database like Redis. We keep the step ID, the number of steps passed, and metadata for each previous step or input.

This allows us to pause and resume execution easily, maintaining the flow logic intact. In cases where there are multiple outputs, we adapt the function to handle specific actions:

const step: any = {
    data: node.data,

height: node.height,
    id: node.id,
    position: node.position,
    positionAbsolute: node.positionAbsolute,
    selected: node.selected,
    type: node.type,
    width: node.width,
    prev: step,
    outputs: {
        0: step,
        1: step
    }
};

This allows for even more detailed and specific management of the constructed flow.

Introducing the Visitor Pattern

For those looking for a more structured and scalable way to iterate over the elements of a visual editor, the Visitor Pattern can be an intriguing solution. The Visitor Pattern allows you to separate algorithms from the objects on which they operate, making it easier to add new operations without modifying the existing elements.

What is the Visitor Pattern?

The Visitor Pattern involves creating a visitor interface that declares a set of visit methods for each type of element. Each element class implements an accept method that accepts a visitor, allowing the visitor to perform the required operation.

Benefits

Simplified Maintenance: Adding new operations only requires creating a new visitor without needing to modify existing elements.
Separation of Responsibilities: Operations are clearly separated from the elements, making the code more modular.
Scalability: Facilitates the addition of new types of elements and operations in an orderly and hassle-free manner.

Basic Example

Here is a basic example of how the Visitor Pattern can be implemented:

class Visitor {
  visitText(element) {}
  visitImage(element) {}
  visitShape(element) {}
}

class ConcreteVisitor extends Visitor {
  visitText(element) {
    console.log('Processing text element');
  }
  visitImage(element) {
    console.log('Processing image element');
  }
  visitShape(element) {
    console.log('Processing shape element');
  }
}

class Element {
  accept(visitor) {}
}

class TextElement extends Element {
  accept(visitor) {
    visitor.visitText(this);
  }
}

class ImageElement extends Element {
  accept(visitor) {
    visitor.visitImage(this);
  }
}

class ShapeElement extends Element {
  accept(visitor) {
    visitor.visitShape(this);
  }
}

function processElements(elements, visitor) {
  for (let element of elements) {
    element.accept(visitor);
  }
}

const elements = [new TextElement(), new ImageElement(), new ShapeElement()];
const visitor = new ConcreteVisitor();
processElements(elements, visitor);

Considering the use of the Visitor Pattern can help keep your code more organized and adaptable to future changes. If you are iterating over a complex set of elements and need to apply multiple operations, this pattern can be an excellent choice.

This article was built based on the BatAnBot project and the recent technical challenge by Vom.

How I used js to dynamically create and delete pods in kubernetes

Ytalo — Sat, 06 Jul 2024 11:47:17 +0000

First of all: Why?

Imagine you have a system where your user needs to execute something private on your server quickly and securely, like deploying bots or what Next.js does. This article is perfect for such cases.

But, how do I do it on localhost?

To develop this code, we will use two approaches: production and development. We will use Minikube for the development environment and Azure for the production environment.

How can we start?

We need to define a development path to avoid getting lost. We will need a way to communicate with Kubernetes, as well as to create and delete pods.

Development

We will use the libraries @kubernetes/client-node and @azure.

First

We start by creating the Kubernetes API. To do this, we define a function getKubernetesApi, and in it, we initiate the Kubernetes configuration, something like:

import { ContainerServiceClient } from "@azure/arm-containerservice";
import { DefaultAzureCredential } from "@azure/identity";
import k8s from "kubernetes-client";

async function getKubernetesClient() {
    const kc = new k8s.KubeConfig();
}

Then we define a try catch block, where we test if the environment is production to configure Kubernetes with Azure, or load the default configuration in the development environment. We create and return the client, something like:

async function getKubernetesClient() {
    const kc = new k8s.KubeConfig();
    try {
        if (process.env.NODE_ENV === "production") {
            const credentials = new DefaultAzureCredential();
            const containerServiceClient = new ContainerServiceClient(credentials);
            const response = await containerServiceClient.managedClusters.listClusterAdminCredentials();
            kc.loadFromString(response.kubeconfigs[0].value);
        } else {
            kc.loadFromDefault();
        }
        const k8sApi = kc.makeApiClient(k8s.CoreV1Api);
        return k8sApi;
    } catch (error) {
        console.error("Error while getting Kubernetes API", error);
        return;
    }
}

Second

Next, we define the functions to create and delete pods using the obtained client. For this, we receive data such as the pod name, pod namespace, and metadata for the pod, something like:

async function createPod(metadata, namespace) {
    try {
        const k8sApi = await getKubernetesClient();
        return await k8sApi.createNamespacedPod(namespace, metadata);
    } catch (error) {
        console.error("Error while creating pod", error);
        return;
    }
}

async function deletePod(name, namespace) {
    try {
        const k8sApi = await getKubernetesClient();
        return await k8sApi.deleteNamespacedPod(name, namespace);
    } catch (error) {
        console.error("Error while deleting pod", error);
        return;
    }
}

Third, but not less important

Finally, we export the useful functions to be used in the code and ensure that they cannot be altered during execution. This can be done as:

export default Object.freeze({
    createPod,
    deletePod,
    getKubernetesClient
});

Final Result

import { ContainerServiceClient } from "@azure/arm-containerservice";
import { DefaultAzureCredential } from "@azure/identity";
import k8s from "kubernetes-client";

async function getKubernetesClient() {
    const kc = new k8s.KubeConfig();
    try {
        if (process.env.NODE_ENV === "production") {
            const credentials = new DefaultAzureCredential();
            const containerServiceClient = new ContainerServiceClient(credentials);
            const response = await containerServiceClient.managedClusters.listClusterAdminCredentials();
            kc.loadFromString(response.kubeconfigs[0].value);
        } else {
            kc.loadFromDefault();
        }
        const k8sApi = kc.makeApiClient(k8s.CoreV1Api);
        return k8sApi;
    } catch (error) {
        console.error("Error while getting Kubernetes API", error);
        return;
    }
}

async function createPod(metadata, namespace) {
    try {
        const k8sApi = await getKubernetesClient();
        return await k8sApi.createNamespacedPod(namespace, metadata);
    } catch (error) {
        console.error("Error while creating pod", error);
        return;
    }
}

async function deletePod(name, namespace) {
    try {
        const k8sApi = await getKubernetesClient();
        return await k8sApi.deleteNamespacedPod(name, namespace);
    } catch (error) {
        console.error("Error while deleting pod", error);
        return;
    }
}

export default Object.freeze({
    createPod,
    deletePod,
    getKubernetesClient
});

Micro Models - How to Build Using This Concept

Ytalo — Wed, 26 Jun 2024 20:50:18 +0000

In my last article (Developing for Scale and Quality with AI Micro-Models), I introduced the concept of micro-models for efficient AI model development. This time, we'll delve deeper into how to find micro-models, how to use them, and the benefits and trade-offs in terms of development time and efficiency. Let's get started!

Defining the Task

In the previous article, we set a fictional goal to "censor people" and broke it down into multiple models and steps. Today, we'll follow a similar approach. Begin by taking the final objective of your model and breaking it down into smaller, manageable tasks. For this example, let's aim to censor people in video clips.

Here's the process we'll follow:

Detect the person.
Crop them.
Apply a blur effect.
Reinsert them back into the original clip. It's evident that using an AI model specifically for detection and handling the rest with code is more efficient than employing a single 'big model' for the entire task.

Searching for the Model

There are numerous sources to find models for development, such as GitHub, arXiv papers, and Hugging Face. I prefer Hugging Face due to its intuitive interface, but any of these platforms should suffice.

Once you've chosen your platform, start searching for models. On Hugging Face, there's a dedicated section for models. I navigated directly to this section and selected the type of model I needed: Object Detection. After that, it's just a matter of choosing a suitable model and learning how to use it.

Applying the Model

Implementing the model was straightforward. I provided the video (or frames of a video) as input. The model output, which included a score and coordinates for cropping the content, was then fed into FFmpeg to handle the blurring of the video frame.

Efficiency and Savings

This approach not only enhances development efficiency but also significantly reduces resource consumption. By leveraging small specialist networks instead of large models, we improve both the speed of development and the execution time of the task.

Size of Micro-Models

There is a trade-off involved. For extremely long and complex tasks, we may need more micro-models, which require more RAM and memory for execution. However, this trade-off results in significantly better execution times and higher precision, as the models are specialized and optimized for their specific tasks.

In summary, micro-models offer a highly efficient way to tackle AI development tasks by breaking down complex problems into smaller, manageable units. This not only speeds up development but also ensures optimal use of resources.

Example

Censor a person in an image

import cv2
import torch

# load our model, in our case, yolo in version 5
model = torch.hub.load('ultralytics/yolov5', 'yolov5x', pretrained=True)

# the model return the boxes and a class for detection, so, we are setting the class to search in landmarks
person_class_id = 0

# function to apply blur
def blur_people(image, boxes):
    for box in boxes:
        x1, y1, x2, y2 = map(int, box)
        image[y1:y2, x1:x2] = cv2.GaussianBlur(image[y1:y2, x1:x2], (51, 51), 50)
    return image

def censor_people_in_image(input_image_path, output_image_path):
    #load the image
    image = cv2.imread(input_image_path)
    if image is None:
        raise FileNotFoundError(f"Não foi possível encontrar ou abrir a imagem {input_image_path}")

    # detect 
    results = model(image)
    boxes = results.xyxy[0].cpu().numpy()  # obter as caixas de detecção

    # filter all person
    person_boxes = [box[:4] for box in boxes if int(box[5]) == person_class_id]

    # apply blur
    censored_image = blur_people(image, person_boxes)
    cv2.imwrite(output_image_path, censored_image)

censor_people_in_image('input.jpg', 'output_image.jpg')

Developing for Scale and Quality with AI - Micro-Models

Ytalo — Sat, 22 Jun 2024 12:41:22 +0000

Written by Eletroswing

Introduction

Certainly, AI-based technologies are here to stay and represent our most imminent technological boom. Much of this is due to OpenAI's launch of its LLM (large language model), ChatGPT. This advancement has motivated the creation of thousands of new services in an industry that, just months before its release, was considered unfeasible. However, this progress comes with a price, one that could be costly in the long run.

Dependence on Large Models

With the emergence of OpenAI and its API, many companies have been built exclusively around these technologies. Many researchers in the field still adhere to the premise of keeping everything in a single large model, despite the daily launch of thousands of new technologies, models being trained, and datasets being created.

Challenges of Training Large Models

When faced with a task, the common approach is to train a single specific model, such as "using the OpenAI API". But why spend so many resources on something that might not be as satisfactory as expected? Consider an example: an eye movement substitution model.

Example of an Eye Movement Substitution Model

If you try to develop this by training a model from scratch, you quickly realize the complexity. Your model needs to identify, track, generate, and substitute the eye in videos. Look at how much work goes into a single objective.

Additionally, suppose your model takes 4 minutes per video. How would you scale this? Your only solution would be hardware, which is expensive and unfeasible.

Micro-Models

This brings us to the concept of micro-models.

Concept of Micro-Models

What if, instead of generalizing everything into a single model, we broke it into stages and utilized existing specialist models? This way, we create a modular and scalable pipeline by adding models and code between them. We could have a queue for identifying, another for tracking, another for generating, and another for substituting.

By preparing between the stages with code, and since the models are already specialists in their respective subjects, we can easily reduce inference time. We can also run parts in parallel if necessary. If tracking is fast but generation is not, we can simply add another consumer to the queue to handle the load.

Transformation with Micro-Models
Basically, we transform this:

Into this:

Benefits of Micro-Models

With this approach, despite seeming more complex, we achieve greater scalability and efficiency in our system. Additionally, we improve integration efficiency and speed, as we can modify only specific parts of the system instead of having to regenerate a model or perform fine-tuning, right?

Conclusion

Adopting micro-models allows for the creation of more scalable and efficient systems, making the most of existing resources and facilitating the continuous maintenance and evolution of AI-based solutions.

The 'dog' philosophy in development

Ytalo — Fri, 22 Mar 2024 13:23:36 +0000

Hello everyone,

You've probably heard of various famous philosophers throughout your life: Pythagoras, Socrates, Plato, among others. However, today let's talk about Diogenes, the Cynic, known as "the madman Socrates," and bring some of his mindset into modern-day development.

Diogenes was known for his ignorance, yet he preferred to live his philosophy in practice, earning him the nickname "Cynic," derived from the Latin for "dog." Despite this, Diogenes laid the foundation of modern philosophy. He preached that humans should learn to appreciate the basic, simplest things, rather than their material possessions. He became famous (and reviled) for taking this so literally, living naked in a barrel and using his actions to complement his words, unlike the common practice of his time to use words to justify actions. Moreover, he already advocated for equality among men, women, slaves, and all members of society, a revolutionary idea for his time. One of Diogenes' most memorable actions was when Plato defined man as "a featherless biped," and Diogenes, in response, threw a plucked chicken at Plato's feet, declaring: "Here is Plato's man."

Now, with this context on Diogenes, let's continue.

His ideas, whether intentional or not, immensely contributed to society, influencing notable figures such as Alexander the Great. Let's bring this into the context of current development practices. Many developers today learn techniques from tools like Tailwind, AWS, and Google Cloud. There's nothing wrong with that, of course, but in a world full of tools, clinging to just one is to limit your long-term potential. New tools will emerge, some better than others, and the one you're accustomed to will become obsolete. Without understanding the fundamentals, you'll be left behind.

Following Diogenes' philosophy, it's crucial to focus on the basics, on the bare minimum needed for something to function. While your team might be using a missile to kill an ant, you'll be labeled a cynic for preferring to use what the project truly requires: simplicity. Abandoning complex tools will allow you to understand how all others work. Like Tailwind? How about starting with basic CSS? Enjoy AWS SQS? Study MQTT and AMQP queues. While other developers around you are stuck with specific tools, you can stand out because you understand the fundamentals. Being adept with CSS, Tailwind, Sass, and applying this principle in everything you do will undoubtedly bring a special touch to all your applications.