DEV Community: LLMGraph

Flowise is shutting down: a migration path for your visual LLM workflows

LLMGraph — Wed, 05 Aug 2026 00:03:44 +0000

Flowise announced on July 29 that it's winding down. The GitHub repo gets archived on August 10, npm packages and Docker images are being deprecated, and August 31 is end of life. The code stays Apache 2.0, and the team's guidance is to fork and maintain it yourself.

If you're running Flowise chatflows in production, that leaves you with two real options: become the maintainer of your own fork (security patches, dependency upgrades, model API churn, all of it) or move to a platform that's actively maintained.

The stated reason, and why I read it differently

The Flowise team's stated reason for shutting down is that "rigid workflow low code approach quickly hits the limit when it comes to complexity." I'd frame it differently: visual workflow builders aren't the problem. Abandoned ones are. Most teams using a tool like Flowise chose it precisely because they don't want to hand-maintain orchestration code. That need didn't go away last week.

What migration actually looks like

I build LLMGraph, a hosted visual LLM workflow builder, and the concepts map over directly:

Chatflows map to Graphs. Same node-and-edge mental model: models, prompts, logic, and data sources wired together on a canvas.
Deployment is built in. A finished graph deploys as a public invoke API or an embeddable chat widget in one click. No Docker, no reverse proxy, no self-hosting.
Hosted and maintained. Model providers, infra, and security patches are our problem, not yours.

Before August 31

If you want to move a Flowise setup over before end of life, every LLMGraph tier starts with a 14-day trial, and I'm happy to personally help port your chatflows. Email founders@llmgraph.ai or ping me in the comments.

Hexagonal architecture in Rust, and why it makes your codebase legible to coding agents

LLMGraph — Tue, 28 Jul 2026 21:13:44 +0000

Coding agents are good at local edits and bad at holding a whole system in their head. Ask one to fix a function and it does fine. Ask it to "add Postgres support" to a codebase where the database calls are smeared across HTTP handlers, and it starts editing in six places, guessing at each, and quietly breaking the seventh. The problem is not the model. The problem is that the codebase never told anyone, human or machine, where things are allowed to live.

Hexagonal architecture is a way of telling them. It is an old idea (Alistair Cockburn named it "ports and adapters" in 2005) that turns out to be exactly the structure an agent needs: narrow contracts, bounded blast radius, and a spec it can read before it writes. This post walks through what it is, how it looks in Rust, and why the same boundaries that keep code readable for people make it tractable for agents.

There is a companion repo that compiles, tests, and runs: llmgraph-ai/hexagonal-rust-template. Everything below is in there.

The one rule

Hexagonal architecture has a lot of vocabulary (ports, adapters, driving side, driven side) but only one rule that matters:

Dependencies point inward. The core depends on nothing.

Your business logic sits in the middle. It defines the interfaces it needs from the outside world (a place to store data, a way to send email) as traits. It does not know or care what implements them. Everything technological (the web framework, the database driver, the JSON) lives at the edge and depends on the core, never the other way around.

Draw it as a picture and it is a set of rings with the arrows all pointing to the center:

server  ->  adapters  ->  application  ->  domain
                \______________________________^

domain is the core: business types and the ports (traits).
application is the use cases, written against those ports.
adapters are the edge: an HTTP handler, a Postgres client, an in-memory fake.
server is the composition root: the one place that wires a specific adapter to the core.

In Rust, the compiler enforces it

Most languages document the dependency rule and hope. Rust lets you make it a compile error. Put each ring in its own crate, and give each crate a dependencies list that only points inward. Now the core cannot import an adapter, because the adapter's crate is not in its dependency graph. The rule stops being a convention that erodes over time and becomes a fact the compiler checks on every build.

Here is the core. Note what is not imported: no axum, no sqlx, no serde. Just the standard library and two small utilities.

// crates/domain/src/lib.rs
pub struct ShortCode(String);

impl ShortCode {
    // The inner String is private, so the only way to get a ShortCode is to
    // parse one. An invalid code cannot exist anywhere in the system.
    pub fn parse(raw: &str) -> Result<Self, DomainError> {
        let ok = (3..=32).contains(&raw.len())
            && raw.chars().all(|c| c.is_ascii_lowercase() || c.is_ascii_digit() || c == '-');
        if ok { Ok(Self(raw.to_string())) } else { Err(DomainError::InvalidCode) }
    }
}

The core also defines the ports: the interfaces it needs from the world, as traits.

// crates/domain/src/ports.rs

// A driven (outbound) port: somewhere to store links. The core does not know
// if this is Postgres, Redis, or a HashMap.
#[async_trait]
pub trait LinkRepository: Send + Sync {
    async fn save(&self, link: &ShortLink) -> Result<(), DomainError>;
    async fn find(&self, code: &ShortCode) -> Result<Option<ShortLink>, DomainError>;
}

// A driving (inbound) port: the use cases the application offers to the world.
#[async_trait]
pub trait Shortener: Send + Sync {
    async fn shorten(&self, target: &str, code: Option<&str>) -> Result<ShortLink, DomainError>;
    async fn resolve(&self, code: &str) -> Result<ShortLink, DomainError>;
}

The application layer implements the driving port by orchestrating the driven one. This file is the behavior of the system with none of the plumbing:

// crates/application/src/lib.rs
pub struct ShortenerService {
    repo: Arc<dyn LinkRepository>, // depends on the port, not a concrete DB
}

#[async_trait]
impl Shortener for ShortenerService {
    async fn shorten(&self, target: &str, code: Option<&str>) -> Result<ShortLink, DomainError> {
        Self::validate_target(target)?;
        let code = match code {
            Some(raw) => ShortCode::parse(raw)?,
            None => ShortCode::parse(&generate_code())?,
        };
        if self.repo.find(&code).await?.is_some() {
            return Err(DomainError::CodeTaken);
        }
        let link = ShortLink { code, target: target.to_string() };
        self.repo.save(&link).await?;
        Ok(link)
    }
    // ...
}

The HTTP adapter depends on the Shortener port, not the concrete service. Its only job is to translate HTTP into a call on the port and translate the result back. There is no business logic here, which is exactly why you can throw it away and replace it with a CLI without touching the core:

// crates/adapters/src/http.rs
async fn create_link(
    State(state): State<AppState>,
    Json(req): Json<CreateLinkRequest>,
) -> Result<(StatusCode, Json<LinkResponse>), ApiError> {
    let link = state.shortener.shorten(&req.target, req.code.as_deref()).await?;
    Ok((StatusCode::CREATED, Json(link.into())))
}

And the composition root, the single file that knows every concrete type:

// crates/server/src/main.rs
let repo: Arc<dyn LinkRepository> = Arc::new(InMemoryLinkRepository::default());
let shortener: Arc<dyn Shortener> = Arc::new(ShortenerService::new(repo));
let app = http::router(shortener);
axum::serve(listener, app).await.unwrap();

Want Postgres instead of memory? Write a PostgresLinkRepository that implements LinkRepository, and change line one. Want a CLI instead of HTTP? Change line three. Nothing upstream notices, because nothing upstream ever knew.

Why agents thrive on this

Now the payoff, and the reason this old pattern is worth revisiting in 2026. Every property that makes hexagonal architecture pleasant for a human maps directly onto something an agent needs.

Tasks map to a single file with a precise contract. "Add a Postgres adapter" is not a vague request that could touch anything. It is: implement this one trait in this one new file. The agent has the exact signature to satisfy and physically cannot reach into business logic while doing it, because the adapter crate does not depend on private application internals. The task is scoped by the architecture, not by your prompt.

The ports are the spec. An agent can write a correct use-case test from the trait definition alone, before a single adapter exists. In the template, the use-case tests run against a five-line fake repository:

#[tokio::test]
async fn shorten_then_resolve_roundtrips() {
    let service = ShortenerService::new(Arc::new(FakeRepo::default()));
    let created = service.shorten("https://example.com", Some("my-code")).await.unwrap();
    let resolved = service.resolve("my-code").await.unwrap();
    assert_eq!(created, resolved);
}

No database, no HTTP server, no fixtures. An agent asked to raise test coverage on the core has everything it needs from the port and nothing it needs to mock away.

Blast radius is bounded by construction. This is the big one. When an agent makes a wrong turn in a well-layered codebase, the mistake stays inside the layer it was working in. A bad decision in the HTTP adapter cannot corrupt the domain, because the compiler will not let the domain see the adapter. You get to be wrong locally, which is the only kind of wrong that is cheap to fix. In a codebase where a database call can appear inside a request handler, there is no such floor. A confused edit propagates.

The structure survives context limits. An agent does not need to load the whole repository to work on one adapter. The relevant surface is the port it implements plus the crate it lives in. The architecture pre-chunks the codebase into units small enough to reason about in isolation, which is the same reason it was pleasant for humans in the first place. Good structure is good structure; agents just raise the stakes on having it.

Where to start

Do not start by rewriting everything into four crates. Start with the one rule and one seam. Pick the dependency that hurts most (usually the database) and define it as a trait in your core. Move the concrete client behind an adapter that implements the trait. Wire it up in one place. You will feel the core get quieter almost immediately, because it stops importing things that have nothing to do with your business.

Then, if it earns its keep, split the rings into crates so the compiler enforces the rule you have been maintaining by hand. Rust makes that enforcement free, and free enforcement is the difference between an architecture you have and an architecture you had six months ago.

The full template compiles, tests, and runs: github.com/llmgraph-ai/hexagonal-rust-template. Clone it, run cargo test, delete the URL shortener, and keep the shape.

This is the kind of structured backend we care about at LLMGraph, where we build LLM and AI-agent workflows that people deploy as APIs. Legible structure is what lets both our team and our tools move fast without breaking the parts that matter.

What it actually costs to run a RAG chatbot

LLMGraph — Tue, 28 Jul 2026 12:06:38 +0000

"How much will this cost to run?" is the question that kills more AI side projects than any technical problem. People build a working RAG chatbot in a weekend, then freeze because they have no idea whether serving it costs 5 dollars a month or 5,000. This post breaks the bill into its real parts so you can estimate before you commit.

I'll use round numbers and current-ish prices. Yours will differ. The point is the shape of the bill, not the exact digits.

The bill has four line items

A retrieval-augmented chatbot spends money in four places:

Embedding your documents (one time, plus updates)
Storing the vectors
Embedding each incoming question
Generating each answer

Three of those four are close to free. One of them is the whole bill. Guess which before you read on.

1. Embedding your documents

You embed each chunk of your documents once. Embedding is cheap. Most providers charge a few cents per million tokens.

Say you have 200 documents averaging 3,000 words each. That's roughly 800,000 tokens. At current embedding prices that is a fraction of a cent to a few cents, total, one time. Even 10,000 documents lands in the low single-dollar range.

If your documents never change, you pay this once and forget it. If they change daily, you re-embed the changed ones, which is still pennies.

Line item: negligible.

2. Storing the vectors

Each chunk becomes a vector, usually 768 or 1536 numbers. A few thousand chunks is a few megabytes. You can hold it in memory, in a local file, or in a hosted vector database.

For a small project, a local store (Chroma, SQLite with a vector extension, FAISS on disk) costs nothing beyond the machine you already run. Hosted vector databases start free and stay cheap until you're storing millions of vectors.

Line item: zero to a few dollars a month until you're at real scale.

3. Embedding each question

Every question a user asks gets embedded so you can search. A question is short, maybe 20 tokens. At embedding prices that is far less than a hundredth of a cent per question.

A thousand questions a day is still cents per month.

Line item: negligible.

4. Generating the answer

Here's the bill.

Every answer is a call to a language model, and the input to that call is not just the question. It's the question plus the chunks you retrieved plus your instructions. That's the expensive part, because generation models charge per token and you're feeding them a few thousand tokens of context every single time.

Rough math. Say each answer sends 2,000 tokens of context and gets back 400 tokens. On a mid-tier model at a few dollars per million input tokens and a bit more for output, one answer costs somewhere around half a cent to two cents depending on the model.

Now multiply by traffic:

100 answers a day: roughly 1 to 6 dollars a month
1,000 answers a day: roughly 15 to 60 dollars a month
10,000 answers a day: roughly 150 to 600 dollars a month

That range is wide because model choice swings it more than anything else. A frontier model can cost 10 to 20 times what a smaller model costs for the same job. For a lot of support and lookup use cases, a smaller model with good retrieval beats a frontier model with sloppy retrieval, and costs a tenth as much.

What actually moves the number

Once you see that generation is the bill, the levers are obvious:

Model choice. The single biggest lever. Try the cheapest model that passes your quality bar before reaching for the expensive one.
Context size. Retrieving 15 chunks instead of 5 triples your input tokens for often worse answers. Fewer, better chunks cost less and read better.
Caching. If the same questions come up again and again, and in support they do, cache the answers. A cache hit costs nothing.
Trimming instructions. A 600-token system prompt sent on every call adds up. Tighten it.

A quick estimate you can do right now

Take your expected answers per day, multiply by 30 for the month, multiply by roughly 1 cent. That's your rough monthly generation bill on a mid-tier model. Everything else on the bill is rounding error at small scale.

So a chatbot answering 300 questions a day costs on the order of 10 dollars a month to run. That's the number that unfreezes the project.

Where this fits

If you'd rather not stand up the embedding, storage, and generation plumbing yourself and meter each piece, that's the kind of thing LLMGraph handles. You build the workflow, it runs the retrieval and the model calls, and you get a deployed API and chat widget. The cost shape above is still what's happening underneath, so it's worth understanding either way.

The takeaway: a RAG chatbot is cheap to run at small scale, generation is the only line item that matters, and model choice is the dial that turns a 10-dollar bill into a 200-dollar one. Estimate before you build, and the "what will this cost" freeze goes away.

Turn a folder of PDFs into a question-answering API in an afternoon

LLMGraph — Tue, 28 Jul 2026 03:58:48 +0000

Someone hands you 200 PDFs and asks for a "chatbot that knows this stuff." You could read all 200. Or you could build a small API that reads them for you and answers questions with citations. This post is the second option, start to finish, with the parts that usually break called out.

I'll show the mechanics first so you understand what's happening, then a shorter path if you don't want to run the plumbing yourself.

What you're actually building

A question-answering API over your own documents is four moving parts:

Ingest. Read each PDF, pull out the text.
Chunk. Split that text into pieces small enough to search.
Embed and store. Turn each chunk into a vector, keep it in a store you can query.
Answer. At question time, find the closest chunks and hand them to a model with the question.

People call the whole thing RAG (retrieval-augmented generation). The name makes it sound heavier than it is. It's search, then a prompt.

Step 1: get the text out

PDFs are a mess. Some are clean text, some are scanned images, some have two columns that extract in the wrong order. Start simple with pypdf:

from pypdf import PdfReader

def read_pdf(path: str) -> str:
    reader = PdfReader(path)
    return "\n".join(page.extract_text() or "" for page in reader.pages)

Where it breaks: scanned PDFs return empty strings because there's no text layer, only pixels. If extract_text() gives you nothing, those files need OCR (Tesseract, or a hosted vision model). Check for empty output early so you don't silently index blank pages.

Step 2: chunk the text

You can't embed a whole document as one vector and expect good answers. Retrieval works on pieces. The size of those pieces matters more than most tutorials admit.

Too small (a sentence) and each chunk loses its context. Too big (a whole page) and the model gets a wall of mostly-irrelevant text and the answer drifts. A reasonable starting point is 500 to 800 tokens per chunk with a small overlap so a sentence split across a boundary isn't lost.

def chunk(text: str, size: int = 600, overlap: int = 80) -> list[str]:
    words = text.split()
    step = size - overlap
    return [" ".join(words[i:i + size]) for i in range(0, len(words), step)]

This splits on whitespace, which is crude but fine to start. If your documents have clear structure (headings, sections), splitting on those boundaries beats a fixed window because a chunk that matches a section is a chunk that makes sense on its own.

Step 3: embed and store

An embedding turns a chunk into a list of numbers that captures its meaning. Similar text lands near similar text. You query by embedding the question the same way and finding the nearest chunks.

For a folder of 200 PDFs you do not need a managed vector database. A local store is plenty:

import chromadb
from chromadb.utils import embedding_functions

client = chromadb.PersistentClient(path="./store")
ef = embedding_functions.DefaultEmbeddingFunction()
col = client.get_or_create_collection("docs", embedding_function=ef)

for doc_id, text in enumerate(all_texts):
    chunks = chunk(text)
    col.add(
        ids=[f"{doc_id}-{i}" for i in range(len(chunks))],
        documents=chunks,
        metadatas=[{"source": sources[doc_id]} for _ in chunks],
    )

Keep the source filename in the metadata. That's what lets you cite where an answer came from, which is the difference between a demo and something people trust.

Step 4: answer with citations

At question time: embed the question, pull the top handful of chunks, put them in the prompt, ask the model to answer only from what it was given.

def answer(question: str) -> str:
    hits = col.query(query_texts=[question], n_results=5)
    context = "\n\n".join(hits["documents"][0])
    sources = {m["source"] for m in hits["metadatas"][0]}

    prompt = (
        "Answer the question using only the context below. "
        "If the context does not contain the answer, say so.\n\n"
        f"Context:\n{context}\n\nQuestion: {question}"
    )
    reply = call_your_model(prompt)  # Claude, GPT, a local model, your choice
    return f"{reply}\n\nSources: {', '.join(sources)}"

Two things that matter here:

n_results. Five is a good default. More context is not better. Past a point you're paying for tokens and diluting the signal.
The "say so" instruction. Without it the model fills gaps with plausible fiction. With it you get "the documents don't cover this," which is the honest answer and the one that keeps people trusting the system.

Wrap answer() in any web framework and you have an API:

from fastapi import FastAPI

app = FastAPI()

@app.post("/ask")
def ask(question: str):
    return {"answer": answer(question)}

Where this gets hard in production

The afternoon version above works. The things that turn it into a real project:

Re-indexing. Documents change. You need a way to update or drop chunks without rebuilding everything.
Bad retrieval. When answers are wrong it's usually retrieval, not the model. The right chunk never made it into the context. Logging what got retrieved for each question is the first debugging tool you'll want.
Access control. If different people should see different documents, retrieval has to filter by permission before it ranks. Bolting this on later is painful.
Cost. Embeddings are cheap. The generation call is not, and it scales with traffic. Caching common questions helps.

The shorter path

If you'd rather not run the ingest, chunking, vector store, and prompt loop yourself, this is the exact shape of thing LLMGraph builds. You point it at your documents, it handles the chunking and retrieval, and you get a REST API and an embeddable chat widget out the other side. The four steps above still happen. You just don't maintain them.

Either way, the mental model is the same: read, chunk, embed, retrieve, answer. Once that clicks, "chatbot that knows our stuff" stops sounding like a research project and starts sounding like an afternoon.

How to Add an AI Customer Support Assistant — Without Writing Code

LLMGraph — Mon, 13 Jul 2026 22:55:46 +0000

Originally published on the LLMGraph blog.

Your customers expect answers fast — at 9am on a Tuesday and at 2am on a Saturday. Meanwhile your support team is finite, and a large share of the questions they field are the same ones, over and over: where's my order, how do I reset this, what does this plan include. Every leader has heard that an AI assistant can take that load off. Most assume it's a months-long engineering project. It isn't anymore.

Here is the entire build, start to finish, in a short demo — a working customer support assistant, no code written:

What an AI customer support assistant actually does

A good support assistant isn't a generic chatbot that guesses. It answers from your knowledge and knows its limits:

Answers instantly, around the clock — in your product's voice, without adding headcount for every timezone.
Grounded in your own content — your docs, help center, and policies, via retrieval (RAG), so it gives the real answer instead of a plausible-sounding one.
Knows its limits — recognizes the complex or sensitive cases that belong with your team and points customers there instead of bluffing.
Lives where customers already are — an embeddable chat widget on your site or app, or wired into your stack through an API.

Why this used to take an engineering team

The reason so many companies shelved the idea is that the traditional build is genuinely involved. To do it yourself you'd choose and host a model, build a retrieval pipeline (chunk your documents, generate embeddings, stand up a vector database, wire up search), tune prompts and guardrails, build the chat widget, then deploy, scale, and monitor the whole thing. That's specialized hires and a quarter of work before a single customer is helped.

How LLMGraph turns it into a visual build

LLMGraph collapses that stack into a visual workflow. You design the assistant as a graph on a canvas — or just describe what you want in chat and let LLMGraph assemble it — and the plumbing is handled for you:

Point it at your documents; LLMGraph does the embeddings and retrieval so answers stay grounded.
Shape the assistant's role, tone, and boundaries without writing code.
Ship it in one click to a live chat widget you paste onto any page, plus a REST API for deeper integration. No infrastructure to manage.

That flow — knowledge in, behavior shaped, deployed — is the whole video above.

What you can stand up this afternoon

A first working assistant is four steps, not four sprints:

Add your knowledge. Upload docs or connect the help content that already answers your common questions.
Shape behavior. Set the assistant's role, tone, and what it should leave to your team.
Test on the canvas. Ask it real customer questions and refine until the answers are right.
Deploy. Drop the chat widget on your site, or call the API from your product.

The business case for leaders

Coverage — 24/7 first-response without hiring for every shift and region.
Deflection — repetitive questions get handled automatically, freeing your team for the complex, high-value cases only a person should touch.
Consistency — the same accurate answer every time, grounded in your single source of truth.
Control — your data, your brand, and a human in the loop where it matters.
Speed to value — build and iterate in an afternoon, then improve it as you learn, instead of committing to a quarter-long project up front.

Start with one workflow

You don't have to automate everything on day one. Pick the ten questions your team answers most, point an assistant at the docs that already answer them, and ship the widget on a single page. Watch what it handles, refine, and expand from there. The first version is the afternoon; the compounding value is everything you add after.

If you want to see it against your own content, start a free trial or look at the plans — every tier includes a 14-day free trial.