DEV Community: Ekemini Samuel

Top 5 Enterprise AI Gateways

Ekemini Samuel — Wed, 14 Oct 2026 06:57:35 +0000

Most companies did not plan their AI infrastructure. It grew as usage across teams increased. Someone wired up the OpenAI API for a prototype, another added Anthropic because it was better at a specific task, and six months later there are a dozen applications, each with its own API keys, its own retry logic, and no shared view of who is spending what.

An enterprise AI gateway is the thing that stops sprawl from becoming a liability: one layer, in front of every model call, that knows who is calling, what they are allowed to do, and what it costs.

This list is not about which gateway has the most features. It is about which ones actually hold up once security, platform, and finance teams are all asking it questions at the same time.

TL;DR

Bifrost is the enterprise AI gateway to use if you want enterprise controls (identity, budgets, guardrails, audit logs) without giving up the open-source core. It is written in Go, licensed Apache 2.0, and adds about 11 microseconds of overhead at 5,000 requests per second, the kind of number that matters once a gateway sits in front of every request your company makes. It also ships Bifrost Edge, an agent that extends the same governance to AI traffic on employee laptops, which none of the other four gateways in this list currently offer.
Portkey has the broadest adoption story in the open-source gateway space, a large community, and a deep enterprise tier. It is a reasonable pick for teams that want a mature product with a long feature list.
Tyk AI Gateway is the choice for teams that are already running Tyk, or any API gateway, for everything else, and want LLM traffic to live under the same policy engine instead of a separate system.
F5 AI Gateway fits organizations that already run F5 for application delivery and security, and want AI traffic folded into that same console rather than standing up a new one.
Cloudflare AI Gateway is the natural fit for teams already running on Cloudflare, with caching, fallback routing, and usage analytics built into infrastructure they are already using.

What enterprises are actually looking for in an AI gateway

The term "AI gateway" covers a wide range of products now, from simple request routers to full governance platforms. Before you compare any of the five below, it helps to know what questions each team in your organization is going to ask once the gateway is actually in production.

Security wants identity on every request. Not an API key shared across a team, but a way to know which application, which service account, or which person sent a given request, and to revoke that access without redeploying anything.
Platform wants the gateway to not be the thing that goes down. That means load balancing across providers and keys, automatic failover when a provider has an outage, and low enough overhead that adding a gateway does not show up as a latency regression in a dashboard somewhere.
Finance wants to know what a model is costing, broken down by team, by project, or by application, before the invoice arrives rather than after. Budgets that can be set in advance, not just logs you can grep through at the end of the month.
Compliance wants an audit trail that holds up. Every request, every response, every policy decision, recorded somewhere that cannot be quietly edited.
Everyone wants deployment flexibility. A gateway that only runs as someone else's SaaS is a hard sell for a company with data residency requirements, a regulated industry, or an internal policy against sending production traffic through a third party's infrastructure. The ability to self-host, including fully air-gapped, is what separates a gateway you can put in front of all traffic from one you can only use for a side project.

Here is how the five stack up at a glance, and where each one actually sits in your request path.

A gateway is the one place security, platform, and finance all get to ask their questions, on every request, regardless of which team sent it or which model answered it.

Comparison across the AI Gateways

Gateway	Core	License	Self-hosted / air-gapped	Best for
Bifrost	Go	Apache 2.0 (enterprise tier)	Yes, including air-gapped	Teams that want full governance without losing the open-source core
Portkey	Open source	MIT (enterprise tier)	Yes, including hybrid and air-gapped	Teams that want a mature, widely adopted gateway with a long feature list
Tyk AI Gateway	Go	Open source (enterprise tier)	Yes, multi-cloud and on-prem	Teams already standardized on Tyk or another API gateway
F5 AI Gateway	Part of F5's platform	Commercial	SaaS and hybrid now; air-gapped planned	Teams already running F5 for application delivery and security
Cloudflare AI Gateway	Cloudflare's global network	Commercial	SaaS only	Teams already running on Cloudflare

1. Bifrost

Bifrost is an open-source AI gateway built in Go, with an Apache 2.0 core and a paid enterprise tier for identity, high availability, and support. It gives you one OpenAI-compatible interface to over 10,000 models across more than 25 providers, including OpenAI, Anthropic, AWS Bedrock, Google Vertex, Azure, Gemini, Groq, Mistral, Cohere, and xAI, plus self-hosted options like vLLM, SGLang, and Ollama. Switching providers is a base URL change, not a rewrite.

What makes Bifrost an enterprise gateway rather than just a fast router is how identity and governance are built in rather than bolted on.

Every caller gets a Virtual Key, and that key is where budgets, rate limits, and model permissions actually live, scoped by team, customer, or individual model, and enforced hierarchically so a sub-team's budget cannot quietly exceed its parent's. A key that runs out returns a clear 402 rather than silently failing or, worse, silently succeeding on someone else's bill.

Bifrost also treats agent tooling as a first-class part of the gateway rather than a separate system. Its MCP Gateway sits in front of tool calls over STDIO, HTTP, and SSE, the same way the rest of Bifrost sits in front of model calls, with filtering at the client, request, and virtual-key level so an agent only ever sees the tools it is allowed to use.

Its Code Mode feature has the model write a short script to orchestrate tools instead of loading every tool definition into context, and in a case study Maxim published, a 508-tool run across 16 MCP servers dropped from $377 to $29, a cut of about 92 percent, while keeping a 100 percent task pass rate. For any team running agents in production, that is the difference between a tool-calling bill that scales with usage and one that does not.

On the operations side: adaptive load balancing with circuit breakers that recalculate provider weights from live latency and error rates, automatic failover configured as a fallbacks array in the request payload rather than a separate piece of infrastructure, semantic caching backed by Redis, Qdrant, Weaviate, or Pinecone, and complexity-based routing that can send a simple query to a cheap model and a harder one to a stronger model automatically.

For multi-region deployments, Bifrost's cluster mode runs as peer-to-peer nodes with no single leader, sharing traffic patterns, rate-limit state, and provider health over a gossip protocol, so there is no coordinator node to become a new single point of failure.

On the compliance side: fine-grained RBAC with custom roles, SSO and SCIM 2.0 support for Okta, Microsoft Entra, Keycloak, Zitadel, Auth0, Google Workspace, and any generic OIDC provider, audit logs that are signed and tamper-resistant with export to S3 or GCS, and in-VPC or fully air-gapped deployment for teams that cannot send traffic, prompts, or telemetry anywhere outside their own network.

Guardrails come built in for content safety, PII, and secrets detection, and extend out to Presidio, Azure AI Language, AWS Bedrock Guardrails, Azure Content Safety, Google Model Armor, CrowdStrike AIDR, Check Point, Lakera Guard, and several others. Secrets themselves can live in AWS Secrets Manager, GCP Secret Manager, or HashiCorp Vault rather than sitting in Bifrost's own database, and budget or rate-limit breaches can alert Slack, Microsoft Teams, or PagerDuty. SOC 2 Type II, ISO 27001, HIPAA, and GDPR are all covered.

One piece that is distinct from every other gateway on this list: Bifrost Edge, an agent that runs on employee laptops and routes AI traffic from desktop apps, browser AI, and coding agents through the same Gateway, so the governance does not stop at the server boundary. It is an add-on to the enterprise tier, priced per endpoint, but it addresses a real gap, the AI usage that never touches your backend at all.

Key capabilities

Unified OpenAI-compatible interface to 10,000+ models across 25+ providers, plus self-hosted vLLM, SGLang, and Ollama.
Virtual Keys for identity, with hierarchical budgets, rate limits, and model scoping.
MCP Gateway with tool filtering and Code Mode for agent tool calls.
Adaptive load balancing with circuit breakers, automatic failover, and complexity-based routing
Peer-to-peer cluster mode for multi-region high availability, with no single coordinator. . Semantic caching and reasoning budget normalization across providers.
RBAC, SSO and SCIM 2.0 (Okta, Entra, Keycloak, Zitadel, Auth0, Google Workspace, generic OIDC).
Signed, tamper-resistant audit logs, exportable to S3 or GCS Bifrost Edge for governing AI traffic on employee devices, not just the backend.
In-VPC and fully air-gapped deployment, SOC 2 Type II, ISO 27001, HIPAA, GDPR.

Best for: Bifrost is built for enterprises running mission-critical AI workloads that require best-in-class performance, scalability, and reliability. It serves as a centralized AI gateway to route, govern, and secure all AI traffic across models and environments with ultra low latency. Bifrost unifies LLM gateway, MCP gateway, and Agents gateway capabilities into a single platform. Designed for regulated industries and strict enterprise requirements, it supports air-gapped deployments, VPC isolation, and on-prem infrastructure. It provides full control over data, access, and execution, along with robust security, policy enforcement, and governance capabilities.

2. Portkey

Portkey is an open-source gateway , with its core project sitting at close to 12,000 stars on GitHub. It started as a lightweight unified API across model providers and has large enterprise feature set: SSO through SAML 2.0, Okta, and Azure AD, role-based access control, PII anonymization, and certifications including SOC 2, ISO 27001, GDPR, and HIPAA.

Deployment is flexible across SaaS, hybrid, and fully air-gapped tiers, and the product line now includes its own MCP Gateway and an Agent Gateway aimed at the same agent-tooling problem Bifrost's MCP layer addresses.

Portkey's recent "Gateway 2.0" release pushed more enterprise-grade functionality back into the open-source core rather than keeping it exclusive to the paid tier, which is a meaningful shift for teams that want to start on the free version and grow into it.

Key capabilities

Unified API across a wide range of model providers, MIT-licensed core.
SSO (SAML 2.0, Okta, Azure AD), RBAC, PII anonymization.
SOC 2, ISO 27001, GDPR, and HIPAA certifications.
MCP Gateway and Agent Gateway for agent tool traffic.
Semantic caching.
SaaS, hybrid, and air-gapped deployment tiers.

Best for: teams that want a mature, broadly adopted gateway with a long enterprise feature list and an active community.

3. Tyk AI Gateway

Tyk is an API and AI gateway paired with Tyk AI Studio. It works with the same policy engine, access control, and multi-cloud deployment model that Tyk customers already use for REST, GraphQL, and gRPC APIs, extended to cover model providers too.

A platform team that already runs Tyk for its API layer does not want a second, unrelated system just for LLM calls, with its own policy language and its own way of defining who can call what. Tyk's AI Gateway folds model traffic into the same Kubernetes-native deployment and policy engine, which is a real simplification for teams in that position. Enterprise users referenced in Tyk's own case studies include organizations like Zeiss, Domino's, and Deloitte, which gives some sense of the scale it is built to run at.

Tyk supports multi-cloud, hybrid, and fully on-premises deployment, and because it grew out of a general-purpose API gateway, it handles a wider range of protocols than most AI-specific gateways, which matters if your LLM traffic is only one part of a broader API surface you need to govern consistently.

Key capabilities

Unified policy engine shared with Tyk's broader API gateway product
Kubernetes-native, with multi-cloud, hybrid, and on-premises deployment.
REST, GraphQL, gRPC, and async protocol support alongside LLM traffic.
Tyk AI Studio for building and managing AI-specific policies.
Enterprise customers at meaningful scale (Zeiss, Domino's, Deloitte referenced in Tyk's own materials).

Best for: teams already running Tyk, or evaluating a general-purpose API gateway, that want LLM traffic governed under the same system rather than a separate one.

4. F5 AI Gateway

F5 is best known for application delivery and security, the layer that sits in front of a company's web and API traffic generally. Its AI Gateway extends that same platform, now branded as F5's Application Delivery and Security Platform, to cover LLM traffic, combining a Model Gateway, an MCP Gateway, and AI-specific guardrails into one console.

The guardrails are where F5's existing security background shows up directly: detection for prompt injection and sensitive data leaks, built on the same inspection capabilities F5 already uses for web application traffic. On cost, F5 claims up to 60 percent reduction in token spend through a combination of routing, model tiering, and semantic caching, though as with any vendor-reported figure, it is worth asking what workload that number was measured against before treating it as a baseline for your own.

Key capabilities

Model Gateway, MCP Gateway, and AI Guardrails unified in one console.
Prompt injection and sensitive data leak detection, built on F5's existing security stack.
Claimed up to 60 percent token spend reduction via routing, tiering, and semantic caching.
SaaS, hybrid SaaS, and hybrid multicloud deployment

Best for: organizations already running F5 for application delivery and security that want AI traffic governed in the same console rather than a new one.

5. Cloudflare AI Gateway

Cloudflare AI Gateway takes the same approach Cloudflare takes to everything else: put it on their global network and let teams configure it through a dashboard or API rather than infrastructure they have to run themselves. It sits in front of model providers as a control plane, with request caching to cut down on repeated calls, fallback routing so a request can move to a backup provider automatically, and unified billing that rolls every provider's cost into one bill instead of several.

The appeal here is proximity to infrastructure a team already has. Routing rules, including which provider handles a request based on latency, cost, or availability, can be changed from the dashboard without a redeploy, and logs come with token counts, cost, duration, and request status attached, so usage analytics are available without wiring up a separate observability stack. Built-in guardrails aim to catch sensitive data before it reaches a provider, without additional setup.

What it does not offer is a self-hosted or air-gapped deployment option. Cloudflare AI Gateway runs on Cloudflare's infrastructure, not yours, which is a real constraint for teams with strict data residency requirements, but less of one for teams already comfortable running production traffic through Cloudflare.

Key capabilities

Request caching with configurable TTL to cut redundant provider calls.
Fallback routing by latency, cost, or availability.
Unified billing across every connected provider.
Logs and usage analytics with token counts, cost, and duration per request.
Built-in guardrails against sensitive data leakage, no extra setup required.

Best for: teams already running on Cloudflare who want an AI gateway that lives in the same dashboard as the rest of their infrastructure.

Which one is right for you

If you are choosing based on what your team already runs, the decision mostly If you are choosing based on what your team already runs, the decision mostly makes itself: Tyk if you already run Tyk, F5 if you already run F5, Cloudflare if your traffic already sits behind Cloudflare.

Those three gateways earn their place by not asking you to operate a second, unrelated system.

If you are choosing without that constraint, the real question is what you are optimizing for. Portkey gives you the most mature, most broadly adopted option, with a long feature list and a large community behind it, though its pending acquisition is worth a direct conversation with the vendor.

Bifrost gives you that same depth of governance (identity, budgets, guardrails, audit), fully self-hostable including air-gapped, without the lock-in that comes from a product tied to one company's broader platform. For a team that wants to own its enterprise AI gateway the way it owns the rest of its stack, rather than renting a slice of someone else's, that is the harder thing to find, and it is what Bifrost is built around.

Frequently asked questions

What is an AI gateway, and how is it different from a model API?
A model API is how you send a request to one provider. An AI gateway sits in front of all of your model providers, handling identity, routing, budgets, and governance for every request, regardless of which model is actually serving it. You call the gateway once and it decides where the request actually goes.

Do I need an AI gateway if I only use one model provider?
Even with a single provider, a gateway gives you identity on every request, rate limits and budgets per team or application, and an audit trail, none of which a provider's own API gives you by default. It also means you are not rewriting application code the day you decide to add a second provider or a fallback.

What does "air-gapped deployment" actually mean for an AI gateway?
It means the gateway can run entirely inside your own network, with no outbound dependency on the vendor's infrastructure to function. For regulated industries or anyone with strict data residency requirements, this is often a hard requirement rather than a preference, which is why it is worth checking whether a vendor offers it today versus on a roadmap.

How much latency does an AI gateway add?
It varies by implementation, but a well-built gateway should add a negligible amount. Bifrost, for example, is benchmarked at about 11 microseconds of overhead at 5,000 requests per second, in Maxim's own published testing. At that level, the gateway is not the thing slowing down your application; the model call itself is. Whatever gateway you evaluate, ask for the benchmark methodology, not just the headline number.

Can an AI gateway reduce my model costs?
Yes, through a few mechanisms: semantic caching avoids paying for a model call when a near-identical one was already answered, complexity-based routing sends simpler queries to cheaper models automatically, and having visibility into spend per team or application in the first place tends to surface waste that was previously invisible.

Do AI gateways work with agents and MCP tools, or just chat completions?
The better ones do. Agent tool calls are a growing share of LLM traffic, and treating them as ungoverned is the same mistake as treating early chat completions as ungoverned. Bifrost's MCP Gateway, Portkey's Agent Gateway, and F5's MCP Gateway are all explicit answers to that, filtering which tools an agent can reach and keeping tool traffic inside the same identity and audit model as everything else.

Next steps

Bifrost is open source and self-hostable, with a quickstart that gets an enterprise AI gateway running in front of your existing model traffic in minutes. Explore the documentation and the GitHub repo, or book a Bifrost demo to see it running in front of your own model traffic.

How SIMD works in Mojo

Ekemini Samuel — Wed, 30 Sep 2026 18:50:31 +0000

Let's say you want to know how many times the word research appears in a document.

You could upload the document to a program and read the file one byte at a time, compare each byte with the first letter of the word, and inspect the promising positions more closely.

Now give the same program ten megabytes of text. Then a hundred megabytes. Then give it the papers, policy reports, interview transcripts, and field notes collected by an entire research team.

The question has not changed. The computer is still asking, “Does the word start here?” The problem is how many times it has to ask.

This is where SIMD starts to make sense.

SIMD means single instruction, multiple data. Instead of comparing one byte and then the next one, the processor can compare several bytes together. The values sit in separate lanes of a vector, and the same operation runs across those lanes.

So SIMD is not really a trick for making a loop move faster. It changes the shape of the work. A scalar loop moves through the data one value at a time. A SIMD operation looks across several values at once.

That sounds simple, and the basic idea is simple. The interesting part begins when we try to use it in a real program.

What happens when the file does not divide neatly into complete vectors? What happens when the first byte matches but the rest of the word does not? How do we distinguish research from researcher?

And after making the code faster, how do we prove that it still returns the right answer?

We will answer those questions by building ResearchLens, a Mojo application that searches research text. It uses SIMD to find possible matches and a scalar verifier to decide which ones are real.

Along the way, we will look at:

what a SIMD lane is
why the vector width is part of a Mojo type
how loads, comparisons, masks, and reductions fit together
why a fast candidate is not automatically a correct match
how to handle the final partial vector safely
how to benchmark the result without making claims the data cannot support.

You do not need a GPU for this tutorial. The SIMD hardware is already inside the CPU in a modern computer. We are going to show Mojo how to use it.

The scalar loop

Let us begin with the obvious version.

def count_first_byte_scalar(data: Span[UInt8, _], first: UInt8) -> Int:
    var count = 0

    for index in range(len(data)):
        if data[index] == first:
            count += 1

    return count

This function is easy to trust. It checks one byte, updates the count, and moves on.

If the input contains eight bytes, the loop asks the same small question eight times:

Is this byte equal to the first byte of the term?

The important word is same. Every comparison is independent. The answer for byte 3 does not depend on the answer for byte 2.

That is the kind of work SIMD is built for.

SIMD does not remove the work. It presents the work to the processor in a shape the hardware can execute more efficiently.

A register with lanes

A SIMD value is one fixed-width value divided into lanes. Every lane contains the same type of element, and an operation is applied element by element.

For example, this value contains eight 16-bit integers:

var values = SIMD[DType.int16, 8](1, 2, 3, 4, 5, 6, 7, 8)
var squared = values * values

print(squared)
# [1, 4, 9, 16, 25, 36, 49, 64]

The multiplication looks ordinary, but its inputs are vector-shaped. Each lane is multiplied by the corresponding lane.

The type tells us two things:

SIMD[DType.int16, 8]

DType.int16 tells Mojo what each lane contains.
8 tells Mojo how many lanes the vector has.

So this is one SIMD value containing eight 16-bit integers. Both values are compile-time parameters. By the time Mojo generates the machine code, it already knows the element type and the number of lanes.

Mojo takes this model further than you might expect. Types such as Float32, Int64, and UInt8 are scalar aliases built on the same SIMD foundation. A scalar is a one-lane SIMD value.

So Mojo does not give us one disconnected model for scalar numbers and another for vectors. It gives us one model that grows from one lane to many.

Note: One vector-shaped operation does not mean one clock cycle. The compiler may map a value to one hardware register, several registers, or another efficient representation. A wider type is not automatically faster.

The useful width depends on the element size, the target CPU, memory behaviour, and the rest of the kernel. Vector width is something to measure, not assume.

The SIMD data path

Most useful SIMD kernels are not one clever instruction. They are a short pipeline.

The stages are separate for a reason:

Stage	Input	Vector work	Correctness rule
Load	Contiguous bytes	Read a complete vector	Never read beyond the buffer
Compare	First target byte	Produce one Boolean per lane	Use an element-wise comparison
Select	Boolean mask	Map `true` and `false` to `1` and `0`	Keep the mask inspectable
Reduce / Iterate	Candidate lanes	Combine lane values	A candidate is not yet a complete match
Verify	Candidate positions	Check the full term	Match the scalar reference

This is the architecture we will use for the ResearchLens project.

Why ResearchLens uses two algorithms

ResearchLens has one job: count exact ASCII terms in UTF-8 research text.

It uses two implementations:

A scalar implementation that is intentionally obvious.
A SIMD implementation that scans several possible starting positions at once.

The scalar version is not code that we keep around for nostalgia. It is the correctness oracle.

If the SIMD version and the scalar version disagree, the SIMD version is wrong.

The ResearchLens project has three areas:

Data layer

Research papers, reports, transcripts, and field notes become a contiguous byte buffer.

The first version is deliberately byte-oriented. UTF-8 text can contain multi-byte characters, but ASCII letters have the same byte representation inside UTF-8. That makes exact ASCII terms a useful, bounded starting point.

It does not make ASCII boundary rules a complete Unicode tokenizer. We will say that plainly instead of hiding it in a footnote.

Decision layer

The SIMD scan checks the first byte of the target term across a complete vector.

If the term is research, each byte equal to r becomes a candidate position.

That narrows the search. It does not prove that the complete word exists.

Verification layer

The verification layer includes a scalar function that checks every candidate:

Does the complete term fit inside the buffer?
Do all bytes match?
Is the byte before the term a word character?
Is the byte after the term a word character?

This is where the program distinguishes research from researcher, or art from partial.

SIMD narrows the search while the verifier owns the meaning.

Building the verifier

Before we write the vector loop, we need the function that defines a correct match.

def is_ascii_word_byte(byte: UInt8) -> Bool:
    return (
        (byte >= 48 and byte <= 57)
        or (byte >= 65 and byte <= 90)
        or (byte >= 97 and byte <= 122)
        or byte == 95
    )


def term_matches_at(
    data: Span[UInt8, _], term: Span[UInt8, _], start: Int
) -> Bool:
    if len(term) == 0 or start < 0 or start + len(term) > len(data):
        return False

    for index in range(len(term)):
        if data[start + index] != term[index]:
            return False

    if start > 0 and is_ascii_word_byte(data[start - 1]):
        return False

    var end = start + len(term)
    if end < len(data) and is_ascii_word_byte(data[end]):
        return False

    return True

This function is not vectorized, and that is fine because the SIMD scan should make the set of positions reaching this function much smaller than the full input. The verifier can then spend more work on the positions that deserve it.

It's a common performance flow to use a cheap parallel filter, then apply a more expensive exact check.

Keep the scalar reference obvious

Now we can write the reference implementation.

def count_term_scalar(data: Span[UInt8, _], term: Span[UInt8, _]) -> Int:
    if len(term) == 0 or len(term) > len(data):
        return 0

    var count = 0
    var limit = len(data) - len(term) + 1

    for start in range(limit):
        if term_matches_at(data, term, start):
            count += 1

    return count

The SIMD implementation has more moving parts, so we need a simple version to test it.

Load, compare, and inspect the mask

Here is the centre of the SIMD path in the src/analyzer.mojo file:

comptime SIMD_WIDTH = 32

var first = term[0]
var ptr = data.unsafe_ptr()

var chunk = ptr.unsafe_load[width=SIMD_WIDTH](offset)
var candidates = chunk.eq(
    SIMD[DType.uint8, SIMD_WIDTH](first)
)

The first line in the code to look at is:

var chunk = ptr.unsafe_load[width=SIMD_WIDTH](offset)

It loads 32 adjacent bytes from data, beginning at offset, into one SIMD[DType.uint8, 32] value.

The next line compares all 32 bytes with the first byte of the search term:

var candidates = chunk.eq(
    SIMD[DType.uint8, SIMD_WIDTH](first)
)

The result is a SIMD mask. Each lane tells us whether the corresponding byte could be the beginning of a complete match.

The name is deliberately honest. Mojo cannot prove that a raw pointer has 32 valid bytes available at this offset.

The loop condition must establish that contract before the load happens.

Then .eq() compares every lane with the first byte of the target.
The explicit method matters. In Mojo, == answers whether two complete SIMD values are equal and returns one Bool. The element-wise .eq() method returns a Boolean SIMD mask, which is what we need here.

Conceptually, the result looks like this:

bytes:       m  o  j  o     r  e  s  e  a  r  c  h
first byte:  r  r  r  r  r  r  r  r  r  r  r  r  r
mask:        F  F  F  F  F  T  F  F  F  F  T  F  F

The true lanes are candidate starting positions. We still have to verify the full term at those offsets.

The complete SIMD loop

The complete function adds the verifier and the tail:

comptime SIMD_WIDTH = 32

def count_term_simd(data: Span[UInt8, _], term: Span[UInt8, _]) -> Int:
    if len(term) == 0 or len(term) > len(data):
        return 0

    var count = 0
    var limit = len(data) - len(term) + 1
    var offset = 0
    var first = term[0]
    var ptr = data.unsafe_ptr()

    while offset + SIMD_WIDTH <= limit:
        var chunk = ptr.unsafe_load[width=SIMD_WIDTH](offset)
        var candidates = chunk.eq(
            SIMD[DType.uint8, SIMD_WIDTH](first)
        )

        for lane in range(SIMD_WIDTH):
            if candidates[lane] and term_matches_at(
                data, term, offset + lane
            ):
                count += 1

        offset += SIMD_WIDTH

    while offset < limit:
        if data[offset] == first and term_matches_at(data, term, offset):
            count += 1
        offset += 1

    return count

The loop does not reduce the mask to one number because we need the actual candidate positions. It reads each Boolean lane and sends only the true positions to the verifier.

Once the tests are stable and the benchmark tells us verification is the bottleneck, we have a precise place to improve.

The boundary is part of the algorithm

A document length is rarely divisible by 32.

Suppose 103 bytes are valid starting positions. The vector loop can safely process three complete 32-byte windows. That covers 96 positions. Seven remain.

An unchecked load at offset 96 would try to read 32 bytes when only seven valid positions remain. That is not a performance issue. It is an invalid memory access.

The loop guard makes the contract explicit:

while offset + SIMD_WIDTH <= limit:
    # A complete vector is safe here.
    ...

while offset < limit:
    # The remaining positions use the scalar path.
    ...

The code above is because limit is not greater than len(data), the first condition also ensures that the 32-byte unsafe_load stays within the input buffer.

The scalar tail is part of the algorithm. A vector can tell us that a lane contains r. It cannot, by itself, tell us that research is a complete word in the application’s tokenization model.
The fast path and the correctness path need a clean boundary between them.

Overview of the data shapes

When you read a SIMD kernel, asking “which line is fast?” is usually less helpful than asking “what shape does the data have here?”

The ResearchLens path is:

Load: 32 adjacent bytes become one typed vector.
Compare: 32 bytes become 32 Boolean answers.
Mask: the answers become inspectable candidate lanes.
Reduce or iterate: the vector result becomes a smaller scalar decision.
Verify: a candidate position becomes a correct application-level match or a rejection.

That is the mental model to keep. Each line changes the shape of the data.

Setting up the project in Mojo

We have discussed how the SIMD path works. Now let us set up the complete project and run it.

ResearchLens was built and tested with Mojo 1.0.0. You can write Mojo in VS Code, Windsurf, or another editor that supports the Mojo extension.

Install Mojo with Pixi

The official Mojo installation guide supports both uv and Pixi for creating and managing a Mojo project. We will use Pixi because it creates a reproducible project environment and generates a lockfile for the dependencies.

If Pixi is not already installed, run:

curl -fsSL https://pixi.sh/install.sh | sh

Restart your terminal after the installation if the pixi command is not immediately available.

Now create the project:

pixi init researchlens \
  -c https://conda.modular.com/max/ \
  -c conda-forge

cd researchlens
pixi add "mojo==1.0.0"
pixi run mojo --version

The final command should print the installed Mojo version:

Mojo 1.0.0 (ed45d567)

The value in parentheses, ed45d567, is the build identifier for this specific Mojo 1.0.0 build.

Mojo may have a newer release by the time you read this. We are using version 1.0.0 because that is the exact version used to build, test, and benchmark this project.

Create the project files

ResearchLens has a small structure:

researchlens/
├── pixi.toml
├── pixi.lock
├── README.md
├── data/
│   └── sample.txt
├── src/
│   └── analyzer.mojo
└── tests/
    └── test_analyzer.mojo

Create the directories:

mkdir -p data src tests

Add the source code to src/analyzer.mojo, the tests to tests/test_analyzer.mojo, and a small research-text sample to data/sample.txt.

The complete project is available in the ResearchLens repository.

Configure the Pixi tasks

Open pixi.toml and use the following configuration:

[workspace]
authors = ["Your Name"]
channels = ["https://conda.modular.com/max", "conda-forge"]
name = "researchlens"
platforms = ["linux-64", "linux-aarch64", "osx-arm64"]
version = "1.0.0"

[dependencies]
mojo = "==1.0.0"

[tasks]
analyze = "mojo run src/analyzer.mojo"
test = "mojo run -I src tests/test_analyzer.mojo"

The two tasks give us shorter commands:

pixi run test
pixi run analyze data/sample.txt research 100

Pinning the Mojo version matters. It means another engineer can clone the repository, install the same dependencies, and reproduce the environment we tested.

Commit pixi.lock to the repository as well. The configuration describes the environment, while the lockfile records the exact packages Pixi resolved.

Test the scalar and SIMD paths together

A fast implementation is only useful if it returns the correct answer.

Instead of testing the scalar and SIMD functions separately, we give both functions the same input and confirm that they return the same match count.

Open tests/test_analyzer.mojo:

from std.testing import assert_equal
from analyzer import count_term_scalar, count_term_simd


def check(text: String, term: String, expected: Int) raises:
    var data = text.as_bytes()
    var needle = term.as_bytes()

    assert_equal(count_term_scalar(data, needle), expected)
    assert_equal(count_term_simd(data, needle), expected)


def main() raises:
    check("", "research", 0)
    check("research", "research", 1)
    check("research research", "research", 2)
    check("researcher research", "research", 1)
    check(
        "A long prefix that crosses a vector tail: research",
        "research",
        1,
    )

    print("ResearchLens: all scalar/SIMD agreement tests passed")

Run the tests:

pixi run test

The result should be:

ResearchLens: all scalar/SIMD agreement tests passed

Notice that check is marked with raises:

def check(text: String, term: String, expected: Int) raises:

Some of the operations called inside check may return an error. Mojo requires the function to make that possibility explicit.

Without raises, the compiler reports that a function which may raise is being called from a context that cannot raise. We encountered this error while building ResearchLens, and adding raises fixed it.

Run the analyzer

With the tests passing, run ResearchLens against the sample corpus:

pixi run analyze data/sample.txt research 100

The three arguments are:

Argument	Meaning
`data/sample.txt`	The text corpus to search
`research`	The term to find
`100`	The number of benchmark iterations

On the machine used for this tutorial, the command produced:

bytes: 244
matches: 3
iterations: 100
scalar total ns: 64000
SIMD total ns: 19000

ResearchLens found three complete occurrences of research. It then ran both implementations 100 times and measured their total execution time.

For this run, the SIMD implementation was approximately 3.37 times faster than the scalar implementation.

Build the executable

So far, Pixi has been running the Mojo source file directly. We can also compile ResearchLens into a standalone executable:

pixi run mojo build src/analyzer.mojo -o researchlens

Run the compiled application:

./researchlens data/sample.txt research 100

On the Apple Silicon machine used for this tutorial, Mojo produced an 86 KB Mach-O arm64 file.

You do not need a GPU for this project. We are using CPU SIMD. The vector hardware is already inside a modern processor, and Mojo gives us a way to express work that can run across those vector units.

Measure the complete application

A vector width of 32 does not exactly mean a 32-times speedup.

The program still has to:

Load the bytes from memory.
Find candidate positions.
Verify each candidate.
Handle the final partial vector.
Count the valid matches.

The compiler may also divide a wide SIMD value into smaller hardware operations. The final performance depends on the processor, the data, the compiler, and the way the algorithm is written.

That is why we benchmark the complete application instead of timing only the most impressive SIMD instruction.

For a useful comparison:

Run the scalar and SIMD implementations on the same bytes.
Confirm that they return the same match count.
Keep compilation and one-time setup outside the timed section.
Run enough iterations to reduce timing noise.
Record the CPU, operating system, Mojo version, corpus size, search term, vector width, and iteration count.

Here are the results from the completed ResearchLens project:

Corpus	Iterations	Matches	Scalar total	SIMD total	Speedup
244 bytes	100	3	64,000 ns	19,000 ns	3.37×
10,485,996 bytes	100	127,878	2,745,454,000 ns	856,933,000 ns	3.20×
104,857,746 bytes	20	1,278,753	5,531,949,000 ns	1,732,811,000 ns	3.19×

The 10 MB and 100 MB tests settled at almost the same ratio: approximately 3.2 times faster for this workload on this machine.

That consistency is more useful than the result from the 244-byte sample. With such a small file, timer noise and fixed overhead account for a larger part of the measurement.

The 100 MB result does not mean Mojo will make every program 3.19 times faster. It means this implementation measured 3.19 times faster with this:

Search term
Corpus
Vector width
Mojo version
Processor
Number of iterations

Change any of those variables and the result may change.

The data also matters

ResearchLens first uses SIMD to find bytes that could begin a match. It then sends those candidate positions to the scalar verifier.

We can describe that relationship with:

candidate ratio =
    candidate starting bytes / valid starting positions

If the search term begins with an uncommon byte, the SIMD filter can reject most positions cheaply.

If it begins with a common byte, more positions reach the scalar verifier, and verification takes a larger share of the execution time.

The value of the optimization therefore depends partly on the data being searched.

Next steps

SIMD is useful when the same small operation must be applied to many independent values.

Mojo makes the parts of that work visible in ordinary code:

The element type
The lane count
The vector load
The comparison mask
The reduction
The scalar verifier
The tail

We can inspect each part, test it, and measure it.

ResearchLens is a small application, but the same pattern appears in delimiter scanning, byte classification, image processing, numerical transformations, filters, checksums, and many other workloads.

The goal is not to vectorize everything.

It is to notice when the computer is being asked the same small question millions of times.

If you build your own version of ResearchLens, try changing the vector width or the distribution of the first byte and share in the comments.

P.S: If you want to learn specifically about SIMD, read Mitchell Hashimoto's article.

Mojo is an innovative, high-performance programming language designed for writing systems-level code for AI workloads. It also comes with a fire emoji file extension .🔥— literally. Learn more about Mojo in this article : -)

Ekemini Samuel — Wed, 30 Sep 2026 09:38:39 +0000

Ekemini Samuel

Aug 11

What is Mojo?

#mojo #ai #programming #software

11 min read

We Built a DevRel Knowledge Base

Ekemini Samuel — Fri, 11 Sep 2026 11:28:29 +0000

Developer Relations & Advocacy is often described as the work of helping developers understand and use a product.

That description is correct, but there is more. In this blog, we will share what we learned while turning a community Developer Relations guide into a structured documentation site with Thally.

A Developer Advocate might write a technical tutorial, deliver a conference talk, review an onboarding journey, support a community, build a sample application, or take recurring developer feedback back to a product team. The work connects people, products, education, and experience.

This range is part of what makes Developer Relations exciting. It is also what makes the field difficult to learn.

Useful knowledge exists across conference talks, community conversations, personal blogs, company playbooks, videos, and the experience of practitioners. Someone entering the field can find definitions easily, but finding a dependable path from understanding DevRel to doing the work is much harder.

Those are some of the problems the DX Guide provides solutions for.

Why we built the DX Guide

The DX Guide is a community-built knowledge base for people learning, practising, or leading Developer Relations.

We wanted to create a resource that did more than define common terms. It needed to help readers understand the work, apply practical frameworks, learn from real community examples, and find useful next steps.

The DX Guide started in 2025 as a project within the DXMentorship cohort. We built its first version with Mintlify, and members of the DX community in that cohort contributed their knowledge and experience to the guide. That beginning shaped the project as a community resource. It was never meant to represent one person's view of Developer Relations.

The first version gave us a valuable foundation. As the material grew, we also saw opportunities to improve how readers moved through it, how contributor work was credited, and how the documentation could be reviewed and maintained. That brought us to the next stage of the guide and, eventually, to Thally.

The guide covers areas such as:

Developer Advocacy
Developer Experience
technical content and documentation
public speaking
community building
feedback collection
measurement and reporting
careers and professional development
real examples and lessons from practitioners

The aim is not to establish one permanent definition of DevRel. The field changes, and different organisations structure the work differently. The guide gives the community a shared place to document what people are learning and improve it through review and contribution.

Why we moved the documentation to Thally

The original documentation contained useful material, but having content is not the same as having a dependable documentation experience.

Some pages repeated their introductions. Navigation needed a clearer structure. Contributor examples needed context and attribution. Product references needed official links and recognisable identities. The site also needed a consistent visual system and a workflow the team could continue to maintain.

The SYNCHACK challenge gave us a practical reason to address those issues through Track 2, which focuses on launching or migrating a documentation project.

We chose Thally because we needed more than a place to display Markdown files.

We wanted a publishing foundation that could support structured navigation, reusable content, search, deployment, and agent-facing ways to discover the same source material.

Our project now contains MDX pages organised into topic-based navigation groups. It includes practical guidance, contributor stories, event photographs, professional links, product resources, and a changelog.

The Books page is one example of the broader goal. It is not an isolated list added to increase the number of pages. It gives readers a route from the guide into deeper learning.

Migration was only the beginning

At first, documentation migration can look like a file-moving task.

Move the Markdown.
Recreate the navigation.
Add the logo.
Deploy the site.

In practice, the more important work was editorial.

We reviewed the guide page by page. We removed repeated introductions and duplicated sections. We replaced emoji headings with a consistent icon system. We revised indirect language so organisations and contributors were described clearly. We checked links, image context, headings, descriptions, and related-page paths.

We also tried to make every example earn its place.

A photograph is not useful simply because it looks good. It becomes useful when the surrounding text explains what a reader can learn from it without inventing details that were never verified.

Joy Ndukwe's DevFest photograph supports the guide's discussion of public speaking and community work. We use the information we can verify and avoid assigning an unconfirmed session title or outcome to the image.

The same principle applies to project and community examples. In the photo above, Ekemini Samuel is presenting a talk at Mojo Africa Community event.

Real work makes the guide more concrete, but accurate context matters more than promotional language.

Designing a technical knowledge base

We wanted the DX Guide to feel technical without feeling cold.

We selected JetBrains Mono as the primary typeface. It gives prose, headings, commands, and examples a consistent voice that feels familiar to developers. We paired it with a restrained colour system built around deep green, lime, and violet.

The DX mark gives the guide a recognisable identity, while the interface keeps the documentation itself in focus.

The image below shows the LinkedIn and Personal Branding chapter in the DX Guide

The technical challenge we did not expect

The production deployment did not work perfectly on the first attempt.

A build failed because the synchronized application expected runtime modules that were missing from the repository snapshot. The error appeared across documentation, search, API, and well-known routes.

Thally provides clear insights into the documentation deployment pipeline

The solution was not to copy a quick runtime fix into the documentation project. Thally's runtime and the authored documentation have different ownership. The repository received a runtime synchronization, and the related static-route tests were aligned with that version.

After the update, the project checks, tests, and production build passed.

This was one of our most useful lessons. Documentation teams need to understand where content responsibility ends and platform responsibility begins. A workaround in the wrong repository can solve today's build while creating a harder maintenance problem tomorrow.

What Thally helped us do

Thally gave the guide a structured route from authored content to published documentation.

It helped us bring together:

MDX content
organised navigation
reusable page components
search and publishing surfaces
deployment checks
agent-facing documentation surfaces

It also gave us a practical environment for reviewing the documentation as a product rather than treating it as a folder of files.

With the analytics, we have insights into the DX documentation, knowing what to improve as we grow

The workflow worked best when automation moved the work forward and people remained responsible for what was published.

What we learned

The biggest lesson was simple: documentation migration is also editorial work.

A successful build does not automatically produce clear documentation. A large page count does not guarantee useful coverage. Search cannot repair an unclear information structure. A polished design cannot replace evidence and attribution.

We learned to ask better questions:

What is the reader trying to accomplish?
Does this page provide a useful next action?
Is this example supported by evidence?
Have we distinguished verified facts from assumptions?
Can another contributor understand and maintain this structure?
Does the downloaded or deployed result behave like the preview?

We also learned that a living guide needs visible maintenance. The changelog, contribution guide, repository history, and review process are part of the documentation experience.

Here is our Changelog in the DX documentation

Building with AI assistance

We used AI assistance during parts of the research process. Google, ChatGPT and Claude helped us inspect patterns, investigate errors, and support validation workflows as we worked through the documentation.

We also used the Thally documentation to research the platform, confirm technical details, and understand the workflow required for the migration.

What comes next

The DX Guide is a living documentation.

We want to continue improving the practical paths through the content, expanding contributor examples with permission, checking the guide's agent-facing surfaces, and making it easier for new community authors to participate.

We would love to see people use the guide to understand a DevRel problem, complete meaningful work, and contribute a lesson that helps the next person.

Explore the guide: advorel.thally.app

Here's a video guide of our DX documentation:

What would make a community Developer Relations guide more useful in your day-to-day work? Please share in the comment!

A big thank you to the SyncHack and Thally team! 🎉

And thanks to Joy, Samuel, Adedotun and Ekemini for working together as we did the migration of the DX documentation to Thally.

Thank you for reading! 💜

Enterprise AI governance is not only about writing the rules. It is about enforcing them once AI traffic starts spreading across teams, applications, models, providers, and agents. That is why an AI gateway matters, giving you a place to see the traffic,

Ekemini Samuel — Tue, 01 Sep 2026 13:03:27 +0000

Ekemini Samuel

Aug 24

Enterprise AI Governance: Governing LLM Traffic at Scale with an AI Gateway

#aigateway #llm #mcp #ai

9 min read

Enterprise AI Governance: Governing LLM Traffic at Scale with an AI Gateway

Ekemini Samuel — Mon, 24 Aug 2026 14:58:04 +0000

Your AI governance problem probably isn't a policy problem; it’s a routing one.

Let’s paint a common scenario: Your team uses OpenAI, another adds Anthropic, then the product team connects another model. Then someone builds an agent with MCP tools. A few months later, the company has several AI applications, multiple providers, and a growing bill.

The question is: who can use which model, how much can they spend, and what happens when a provider fails?

That is when AI governance stops being a policy document and becomes an infrastructure problem.

In this article, we'll look at why enterprise AI governance gets harder as LLM traffic grows, where an AI gateway fits into the architecture, and how Bifrost can govern access, routing, budgets, rate limits, observability, and AI agent tools.

Bifrost is an open-source AI gateway built in Go by Maxim AI. It sits between your applications and AI model providers, routing requests through a unified API instead of making each application manage every provider separately.

Let’s get started.

Why enterprise AI governance becomes an infrastructure problem

The first generation of enterprise AI was mostly about choosing models. The problem is different now.

LLM traffic is not like ordinary API traffic. Cost depends on tokens. Latency varies by model and provider. A single agent can make many model and tool calls. Providers have different limits and capabilities. Teams may want different models for different jobs.

So the traffic itself becomes something the platform team needs to manage.

At scale, the same request pattern repeats across applications, teams, providers, and agents. Governance gets difficult simply because the system grows.

The problem with governing AI in every application

Every AI request an enterprise makes needs an answer to questions like: who is allowed to make it, which model or provider it's allowed to use, and whether anyone can find a record of it afterward.

That's what "governing AI traffic" means in practice: not a policy document, but those questions getting answered correctly, every time, for every request.

The first move might be to answer them inside each application. Give the app its own provider keys, its own retry logic, its own spending limits, its own logging. For one team, on one service, this works fine.

It stops working once there are twenty teams doing it. Each one solves the same questions once, at different times, in a slightly different way, because nothing forces any two of them to do it the same way.

So when a provider changes its API, or a new model gets approved, or security wants a new rule, that's not one change anymore. It's twenty changes, made by twenty people, on twenty schedules, and every application has quietly become part of the governance system whether it was built for that job or not.

The limit is not the volume of traffic, but not having a single place where any of this policy lives. That's the gap an AI gateway is built to close.

The AI gateway as the control plane

A better way to go about handling different requests, as shown in the diagram above, is to route applications and agents through an AI gateway connected to the AI model providers.

The gateway is the control plane where identity, access, routing, budgets, rate limits, guardrails, and observability can be applied consistently.

It’s not simply having another proxy; rather, it’s having a place where policy meets the request while the request is happening.

A typical request can move through:

Identity: who is making the request, usually a Virtual Key tied to an app, team, or agent.
Access policy: which providers, models, and tools that identity is allowed to use.
Budget and rate limit: whether the request fits within the spending and volume limits set for it.
Routing: which provider and model should actually handle the request.
Guardrails: a check for policy violations or sensitive data before anything reaches the user.
Provider: the model provider, like Groq, OpenAI, or Anthropic, that generates the response.
**Observability: **a record of the request, response, cost, and outcome for later audit.

If a check fails, the request stops there, emphasizing the principle of one governed path, where the idea is to effectively manage traffic along its existing route.

What should an AI gateway govern?

When traffic passes through one layer, the governance questions become much easier to answer. You can start with something as simple as knowing who is making a request. That might be a user, an application, a team, or a Virtual Key. From there, you can decide what that caller is allowed to access and keep its usage separate from everyone else's.

The same applies to models and providers. Not every application needs access to everything. A support application might only need a few approved models, while an internal agent might need access to a different set.

There is also the question of how much traffic each application should generate. Budgets can keep spending under control, while request and token limits prevent a single application from consuming more than its share. The Budget and Limits controls let those limits be applied at different levels.

With several models and providers in the mix, the gateway also needs to decide where each request should go. Routing can direct traffic to particular providers or models, distribute requests, and provide a fallback when something goes wrong.

Some requests should never make it that far. Guardrails can inspect what is going into and coming out of a model and block requests that violate the rules.

And after the request has been handled, you still need to understand what happened. How much traffic did an application generate? Which model handled it? How long did it take? Did a provider fail? Good observability turns those questions from guesses into things you can actually answer.

That is what governance looks like at the traffic layer: knowing who is making the request, what they can do, how much they can use, where the request goes, and what happened along the way.

Agents make the governance problem bigger

As AI agents get more development and adoption, the governance challenge gets bigger. At first, the question was simple: "Can this application call GPT?" But for agents, that's just the start.

These agents can do a lot more than just call a model. They can read files, query databases, search internal systems, use Git, and call other tools through the Multi-Channel Protocol. Because of this, governance needs to include control over tool access as well.

For example, a research agent might only need to read files but shouldn’t be able to delete them. On the other hand, an engineering agent might need access to Git.

Bifrost’s MCP Tool Filtering lets you set up a Virtual Key. This key can explicitly allow certain clients and tools to be accessed. If you don’t set up any MCP configuration, the default is to deny access to everything; this keeps things secure.

So instead of asking, "Which model can this application use?" we need to ask, "What can this AI system actually do?" This shift means we have to think more about how to govern these agents and what permissions they really need to work safely and effectively.

Cost and reliability belong in the same layer

The gateway is also where governance meets business economics. If one team's agent suddenly makes ten times as many requests. With gateway-level budgets and usage data, you can see which key, team, provider, or workflow caused the increase.

The same layer can handle reliability. Bifrost's routing controls can distribute requests across configured provider or model paths, which also caters to any fallback behavior when a provider path is no longer available.

Governance is not only about saying no. It also keeps approved traffic reliable.

The governed AI traffic layer

This is where Bifrost comes in as an open-source, high-performance AI gateway. It gives applications a unified API and interface to connect across the AI model providers.

To learn more about the engineering story behind Bifrost, watch the Runtime episode on YouTube. It covers why the team built Bifrost in Go, the unified API, routing and failover, observability, MCP, and the infrastructure decisions behind a high-throughput AI gateway.

Let's put this into practice

We built a Customer Support Assistant using Next.js and Groq for AI inference. The application does not call a provider directly. Its baseURL points to Bifrost, and the Bifrost Virtual Key identifies the workload, with the API key credential entered in the Bifrost dashboard.

The Customer Support application only sends chat requests through the AI gateway.

Also, the governance policy includes a request and token limit, a monthly budget, and an approved model list configured in a config.json file and also managed in the Bifrost dashboard.

The Bifrost Gateway dashboard runs at http://localhost:8080

The Virtual Key Rankings attribute requests, tokens, and cost to the Customer Support Assistant workload.

In the image above, the enterprise-support-assistant Virtual Key accounts for all 35 attributed requests and $0.0019 spent. This observability enforces policy and gives finance and platform teams a shared view of LLM usage.

The Virtual Key Rankings tab also shows how Bifrost attributes traffic back to each workload. In this demo, every request from the Customer Support Assistant arrives with the Virtual Key, and Bifrost maps that key to the workload name, rolls up requests, tokens, and costs in one place.

From the LLM logs tab in the dashboard, we can view total requests, success rate, average latency, total tokens, and cost, along with a log and timestamp for every request the application sent.

An example of a policy block is the HTTP 429 code when the rate limit has been exceeded. This enforcement is done from the AI gateway before the request is sent to the AI provider.

When an AI model is allowed in the Bifrost configuration, it shows an HTTP 403 message, as shown below.

The LLM logs also show the 403 error in the dashboard, with the Groq logo displayed like so:

You can also view the AI model rankings in the Bifrost dashboard and export the data when needed.

For the Customer support application use case, this shows which AI models have been used for the responses.

From the image, we can see that there are two models in use:

openai/gpt-oss-20b
openai/gpt-4o-mini

It also shows the requests on each model and the success rates, tokens used, costs, average latency, and throughput. This gives more insights into the AI models, which help with better understanding and decision-making.

As shown in the Customer support application above, Enterprise governance is enforced centrally, in the request path. The LLM traffic flows through one gateway; budgets and limits control growth; model and tool access can be restricted; routing can keep approved traffic moving; and requests remain traceable.

The LLM provider (for instance, OpenAI, Anthropic) the application needs is defined through a provider configuration. Then the application gets its own identity with a Virtual Key, which gives a clear boundary for access and usage.

From there, you can put required limits around the traffic. The budgets and rate limits help keep an application or team from using more than it should.

As the system grows and you add more models or providers, and introduce routing so requests can go to the right place, with fallback when an LLM provider is unavailable.

And once the application becomes an agent, MCP Tool Filtering lets you decide which tools that agent can actually access.

You can get started with one application and one provider, then add the controls as the traffic and the system grow.

Next steps

Enterprise AI governance is not only about writing the rules. It is about enforcing them once AI traffic starts spreading across teams, applications, models, providers, and agents.

That is why an AI gateway matters, giving you a place to see the traffic, apply policies, control access and spend, route requests, govern agent tools, and keep a record of what happened.

Instead of asking every application to solve these problems on its own, you can handle them at the layer every request already passes through, and your organization can scale AI securely without slowing innovation.

To get started with Bifrost, check out the documentation.

What is Mojo?

Ekemini Samuel — Tue, 11 Aug 2026 19:45:23 +0000

The Collins Dictionary defines Mojo as personal power or influence over people. Other definitions describe it as confidence, energy, self-belief, or the ability to make things happen.

Mojo is an interesting name for a programming language, and that was the first thing that made me curios about Mojo when I discovered it while doing some research.

In this article we are referring to the Mojo programming language developed by Chris Lattner and the Modular team. Mojo was launched on the 2nd of May 2023, and celebrated it's 3rd birthday this year.

If you've heard about Mojo (or not) but aren't quite sure what it is, or why people are excited about it, this article is for you.

Let's begin!

Mojo (the programming language)

The Modular team defines their vision for Mojo like so:

Our vision for Mojo is to be the one programming language developers need to target diverse hardware—CPUs, GPUs, and other accelerators—using Python's intuitive syntax combined with modern systems programming capabilities.

For a while now, Software engineers building AI systems had to choose to either code in Python for simplicity or in C++ and CUDA for performance. With Mojo, those trade-offs don't have to be made.

In simple terms, Mojo is a high performance Python superset programming language powering the next generation of AI infrastructure and making GPU programming accessible to everybody.

This aligns with Modular's mission:

Enable AI to be used by anyone, anywhere

Mojo is an innovative, high-performance programming language designed for writing systems-level code for AI workloads. It also comes with a fire emoji file extension .🔥— literally.

You can learn more about GPU Programming with Mojo in this video:

💡 The easiest way to think about Mojo is this:

Mojo keeps what software engineers love about Python while making it capable of building the next generation of AI infrastructure, and giving the engineers more control over how their programs run.

Why Mojo?

Today, millions of people use AI every day.

Whether it's ChatGPT helping someone write an email or designing a flyer, Windsurf or Claude Code assisting software engineers, AI is gradually becoming part of everyday life.

Just as social media became mainstream over the last two decades, AI is becoming part of how we work, learn, communicate, and build. Which leads to an important question: What powers all of these AI systems?

Behind every chatbot, recommendation engine, image generator, AI Agent or autonomous system is an enormous amount of infrastructure working together.

There are:

Massive data centers filled with GPUs.
Servers processing billions of requests.
Frameworks serving AI models in real time.
Programming languages that tell all of this hardware exactly what to do.
People across different fields of work collaborating from different locations in the world.

This is AI infrastructure, the people, software, and hardware foundation that makes AI technology possible.

As AI continues to grow, building this infrastructure efficiently becomes just as important as building smarter models.

Beyond the intelligence of the AI models, we have to consider how efficiently and quickly we can train and deploy the models to fully utilize AI hardware.

Achieving this requires new tools and frameworks such as Mojo and MAX (Modular Accelerated Xecution).

Mojo is a programming language designed for AI technology.

It was created by Modular to make it easier for engineers to build software that takes full advantage of modern AI hardware; TPUs, NPUs, GPUs, like the NVIDIA H100, while keeping the simplicity that made Python one of the world's most popular programming languages.

Here's an example of a SIMD-vectorized kernel squaring array elements in place in Mojo

# SIMD-vectorized kernel squaring array elements in place.

def mojo_square_array(array_obj: PythonObject) raises:
    comptime simd_width = simd_width_of[DType.int64]()
    var ptr = array_obj.ctypes.data.unsafe_get_as_pointer[DType.int64]()

    def pow[width: Int](i: Int) {mut ptr}:
        var elem = ptr.load[width=width](i)
        ptr.store[width=width](i, elem * elem)

    vectorize[simd_width](len(array_obj), pow)

You don't need to understand every line of this example yet. The important thing is to see what Mojo is trying to make possible: a programming experience that remains approachable while giving developers much more control over how their code interacts with the underlying hardware.

Then we have MAX, which is Modular's AI framework, designed specifically around high-performance AI inference and deployment. It provides the tools needed to serve generative AI models and deploy them efficiently across different environments, while Mojo provides the programming language and systems-level capabilities underneath the stack.

You can think about it simply: Mojo gives you the programming language. MAX gives you the framework for running AI efficiently.
Together, they are part of Modular's broader vision for building a more unified AI software stack.

MAX is the framework that is natively integrated with Mojo, and is designed specifically for inference rather than training like PyTorch. It delivers incredible performance for serving GenAI models, deploying and scaling with ease.

You can learn more about MAX in the video by Chris Lattner:

Together, Mojo and MAX form an important part of Modular's approach to building modern AI infrastructure. And the story around Modular itself has recently entered a new chapter.

On July 29, 2026, Qualcomm announced that it had completed its acquisition of Modular. Modular is now a Qualcomm company, while Mojo, MAX, and Modular Cloud continue as products and brands. According to Modular and Qualcomm, the acquisition is intended to combine Modular's AI-native software platform with Qualcomm's high-performance, energy-efficient computing platforms and expand the reach of Mojo and MAX across CPUs, GPUs, NPUs, and other hardware architectures.

This is an important development for anyone building with Mojo. The technology is no longer being built only by an independent AI infrastructure startup. It is now part of a much larger computing company, while the Modular team continues its work on the software stack.

And for me, that makes the question even more interesting:

Why did Modular decide that the AI industry needed a new programming language in the first place? We will answer this in the article.

Where did Mojo come from?

To understand the origins of Mojo, we must first explore the problem that Modular aimed to solve. Founded in 2022 by Chris Lattner and Tim Davis, Modular sought to create a more unified, efficient, and user-friendly AI infrastructure across various types of hardware.

As the team delved into this challenge, they discovered that the programming layer itself contributed to the issue. AI developers commonly use Python for its productivity and ease of use. However, when they needed to optimize performance and gain closer access to hardware like CPUs and GPUs, they were often forced to switch to lower-level languages such as C++ and CUDA.

This duality created a productivity problem in the AI industry, as developers worked with different tools for performance and ease of use. Modular decided to address this by rethinking the programming language altogether.

What if developers could retain the familiarity and productivity of Python while also gaining the performance and hardware control necessary to construct modern AI infrastructure? This innovative concept led to the development of Mojo.

The thought process behind Mojo is strongly influenced by Chris Lattner, who has a remarkable background in the technology field. He is well-known for creating LLVM and Clang, which are foundational components of modern compiler infrastructure. He also created the Swift programming language, used by millions of Apple's iOS devices.

Mojo was developed as part of Modular's broader effort to making AI and AI infrastructure accessible, bringing together a team with deep experience in programming languages, compilers, systems, and modern hardware.

How to learn Mojo

As with every skill and programming language, it starts with Why.
Why do you want to learn a new programming language? If you've read this far into the article, probably you have an interest in the Mojo Programming language, that's great!

The Modular team is currently running a Mojo 101 series, which started on the 9th of July (my birthday)😎

It's a great way to learn the basics of Mojo and how to get started building with Mojo. Check out the Mojo 101 series here:

Thanks to the Modular team for this series! 😎

There's also a talk by Chris Lattner on how Mojo & MAX contributes to a modern AI-first software development ecosystem

You can explore the official Mojo documentation and start experimenting with the language yourself.

You don't need to understand everything before you start, just get started, one day at a time.

Mojo Africa

Mojo Africa is a platform and community, where we are building the future of AI infrastructure in Africa, and educating on Mojo, the programming language for AI, GPUs, and MAX, the next-generation AI framework.

We launched in March 2025, and in 16 months, we've had two Mojo Africa meetup events, with sponsorship from the Modular team.

In August 2025, we had our first Mojo Africa meetup, where we introduced Mojo to the community.

We had our second Mojo Africa meetup in May 2026, where we learned and discussed the Role of Mojo and Max in building and scaling AI infrastructure. Great energy at the Mojo Africa meetup! ✨

Mojo Africa is about more than teaching Africans about Mojo. We want to help redefine the narrative.

Africa should not only consume AI and technology. We should also be actively contributing, building AI projects, developing AI infrastructure, and creating the technologies that will shape the future.

We are building Mojo Africa to create an environment where Africans can learn, experiment, contribute, and build the AI technology and infrastructure of the future. And it starts with Mojo.

That is the vision. Join the community, more updates are coming.

Why Now?

If there was ever a time to pay attention to Mojo and AI infrastructure, I believe it is now.

AI is moving incredibly fast, and the conversation is gradually shifting from simply building better models to figuring out how we can run those models efficiently, across different hardware and at much larger scale.

Imagine a company collecting millions of data points from sensors, devices, or other systems. They may want to process that data, run models against it, and generate insights in real time. The performance of the software processing that information can become just as important as the model itself. Mojo is built to excel at use cases like this.

Modular has also reached some major milestones.

In September 2025, Modular raised $250 million in funding, bringing its total capital raised to $380 million and its valuation to $1.6 billion. Then, in July 2026, Qualcomm completed its acquisition of Modular, bringing the company, Mojo, and MAX into the Qualcomm ecosystem, with Chris Lattner taking on the role of EVP of Advanced AI Software and Platforms and Tim Davis becoming SVP and GM of Modular.

And now, another major milestone has arrived. Today, 11th August 2026 as I write and publish this article, Mojo 1.0 is now officially here 🔥, Modular announced it today!

Mojo 1.0 provides a stable, production-ready foundation.

After years of development and contributions from the community, Modular has officially released Mojo 1.0 as a stable foundation for developers building with the language. Nearly 200 contributors have landed more than 1,100 pull requests affecting over 200,000 lines of code since the standard library was open-sourced, with more than 1,000 people also contributing suggestions, issues that helped shape the language.

For me, this is significant because 1.0 changes the conversation.

Mojo is no longer simply an interesting programming language to watch while it is being developed. Developers can now build on a more stable foundation, with Modular committing to primarily additive changes throughout the 1.x era and carefully managing breaking changes.

Also the Modular team mentioned during the Ask Us Anything: Modular and Qualcomm that the Mojo compiler and toolchain will be progressively open-sourced in 2026.

There is more coming.

Modular's ModCon 2026, happening on August 18 in San Francisco, is focused on the next frontier of AI infrastructure, with major announcements and updates around Mojo, MAX, open source, hardware, and the broader AI ecosystem.

This is why I think the timing matters.

The technology is evolving quickly. The company behind it has entered a new chapter with Qualcomm. Mojo has reached 1.0. More of the stack is moving toward open source. And developers around the world are beginning to build on top of it.

We are still early.

And that is exactly why I believe now is a good time to learn, experiment, contribute, and build.

For anyone interested in the future of AI infrastructure, this is a good time to start paying attention.

My Thoughts

AI is becoming more powerful, but that also means the computing infrastructure underneath it has to become more efficient. We need software that can take advantage of increasingly diverse hardware without making developers rewrite everything every time the hardware changes.

Learning Mojo does not mean you have to stop using Python, C++, CUDA, or any other language you already know.

For me, the value is in understanding another part of the stack.

The more I learn about AI, the more I realize that the future will not only be about using AI tools. There will be enormous opportunities for people who understand how these systems are actually built.

And that is why we started Mojo Africa.

I want to learn, build, and create with other people who are interested in this future.

Next Steps

If you've read this far, I hope you now have a better understanding of what Mojo is, why it exists, where it fits into AI infrastructure, and why I believe it is worth paying attention to.

The next step is simple: try it.

Start with the Mojo 101 series, explore the official Mojo documentation, and write your first Mojo program.

And if you want to learn and build with other people, join Modular's Discord community.

Thank you for reading ❤️. You can connect with me on X or LinkedIn.

Clone MedTalk: HIPAA-Ready Video and Chat Consultations in Flutter

Ekemini Samuel — Mon, 12 Jan 2026 21:28:52 +0000

Telehealth is transforming the way patients and providers connect, offering faster access to care and reducing barriers caused by distance or scheduling. A critical part of this experience is enabling secure, real-time video consultations alongside features like chat messaging for sharing updates, questions, and follow-ups.

With Stream's healthcare chat solution, developers can build HIPAA-ready communication features into their apps.

In this tutorial, you'll learn how to clone MedTalk, a telehealth platform designed for seamless communication between doctors and patients. We'll use Flutter Web to deliver a responsive cross-platform experience and Stream's Chat & Video SDKs to power HIPAA-ready messaging and video calls.

Here’s a demo video of the complete application:

Prerequisites

A free Stream account and API key
VS Code
Android Studio
Flutter SDK installed on your computer

Set Up Your Development Environment

To install Flutter on your computer, first open flutter.dev.

Note:

If you already have your development environment set up, you can clone the project and get started quickly.

git clone <https://github.com/Tabintel/medtech.git>

cd medtech

Run this command to install the dependencies needed for the app:

flutter pub get

If not, then continue with the steps below.

First, download the version compatible with your computer’s operating system: Windows, macOS, Linux, or ChromeOS.

Then choose the type of app you’ll be building. For this tutorial, we’re building a Flutter web application, so select Web from the list.

After selecting the type of app you’re building, the next step is setting up the requirements to begin building with Flutter. Follow the steps in the Flutter documentation to proceed.

If you’re using another operating system, follow the guide for your OS.

Specific requirements for a Windows OS are:

Use Git for Windows to manage Flutter versions and your source code versioning.
Google Chrome to debug JavaScript code for web apps.
Visual Studio Code with the Flutter extension for VS Code.
Also, note that we are installing Flutter with VS Code.
Once you meet all these requirements, you’re ready to start building.

Install the Flutter SDK

In your VS Code editor, run the Ctrl + Shift +p command.

Then select Download when this window shows.

After downloading and installing the Flutter SDK, you will choose the type of Flutter template you want to install.

For this project, select Application as shown below.

Now that we have the Flutter SDK, we can proceed to further develop the Virtual Health consultation platform with Stream and Flutter.

If you need any clarification, read the Flutter docs.

To confirm that you installed Flutter correctly, run flutter doctor -v in your command prompt terminal.

It should print out a similar response like so:

Follow these steps to set up the project for building with Flutter Web.

Create a medtech folder on your computer, and open it in the VS Code editor. In your command prompt, run:

flutter create . \--platforms web

The output then displays like so:

It creates the structure for the Flutter web project. It also makes a web/ directory containing the web assets used to bootstrap and run your Flutter app, like so:

In the following steps, we will look at how to set up Stream for the virtual health consultation platform with Flutter SDK.

Get Started with Stream

First, create an account in the Stream dashboard.

Then click on Create App to create a new application.

Enter the details of the application.

After creating the Stream app, go to your dashboard and click Chat Messaging in the top right. You will then see the App Access Keys displayed.

Create an .envfile in your project’s root directory, copy the App key and secret, and save it like so:

STREAM\_API\_KEY=your-api-key  
STREAM\_API\_SECRET=your-api-secret

Now you’re ready to build and integrate Stream using the Flutter SDK.

In your terminal, run this command:

flutter pub get

This command installs all the dependencies listed in your pubspec.yaml, making packages like stream_chat_flutter, flutter_dotenv, available to your app. It’s required after any change to pubspec.yaml before you can use new packages in your code.

To integrate prebuilt UI components for chat in Flutter, use the stream_chat_flutter package. This package includes reusable and customizable UI components already integrated with Stream’s API.

Add the package to your dependencies like so:

dependencies:  
  stream\_chat\_flutter: ^1.0.1-beta

This package provides full chat UI widgets you can use directly in your app for fast integration.

For video, UI components to Stream Video in Flutter, check out the documentation.

Build the Role Selection Screen

As we develop the app, the codebase will increase to what is in the image below.

In the lib/screen/auth directory, create a file called role_selection_screen.dart and enter the code below:

import 'package:flutter/material.dart';  
import '../../stream\_client.dart';  
class RoleSelectionScreen extends StatefulWidget {  
  const RoleSelectionScreen({super.key});  
  @override  
  State\<RoleSelectionScreen\> createState() \=\> \_RoleSelectionScreenState();  
}  
class \_RoleSelectionScreenState extends State\<RoleSelectionScreen\> {  
  bool \_loading \= false;  
  Future\<void\> \_connectAndNavigate(String userId, String route, {String? name}) async {  
    setState(() \=\> \_loading \= true);  
    await StreamClientProvider.connectUser(userId, name: name);  
    setState(() \=\> \_loading \= false);  
    Navigator.pushNamed(context, route);  
  }  
  @override  
  Widget build(BuildContext context) {  
    return Scaffold(  
      backgroundColor: const Color(0xFFF7F9FC),  
      body: Center(  
        child: Padding(  
          padding: const EdgeInsets.all(24.0),  
          child: Column(  
            mainAxisAlignment: MainAxisAlignment.center,  
            children: \[  
              Image.asset(  
                'assets/images/medtalk\_logo.png',  
                height: 120,  
              ),  
              const SizedBox(height: 32),  
              const Text(  
                'MedTalk',  
                style: TextStyle(fontSize: 32, fontWeight: FontWeight.bold, color: Color(0xFF2A4D9B)),  
              ),  
              const SizedBox(height: 8),  
              const Text(  
                'Talk to a doctor anytime, anywhere',  
                style: TextStyle(fontSize: 16, color: Color(0xFF4A4A4A)),  
                textAlign: TextAlign.center,  
              ),  
              const SizedBox(height: 48),  
              \_loading  
                  ? const CircularProgressIndicator()  
                  : Column(  
                      children: \[  
                        ElevatedButton(  
                          onPressed: () \=\> \_connectAndNavigate('doctor', '/doctor', name: 'Dr. Sarah Lee'),  
                          style: ElevatedButton.styleFrom(  
                            minimumSize: const Size.fromHeight(48),  
                            backgroundColor: Color(0xFF2A4D9B),  
                            foregroundColor: Colors.white,  
                            textStyle: const TextStyle(fontWeight: FontWeight.bold),  
                            shape: RoundedRectangleBorder(  
                              borderRadius: BorderRadius.circular(12),  
                            ),  
                          ),  
                          child: const Text('Continue as Doctor'),  
                        ),  
                        const SizedBox(height: 16),  
                        OutlinedButton(  
                          onPressed: () \=\> \_connectAndNavigate('patient', '/patient', name: 'Patient'),  
                          style: OutlinedButton.styleFrom(  
                            minimumSize: const Size.fromHeight(48),  
                            foregroundColor: Color(0xFF2A4D9B),  
                            textStyle: const TextStyle(fontWeight: FontWeight.bold),  
                            side: const BorderSide(color: Color(0xFF2A4D9B), width: 2),  
                            shape: RoundedRectangleBorder(  
                              borderRadius: BorderRadius.circular(12),  
                            ),  
                          ),  
                          child: const Text('Continue as Patient'),  
                        ),  
                      \],  
                    ),  
            \],  
          ),  
        ),  
      ),  
    );  
  }  
}

Note: You can get the asset in assets/images/medtalk_logo.png from this GitHub repository. Download the medtalk_logo.png file and add it to the /assets/images directory.

The role selection screen is the entry point to the virtual health consultation application. It allows users to choose between doctor and patient roles. This screen sets the user's identity and routes to the appropriate dashboard.

The state management uses Flutter's built-in StatefulWidget to manage UI states and tracks the loading state with a _loading boolean to show/hide the loading indicator.

class \_RoleSelectionScreenState extends State\<RoleSelectionScreen\> {  
  bool \_loading \= false;  
  // ...  
}

We import flutter/material.dart for UI components and ../../stream_client.dart to manage connections to Stream Chat using a user’s ID and name.

The screen is built as a stateful widget to handle dynamic changes, such as displaying a loading spinner during Stream connection, with a _loading boolean (shown in the code snippet above) that tracks this connection status.

_connectAndNavigate function sets _loading to true, calls StreamClientProvider.connectUser(userId, name: name) to connect the user, then navigates to either /doctor or /patient once connected.

The UI has an off-white background, the MedTalk logo, and a tagline. If _loading is true, a CircularProgressIndicator is shown; otherwise, two buttons appear — “Continue as Doctor” (blue, connects with Dr. Sarah Lee) and “Continue as Patient” (outlined, connects with Patient).

This screen is the entry point to the virtual health consultation platform, guiding users with a personalized experience based on their role; Doctor or Patient.

Build the Doctor’s Home Screen

After users select their role and connect to Stream, doctors are taken to the DoctorHomeScreen. This is the main dashboard for doctors, where they can see demo patients, start chats, and view existing conversations.

In lib/screens/doctor/ create a file: doctor_home_screen.dart, and enter the code below:

The full code is in the GitHub repository.

import 'package:flutter/material.dart';  
import 'package:stream\_chat\_flutter/stream\_chat\_flutter.dart';  
import '../../stream\_client.dart';  
import '../../demo\_users.dart';  
class DoctorHomeScreen extends StatefulWidget {  
  const DoctorHomeScreen({Key? key}) : super(key: key);  
  @override  
  State\<DoctorHomeScreen\> createState() \=\> \_DoctorHomeScreenState();  
}  
class \_DoctorHomeScreenState extends State\<DoctorHomeScreen\> {  
  late final StreamChannelListController \_channelListController;  
  @override  
  void initState() {  
    super.initState();  
    \_channelListController \= StreamChannelListController(  
      client: StreamClientProvider.client,  
      filter: Filter.in\_('members', \[StreamClientProvider.client.state.currentUser\!.id\]),  
      limit: 20,  
    );  
  }  
  @override  
  Widget build(BuildContext context) {  
    final currentUser \= StreamClientProvider.client.state.currentUser;  
    return Scaffold(  
      backgroundColor: const Color(0xFFF7F9FC),  
      appBar: AppBar(  
        title: const Text('Doctor', style: TextStyle(color: Color(0xFF2A4D9B), fontWeight: FontWeight.bold)),  
        backgroundColor: Colors.white,  
        iconTheme: const IconThemeData(color: Color(0xFF2A4D9B)),  
        elevation: 1,  
      ),  
      body: Padding(  
        padding: const EdgeInsets.all(8.0),  
        child: Column(  
          children: \[  
            // Patient list  
            SizedBox(  
              height: 120,  
              child: ListView.separated(  
                scrollDirection: Axis.horizontal,  
                itemCount: demoPatients.length,  
                separatorBuilder: (\_, \_\_) \=\> const SizedBox(width: 16),  
                itemBuilder: (context, index) {  
                  final patient \= demoPatients\[index\];  
                  return Card(  
                    elevation: 2,  
                    shape: RoundedRectangleBorder(borderRadius: BorderRadius.circular(12)),  
                    child: Container(  
                      width: 200,  
                      padding: const EdgeInsets.all(12),  
                      child: Column(  
                        mainAxisAlignment: MainAxisAlignment.center,  
                        children: \[  
                          Text(patient.name, style: const TextStyle(fontWeight: FontWeight.bold)),  
                          const SizedBox(height: 8),  
                          ElevatedButton(  
                            onPressed: () async {  
                              final client \= StreamClientProvider.client;  
                              final currentUser \= client.state.currentUser;  

                              if (currentUser \== null) {  
                                ScaffoldMessenger.of(context).showSnackBar(  
                                  SnackBar(content: Text('Current user not found. Please log in again.')),  
                                );  
                                return;  
                              }  
                              try {  
                                // Create the channel  
                                final channel \= client.channel(  
                                  'messaging',  
                                  extraData: {  
                                    'members': \[currentUser.id, patient.id\],  
                                    'created\_by\_id': currentUser.id,  
                                  },  
                                );  

                                // Create the channel on Stream  
                                await channel.create();  

                                // Navigate to the chat  
                                if (\!mounted) return;  
                                Navigator.push(  
                                  context,  
                                  MaterialPageRoute(  
                                    builder: (context) \=\> StreamChannel(  
                                      channel: channel,  
                                      child: const StreamChannelPage(),  
                                    ),  
                                  ),  
                                );  
                              } catch (e) {  
                                if (\!mounted) return;  
                                ScaffoldMessenger.of(context).showSnackBar(  
                                  SnackBar(content: Text('Error creating chat: ${e.toString()}')),  
                                );  
                              }  
                            },  
                            child: const Text('Start Chat'),  
                          ),  
                        \],  
                      ),  
                    ),  
                  );  
                },

We import four files:

flutter/material.dart for UI components.
stream_chat_flutter.dart for prebuilt Stream Chat widgets and controllers.
stream_client.dart for the connected Stream client instance.
demo_users.dart for sample patient data, so the demo works without a real backend.

DoctorHomeScreen is a stateful widget because it manages a StreamChannelListController, which fetches channels based on filters. In initState(), the controller is created with:

client: StreamClientProvider.client — the active Stream connection.
filter: Filter.in_('members', [currentUser.id]) — returns only channels that include the logged-in doctor.
limit: 20 — loads the latest 20 channels.

In the build method, we get the currentUser to display their info and use it when creating new chats. The UI has two main sections:

1. Patient List (top horizontal list)
A scrollable row of demo patients from demoPatients. Each patient card shows their name and a Start Chat button. It then:

Creates a messaging channel with members [doctorId, patientId].
Calls await channel.create() to set up the conversation in Stream.
Navigates to StreamChannelPage, which shows the chat UI if there’s an error, and a SnackBar displays the message.

2. Chat List (main section)
A StreamChannelListView that automatically lists all the doctor’s existing chats based on the controller’s filter. Tapping a channel opens it in StreamChannelPage.

The StreamChannelPage
This uses three widgets from stream_chat_flutter:

StreamChannelHeader(): shows the other user’s name and a back button.
StreamMessageListView(): displays real-time messages.
StreamMessageInput(): lets users send new messages with emoji and attachments.

Stream Chat’s Flutter SDK handles message syncing, typing indicators, history, and UI updates, so we don’t build these features manually.

This setup allows doctors to see their patients, start conversations, view existing chats, and message in real time. Because this is Flutter Web, you can even open two browser tabs: one as a doctor and one as a patient.

Next, we’ll switch to the patient’s view in lib/screens/patient/patient_home_screen.dart to see how the experience mirrors the doctor’s while keeping roles separate.

Build the Patient’s Home Screen

Just like the doctor’s view, our patient home screen gives users two main capabilities:

See available doctors.
Start or continue chats with them.

Create a patient_home_screen.dart file in the lib/screens/patient directory and enter the code below:

import 'package:flutter/material.dart';  
import 'package:stream\_chat\_flutter/stream\_chat\_flutter.dart';  
import '../../stream\_client.dart';  
import '../../demo\_users.dart';  
class PatientHomeScreen extends StatefulWidget {  
  const PatientHomeScreen({Key? key}) : super(key: key);  
  @override  
  State\<PatientHomeScreen\> createState() \=\> \_PatientHomeScreenState();  
}  
class \_PatientHomeScreenState extends State\<PatientHomeScreen\> {  
  late final StreamChannelListController \_channelListController;  
  @override  
  void initState() {  
    super.initState();  
    \_channelListController \= StreamChannelListController(  
      client: StreamClientProvider.client,  
      filter: Filter.in\_('members', \[StreamClientProvider.client.state.currentUser\!.id\]),  
      limit: 20,  
    );  
  }  
  @override  
  Widget build(BuildContext context) {  
    final currentUser \= StreamClientProvider.client.state.currentUser;  
    return Scaffold(  
      backgroundColor: const Color(0xFFF7F9FC),  
      appBar: AppBar(  
        title: const Text('Patient', style: TextStyle(color: Color(0xFF2A4D9B), fontWeight: FontWeight.bold)),  
        backgroundColor: Colors.white,  
        iconTheme: const IconThemeData(color: Color(0xFF2A4D9B)),  
        elevation: 1,  
      ),  
      body: Padding(  
        padding: const EdgeInsets.all(8.0),  
        child: Column(  
          children: \[  
            // Doctor list  
            SizedBox(  
              height: 120,  
              child: ListView.separated(  
                scrollDirection: Axis.horizontal,  
                itemCount: demoDoctors.length,  
                separatorBuilder: (\_, \_\_) \=\> const SizedBox(width: 16),  
                itemBuilder: (context, index) {  
                  final doctor \= demoDoctors\[index\];  
                  return Card(  
                    elevation: 2,  
                    shape: RoundedRectangleBorder(borderRadius: BorderRadius.circular(12)),  
                    child: Container(  
                      width: 200,  
                      padding: const EdgeInsets.all(12),  
                      child: Column(  
                        mainAxisAlignment: MainAxisAlignment.center,  
                        children: \[  
                          Text(doctor.name, style: const TextStyle(fontWeight: FontWeight.bold)),  
                          const SizedBox(height: 8),  
                          ElevatedButton(  
                            onPressed: () async {  
                              final client \= StreamClientProvider.client;  
                              final currentUser \= client.state.currentUser;  

                              if (currentUser \== null) {  
                                if (\!mounted) return;  
                                ScaffoldMessenger.of(context).showSnackBar(  
                                  const SnackBar(content: Text('Error: User not logged in')),  
                                );  
                                return;  
                              }  

                              try {  
                                // Create the channel  
                                final channel \= client.channel(  
                                  'messaging',  
                                  extraData: {  
                                    'members': \[currentUser.id, doctor.id\],  
                                    'created\_by\_id': currentUser.id,  
                                  },  
                                );  

                                // Create the channel on Stream  
                                await channel.create();  

                                // Navigate to the chat  
                                if (\!mounted) return;  
                                Navigator.push(  
                                  context,  
                                  MaterialPageRoute(  
                                    builder: (context) \=\> StreamChannel(  
                                      channel: channel,  
                                      child: const StreamChannelPage(),  
                                    ),  
                                  ),  
                                );  
                              } catch (e) {  
                                if (\!mounted) return;  
                                ScaffoldMessenger.of(context).showSnackBar(  
                                  SnackBar(content: Text('Error creating chat: ${e.toString()}')),  
                                );  
                              }  
                            },  
                            child: const Text('Start Chat'),  
                          ),

Get the full code in GitHub.

When the screen loads, initState() creates a StreamChannelListController that acts as the inbox manager. It connects to StreamClientProvider.client (set up in role_selection_screen.dart) and filters for any channels where the logged-in patient is a member:

filter: Filter.in\_('members', \[StreamClientProvider.client.state.currentUser\!.id\]),

This works because the patient is connected to Stream through getstream-token-server.js. (We will explain this later in the article.)

The Node server takes the patient’s ID ("patient"), uses the Stream API key and secret from .env, generates a user token, and registers them with Stream. Without that step, currentUser would be null and chats wouldn’t work.

The UI starts with a horizontal list of doctors pulled from demoDoctors.

When a patient taps Start Chat:

We create or retrieve a messaging channel with both patient and doctor IDs.
If it doesn’t exist, Stream creates it automatically.
We call await channel.create() to finalize it.
We navigate to StreamChannelPage, which uses StreamMessageListView() for real-time messages and StreamMessageInput() for sending new ones.

At the bottom, StreamChannelListView displays all active chats, including ones started by doctors. Stream’s SDK updates this list instantly, so new messages appear in real time.

With both doctor and patient home screens complete, we have a two-way chat system using Flutter Web, Stream Flutter SDK, and a Node token server. The next step is to add a shared chat screen functionality.

Build the Shared Chat Screen

Both doctors and patients use the shared chat screen. It provides a reusable UI for a single channel, powered by the Stream Chat Flutter SDK. You pass in a channelId and userId, and the screen connects to that channel, showing real-time messages with Stream’s prebuilt widgets.

Create a chat_screen.dart file in the lib/screens/shared directory and enter the code below:

import 'package:flutter/material.dart';  
import 'package:stream\_chat\_flutter/stream\_chat\_flutter.dart';  
import '../../stream\_client.dart';  
class ChatScreen extends StatefulWidget {  
  final String channelId;  
  final String userId;  
  const ChatScreen({super.key, required this.channelId, required this.userId});  
  @override  
  State\<ChatScreen\> createState() \=\> \_ChatScreenState();  
}  
class \_ChatScreenState extends State\<ChatScreen\> {  
  late Channel channel;  
  @override  
  void initState() {  
    super.initState();  
    channel \= StreamClientProvider.client.channel(  
      'messaging',  
      id: widget.channelId,  
      extraData: {'members': \[widget.userId\]},  
    );  
    channel.watch();  
  }  
  @override  
  Widget build(BuildContext context) {  
    return StreamChannel(  
      channel: channel,  
      child: Scaffold(  
        appBar: StreamChannelHeader(),  
        body: Column(  
          children: \[  
            Expanded(child: StreamMessageListView()),  
            StreamMessageInput(),  
          \],  
        ),  
      ),  
    );  
  }  
}

When the widget loads, initState() creates a Channel from the global client in StreamClientProvider.

It calls:

client.channel(  
  'messaging',  
  id: widget.channelId,  
  extraData: {'members': \[widget.userId\]},  
)

This communicates with Stream to:

“Get or create this messaging channel with these members.”

If the channel doesn’t exist yet, Stream will create it when channel.create() is called earlier in the flow. The extraData helps identify channel members and supports filtering and admin logic.

After creating the channel object, the code calls channel.watch(). This starts streaming channel state and events—messages, typing indicators, presence updates, and reactions—over a WebSocket connection. Updates arrive instantly without polling.

In the build() method, the UI is wrapped in StreamChannel(channel: channel, child: ...). StreamChannel is a provider widget that makes the channel available to its children. Inside, the prebuilt widgets handle the chat experience:

StreamChannelHeader() → shows the channel name and back button.
StreamMessageListView() → displays the live message feed with avatars, ordering, and read receipts.
StreamMessageInput() → sends messages, attachments, and reactions.

These widgets handle sending and receiving messages, rendering updates, retry logic, and syncing with Stream’s servers.

This screen assumes the user is already connected to Stream through StreamClientProvider—a step that happens earlier (for example, in role_selection_screen.dart). That connection involves calling your Node-based token server (getstream-token-server.js), which uses your API secret to generate a user token. The Flutter app then calls client.connectUser(User(id: ...), token) to authenticate.

When a doctor or patient clicks “Start Chat” from their home screen, the app either creates the channel there or navigates into this ChatScreen with the existing channelId. As long as both members are included in the channel’s members list, Stream will route messages correctly and store them in your dashboard.

Connect to Stream Client

To connect the Flutter app to the Stream client, create a stream.dart file in the lib/stream directory and enter the code below:

import 'package:stream\_chat\_flutter/stream\_chat\_flutter.dart';  
import 'package:flutter\_dotenv/flutter\_dotenv.dart';  
import 'dart:convert';  
import 'package:http/http.dart' as http;  
class StreamClientProvider {  
  static final StreamChatClient client \= StreamChatClient(  
    dotenv.env\['STREAM\_API\_KEY'\]\!,  
    logLevel: Level.INFO,  
  );  
  /// Connect a user to the Stream Chat client (production, fetch token from backend)  
  static Future\<void\> connectUser(String userId, {String? name}) async {  
    if (client.state.currentUser?.id \== userId) return;  
    await client.disconnectUser();  
    // Fetch token from backend  
    final tokenEndpoint \= dotenv.env\['TOKEN\_ENDPOINT'\] ?? 'http://localhost:3000/token';  
    final response \= await http.get(Uri.parse('$tokenEndpoint?user\_id=$userId'));  
    if (response.statusCode \!= 200\) {  
      throw Exception('Failed to fetch Stream Chat token: ${response.body}');  
    }  
    final token \= jsonDecode(response.body)\['token'\];  
    await client.connectUser(  
      User(id: userId, name: name ?? userId),  
      token,  
    );  
  }  
}

This connects the Flutter app and the Stream Chat API. It loads your API key from .env, initializes a single Stream client for the whole app, and connects users by fetching their token from your backend.

At the top, we import:

import 'package:stream\_chat\_flutter/stream\_chat\_flutter.dart';  
import 'package:flutter\_dotenv/flutter\_dotenv.dart';  
import 'dart:convert';  
import 'package:http/http.dart' as http;

stream_chat_flutter – The official Flutter SDK for Stream Chat.
flutter_dotenv – Loads environment variables from .env, such as your API key and backend URL.
dart:convert – Decodes JSON responses from the server.
http – Makes HTTP requests to your backend to get the chat token.

The main class is StreamClientProvider, which creates one StreamChatClient for the entire app:

static final StreamChatClient client \= StreamChatClient(  
  dotenv.env\['STREAM\_API\_KEY'\]\!,  
  logLevel: Level.INFO,  
);

Here, the API key comes from .env, and the log level is set to INFO so you can see helpful debug messages about chat events.

The connectUser method is where the actual login happens:

static Future\<void\> connectUser(String userId, {String? name}) async {  
  if (client.state.currentUser?.id \== userId) return;  
  await client.disconnectUser();

First, it checks if the user is already connected with the same ID. If so, it skips the rest. If not, it disconnects any existing users to avoid conflicts.

Next, it fetches the token from your backend:

final tokenEndpoint = dotenv.env['TOKEN_ENDPOINT'] ?? 'http://localhost:3000/token';

final response = await http.get(Uri.parse('$tokenEndpoint?user_id=$userId'));

The token endpoint is read from .env (falling back to localhost for development). The request includes the user ID as a query parameter.

If the server responds with anything other than 200 OK, it throws an error. Otherwise, it extracts the token from the JSON:

if (response.statusCode != 200) {
  throw Exception(
    'Failed to fetch Stream Chat token: ${response.body}',
  );
}
final token = jsonDecode(response.body)['token'];

Finally, it connects the user to Stream:

await client.connectUser(  
  User(id: userId, name: name ?? userId),  
  token,  
);

The SDK uses this ID, optional display name, and the JWT token from your backend to log in securely. The token is generated server-side using your API key and secret, so those sensitive credentials never touch the Flutter app.

Basically, the stream_client.dart file loads your API key from .env, requests a secure token from the server, and connects the user to Stream Chat so they can send and receive messages in real time.

Set Up the Main App Entry Point

In the lib directory, create a main.dart file. This will serve as the entry point for your Flutter application.

import 'package:flutter/material.dart';  
import 'package:flutter\_dotenv/flutter\_dotenv.dart';  
import 'package:stream\_chat\_flutter/stream\_chat\_flutter.dart';  
import 'package:flutter\_localizations/flutter\_localizations.dart';  
import 'stream\_client.dart';  
import 'screens/auth/role\_selection\_screen.dart';  
import 'screens/doctor/doctor\_home\_screen.dart';  
import 'screens/patient/patient\_home\_screen.dart';  
import 'screens/shared/video\_call\_screen.dart';  
Future\<void\> main() async {  
  WidgetsFlutterBinding.ensureInitialized();  
  await dotenv.load(fileName: '.env');  
  runApp(MedTalkApp());  
}  
class MedTalkApp extends StatelessWidget {  
  MedTalkApp({super.key});  
  final StreamChatClient client \= StreamClientProvider.client;  
  @override  
  Widget build(BuildContext context) {  
    return MaterialApp(  
      title: 'MedTalk',  
      theme: ThemeData(  
        colorScheme: ColorScheme.fromSeed(seedColor: Colors.blue),  
        useMaterial3: true,  
      ),  
      localizationsDelegates: \[  
        GlobalMaterialLocalizations.delegate,  
        GlobalWidgetsLocalizations.delegate,  
        GlobalCupertinoLocalizations.delegate,  
      \],  
      supportedLocales: \[  
        Locale('en'),  
      \],  
      initialRoute: '/',  
      routes: {  
        '/': (context) \=\> const RoleSelectionScreen(),  
        '/doctor': (context) \=\> const DoctorHomeScreen(),  
        '/patient': (context) \=\> const PatientHomeScreen(),  
        '/video': (context) \=\> const VideoCallScreen(participantName: 'Dr. Sarah Lee'),  
      },  
      builder: (context, child) \=\> StreamChat(  
        client: client,  
        child: child\!,  
      ),  
    );  
  }  
}

We start by importing the required packages:

flutter/material.dart for core Flutter UI components
flutter_dotenv to load environment variables from the .env file
stream_chat_flutter for integrating the Stream Chat SDK
flutter_localizations to add localization support
stream_client.dart` (local file) and screen widgets for different roles and features.

Next, we set up the main() function, which ensures Flutter’s bindings are initialized, loads our environment variables, and finally runs the MedTalkApp.

The MedTalkApp widget extends StatelessWidget and defines the app’s overall configuration. It creates a StreamChatClient instance (from StreamClientProvider.client) and sets up the app theme, localization delegates, and supported locales.

For navigation, we define our initial route ('/'), which takes the user to the Role Selection Screen, and register other routes for the Doctor Home, Patient Home, and Video Call screens. This makes navigating between different app parts easy, depending on the user’s role and actions.

The builder property of MaterialApp wraps the child widget with the StreamChat widget, passing it on to the client. This ensures the Stream Chat SDK is available throughout the app’s widget tree, enabling messaging and chat features in any screen.

With the app’s entry point ready, the next step is to create a node server to generate Stream Chat tokens.

Set Up the Node.js Server

Create a getstream-token-server.js file in the root directory and enter the code in the GitHub repository.

This runs a Node.js + Express server that generates secure JWT tokens for authenticating users with the Stream Chat API.

Note: In production, do not hardcode your Stream API secret directly in your Flutter app. Instead, the server/backend generates a timed token for the authenticated user, and your Flutter app uses that token to connect to Stream. This is exactly what this Node token server does in this context.

We start by importing the required packages:

express to create a simple HTTP server
stream-chat to talk to Stream’s API and create tokens
cors so our Flutter web app can call the server without CORS issues
dotenv so we can store our API key and secret safely in a .env file

Next, we load the environment variables and check if the STREAM_API_KEY and STREAM_API_SECRET are set. The server will stop immediately if they're missing; we can’t run without them.

We then initialize a StreamChat server client using the API key and secret. This client will be able to create tokens, seed users, and perform admin operations.

The server exposes two main endpoints:

/token?user_id=USER_ID – This generates a JWT token for the given user_id. Flutter will call this when connecting to Stream.
/ – A simple HTML page just to verify that the server is running.

We also added:

Request logging so you can see when tokens are requested.
A *user seeding script *(seedUsers) that adds demo doctors and patients to Stream. These IDs and names match the ones in our Flutter app, so you can test right away.
Graceful shutdown handling so the server closes cleanly when you stop it.

Open a terminal and start the server with this command:

node getstream-token-server.js

Now your Flutter app can request tokens securely and connect to Stream Chat.

Set Up Demo Users for MedTalk

To keep development simple, we’ll define a set of demo doctors and patients that our MedTalk app can use for testing. This will allow us to quickly switch between roles (doctor or patient).

In the lib directory, create a new file called demo_users.dart and add the following code from the GitHub repository.

With this setup, we can easily pull a list of doctors or patients in the UI, making it much faster to test the application.

Now the full Flutter app is set up, and we can begin testing it!

Test the Virtual Health Consultation App

Select Chrome as your app's target device to run and debug a Flutter web app:

bash flutter run \-d chrome

You can also choose Chrome as a target device in your IDE.

If you prefer, you can use the edge device type on Windows, or use web-server to navigate to a local URL in the browser of your choice.

Run From the Command Line

If you use Flutter run from the command line, you can now run hot reload on the web with the following command:

bash flutter run \-d chrome \--web-experimental-hot-reload
When hot reload is enabled, you can restart your application by pressing "r" in the running terminal or "R".

The Flutter web app loads and opens this URL in the Chrome browser:

http://localhost:60767/

This is the application's landing page.

When you click on either Continue as Doctor or Continue as Patient, it then redirects to a loading screen for a few seconds, like so:

Which then redirects to the chat page, with this URL (if you selected Patient role), like so:

http://localhost:57331//#/patient

At the moment, there are no active chats yet.

The users then show:

Then the patients:

And the Patient/Doctor chat loads:

From the Stream dashboard, check the chat logs to confirm the application is integrated with Stream through the Flutter SDK.

Note: We built a Node.js server for handling the Stream tokens for chat/video.

Test the endpoint in your browser or with curl:

You should get a JSON response like:

How to Run the Flutter Web App

To connect the Flutter web app with the Stream Flutter SDK, we set up a server using Node.js, as shown in the step above. This server generates tokens from Stream Chat using the Stream API, which is connected through the API key set up in the .env file at the root of the project directory codebase.

To do this, we first need to run the Node.js server, seed some users (Doctors and Patients), and assign Stream tokens to each user. This enables users to chat asynchronously on the platform.

To get started running the app, first run the Node.js server in a different terminal.

Note: We use two command prompts in a terminal to run this application.

In the first terminal, run this command:

bash node getstream-token-server.js

This starts the node server and shows an output like so:

To test an endpoint, open this URL in your browser: http://localhost:3000/token?user_id=doctor_anna

It then loads this HTML page, and when you click Get token for doctor anna.

It generates a token like so:

You can also view the server logs in the Node.js terminal.

The token server is now up and running perfectly with this setup:

Server is listening on http://localhost:3000
It has successfully generated a token for a Doctor user, doctor_anna
All demo users are seeded in Stream Chat

Next, let's test the Flutter app, which should now be able to:

Connect to the token server
Authenticate users
Create chat channels

Run the Flutter app with this command:

flutter run \-d chrome

The Flutter app begins to start and shows these logs in your terminal. If you look closely at the log messages, you will see that the Flutter app is connected to the Stream API.

The Flutter app then loads and opens in your Chrome browser.

Click on either Continue as Doctor or Patient, and then try starting a new chat. If you select the Doctor role, the chat loads, showing the history of previous chats with Patients.

Click on any user, say Bob Brown, to start a chat

You can also confirm that the messages are sent through Stream by viewing the real-time chat logs in your dashboard.

Now let’s try out the Patient's chat!

Navigate to the homepage of the application with http://localhost:64150, and then click on Continue as Patient, which then loads the view as shown below:

The direct URL to the patient view of the app is: http://localhost:64150/#/patient

Click on any of the Doctors, say Doctor Emily, to start a conversation:

1. As a Doctor:

Click "Start Chat" with a patient
Send a message in the chat
The chat should now appear in your chat list

2. As a Patient:

Log in as the patient you chatted with
You should see the chat in your chat list
You can reply, and the doctor will see it immediately

3. Verify Real-time Sync:

Open the same chat on two devices (or two different user accounts)
Messages should appear instantly on both devices when sent from either side

You can view the total number of messages on the Stream chat dashboard.

To run the app locally, clone the GitHub repository with this command.

git clone

Note: As mentioned in the article earlier, make sure you have Flutter set up and installed on your computer.

Navigate to the project directory, then run this command to install the dependencies needed for the app.

flutter pub get

Also, ensure your Stream API keys are added to an .env file in the project directory.

Then run the application with:

flutter run \-d chrome

And that’s it! You have a working virtual consultation app, fully set up.

Conclusion

In this tutorial, you built a MedTalk clone, a HIPAA-compliant telehealth application powered by Flutter and Stream. By integrating Stream Chat and Stream Video, you enabled secure messaging and real-time video consultations between doctors and patients. This prototype shows how quickly you can deliver compliant, modern healthcare communication experiences with Stream’s SDKs.

Build AI-Powered Smart Replies with React and Synthetic

Ekemini Samuel — Fri, 19 Dec 2025 07:59:08 +0000

You’ve probably used Gmail’s Smart Reply or Slack’s suggested messages, which are short, AI-generated responses like “Sounds good 👍” or “Let’s do it!”. With a single tap response, you can save time and keep conversations flowing.

By reducing time spent typing, conversations stay active, drop-off rates decrease, and chat apps feel more innovative and modern.

Smart replies can be used in customer support apps (agents pick relevant replies and respond faster), marketplaces or communities (buyers and sellers exchange quick transactional messages), or even productivity tools or workplace chat platforms (users save time on repetitive confirmations like “Yes,” “Got it,” or “On it”).

In this tutorial, we’ll use Stream’s React SDK and Synthetic’s inference API to bring smart replies into chat. You’ll build a chat application that generates real-time, context-aware reply suggestions directly within conversations.

Watch a demo of what you’ll build:

Prerequisites

To follow along, make sure you have the following prerequisites:

Node.js (version 16 or higher) and NPM installed
A free Stream account with your API key
A Synthetic API key for accessing the inference API
Basic knowledge of React, Next.js, and Typescript

Ready? Let’s begin!

Stream is a platform that provides APIs and SDKs for developers to build real-time chat, video, and activity feeds into their applications.

To get started with Stream, create your free account.

Learn how to set up your Stream account in this guide.

After you’ve signed up, navigate to your dashboard and click on the Create App button in the top right of the page, like so:

This modal then loads, where you enter details of the new app you’re creating. Enter the app name and then select the chat & video data region, feed data storage location, and environment (you can choose 'development'). Click 'Create App' to proceed.

Next, you’ll get your App API Access Keys from the Chat Overview in your dashboard. Copy the Key and Secret and add it to your .env file in your codebase.

STREAM_API_KEY=”your_stream_api”
STREAM_API_SECRET=”your_stream_secret”

We will be using the Stream React Chat SDK for this application. Learn more about it in the documentation.

Setting Up the React Project

Open your terminal and enter this command to create a new app:

npx create-next-app@latest smart-chat --typescript --tailwind --eslint --app --src-dir --import-alias "@/*"

This creates a new Next.js app in your directory called “smart-chat”, with Typescript and Tailwind configuration. It also adds an Import alias.

Note: Import aliases help with better imports and readability in your codebase. When adding new imports, simply update the alias configuration in tsconfig.json or jsconfig.json, rather than updating import paths throughout your codebase.

Next, navigate to the new project directory we just created with this command.

cd smart-chat

Then, install the Stream Chat React SDK with this command:

npm install stream-chat stream-chat-react

This command installs the stream-chat and stream-chat-react packages. We will import the required components, such as Chat, Channel, MessageList, and MessageInput, from stream-chat-react into the main application file, which is the page.tsx file.

Next, create a stream-chat-container.tsx file in the components/ directory and import the necessary components from stream-chat-react to build the chat interface:

import { StreamChat } from "stream-chat"
import {
  Chat,
  Channel,
  ChannelHeader,
  MessageList,
  MessageInput,
  Window,
} from "stream-chat-react"
import "stream-chat-react/dist/css/v2/index.css"

The stream-chat package provides the core Stream Chat client for initialising and managing chat functionality, while stream-chat-react offers pre-built React components like Chat,Channel, MessageList, and MessageInput.

Chat component acts as a provider, wrapping the chat interface to manage the Stream client.
Channel component renders a specific chat channel.
MessageList displays the conversation history, and <MessageInput> allows you to compose messages.

The full implementation is available in the GitHub repository.

In your project’s root directory, create a page.tsx file in the /app directory and enter the code below:

"use client"

import { useState } from "react"
import { StreamChatContainer } from "@/components/stream-chat-container"

export default function Home() {
  const [isLoggedIn, setIsLoggedIn] = useState(false)
  const [userId, setUserId] = useState("")
  const [userName, setUserName] = useState("")
  const [inputUserId, setInputUserId] = useState("")
  const [inputUserName, setInputUserName] = useState("")

  const handleLogin = () => {
    if (inputUserId && inputUserName) {
      setUserId(inputUserId)
      setUserName(inputUserName)
      setIsLoggedIn(true)
    }
  }

  if (isLoggedIn) {
    return <StreamChatContainer userId={userId} userName={userName} />
  }

}

Due to the length of the code file above, we have included only a snippet here. The complete code is available in the GitHub repository.

This sets up the main entry point for the application by prompting you to enter a User ID and Display Name before rendering the chat interface.

It uses React’s useState hook to manage user login state. When you enter a User ID and Display Name and click the Join Chat button, the handleLogin function updates the state and renders the component, passing the userId and userName props.

For the UI, we’re adapting a Discord-like interface.
To set up the layout for this application, create a layout.tsx file also in the /app directory, and enter this code:

import type React from "react"
import type { Metadata } from "next"
import "./globals.css"


import { Geist, Geist_Mono } from 'next/font/google'


// Initialize fonts
const geist = Geist({
  subsets: ["latin"],
  variable: "--font-geist",
  weight: ["100", "200", "300", "400", "500", "600", "700", "800", "900"],
})


const geistMono = Geist_Mono({
  subsets: ["latin"],
  variable: "--font-geist-mono",
  weight: ["100", "200", "300", "400", "500", "600", "700", "800", "900"],
})


export const metadata: Metadata = {
  title: "AI-Powered Live Chat with Intelligent Replies",
  description: "Real-time chat with context-aware AI reply suggestions powered by Synthetic",
}


export default function RootLayout({
  children,
}: Readonly<{
  children: React.ReactNode
}>) {
  return (
    <html lang="en" className={`${geist.variable} ${geistMono.variable}`}>
      <body className="font-sans">{children}</body>
    </html>
  )
}

Connecting to the Chat App

Next, we initialise the Stream client with your API Key from the app we created earlier in the Stream Dashboard above.

In the app directory, create a file named api/token/route.ts to handle user authentication for Stream’s Chat React SDK. Add the following code:

/token/route.ts

import { type NextRequest, NextResponse } from "next/server"
import { StreamChat } from "stream-chat"


const apiKey = process.env.NEXT_PUBLIC_STREAM_API_KEY
const apiSecret = process.env.STREAM_API_SECRET


export async function POST(request: NextRequest) {
  try {
    if (!apiKey || !apiSecret) {
      console.error("Missing Stream credentials - NEXT_PUBLIC_STREAM_API_KEY or STREAM_API_SECRET not set")
      return NextResponse.json(
        {
          error:
            "Stream credentials not configured. Please add NEXT_PUBLIC_STREAM_API_KEY and STREAM_API_SECRET environment variables.",
        },
        { status: 500 },
      )
    }


    const { userId, userName } = await request.json()


    if (!userId || !userName) {
      return NextResponse.json({ error: "userId and userName are required" }, { status: 400 })
    }


    console.log("Generating token for user:", userId)


    // Use Stream official server client to create a user token
    const serverClient = StreamChat.getInstance(apiKey, apiSecret)
    // Ensure the user exists/updated on Stream
    await serverClient.upsertUser({ id: userId, name: userName })
    const token = serverClient.createToken(userId)


    console.log("Token generated successfully")


    return NextResponse.json({
      token,
      userId,
      userName,
    })
  } catch (error) {
    console.error("Token generation error:", error)
    return NextResponse.json(
      { error: error instanceof Error? error.message : "Failed to generate token" },
      { status: 500 },
    )
  }
}

This API route authenticates users for the React SDK. It uses the StreamChat.getInstance method to create a server-side client with your NEXT_PUBLIC_STREAM_API_KEY and STREAM_API_SECRET (set these in your .env file from your Stream dashboard).

The createToken method generates a JWT token for the user (from the API key), which is used to connect to Stream’s chat service. This token is returned to the client for use in components/stream-chat-container.tsx. The route includes error handling for missing credentials or invalid requests to ensure secure authentication.

Then, in components/stream-chat-container.tsx, initialise the Stream client and connect the user like so:

const apiKey = process.env.NEXT_PUBLIC_STREAM_API_KEY

useEffect(() => {
  const init = async () => {
    if (!apiKey) {
      setError("Stream API key is not configured. Please check your environment variables.")
      setIsLoading(false)
      return
    }

    try {
      setIsLoading(true)
      setError(null)

      const res = await fetch("/api/token", {
        method: "POST",
        headers: { "Content-Type": "application/json" },
        body: JSON.stringify({ userId, userName }),
      })

      if (!res.ok) {
        const errorData = await res.json().catch(() => ({}))
        throw new Error(errorData.error || "Failed to get authentication token")
      }

      const { token } = await res.json()
      const c = StreamChat.getInstance(apiKey)

      await c.connectUser({ id: userId, name: userName }, token)

      if (!c.userID) {
        throw new Error("Failed to establish connection with Stream Chat")
      }

      setClient(c)
    } catch (e) {
      const errorMessage = e instanceof Error ? e.message : "An unknown error occurred"
      console.error("Stream initialization error:", e)
      setError(`Failed to initialize chat: ${errorMessage}`)
      setClient(null)
    } finally {
      setIsLoading(false)
    }
  }
  init()

  return () => {
    if (client) client.disconnectUser()
  }
}, [userId, userName, apiKey])

In components/stream-chat-container.tsx, the useEffect hook initialises the Stream client using StreamChat.getInstance with the NEXT_PUBLIC_STREAM_API_KEY. It fetches a user token from the api/token/route.ts endpoint, passing userId and userName from the login form at app/page.tsx.

The connectUser method authenticates the user with the token, establishing a real-time connection to Stream’s chat service.

The useMemo hook creates an AI Smart Replies channel and adds the authenticated user as a member:

const channel = useMemo(() => {
  if (!client) return null
  return client.channel("team", channelId, {
    name: "AI Smart Replies",
    members: [userId],
    created_by_id: userId
  })
}, [client, channelId, userId])

The full initialisation logic, including error handling and loading states, is in the GitHub repository.

Still in the /app directory, create an app folder, which will have these sub-folders and files:

/smart-replies/route.ts

import { type NextRequest, NextResponse } from "next/server"
import { generateSmartReplies } from "@/lib/synthetic-ai"


export async function POST(request: NextRequest) {
  try {
    const { messages } = await request.json()


    if (!messages || !Array.isArray(messages)) {
      return NextResponse.json({ error: "messages array is required" }, { status: 400 })
    }


    const result = await generateSmartReplies(messages)


    return NextResponse.json({
      replies: result.replies,
      isAI: result.isAI,
      model: result.model,
    })
  } catch (error) {
    console.error("Smart replies API error:", error)
    return NextResponse.json({ error: "Failed to generate smart replies" }, { status: 500 })
  }
}

Here is an overview of how the project’s directory looks:

Creating the Chat UI

To set up the chat UI, we start with the <Chat> component we added earlier in the stream-chat-container.tsx file. This wraps the interface, passing the initialised client to provide access to Stream’s chat functionality.

The <Channel> component renders the active channel (created in the useMemo hook), and the <Window> organises the layout.

"use client"
import { useEffect, useMemo, useState } from "react"
import { StreamChat } from "stream-chat"
import {
  Chat,
  Channel,
  ChannelHeader,
  MessageList,
  MessageInput,
  Window,
} from "stream-chat-react"
import "stream-chat-react/dist/css/v2/index.css"
import { SmartReplyBar } from "./smart-reply-bar"

return (
  <Chat client={client} theme="str-chat__theme-dark">
    <Channel channel={channel}>
      <Window>
        <ChannelHeader title="Live Stream Chat" live />
        <MessageList />
        <SmartReplyBridge />
        <MessageInput focus />
      </Window>
    </Channel>
  </Chat>
)

ChannelHeader displays the channel name (Live Stream Chat) with a live indicator, MessageList shows the conversation history, and MessageInput provides a text input for sending messages. For custom styling, we use styles/globals.css to define a dark theme with Tailwind CSS:

.dark {
  --background: oklch(0.145 0 0);
  --foreground: oklch(0.985 0 0);
  --primary: oklch(0.985 0 0);
  --primary-foreground: oklch(0.205 0 0);
  // ... (Other theme variables)
}

The complete code for the UI is available in the GitHub repository.

For the chat UI, we adapt a Discord-like feel to the interface, create a discord-chat-layout.tsx file in the /components directory, and enter the code below:

"use client"

import { useEffect, useRef } from "react"
import { StreamChat } from "stream-chat"
import { Chat, Channel, MessageList, MessageInput, useChatContext, useChannelStateContext } from "stream-chat-react"
import { SmartReplyBar } from "./smart-reply-bar"

// Component to bridge Stream messages to SmartReplyBar
function MessageBridge() {
  const { messages = [] } = useChannelStateContext()
  const { channel } = useChatContext()

  const handleSelectReply = (reply: string) => {
    if (channel) {
      void channel.sendMessage({ text: reply })
    }
  }

  const simplifiedMessages = messages.map((m: any) => ({
    id: m.id,
    userId: m.user?.id || '',
    userName: m.user?.name || m.user?.id || 'User',
    text: m.text || '',
    timestamp: m.created_at ? new Date(m.created_at) : new Date(),
  }))

  return <SmartReplyBar messages={simplifiedMessages} onSelectReply={handleSelectReply} />
}

export function DiscordChatLayout({ userId, userName, client, channel }: DiscordChatLayoutProps) {
  const messagesEndRef = useRef<HTMLDivElement>(null)

  // Auto-scroll to bottom when messages change
  useEffect(() => {
    messagesEndRef.current?.scrollIntoView({ behavior: "smooth" })
  }, [channel?.state?.messages])

  return (
    <div className="flex h-screen bg-[#1e1f22]">
      {/* Sidebar */}
      <div className="w-16 bg-[#2b2d31] flex flex-col items-center py-4">
        {/* Server icon and channels would go here */}
      </div>

      {/* Main Chat Area */}
      <div className="flex-1 flex flex-col">
        {/* Channel Header */}
        <div className="h-14 border-b border-[#1e1f22] bg-[#2b2d31] flex items-center px-4">
          <h2 className="text-white font-semibold"># general</h2>
        </div>

        {/* Messages */}
        <div className="flex-1 overflow-y-auto p-4 bg-[#313338]">
          <MessageList />
          <div ref={messagesEndRef} />
        </div>

        {/* Message Input */}
        <div className="p-4 border-t border-[#1e1f22] bg-[#383a40]">
          <MessageInput />
          <MessageBridge />
        </div>
      </div>
    </div>
  )
}

In the code above, we adapted a Discord-style UI for the interface with a dark theme and a message display area, styled with Tailwind. We customised Stream’s built-in components, such as MessageList and MessageInput, to align with Discord’s aesthetic while maintaining their real-time functionality.

We also added a connector component that bridges Stream’s message system with our AI Smart Reply feature. This component handles message data transformation to make sure that AI-generated replies appear seamlessly within the chat flow.

Now that we have the chat set up and integrated with Stream’s React SDK, let’s set up Synthetic, which we will use for the AI smart replies.

Setting up Synthetic

Synthetic is an AI Inference platform that provides a solution for running open-source AI LLMs, such as Llama, Mistral, and DeepSeek.

Visit the synthetic website and click on “Sign up” like so:

The authentication page will then load. You can sign up using your email address or use Google social login.

After signing up, you will be prompted to enter your username like so:

Next, your Synthetic dashboard loads like so:

Click on the icon in the top left to open the sidebar shown below, where you will get your API key.

In the API settings, copy your API key and add it to your environment variable file .env in the project’s codebase.

SYNTHETIC_API_KEY="your_api_key"

STREAM_API_KEY=”your_stream_api”
STREAM_API_SECRET=”your_stream_secret”

For this project, we will utilise the OpenAI LLM and its API URL within the codebase. You can find all the necessary information on the page shown above.

OpenAI API Base URL: https://api.synthetic.new/openai/v1

Note:

These are the available OpenAI-compatible endpoints to use with the base URL from Synthetic:

/models - Lists all always-on models and any recently used on-demand models
/chat/completions - Chat-based completions with conversation history
/completions - Traditional text completions
/embeddings - Transforms text into vector embeddings

We are integrating the chat completions feature in the app we’re building here in this guide. The chat completion from the LLM model will create a model response for a given chat conversation. The Synthetic documentation provides a guide on how to integrate this feature.

In the code, we will be making a POST request to the API base URL endpoint like so:

POST https://api.synthetic.new/openai/v1/chat/completions

When adding the LLM URLs, make sure they are added with this prefix, like so:

hf:openai/gpt-oss-120b

hf:meta-llama/Llama-3.1-405B-Instruct

These are the Always-on models available on Synthetic:

You can also use a specific LLM model by copying the model’s name to be used in the API. Learn more in the Synthetic API documentation.

Synthetic also supports LoRA models, which are Low-rank adapters, called "LoRAs"; they are small, efficient fine-tunes that run on top of existing models. They can modify a model to be much more effective at specific tasks.

Learn about other models like Embedding Models and On-Demand Models here.

Note: Depending on the length of the content you’re prompting using the AI inference API from Synthetic, you may be required to use a pay-as-you-go service or upgrade to a pro plan on Synthetic.

So, don’t fret if you come across errors like this 🙂

fetch to https://api.synthetic.new/openai/v1/chat/completions failed with status 402 and body: {"error":"Insufficient credits and no active subscription. Due to context lengths, a minimum balance of $1.1179648 is required to run meta-llama/Llama-3.3-70B-Instruct. Your current balance is $0.00. Go to https://synthetic.new/billing to subscribe or purchase credits."}

Synthetic API error (HTTP ${response.status}): {"error":"Insufficient credits and no active subscription. Due to context lengths, a minimum balance of $1.1179648 is required to run meta-llama/Llama-3.3-70B-Instruct. Your current balance is $0.00. Go to https://synthetic.new/billing to subscribe or purchase credits."}

at generateSmartReplies (/lib/synthetic-ai)

Now that we have the Synthetic setup, let’s integrate it into the codebase for the smart chats application.

Capturing Messages and Sending to Synthetic

We’ll use Stream’s message.new event to capture incoming messages from the chat, and then forward the message text and context (recent conversation history) to Synthetic’s inference API.

In components/stream-chat-container.tsx, create a component to capture messages and send them to Synthetic’s API. Add this code within components/stream-chat-container.tsx:

function SmartReplyBridge() {
  const { useChannelStateContext } = require("stream-chat-react") as typeof import("stream-chat-react")
  const { messages } = useChannelStateContext()

  type SRMessage = { id: string; userId: string; userName: string; text: string; timestamp: Date }
  const simplified: SRMessage[] = (messages ?? []).slice(-10).map((m: any) => ({
    id: m.id,
    userId: m.user?.id ?? "",
    userName: m.user?.name ?? m.user?.id ?? "User",
    text: m.text ?? "",
    timestamp: new Date(m.created_at || Date.now()),
  }))

  const { useChatContext } = require("stream-chat-react") as typeof import("stream-chat-react")
  const { channel } = useChatContext()

  const handleSelect = async (reply: string) => {
    await channel?.sendMessage({ text: reply })
  }

  return <SmartReplyBar messages={simplified} onSelectReply={handleSelect} />
}

The component uses Stream’s useChannelStateContext hook to access the latest messages in the channel. It captures the last 10 messages, mapping them to a simplified format (SRMessage) with IDs, user IDs, user names, texts, and timestamps.

These messages are passed to the component, which sends them to Synthetic’s API via the [api/smart-replies/route.ts](https://github.com/Tabintel/smart-chat/tree/master/app/api/smart-replies) endpoint. The useChatContext hook provides access to the channel object, used later for sending replies.

This setup listens for Stream’s message component implicitly through the reactive messages state, with real-time updates.

Generating Smart Reply Suggestions with Synthetic API

We define a function to call Synthetic’s API endpoint with the chat context and parse the API response to extract multiple thoughtful reply options.

In your lib directory, create a file named synthetic-ai.ts to integrate Synthetic’s LLM models, and enter the code below:

interface SimpleMessage {
  user: string
  text: string
}

export interface SmartReplyResult {
  replies: string[]
  isAI: boolean
  model?: string
}

export async function generateSmartReplies(messages: SimpleMessage[]): Promise<SmartReplyResult> {
  try {
    const apiKey = process.env.SYNTHETIC_API_KEY

    if (!apiKey) {
      return {
        replies: getContextualReplies(messages),
        isAI: false,
      }
    }

    const conversationContext = messages
      .slice(-10)
      .map((msg) => `${msg.user}: ${msg.text}`)
      .join("\n")

    const systemPrompt = `You are a smart reply assistant for a live chat application. Your job is to generate 3 short, contextually relevant, and natural reply suggestions based on the conversation.

Guidelines:
- Each reply should be 5-50 characters
- Make them casual, friendly, and conversational
- Match the tone of the conversation
- Use emojis sparingly and naturally
- Avoid generic responses
- Be authentic and human-like

Return ONLY a valid JSON array of 3 strings, nothing else. Example: ["That's awesome!", "Let's go!", "I'm down"]`

    const response = await fetch("https://api.synthetic.new/openai/v1/chat/completions", {
      method: "POST",
      headers: {
        "Content-Type": "application/json",
        Authorization: `Bearer ${apiKey}`,
      },
      body: JSON.stringify({
        model: "hf:meta-llama/Llama-3.3-70B-Instruct",
        messages: [
          {
            role: "system",
            content: systemPrompt,
          },
          {
            role: "user",
            content: `Recent conversation:\n${conversationContext}\n\nGenerate 3 smart reply suggestions as a JSON array.`,
          },
        ],
        temperature: 0.7,
        max_tokens: 200,
      }),
    })

    // ... (Error handling and response parsing as provided)
  } catch (error) {
    console.error("Synthetic API error:", error)
    return {
      replies: getContextualReplies(messages),
      isAI: false,
    }
  }
}

The generateSmartReplies function integrates Synthetic’s LLM model: hf:meta-llama/Llama-3.3-70B-Instruct, to generate context-aware smart replies.

It retrieves the last 10 messages, formats them within a conversation context, and sends them to Synthetic’s API endpoint: https://api.synthetic.new/openai/v1/chat/completions, along with a system prompt that defines guidelines for concise, natural replies.

The API returns a JSON array of three replies, which are parsed and returned with isAI: true and the model name. If the API call fails (for example, due to a missing SYNTHETIC_API_KEY or rate limits), the getContextualReplies function provides fallback replies based on the conversations.

Here is the LLM system prompt in use for the chat app:

You are a smart reply assistant for a live chat application. Your job is to generate 3 short, contextually relevant, and natural reply suggestions based on the conversation.


Guidelines:
- Each reply should be 5-50 characters
- Make them casual, friendly, and conversational
- Match the tone of the conversation
- Use emojis sparingly and naturally
- Avoid generic responses
- Be authentic and human-like


Return ONLY a valid JSON array of 3 strings, nothing else. Example: ["That's awesome!", "Let's go!", "I'm down"]`

You can access the full code in this GitHub repository.

Next, in app/api/smart-replies/route.ts, create an API route to handle this logic:

import { type NextRequest, NextResponse } from "next/server"
import { generateSmartReplies } from "@/lib/synthetic-ai"

export async function POST(request: NextRequest) {
  try {
    const { messages } = await request.json()

    if (!messages || !Array.isArray(messages)) {
      return NextResponse.json({ error: "messages array is required" }, { status: 400 })
    }

    const result = await generateSmartReplies(messages)

    return NextResponse.json({
      replies: result.replies,
      isAI: result.isAI,
      model: result.model,
    })
  } catch (error) {
    console.error("Smart replies API error:", error)
    return NextResponse.json({ error: "Failed to generate smart replies" }, { status: 500 })
  }
}

This route receives messages from SmartReplyBar, calls generateSmartReplies, and returns the replies. The integration combines Stream’s real-time message data with Synthetic’s AI inference for dynamic suggestions.

Displaying the Smart Replies in the Chat

Next, we create a custom SmartReplyBar component and render the AI reply suggestions as clickable buttons under the message input.

In your /components directory, create a file named smart-reply-bar.tsx to display AI-generated reply suggestions, then add the following code:

"use client"

import { useState, useEffect } from "react"

interface Message {
  id: string
  userId: string
  userName: string
  text: string
  timestamp: Date
}

interface SmartReplyBarProps {
  onSelectReply: (reply: string) => void
  messages: Message[]
}

export function SmartReplyBar({ onSelectReply, messages }: SmartReplyBarProps) {
  const [replies, setReplies] = useState<string[]>([])
  const [loading, setLoading] = useState(false)
  const [isAI, setIsAI] = useState(false)

  useEffect(() => {
    if (messages.length === 0) return
    const t = setTimeout(() => {
      void generateReplies()
    }, 400)
    return () => clearTimeout(t)
  }, [messages])

  const generateReplies = async () => {
    setLoading(true)
    try {
      const formattedMessages = messages.slice(-5).map((msg) => ({
        user: msg.userName,
        text: msg.text,
      }))

      const response = await fetch("/api/smart-replies", {
        method: "POST",
        headers: { "Content-Type": "application/json" },
        body: JSON.stringify({ messages: formattedMessages }),
      })

      if (response.ok) {
        const data = await response.json()
        setReplies(data.replies || [])
        setIsAI(data.isAI || false)
      } else {
        setReplies([])
        setIsAI(false)
      }
    } catch (error) {
      setReplies([])
      setIsAI(false)
    } finally {
      setLoading(false)
    }
  }
}

The SmartReplyBar component displays AI-generated replies as clickable Button components from the components/ui/button library.tsx below the message input. useEffect hook triggers generateReplies when new messages arrive from Stream’s messages via <SmartReplyBridge>.

generateReplies function sends the last five messages to the [api/smart-replies/route.ts](https://github.com/Tabintel/smart-chat/tree/master/app/api/smart-replies) endpoint, which calls Synthetic’s LLM to generate replies. AI replies are then stored in state and rendered as buttons with a dynamic style. The components/ui/button.tsx file defines reusable button styles:

const buttonVariants = cva(
  "inline-flex items-center justify-center gap-2 whitespace-nowrap rounded-md text-sm font-medium transition-all disabled:pointer-events-none disabled:opacity-50 [&_svg]:pointer-events-none [&_svg:not([class*='size-'])]:size-4 shrink-0 [&_svg]:shrink-0 outline-none focus-visible:border-ring focus-visible:ring-ring/50 focus-visible:ring-[3px] aria-invalid:ring-destructive/20 dark:aria-invalid:ring-destructive/40 aria-invalid:border-destructive",
  {
    variants: {
      variant: {
        default: 'bg-primary text-primary-foreground hover:bg-primary/90',
        // ... (Other variants)
      },
      size: {
        default: 'h-9 px-4 py-2 has-[>svg]:px-3',
        // ... (Other sizes)
      },
    },
    defaultVariants: {
      variant: 'default',
      size: 'default',
    },
  },
)

This ensures consistent, accessible buttons for the AI smart replies from Synthetic’s Inference.

Sending Suggested Replies

We use sendMessage and the button click to call channel.sendMessage() with the chosen reply. The reply then gets added to the conversation instantly.
In components/stream-chat-container.tsx, the component handles sending selected replies like so:

const { useChatContext } = require("stream-chat-react") as typeof import("stream-chat-react")
const { channel } = useChatContext()

const handleSelect = async (reply: string) => {
  await channel?.sendMessage({ text: reply })
}

return <SmartReplyBar messages={simplified} onSelectReply={handleSelect} />

When you click a smart reply button in SmartReplyBar, the onSelectReply callback triggers handleSelect, which uses Stream’s channel.sendMessage method to send the selected reply to the channel. This integrates Stream’s Chat React SDK with Synthetic’s AI-generated replies, instantly adding the reply to the conversation. The useChatContext hook provides access to the channel object, ensuring that all messages in the chat are sent properly, along with smart AI-suggested replies.

Running the App

Now that we have set up the chat application and Synthetic API integration, let’s run the application.

To start the application, open your terminal (command prompt) and enter this command npm run dev like so:

Copy http://localhost:3000 and open it in your browser. The app then loads like so:

Enter a random user ID and Display Name, then click on Join Chat:

After you enter your ID and username, the app shows the “Connecting to Stream chat” loader:

The chat app then loads like so:

We used a Discord-like interface for the Frontend UI. Now, send a message in the chat space. When typing, you’ll notice some quick replies you can send, which are done with AI Inference using Synthetic.

As you send messages in the app, you can see the chat logs on your Stream dashboard.

Cloning the GitHub repository

To get up to speed, you can also clone the GitHub repository for the project directory, install the required dependencies, and run the chat application.

Open your command terminal and run this command:

git clone https://github.com/Tabintel/smart-chat.git

Then navigate to this directory.

cd smart-chat

After opening the directory, run this command to install all the dependencies needed for the project.

npm install --legacy-peer-deps

We use the “legacy-peer-deps” command to instruct npm to ignore peer dependency conflicts during package installation.

Once the dependencies are installed, run the npm run dev command to start the application. You can then open the 'http://localhost:3000' URL in your browser to test the chat app.

Next Steps

In this guide, we created a chat app using Stream and Synthetic, with AI-powered smart replies that save users time with one-tap responses, keep conversations active, and make your app feel more innovative and modern.

You can extend the features we’ve built to:

Include syncing responses across different channels.
Add multiple AI agents to handle diverse use cases, such as customer support, marketplaces, or workplace chat.

Happy coding! 🎉

Speed up your designs to functional web apps with Kombai

Ekemini Samuel — Wed, 03 Dec 2025 19:20:52 +0000

Ekemini Samuel

Oct 7 '25

From Figma to Production: How I Optimized Frontend Development Time by 70%

#webdev #programming #ai #kombai

11 min read

The Best AI Agent For Frontend - Kombai

Ekemini Samuel — Mon, 24 Nov 2025 09:31:33 +0000

AI Agents have evolved a lot from when I started using them in 2022 till now. If you pay close attention, you will notice that these agents are growing not just in capabilities, that is, the kind of task they can do, but in understanding, which in my opinion is far important than the type of tasks they do. I say this for two reasons:

An AI agent is as good as the person using it.
An AI agent with a deep understanding of a particular context or subject matter will outperform generalistic AI agents long-term.

If you look at this from a wide view, the same applies to humans. A person who has developed knowledge, understanding and expertise in one area, for example, Finance, has built not just skill but discipline over an extended period of time. Say that person wants to learn Product design in 6 months, it won’t be as difficult for someone who has not first focused on a particular skill.

Now to the main purpose of this article. In the coding space, there are lots of AI coding agents and assistants like v0, Cursor, Claude Code, Lovable, Bolt, and Anything. These AI agents can write code and build websites, web apps and mobile apps quite alright. However, if you examine the UI/UX and Frontend, you will notice that they are similar. Either in terms of colour gradients, fonts, typography, framework (mostly Next.js/Vue) and even more technical aspects like state management.

They get the job done, yes, but if you want an actual product you and other people will use, then you definitely have to be unique and quite technical (or you get a technical friend) to help you out.

I have been using the Kombai AI agent for some time now, specifically for the Frontend of my web application, and in this guide, I will show you why it’s the best AI Agent for frontend development while building the frontend of a bookstore application.

Let’s get started!

Prerequisites

To follow along, you need these:

Basic understanding of JavaScript.
NPM Installed.
An IDE like Windsurf, VS Code.
A Kombai account.

Kombai is an AI agent built specifically for frontend development. It converts Figma designs, images, or text into production-ready code with modern web frameworks like React, Next.js, and Tailwind. With Kombai, you have the flexibility to choose the tech stack or framework you want to work with.

Learn more about Kombai in this resource.

What Kombai Does (and Doesn’t Do)

Before we build, it is important to define the scope. To claim Kombai is the "best," we must clarify what it is best at.

Kombai is a specialised Frontend AI Agent.

What it does: It specialises in UI, layout, client-side logic, and component architecture. It is designed to work safely within production repositories, understanding your specific design system.
What it doesn’t do: It is not a backend or database agent. It won't configure your AWS servers or write complex SQL queries.

By focusing entirely on the frontend, Kombai achieves a level of fidelity in CSS, React state, and accessibility that generalist full-stack agents simply cannot match.

What makes Kombai the best AI coding agent for frontend

Many AI tools can generate code, but their underlying architectures differ. General-purpose models are trained on broad datasets, while some newer agents are highly specialised. This specialisation is key, as the agent's architecture is purpose-built for a specific domain, in this case, frontend development. This focused approach is evident in its core technology.

Let's break down the specific features that result from this specialisation. Here are some reasons why Kombai is different from other AI agents:

Specialised Context Engineering

Instead of generating a generic <button>, the agent can read a project's theme.js file and output a custom <StyledButton variant="primary">. This is the result of specialised context engineering.

Before generating code, the system scans the repository to learn its structure. It identifies the component library, ingests design system tokens (colours, fonts, spacing), and understands existing custom components. The goal is to produce code that is idiomatic to the specific project, helping it align with established patterns and reducing the need for manual refactoring.

Planning and Preview Workflow

The Kombai AI workflow is designed to be more transparent than a simple prompt-to-code process. It operates in stages. First, it generates a step-by-step plan in markdown, detailing how it will build the feature, for example:

Create UserProfileCard.jsx
Add props for userName and avatarUrl
Style using Tailwind CSS

The Frontend developer can then review and edit this plan before execution. Once approved, the system generates the code and presents it in a live preview within an isolated browser sandbox. This allows for a verification step to catch runtime issues or styling problems before committing the code to the local branch.

Figma Interpretation Engine

A common challenge for design-to-code tools is handling imperfect design files if layers aren't clearly grouped.

Kombia’s AI agent's interpretation engine is designed to be more resilient, directly interpreting and converting the Figma design files to code. This approach effectively converts design frames into responsive components with logical DOM structures, rather than relying on pixel-based or absolute positioning.

Built-in RAG for Frontend Libraries

Another key technology at play is (RAG) Retrieval-Augmented Generation. Many general-purpose models may "hallucinate" or use outdated syntax when trying to implement a specific library, as their knowledge is based on a static, broad training dataset.

A RAG-based system addresses this by connecting the language model to an external, up-to-date knowledge base. Here’s how it works:

Retrieval: When you ask the agent to build a component using, say, Chakra UI, it doesn't just rely on its general knowledge. It first retrieves relevant, up-to-date information and code patterns from its own curated and human-tested database of over 30 frontend libraries (like Tailwind, MUI, Shadcn/ui).
Generation: It then augments the prompt. It essentially tells its AI model:

“Here is the user's request, and here is the correct, best-practice pattern for doing this in Chakra UI. Now, generate the code."

This RAG-based approach improves the model's output in verified patterns. It's not remembering how to use a library; it's referencing a correct implementation in real-time. This method is designed to improve accuracy and ensure the generated code follows the best practices for that specific library.

Enterprise-Ready Security

For enterprise teams that use AI coding agents, security is non-negotiable. Kombai has these security features implemented from the beginning:

No Training on Your Code: Kombai does not train its base models on your private codebase.
SOC 2 Compliance: It adheres to strict security standards, making it safe for enterprise deployment.
Local Context: It builds context from your local repo to answer questions, keeping data boundaries respectful.

Pricing & Plans

Kombai offers a pricing tier for every stage of development:

Free Plan: Great for students and indie devs. Includes a set amount of credits/month to build small projects.
Pro Plan: Designed for solo founders and freelancers who need higher limits and faster processing.
Enterprise: For teams that need SSO, enhanced security, and custom context limits.

Building the Bookstore app with Kombai

We will be building a bookstore web app with authentication, book listings, a dashboard, and a checkout flow.

First, let’s set up Kombai in your IDE (Windsurf is the IDE used in this article)

Create your Kombai account, sign in and install the extension for your IDE, whether it’s VS Code, Cursor, Windsurf or Trae.

If you’re using Windsurf, when you click on the Kombai icon in the left sidebar, it opens like so:

On the right, you can read about the latest updates to the Kombai AI agent and learn how to use the features in your IDE.

Click Open Folder to open a folder or workspace and start using Kombai.

In your computer, create a folder and name it “bookstore”. Then open it in the Windsurf IDE with Kombai.

When you start the Kombai extension, it scans your workspace for context awareness. It loads and shows you this modal regarding the Plan Mode. This mode generates a detailed code plan for the task (from your prompts) before the AI agent starts generating the code.

Now that Kombai is set up, let’s proceed with building the bookstore app.

Planning before generating code

With most AI tools, the flow for building apps is mostly from text/prompt to action. When you prompt the AI tool, it immediately starts generating the code for the application. While that is faster, it does not give room to plan the task properly, which is helpful for making iterations during frontend development.

Kombai uses a more professional workflow: Plan, Code and Ask.

Plan: You describe the feature. Kombai generates a detailed markdown plan. This makes sure planning is done before any code generation.
Code: You review and approve the plan. Kombai then executes it, generating files and fixing its own errors.
Ask: You can ask questions about your repo without changing code.

In the Kombai chat, enter this prompt:

We will be building the frontend of a modern bookstore app. Let's plan the bookstore app and UI.
List all required screens, components, and user states for authentication, dashboard, checkout, and book browsing. 
Use React.js + Tailwind + TypeScript. Focus on clean visual hierarchy and modularity.

Note: You can also instruct Kombai to turn your designs into code by either entering a link to the Figma file or directly connecting Figma by clicking on the Connect Figma button in the image.

Before you send the prompt, click on the Code dropdown and select Plan. This will create a structured plan before coding.

When you have selected the Plan mode, send the prompt to Kombai. It then lists the tech stack that will be used. You can either let Kombai AI scan the workspace for any files, configure the tech stack to be used and then proceed.

Now click on Proceed. The AI agent then prepares a comprehensive plan, including all of these. Scroll down the plan.md document or click on the file icon at the top right to expand the view.

Design philosophy: For the design philosophy, the AI agent suggests the visual identity as a modern, clean, editorial-inspired design with a focus on book discovery.

Screen architecture: The screen architecture shows the screens for the bookstore app, the routes where authentication is required, and the purpose of each screen.

Component breakdown: The AI agent also lists all the components needed, like the layout, authentication, book components, shopping, dashboard, the shared UI, and other components needed in the bookstore app

User state management: Next is the state management, which includes the state domains, storage, key data and type definitions.

Authentication flow: The AI agent also creates a visual representation of the authentication flow, from when the user visits the bookstore app, authenticates, adds a book to the cart and checks out. It also includes the Auth States: idle, loading, authenticated, error.

Checkout user journey: Next, we have the checkout user journey, which includes the steps, screens, actions and validation. The AI agent also includes a progress indicator and a stepper component (1/6 → 6/6).

Browse page layout: There is also the browse page layout, like the header (search, cart, user), filters sidebar, the book genre, price for each book and ratings.

Reusable patterns: The AI agent also lists reusable patterns, their usage and props to be used.

Accessibility checklist: It also adds the accessibility checklist, like ARIA labels for icon buttons, Alt text for all book covers, and screen reader announcements for cart updates.

Performance optimisations: The AI Agent also lists the performance optimisations with the strategies and how they will be implemented in the bookstore application.

The above metrics are important when developing the frontend of a web application, and it’s necessary to consider them before building. Instead of going directly to generating code.

The Kombai AI agent also lists the Todos (10) needed for the bookstore web app.

When you’re okay with the plan the AI agent generated, click on Approve Plan.

The AI agent now begins creating the bookstore application with the plan we approved.

Setting up the bookstore project

The Kombai AI agent begins creating the application and installing the needed dependencies with the npm command. Click Run like so:

While installing the dependencies, the AI agent loads the terminal to the right side and continues generating the code for the bookstore app. It lists every file it’s creating and the purpose for each one.

When the AI agent is done setting up the bookstore app project and generating the code files, it then shows a prompt to run the app with the npm run dev command. Click on the Run button as shown in the image below:

After the AI agent runs the bookstore application, it provides a local URL: http://localhost:5173 you can open in your browser to test the application. Also, you can open the preview in the Kombai Browser by clicking on the Kombai Browser.

The bookstore app then loads in the browser.

You can view all the books displayed like so:

Now we have the frontend of the bookstore developed with the Kombai AI agent. Next, we will check the agent-controlled and user-controlled browser features of Kombai.

Agent-controlled and User-controlled Browsers

The Kombai team shipped an update recently called the Kombai browser, which transforms how AI-assisted Frontend development works. It involves an Agent-controlled and User-controlled browser.

Let’s go deeper into the capabilities and use cases of each browser.

Agent-controlled Browser

The idea for the Agent-controlled browser is simple: instead of you running your app and manually checking everything, the agent can open your app and use it like a real user.

Watch this video to see the Kombai Agent-controlled browser in action.

It can navigate, click, type, scroll, and perform actions on the web application.

For the bookstore app we’re building, I asked the Agent to:

Open the browser, sign up, log in, and add a book to your cart

With the Agent-controlled browser, it loaded the bookstore app, created an accreated a new account, then logged in and added a book to the cart.

After the AI agent adds the book to the cart, it displays like so:

The AI agent also provides a summary of all the actions it took on the browser.

Here is a video showing how the Agent-controlled browser works when we give it the prompt

User-controlled Browser

The Kombai User-controlled browser lets you preview your app’s local deployment in the browser (optimised for Google Chrome and Chromium-based browsers), with built-in listeners so you can iterate fast by sending elements and errors back to Kombai as context.

To open the preview, follow these steps:

Click the Start Preview button below the Kombai input box. Kombai will start the dev server.

Once the server is running, click the Kombai Browser button to open the browser preview. Alternatively, press the Kombai Browser button that appears after the code generation process is completed.

You can also prompt the AI agent to make changes to the web app in real-time using the chat window, as shown in the image below.

You can also use the select elements button highlighted below to select any section that needs an update. This makes implementing additional changes faster using the user-controlled browser.

You can also initialise a GitHub repository directly with the AI agent by giving it a prompt and clicking on the Run button.

Initialise a GitHub repository, git add. git commit, and git push

You can clone the GitHub repository, install the required dependencies and run the bookstore application.

git clone https://github.com/Tabintel/bookstore.git

cd bookstore
npm install
npm run dev

And that’s it! We have built the frontend of a bookstore application with Kombai and tested it out using the Agent and User-controlled browsers.

Next Steps

The major takeaway from developing the frontend of applications with Kombai is that the AI agent works as an AI coding “partner”, not a plugin for an IDE. You can make updates to the bookstore by adding a community feature where other users can post comments on books they buy.

Optimise your frontend development today, start building with Kombai!

Analysing European Soccer Data with Deepnote in Windsurf IDE

Ekemini Samuel — Tue, 11 Nov 2025 19:09:43 +0000

Word on the street? Deepnote is now open-source! Jakub Jurových, the Founder of Deepnote, announced on LinkedIn this week.

During my research, I found out that the European Soccer Database from Kaggle has 25K+ matches, 10K+ players, and 2008–2016 seasons across 11 leagues. In this guide will use Deepnote in Windsurf to analyse the European sports data from Kaggle for three use cases:

To find out which team improved most post-2012 from the data?
To predict "star players" and their value, with the Top 10 players by goal impact
To analyse win rates, goal differences, and home/away advantages over seasons for the teams

Let's get started!

Prerequisites

You will need the following to proceed

A Kaggle account
Windsurf account and IDE installed on your computer
A Deepnote account

What is Deepnote?

Deepnote is a cloud-based, collaborative data science notebook platform that enhances Jupyter notebooks with real-time co-editing, integrated data connections, and simplified sharing, acting as a central workspace for teams to develop and deploy data-driven products. It supports Python, R, and SQL

To get started, we will download the European soccer database from Kaggle. Click on the Download button like so

Then click on Download dataset as zip

When the dataset is downloaded on your computer, extract it to your preferred directory. For this project, I created a directory on my computer called windsurf_deepnote. You can do the same and extract the dataset to that specific folder.

After the dataset is extracted, you get a file named database.sqlite in the directory.

Now that we have the dataset, lets setup Windsurf.

Setting up Windsurf IDE

Windsurf is an integrated development environment (IDE) for coding. It is similar to VS Code, with AI-first features.

Go to the Windsurf website, and download the application suited for your operating system (OS).

After it has downloaded, sign up with your preferred method, and open the Windsurf IDE with the directory you want to work with.

You can also view your Windsurf account activity in the profile section like so

Using Deepnote in Windsurf

Now we have the dataset and Windsurf setup, let's proceed to working with Deepnote in Windsurf.

Sign up here to create your Deepnote account

In the Windsurf IDE, go to the extensions tab at the left sidebar, search for Deepnote and click on the Install button like so

After it has installed, this tab loads. It has information on how to get started using Deepnote in Windsurf

Next, use the Ctrl+Shift+P command to create a new Deepnote project, and name it soccer.

It then creates a new project called soccer and loads a new Deepnote block, just as in the Deepnote web app UI

We are using Kaggle to get the football/soccer data that we will be analysing.

You can enter python code in blocks and run like so

If you want to convert a notebook to a deepnote notebook, open your terminal and run this command:

npx @deepnote/convert notebook.ipynb # This will convert the notebook and create notebook.deepnote

Enter yes (y) for the prompts that come up during installation

After it has been installed, Then open your .deepnote file in Windsurf IDE.

Implementing the data analysis of the three use Cases

From the soccer dataset, we will run each of this Python codes to analyse the data.

Find Teams That Improved Most Post-2012

We will find which teams improved most after 2012, by comparing average team performance (win rate, goals) before and after 2012

Enter this code:

import sqlite3
import pandas as pd
import matplotlib.pyplot as plt

# Connect to the SQLite database
conn = sqlite3.connect("database.sqlite")

# Load match data
matches = pd.read_sql_query("""
SELECT
    m.id, m.season, m.home_team_api_id, m.away_team_api_id,
    m.home_team_goal, m.away_team_goal,
    t.team_long_name AS home_team
FROM Match m
JOIN Team t ON m.home_team_api_id = t.team_api_id
""", conn)

# Compute win rate per team per season
matches["result"] = matches.apply(
    lambda x: "win" if x["home_team_goal"] > x["away_team_goal"] else 
              ("loss" if x["home_team_goal"] < x["away_team_goal"] else "draw"), axis=1)

team_stats = matches.groupby(["home_team", "season"])["result"].value_counts().unstack().fillna(0)
team_stats["win_rate"] = team_stats["win"] / (team_stats["win"] + team_stats["loss"] + team_stats["draw"])

# Split pre- and post-2012 averages
pre_2012 = team_stats[team_stats.index.get_level_values("season") < "2012/2013"].groupby("home_team")["win_rate"].mean()
post_2012 = team_stats[team_stats.index.get_level_values("season") >= "2012/2013"].groupby("home_team")["win_rate"].mean()

improvement = (post_2012 - pre_2012).dropna().sort_values(ascending=False).head(10)

# Visualize
plt.figure(figsize=(10,5))
improvement.plot(kind="bar", color="green")
plt.title("Top 10 Most Improved Teams Post-2012")
plt.ylabel("Win Rate Improvement")
plt.tight_layout()
plt.show()

After we run the code block above, we get this visualization:

From the data, we confirm that Juventus showed the biggest leap after 2012, improving their win rate by +30%.

Borussia Mönchengladbach and AS Saint-Étienne followed closely, both gaining around 25%, proving that the post-2012 era rewarded better management, new ownership money, and tactics.

Predict Star Players by Goal Impact

Next we predict star players and rank the Top 10 by goal impact score, by combining player attributes and match stats, then using regression to find impact.

Enter this python code block and run it:

import sqlite3
import pandas as pd
from sklearn.linear_model import LinearRegression
import matplotlib.pyplot as plt

conn = sqlite3.connect("database.sqlite")

# Load player data with names
players = pd.read_sql_query("""
    SELECT p.player_api_id, p.player_name, 
           pa.overall_rating, pa.potential 
    FROM Player p
    JOIN Player_Attributes pa ON p.player_api_id = pa.player_api_id
""", conn)

# Load match data
goals = pd.read_sql_query("""
    SELECT home_player_1, home_player_2, home_player_3, home_team_goal 
    FROM Match
""", conn)

# Aggregate goals by player
player_goals = goals.melt(
    value_vars=['home_player_1', 'home_player_2', 'home_player_3'],
    var_name='position', 
    value_name='player_api_id'
)
player_goals = player_goals.groupby('player_api_id').size().reset_index(name='goal_count')

# Merge with player data and calculate impact score
merged = players.groupby(['player_api_id', 'player_name']).mean().reset_index()
merged = merged.merge(player_goals, on='player_api_id', how='inner').dropna()

# Model goal impact
X = merged[['overall_rating', 'potential']]
y = merged['goal_count']
model = LinearRegression().fit(X, y)
merged['impact_score'] = model.predict(X)

# Get top 10 players
top_players = merged.sort_values('impact_score', ascending=False).head(10)

# Plot with player names
plt.figure(figsize=(12, 6))
plt.barh(top_players['player_name'], top_players['impact_score'], color='orange')
plt.title("Top 10 Star Players by Goal Impact Score", pad=20)
plt.xlabel("Predicted Impact Score")
plt.ylabel("Player Name")
plt.gca().invert_yaxis()
plt.tight_layout()
plt.show()

After running the code, we get this visualisation:

From the data, we confirm that Iker Casillas tops the list as the star player with the highest predicted goal-impact score (~41), followed closely by Sergio Ramos, Manuel Neuer, and Petr Čech. John Terry also rounds out the top spot.

Win Rate & Home Advantage Analysis

Then we analyse win rates, goal differences, and home/away advantages across seasons. We do this by comparing home vs away wins by season and visualize trends.

Run this python code block:

import sqlite3
import pandas as pd
import matplotlib.pyplot as plt

# Connect to the database
conn = sqlite3.connect("database.sqlite")

# Load match data with team names
matches = pd.read_sql_query("""
    SELECT 
        m.season,
        t1.team_long_name as home_team,
        t2.team_long_name as away_team,
        m.home_team_goal,
        m.away_team_goal
    FROM Match m
    JOIN Team t1 ON m.home_team_api_id = t1.team_api_id
    JOIN Team t2 ON m.away_team_api_id = t2.team_api_id
    WHERE m.season IS NOT NULL
""", conn)

# Calculate match results
matches['home_win'] = matches['home_team_goal'] > matches['away_team_goal']
matches['draw'] = matches['home_team_goal'] == matches['away_team_goal']
matches['away_win'] = matches['home_team_goal'] < matches['away_team_goal']

# Calculate win rates per team
home_stats = matches.groupby('home_team').agg({
    'home_win': 'sum',
    'home_team_goal': 'sum',
    'away_team_goal': 'sum'
}).rename(columns={
    'home_win': 'wins',
    'home_team_goal': 'goals_for',
    'away_team_goal': 'goals_against'
})

away_stats = matches.groupby('away_team').agg({
    'away_win': 'sum',
    'away_team_goal': 'sum',
    'home_team_goal': 'sum'
}).rename(columns={
    'away_win': 'wins',
    'away_team_goal': 'goals_for',
    'home_team_goal': 'goals_against'
})

# Combine home and away stats
team_stats = home_stats.add(away_stats, fill_value=0)
team_stats['total_matches'] = matches['home_team'].value_counts().add(
    matches['away_team'].value_counts(), fill_value=0)
team_stats['win_rate'] = (team_stats['wins'] / team_stats['total_matches']) * 100
team_stats['goal_difference'] = team_stats['goals_for'] - team_stats['goals_against']

# Get top 10 teams by win rate
top_teams = team_stats.nlargest(10, 'win_rate').sort_values('win_rate')

# Plotting
plt.figure(figsize=(12, 8))

# Create bar plot
bars = plt.barh(top_teams.index, top_teams['win_rate'], color='skyblue')
plt.title('Top 10 Teams by Win Rate', fontsize=16, pad=20)
plt.xlabel('Win Rate (%)', fontsize=12)
plt.ylabel('Team', fontsize=12)

# Add value labels
for bar in bars:
    width = bar.get_width()
    plt.text(width + 0.5, 
             bar.get_y() + bar.get_height()/2, 
             f'{width:.1f}%',
             va='center',
             fontsize=10)

plt.grid(True, linestyle='--', alpha=0.6)
plt.tight_layout()

# Save the plot
plt.savefig('team_win_rates.png', bbox_inches='tight', dpi=100)
plt.close()

print("Top 10 Teams by Win Rate:")
print(top_teams[['win_rate', 'total_matches', 'goal_difference']].sort_values('win_rate', ascending=False).to_string())

print("\nPlot saved as 'team_win_rates.png'")

After running the code, we get the visualization of the dataset like so:

We confirm that FC Barcelona dominates with a 77% overall win rate across the entire period, followed by Real Madrid (75%) and SL Benfica (74.6%). The data shows the classic home advantage of roughly 55–60% win rate for home teams throughout 2008–2016.

And that's it! We have a great analysis of the soccer data!

Next steps

Show the Deepnote some support and star the GitHub repository !😎

Thank you for reading!