DEV Community: Studio1

Building a Zulip Style Collaborative Chat App with Next.js and Velt

Astrodevil — Tue, 05 May 2026 17:43:33 +0000

Zulip is known for keeping conversations organized. Topic-based threads, clear context, and async-friendly discussions make it a favorite for technical and distributed teams. Unlike traditional chat apps, conversations in Zulip stay readable even as teams scale.

Recreating this experience can be a bit difficult. Real-time messaging, user presence, inline comments, and notifications usually require a complex backend and real-time infrastructure.

In this tutorial, we will build a Zulip-style collaborative chat application using Next.js, Tailwind CSS, and Velt. Next.js powers the UI and application structure, while Velt adds collaboration features like presence, comments, and notifications without writing backend code.

By the end, you will have a working chat interface inspired by Zulip, with real-time collaboration built in and ready to extend.

Why Zulip’s Approach Works

Zulip’s design solves a problem most chat tools struggle with: conversations getting messy over time.

Instead of long, linear message streams, Zulip organizes discussions into topics. Each message belongs to a clear context, so conversations stay focused and easy to follow even days or weeks later.

Three ideas make this work especially well:

Real-time collaboration: Messages, comments, and updates appear instantly for everyone in the channel.
Context-rich discussions: Replies stay tied to a specific topic or message, so feedback does not get lost in the noise.
Visible presence: You always know who is online and actively participating, which makes collaboration feel immediate and shared.

What makes this challenging to build in a SaaS product is not the UI, but the infrastructure behind it. Real-time messaging, presence tracking, comments, and notifications typically require WebSockets, backend event systems, data synchronization, and careful handling of concurrent users. Building and maintaining this reliably can take significant engineering effort.

In this tutorial, we recreate Zulip’s collaborative behavior without building that infrastructure ourselves, by using Velt as the collaboration layer.

What We’re Building With

Before we dive into the code, let’s quickly look at the tools we’ll use and why they fit this project well.

Next.js: Next.js gives us a solid foundation for building interactive applications. With the App Router, we get a clean layout structure and client-side interactivity that works well for chat-style interfaces.
React: React handles the UI composition and state updates. The chat interface, message list, and user interactions all rely on simple, predictable React patterns.
Tailwind CSS: Tailwind helps us build a clean and modern UI quickly. It keeps styling close to the components and makes it easy to adjust layouts, spacing, and themes without writing custom CSS.
shadcn ui (powered by Radix UI): These provide accessible, reusable UI primitives like buttons, dropdowns, and avatars. They give us a polished look without locking us into heavy component libraries.
Zustand: Zustand is used for lightweight state management. In this project, it manages demo users and allows us to switch between them to test real-time collaboration.
Velt: Velt is the key piece. Instead of building real-time infrastructure ourselves, Velt provides presence, comments, notifications, and collaboration out of the box. Once integrated, features like reactions, read status, and threaded comments work automatically without writing extra backend or real-time code.

Together, this stack lets us focus on the chat experience and UI, while Velt handles the collaboration layer behind the scenes.

Prerequisites

To follow along with this tutorial, you’ll need:

Node.js (v16 or later) installed on your machine
Basic familiarity with Next.js, React, and TypeScript
A Velt account (you can sign up for free at velt.dev)
Working knowledge of Tailwind CSS fundamentals

You do not need:

Prior experience with Velt
Any backend or database setup
Experience building real-time systems

We’ll walk through the collaboration setup step by step, and the app runs entirely on the frontend.

Project Setup

Instead of scaffolding a new project from scratch, we’ll start from an existing Zulip style chat app that already has the UI and collaboration logic wired up. This lets us focus on understanding how the pieces fit together.

Clone the Repository

Begin by cloning the project and moving into the directory:

gitclone https://github.com/Studio1HQ/zulip-velt
cd zulip-velt

Install Dependencies

Install all required dependencies using npm:

npm install

This will install Next.js, Tailwind CSS, shadcn ui components, Zustand for state management, and the Velt SDK used for collaboration features.

Configure the Velt API Key

Velt requires a public API key to enable collaboration features like presence, comments, and notifications.

Create a .env.local file in the root of the project and add:

NEXT_PUBLIC_VELT_ID=your_velt_api_key_here

You can generate this key from the Velt dashboard after creating a free account.

Once the key is added, restart the development server if it is already running.

Run the App Locally

Start the development server:

npm run dev

Open your browser and visit:

http://localhost:3000

You should now see a Zulip-style chat interface with channels on the left, a message area in the center, and collaboration controls in the header.

At this point, the UI is already functional. In the next sections, we’ll break down how the app is structured and how Velt is integrated to power real-time collaboration without any backend setup.

Project Setup

At a high level, the project is divided into five main parts:

app: Handles routing, layouts, and global configuration using the Next.js App Router
components: Contains all reusable UI and chat-related components
helper: Manages demo user data and user switching logic
hooks: Stores custom React hooks such as theme management
lib: Holds small utility functions used across the app

Let’s take a closer look at the most important folders we’ll work with in this tutorial.

Understanding the App Router and Layout Setup

This project uses the Next.js App Router, which organizes the application using layouts and pages instead of traditional route files. If you are new to the App Router, don’t worry. We’ll focus only on what matters for this app.

The key idea is simple: layouts wrap pages, and this is where we place Velt and theme handling.

In this project, there are two layout files, each with a different responsibility.

layout.tsx is the root layout for the entire application. It defines the HTML structure, metadata, and global styles.

"use client";

import { ThemeProvider } from "@/hooks/use-theme";
import { VeltProvider } from "@veltdev/react";

export default function RootLayout({
  children,
}: {
  children: React.ReactNode;
}) {
  return (
    <VeltProvider apiKey={process.env.NEXT_PUBLIC_VELT_ID || ""}>
      <ThemeProvider>{children}</ThemeProvider>
    </VeltProvider>
  );
}

page.tsx - This file renders the main chat interface. You’ll see it importing layout and chat components and placing them on the page.

"use client"

import { useState, useEffect } from 'react'
import { Header } from '@/components/layout/header'
import { Sidebar } from '@/components/layout/sidebar'
import { ChatArea } from '@/components/chat/chat-area'

export default function Home() {
  const [isSidebarOpen, setIsSidebarOpen] = useState(true)
  const [isSidebarCollapsed, setIsSidebarCollapsed] = useState(false)
  const [isMobile, setIsMobile] = useState(false)

  useEffect(() => {
    const checkMobile = () => {
      const mobile = window.innerWidth < 768
      setIsMobile(mobile)
      if (mobile) {
        setIsSidebarOpen(false)
        setIsSidebarCollapsed(false)
      }
    }

    checkMobile()
    window.addEventListener('resize', checkMobile)
    return () => window.removeEventListener('resize', checkMobile)
  }, [])

  const toggleSidebar = () => {
    if (isMobile) {
      setIsSidebarOpen(!isSidebarOpen)
    } else {
      if (isSidebarCollapsed) {
        setIsSidebarCollapsed(false)
      } else if (isSidebarOpen) {
        setIsSidebarCollapsed(true)
      } else {
        setIsSidebarOpen(true)
      }
    }
  }

  const closeSidebar = () => {
    if (isMobile) {
      setIsSidebarOpen(false)
    }
  }

  return (
    <div className="h-screen flex flex-col bg-background">
      <Header 
        onToggleSidebar={toggleSidebar}
        isSidebarOpen={isSidebarOpen}
        isMobile={isMobile}
      />
      <div className="flex-1 flex overflow-hidden">
        <Sidebar 
          isOpen={isSidebarOpen}
          isCollapsed={isSidebarCollapsed}
          isMobile={isMobile}
          onClose={closeSidebar}
        />
        <ChatArea />
      </div>
    </div>
  )
}

app/layout.tsx is the global shell
app/(app)/layout.tsx is where Velt is initialized
page.tsx is where the chat UI begins
Providers are placed at layout level so all components can access them

Building the Chat Interface with Components

Most of the Zulip-style experience in this app is built using reusable React components. Each part of the interface has a clear responsibility, which makes the code easier to read and extend.

We’ll focus on three areas:

The chat area where messages appear
The message component itself
The layout components that shape the overall UI

Chat Area:

chat-area.tsx - This component is responsible for rendering the list of messages and attaching collaboration features to them.

message.tsx - Each message in the chat is rendered as its own component. This keeps the UI modular and makes it easy to attach collaboration features at the message level.

Layout Components:

The layout of the app is built using two main components: a header and a sidebar.

The sidebar represents Zulip style channels and navigation.
The header is where collaboration controls live.

Inside the header, you’ll see Velt components like:

<VeltPresence />
<VeltNotificationsTool />
<VeltCommentsSidebar />
<VeltSidebarButton />

What these do:

VeltPresence shows who is currently online
VeltNotificationsTool displays collaboration notifications
VeltCommentsSidebar Opens a panel with all comments
VeltSidebarButton toggles the comments sidebar

All of these work automatically once Velt is initialized at the layout level.

import {
  VeltPresence,
  VeltNotificationsTool,
  VeltSidebarButton,
  VeltCommentsSidebar,
  useVeltClient,
} from "@veltdev/react";
import { names, userIds, useUserStore } from "@/helper/userdb";

interface HeaderProps {
  onToggleSidebar: () => void;
  isSidebarOpen: boolean;
  isMobile: boolean;
}

export function Header({
  onToggleSidebar,
  isSidebarOpen,
  isMobile,
}: HeaderProps) {
  const { theme } = useTheme();
  const { user, setUser } = useUserStore();
  const { client } = useVeltClient();
  const prevUserRef = useRef(user);
  const isInitializingRef = useRef(false); // Prevent overlapping initialization calls

Client Initialization, User Identification, and Document Setting

 // Handle Velt client initialization, user identification, and document setting
  useEffect(() => {
    if (!client || !user || isInitializingRef.current) {
      console.log("Velt init skipped:", {
        client: !!client,
        user: !!user,
        initializing: isInitializingRef.current,
      });
      return;
    }

    const initializeVelt = async () => {
      isInitializingRef.current = true;
      try {
        // Detect user switch
        const isUserSwitch = prevUserRef.current?.uid !== user.uid;
        prevUserRef.current = user;

        console.log("Starting Velt init for user:", user.uid, { isUserSwitch });

        // Re-identify the user (handles initial and switches)
        const veltUser = {
          userId: user.uid,
          organizationId: "organization_id",
          name: user.displayName,
          email: user.email,
          photoUrl: user.photoUrl,
        };
        await client.identify(veltUser);
        console.log("Velt user identified:", veltUser.userId);
        await client.setDocuments([
          {
            id: "zulip-velt",
            metadata: { documentName: "zulip-velt" },
          },
        ]);
        console.log("Velt documents set: zulip-velt");
      } catch (error) {
        console.error("Error initializing Velt:", error);
      } finally {
        isInitializingRef.current = false;
      }
    };

    initializeVelt();
  }, [client, user]); // Re-run on client or user change

Reusable UI Components

To keep the chat interface clean and consistent, this project uses a set of reusable UI components. These components handle common UI patterns like buttons, inputs, avatars, and scroll areas.

Instead of building these from scratch, the project uses shadcn ui, which is a collection of accessible, unstyled components built on top of Radix UI. This approach gives us flexibility while keeping the UI consistent.

All reusable UI components live inside the components/ui folder.

Reusable UI primitives: Shared building blocks like buttons, inputs, avatars, and dropdowns used across the app to keep styling consistent.
Built with shadcn ui (powered by Radix UI): Accessible, unstyled components that provide behavior and keyboard support without locking the design.
Scroll handling for chat: The ScrollArea component manages smooth scrolling for long message lists in the chat view.
Separation of concerns: UI components handle appearance and interaction, while chat and layout components focus on structure and logic.
Easy to extend and customize: Updating a UI component in this folder automatically updates its usage across the entire app.

User Management for Collaboration Testing

To test real-time collaboration features like presence, comments, and notifications, we need a way to simulate multiple users. Instead of setting up a full authentication system, this project uses a simple and beginner-friendly approach with predefined users.

User state is managed using Zustand, a lightweight state management library.

import { create } from "zustand";
import { persist } from "zustand/middleware";

export type User = {
  uid: string;
  displayName: string;
  email: string;
  photoUrl?: string;
};

export interface UserStore {
  user: User | null;
  setUser: (user: User) => void;
}

export const userIds = ["user001", "user002"];
export const names = ["Nany", "Mary"];

export const useUserStore = create<UserStore>()(
  persist(
    (set) => ({
      user: null,
      setUser: (user) => set({ user }),
    }),
    {
      name: "user-storage",
    }
  )
);

Theme Management with `useTheme`

Modern apps usually support both light and dark themes, and this project follows the same pattern. Theme state is managed using a custom React hook, which keeps the logic reusable and easy to maintain.

The theme logic lives inside a single file.

The theme is:

Stored in React state
Loaded from localStorage on page load
Saved back to localStorage when changed

This ensures the user’s theme preference persists across page refreshes.

Syncing Theme with Velt

The theme value is also passed to Velt components, allowing them to automatically switch between light and dark modes.

This means:

The app UI and Velt UI stay visually consistent
No extra styling logic is required for collaboration components
Once the hook is set up, theme changes propagate everywhere.

Utility Functions

The lib folder contains small helper utilities that are shared across the application. In this project, it mainly exists to support reusable UI components and keep common logic out of individual files. The utilities in this file are primarily related to class name handling. When working with Tailwind CSS and reusable components, it’s common to conditionally apply multiple class names.

Running the App and Testing the Output

Now that we’ve walked through the project structure and key files, it’s time to run the app and verify that everything works as expected.

Seeing the app in action makes the collaboration features easier to understand in the next sections.

From the project root, run:

npm run dev

Once the server starts, open your browser and navigate to:

http://localhost:3000

You should see a Zulip style chat interface with:

A sidebar for channels
A header with user controls
A central chat area for messages

To test collaboration features, open the app in two browser windows or one normal window and one incognito window.

Use the user switcher in the header to change between demo users such as Nany and Mary.

Each window should represent a different user.

Try the following actions:

Presence: Notice user avatars appear in the header when multiple users are online.
Comments: Hover over a message and add a comment. The comment should appear instantly in both windows.
Notifications: Add a comment in one window and check the notification bell in the other.
Theme Switching: Toggle between light and dark mode and observe that both the app and Velt components update correctly.

How It Works

Here’s a simple view of what happens when the app runs:

The app loads and initializes Velt using the API key in the layout
A demo user is selected using the Zustand store
Velt identifies the active user and associates them with the app
A shared document context is set for collaboration
Messages render in the chat area
Comments, presence, and notifications are automatically tracked by Velt
Updates sync instantly across all connected clients

The key idea is that the app focuses on UI and user flow, while Velt handles real-time synchronization and collaboration logic.

Collaborative Features You Get Automatically

Once Velt is integrated, collaboration features start working without writing additional backend or real-time code.

These features are available out of the box:

Inline comments: Add comments directly on chat messages and view them in context.
User presence: See who is currently online and active in the chat.
Notifications: Get notified when someone comments or interacts.
Reactions and comment status: React to comments and mark them as resolved or active.
Read awareness: See when comments have been viewed by other users.

All of these features work automatically once Velt is set up.

Things to Note Before Shipping to Production

Before using this setup in a real product, keep the following in mind:

Replace demo users with a real authentication system
Identify users using real user IDs and metadata
Use dynamic document IDs instead of a hardcoded value
Add permissions to control who can view or comment
Handle error states when collaboration services are unavailable
Review Velt customization options to match your product UI
Test performance with multiple concurrent users

These steps ensure the app scales safely beyond a demo environment.

Demo Video

You can see the completed Zulip style chat app with real-time collaboration in action here:

Live Demo:

https://zulip-velt-chat.vercel.app/

This demo shows user switching, comments, presence, notifications, and theme support working together.

Conclusion

Zulip demonstrates how powerful chat can be when collaboration is built into the experience. Organized conversations, clear context, and real-time awareness turn chat from simple messaging into a system teams can rely on.

For builders, the real challenge is not the UI. It is the collaboration layer behind it. Features like real-time sync, presence tracking, inline comments, notifications, and reactions usually require complex infrastructure and long development cycles. This is especially true when building a chat-based SaaS product.

In this tutorial, we focused on that exact problem. Instead of rebuilding collaboration from scratch, we showed how to add Zulip-style collaborative features inside a chat application using Next.js and a modern frontend stack, while delegating the hard real-time problems to Velt.

The result is a working chat app where collaboration feels native, but the code stays simple and frontend-focused.

Key Takeaways

Clean component structure makes complex apps easier to understand
Collaboration does not need custom real-time infrastructure
Velt enables comments, presence, notifications, and more out of the box
Frontend-focused development speeds up experimentation and iteration

If you are building a chat app, internal tool, or collaborative product, Velt lets you focus on your core experience instead of infrastructure.

Try building with Velt and add collaboration to your app in minutes at velt.dev.

Scaling Karpathy’s AutoResearch Using Nebius Token Factory

Arindam Majumder — Sun, 03 May 2026 13:50:44 +0000

Introduction

We ran about 250 prompt optimization experiments overnight using a small AI agent loop. The idea was simple: let an AI system propose an experiment, run it, evaluate the result, and then try again with a better idea. Instead of manually testing prompts one by one, the system keeps improving its own attempts over multiple iterations.

This idea comes from Andrej Karpathy’s AutoResearch, where an AI agent can automate the typical machine learning research cycle. In a normal workflow, researchers adjust parameters, run experiments, observe the results, and repeat the process many times before reaching a good configuration. AutoResearch shows that this repetitive process can be handled by an intelligent agent.

In this article, we will walk through how we built a cloud-native AutoResearch loop using Nebius Token Factory for LLM inference, allowing the agent to run hundreds of experiments automatically while keeping structured records of every attempt.

Pitfalls in the Original AutoResearch Implementation

Karpathy’s AutoResearch project is a very interesting idea and a great starting point to understand how AI agents can run experiments automatically.

The repository mainly consists of simple files such as program.md (which defines the research goal), train.py (which runs the experiment), and a lightweight results log that records experiment outcomes. The agent reads the goal, modifies the experiment code, runs it, and stores the results, demonstrating how an AI system can iterate through experiments automatically.

It was built as a research prototype to demonstrate the concept, not as a full system for running large-scale experiments in real workflows.

As a result, a few limitations arise when we try to scale the idea.

Experiment tracking using TSV files: In the original implementation, experiment results are stored in a simple TSV (tab-separated values) file. This file usually contains basic fields like experiment ID, score, and parameters. While this is easy to implement, it becomes difficult to manage as the number of experiments grows.
Limited structure for experiment metadata: A flat TSV file does not provide structured storage for experiment details such as prompts, responses, timestamps, or reasoning steps.
Local model dependency: The original workflow usually depends on running models locally. This means developers often need access to local GPUs or preconfigured environments to run inference.
Difficulty scaling experiment loops: Because of the local infrastructure and simple logging system, running hundreds of experiments in a controlled way becomes harder.

// Detect dark theme var iframe = document.getElementById('tweet-2030371219518931079-421'); if (document.body.className.includes('dark-theme')) { iframe.src = "https://platform.twitter.com/embed/Tweet.html?id=2030371219518931079&theme=dark" }

What We Are Going to Build

The use case for this project is prompt optimization. In many real workflows, developers often try multiple prompts manually to get the best response from a language model. This usually involves repeated trial-and-error, where prompts are slightly modified, tested, and evaluated before arriving at a good result. When the number of experiments increases, managing and tracking these attempts becomes difficult.

In this tutorial, we will build a small system that keeps the same AutoResearch idea while addressing a few of those practical limitations.

AutoResearch-style experiment loop: We implement the same research cycle where an agent proposes an experiment, runs it, evaluates the result, stores the outcome, and then proposes the next attempt based on previous results.
Cloud-based reasoning using Nebius Token Factory: Instead of relying on local models, the agent uses Nebius Token Factory to generate experiment ideas and responses. This removes the dependency on local GPUs and makes the research loop easier to run.
Structured experiment tracking using JSON: Instead of logging experiments in a flat TSV file, each experiment is stored as a structured JSON record. This allows us to track prompts, responses, scores, and timestamps more clearly.
Prompt optimization as the experiment domain: For this tutorial, the system will try to generate the best explanation for vector databases. Each experiment proposes a different prompt and evaluates how well the response matches our criteria.
Running a large experiment loop: The system runs around 250 iterations, allowing the agent to gradually improve prompts by learning from previous experiment results.

Why Nebius Token Factory

To run the AutoResearch loop properly, the agent needs a reliable way to generate experiment ideas and responses. Instead of running models locally, we use Nebius Token Factory as the inference layer.

Managed model inference: Nebius Token Factory lets us run open models through an API. We do not need to download models or manage GPUs locally.
Access to open models: The platform provides models such as Llama, DeepSeek, and Qwen, which can be used for reasoning, prompt generation, and experiment responses.
OpenAI-compatible API: The API follows the same structure used in the OpenAI SDK. This makes integration simple in Python applications.
Agent reasoning backend: In our system, the AutoResearch agent calls Nebius Token Factory to analyze previous experiments and propose the next prompt.
Supports large experiment loops: Because inference runs in the cloud, we can run hundreds of experiment iterations without worrying about local compute limits.
No local GPU requirement: Developers can run the experiment loop directly from their machine while the model inference happens in Nebius infrastructure.

For more demos, refer to the cookbooks here.

Tutorial: Building an AutoResearch-like System with Nebius Token Factory

Step 1 — Clone the Project Repository

Clone the repository.

git clone https://github.com/studio1hq/Nebius_autoresearch.git

Install required dependencies.

pip install requests python-dotenv

Set up a working environment.
Review the Project Structure

File	Purpose	What it Does
program.md	Defines the research goal	Contains the task the agent is trying to optimize. In this project, the goal is to generate the best prompt for explaining vector databases.
agent.py	Generates the next experiment	Uses Nebius Token Factory to analyze previous experiment results and propose the next prompt to test.
experiment.py	Runs the experiment	Sends the generated prompt to Nebius Token Factory and retrieves the model’s response.
scorer.py	Evaluates experiment output	Scores the response based on simple rules such as response length and presence of relevant keywords. Returns a numeric score.
main.py	Controls the AutoResearch loop	Loads previous experiment history, runs the experiment cycle, logs results, and repeats the process for multiple iterations.
results.json	Stores experiment history	Saves structured experiment records including prompt, response, score, and timestamp. Easier to analyze compared to flat TSV logs.

Step 2 — Configure Nebius Token Factory

Create a Nebius Token Factory API key

Open the Token Factory dashboard:

Navigate to the API Keys section in the left sidebar and click Get API key.

Create a new key for your project. This key will be used by the application to authenticate API requests sent to Nebius Token Factory.

Store the API key as an environment variable

Create a .env file in the root of the project and store the key as an environment variable:

NEBIUS_API_KEY=your_api_key

Configure the Token Factory client

Nebius Token Factory provides an OpenAI-compatible API, so we can use the same client structure used for OpenAI integrations.

Initialize the client using the Nebius API base URL and the API key stored in the environment variable.

The request sends the prompt to the selected model (for example, llama-3-70b-instruct) and receives the generated response.

In this system, these API calls power two parts of the AutoResearch loop:

Agent reasoning: Where the agent proposes the next experiment prompt (agent.py)

from typing import List, Dict, Any
from client import generate_response

def propose_experiment(history: List[Dict[str, Any]], goal: str) -> str:
    """
    Proposes a new system prompt based on the research goal and experiment history.

    Args:
        history: List of previous experiment results (dicts with 'prompt' and 'score').
        goal: The research goal string.

    Returns:
        str: The proposed system prompt for the next experiment.
    """
    # Format history for the prompt context
    if not history:
        history_text = "No previous experiments."
    else:
        # Use the last 3 experiments to provide context without overflowing context window
        recent_history = history[-3:]
        history_text = "\n".join([
            f"Attempt {i+1}:\n"
            f"Prompt: {r.get('prompt', 'Unknown')}\n"
            f"Score: {r.get('score', 0)}/10\n" 
            for i, r in enumerate(recent_history)
        ])

    system_instruction = (
        "You are an expert AI researcher optimizing a system prompt to achieve a specific goal.\n"
        "Analyze the previous attempts and their scores. Identify what worked and what didn't.\n"
        "Then, generate a NEW, improved system prompt that is likely to achieve a higher score.\n"
        "Do not repeat previous prompts. Be creative and precise.\n\n"
        "Format your response exactly as follows:\n"
        "THOUGHT: <your analysis and plan>\n"
        "PROMPT: <the actual system prompt text>"
    )

    user_message = (
        f"Research Goal:\n{goal}\n\n"
        f"Experiment History:\n{history_text}\n\n"
        "Based on the above, generate the next system prompt to test.\n"
        "Ensure you include your thought process before the prompt."
    )

The agent builds a context using the goal + past experiments.
This prompt is sent to Nebius using generate_response().
The model suggests the next experiment (new prompt).
This is where the “thinking” of the agent happens.

Experiment execution: where the prompt is sent to the model, and the response is evaluated. (experiment.py)

from client import generate_response

def run_experiment(system_prompt):
    """
    Runs the experiment by using the proposed system prompt to explain vector databases.
    """
    test_prompt = f"{system_prompt}\n\nExplain vector databases. Respond in under 120 words."
    return generate_response(test_prompt)

The generated prompt is sent again to Nebius.
The model produces the actual output for evaluation.
This response is later scored and stored.
This is the execution step of the experiment.

With this configuration in place, the AutoResearch loop can use Nebius Token Factory as its inference backend.

For example, (refer to client.py)

api_key = os.environ.get("NEBIUS_API_KEY")
    if not api_key:
        raise ValueError("NEBIUS_API_KEY environment variable not set")

    # Nebius Token Factory OpenAI-compatible endpoint
    url = "https://api.tokenfactory.nebius.com/v1/chat/completions"

    headers = {
        "Content-Type": "application/json",
        "Authorization": f"Bearer {api_key}"
    }

    payload = {
        "model": "meta-llama/Llama-3.3-70B-Instruct",
        "messages": [
            {"role": "user", "content": prompt}
        ],
        # Optional parameters for generation control
        "temperature": 0.7,
        "max_tokens": 4096
    }

You can also use an OpenAI-compatible API interface. The same client patterns used in OpenAI-based applications can be used here with minimal changes, making integration into existing workflows straightforward.

from openai import OpenAI
client = OpenAI(base_url="https://api.tokenfactory.nebius.com",
        api_key="NEBIUS_API_KEY")

completion = client.chat.completions.create(
    model="llama-3-70b-instruct",
    messages=[{
      "role":"user",
      "content":"What is the answer to all questions?"
    }]
)
print(completion.choices[0].message.content)

Step 3 — Understanding the AutoResearch Loop

The system follows a simple research loop where the agent proposes experiments, evaluates the results, and improves the next attempt based on previous runs.

Load the research goal: The system reads the task defined in program.md. In this case, the goal is to generate the best prompt for explaining vector databases.
Load previous experiment history: The program reads results.json to understand what prompts were already tested and how they performed.
The agent proposes the next experiment: The agent uses Nebius Token Factory to analyze the previous results and suggest a new prompt to try.
Run the experiment: The new prompt is sent to the model through the Token Factory API, and the response is generated.
Score the result: The response is evaluated using the scoring function defined in scorer.py.
Store the experiment record: The prompt, response, score, and timestamp are saved in results.json.
Repeat the loop: The system continues this cycle for around 250 experiment iterations, allowing the agent to gradually improve the prompts.

Step 4 — Run the System

Command:

python main.py

Terminal output shows:

iteration number
proposed prompt
generated response
score.

Every 25 experiments, the system prints a short report in the terminal showing the number of experiments completed, the best score so far, and the prompt that produced the best result. This helps track how the agent is improving over time.

Each experiment is stored as a structured record in results.json. The record contains the prompt used, the response generated by the model, the score assigned by the evaluation function, and the timestamp of the run. This structured format makes it easier to inspect and analyze experiment history later.

Instead of starting from scratch every time, the system loads the existing results.json file when the program starts. It detects the last completed experiment and continues from the next iteration. This allows the experiment loop to resume without losing previous results.

Key Takeaways

AutoResearch shows how an AI agent can run experiments automatically by proposing ideas, testing them, and learning from the results.

In this tutorial, we extended that idea by using Nebius Token Factory for cloud-based model inference and structured experiment logging for better tracking. This makes the research loop easier to run, easier to observe, and more practical for developers experimenting with AI workflows.

If you want to try similar experimentation workflows, you can start with Nebius Token Factory to run open models through a simple API without managing GPUs. The broader Nebius Cloud platform also provides GPU infrastructure, scalable inference, and tools for building AI applications.

Explore the available services, experiment with different models, and build your own AI-driven systems using the Nebius ecosystem.

Build a HackMD-Style Collaborative Markdown Editor with React, Antigravity IDE & Velt

Arindam Majumder — Mon, 27 Apr 2026 07:21:30 +0000

TL;DR

Building real-time collaboration from scratch takes significant effort. You need sync logic, presence, comments, and infrastructure before you even ship the feature.

In this guide, we generate a pixel perfect HackMD style editor UI using Antigravity, connect live markdown preview in React, and then use Velt to add presence, live sync, and comments in just a few steps.

What We’re Building

We are building a HackMD style markdown editor with a clean two pane layout. On the left, users can write markdown. On the right, they see a live rendered preview. The interface follows a dark theme and mirrors the structure and layout of HackMD closely.

This is not just a static clone. The final result will support real time collaboration, allowing multiple users to edit, comment, and stay aware of each other inside the same document.

Tech Stack

We use a focused, minimal stack:

React with Vite and TypeScript: Provides a fast development setup and a clean component-based architecture.
**Antigravity:** Used to generate a pixel-accurate editor UI directly from a reference image. This allows us to replicate the layout precisely without manual design iteration.
**Velt React SDK:** Adds the collaboration layer. We use it to enable presence, live state sync, and contextual comments without building real-time infrastructure from scratch.

Step 1: Generating a Pixel-Perfect UI with Antigravity

Antigravity is an AI-powered development platform and “agent-first” IDE where AI agents assist with coding tasks across your editor, terminal, and browser, moving beyond simple code completion toward autonomous execution of complex software workflows.

It lets you generate and modify real code based on high-level instructions, orchestrating planning, editing, and validation with minimal manual effort.

Why Use Antigravity?

Cloning an interface like HackMD manually is time-consuming. Matching spacing, typography, layout, and dark mode details takes careful iteration.

We used Antigravity to generate the editor UI directly from the reference image. The prompt enforced strict visual fidelity. No redesign. No interpretation.

This gave us:

Rapid UI cloning: Full split layout with header, editor, preview, and status bar in minutes.
Pixel-accurate output: Layout and styling matched the reference closely.
No design drift: The UI stayed consistent with the original.

With the UI ready, we could move straight to functionality and collaboration.

The Prompt Strategy

The prompt was written with strict visual constraints. Every layout detail, spacing rule, and styling decision had to follow the reference image exactly.

We enforced a simple rule: the image always wins. If there was any conflict between best practice and the screenshot, the screenshot was treated as the authority.

You are an expert frontend engineer and UI pixel-perfect implementer.

Your task is to build **only the editor UI** of a HackMD-style markdown editor **exactly matching the provided image**. This is **not** a redesign, interpretation, or approximation. This must be a **visual and behavioral clone** of the image.

---

### CRITICAL INSTRUCTIONS (DO NOT IGNORE)

* **DO NOT make any assumptions** about layout, spacing, colors, typography, sizing, or behavior.
* **DO NOT invent UI elements** that are not visible in the image.
* **DO NOT omit UI elements** that are visible in the image.
* **DO NOT restyle or “improve” anything.**
* **DO NOT change colors, icons, padding, fonts, or alignment.**
* **DO NOT guess breakpoints** — infer responsiveness strictly from the image and standard proportional scaling.
* **Follow the image exactly as it is.** If something is unclear, replicate it as faithfully as possible from the visual evidence alone.

---

### INPUT CONTEXT

* You are working inside a **basic React project**.
* You are building **only the editor UI** (no authentication, no backend, no real GitHub integration).
* The editor consists of:

  * **Left pane**: Markdown editor
  * **Right pane**: Live markdown preview
* The provided image is the **single source of truth**.

---

### REQUIRED OUTPUT

Produce **production-ready React code** that recreates the UI **pixel-perfectly**.

You must:

1. Use **React functional components**
2. Use **CSS (or CSS Modules / styled-components)** to precisely match styles
3. Ensure the layout is **fully responsive** to all screen sizes while preserving proportions
4. Match:

   * Background colors
   * Pane widths
   * Divider behavior
   * Toolbar icons and placement
   * Font family, size, weight
   * Line height
   * Button styles
   * Hover/focus states (only if visible/implied)
   * Spacing and margins
5. Implement:

   * Markdown input on the left
   * Live preview rendering on the right
6. Match **dark mode styling exactly** as shown
7. Match scrollbar appearance as closely as possible
8. Use **no external UI libraries** unless strictly necessary for markdown parsing
9. Use semantic HTML where applicable

---

### LAYOUT REQUIREMENTS

* Two-column split layout
* Left: editable markdown text area
* Right: rendered markdown preview
* Divider exactly positioned as shown
* Toolbar at the top exactly matching icon order, spacing, and alignment
* Bottom GitHub buttons and template buttons must appear exactly as shown (visual only)

---

### RESPONSIVENESS REQUIREMENTS

* On smaller screens:

  * Maintain proportional scaling
  * Preserve visual hierarchy
  * Do NOT collapse, remove, or redesign panes unless explicitly shown in the image
* No mobile-specific UI unless clearly implied by the image

---

### FUNCTIONAL REQUIREMENTS

* Markdown typing updates preview in real time
* Toolbar buttons do NOT need real functionality unless explicitly visible
* GitHub buttons are **visual only**
* No routing, no persistence, no API calls

---

### DELIVERY FORMAT

* Return:

  * React component(s)
  * CSS
  * Brief explanation of structure
* Code must be clean, readable, and copy-paste ready

---

### FINAL RULE

If there is ever a conflict between best practices and the image:
**THE IMAGE ALWAYS WINS.**

Use the provided image as the **absolute authority** and replicate it **exactly**.

Generated Component Structure

Antigravity generated a clean, modular React structure instead of a single large file. The UI was split into focused components, each responsible for one section of the editor. This made the layout easy to reason about and ready for collaboration features.

Header.tsx — Top navigation and toolbar
Editor.tsx — Left pane markdown input
Preview.tsx — Right pane rendered markdown
StatusBar.tsx — Bottom metadata bar
Layout.tsx — Structural wrapper composing the editor layout

This structure is accurate based on the repo you shared. It matches the actual src/components breakdown and reflects how the UI is assembled in App.tsx.

Implementing Markdown + Live Preview

The editor follows a simple two-pane model. The left pane is a controlled textarea where users write markdown. The right pane renders the parsed markdown in real time.

State is lifted to a shared parent component so that every keystroke updates both the editor and the preview instantly. This keeps the UI predictable and ensures the preview always reflects the latest content.

Step 2: Making the Editor Collaborative with Velt

Once the local markdown editor was working, the next step was to make it collaborative. Instead of building real-time infrastructure from scratch, we integrated Velt to handle sync, presence, and comments.

Why Use Velt?

Velt is a collaboration SDK that lets developers embed real-time collaboration features into web products quickly and efficiently. It provides fully managed components and backend support so you can add multiplayer-style experiences without building real-time infrastructure from scratch.

Key Features of Velt:

Live Sync – Real-time shared state across users so everyone sees updates instantly.
Comments – Contextual commenting components like those in Figma, Google Docs, and spreadsheet tools.
Presence & Cursors – Shows active users and cursor positions in shared sessions.
Multiplayer Editing – Multiple users can edit content concurrently with conflict resolution.
Notifications – Built-in support for alerts and updates (mentions, replies).
Recording & Huddles – audio/video/screen recording and in-app collaborative sessions.
Customizable SDK – Components and behavior can be styled and extended to match your product.

Agent Skills and MCP Integration

Velt recently introduced Agent Skills and an implementation MCP that allow collaboration features to be integrated using AI agents. Instead of manually wiring presence, comments, and live sync, agents can now orchestrate much of the integration flow.

Installing & Setting Up Velt

We started by installing the Velt React SDK and adding it to the project. This gives us access to collaboration primitives such as live state, presence, and comments.

npm install @veltdev/react
# Optional: npm install --save-dev @veltdev/types

Next, we wrapped the root of the application with the Velt provider. This initializes the collaboration layer and connects the app to Velt using an API key.

From this point on, collaboration features can be layered into existing components without restructuring the entire application.

Since Velt also supports Agent Skills and MCP-based implementations in an agent-enabled environment, collaboration features can be scaffolded automatically, without manually wiring every component. The agent can configure provider setup, inject components, and connect live state with minimal manual steps.

In other words:

Manual SDK setup → Explicit integration in code
Agent Skills / MCP → AI-assisted integration with reduced setup effort

For this project, we used the manual SDK approach. But teams using agent driven workflows can accelerate collaboration integration even further.

Adding Text Comments

With Velt initialized, the next step was enabling inline comments inside the document.

We wrapped the editor layout with the VeltComments component in text mode. This attaches a collaborative comment layer directly to the markdown content without changing the editor’s internal logic.

Contextual inline comments: Users can select text and leave feedback directly within the document.
Anchored collaboration: Comments stay attached to specific sections even as content evolves.
Multi-user discussion: Multiple users can comment and reply in the same document in real time.

At this point, the editor moves from being a single user tool to a shared workspace.

function App() {
  const apiKey = import.meta.env.VITE_VELT_API_KEY;
  const [currentUser, setCurrentUser] = useState(staticUsers[0]);

  const switchUser = (user: VeltUser) => {
    setCurrentUser(user);
    localStorage.setItem("hackmd-current-user", user.userId);
  };

  // Load user preference on app start
  useEffect(() => {
    const storedUserId = localStorage.getItem("hackmd-current-user");
    const user = storedUserId
      ? staticUsers.find((u) => u.userId === storedUserId) || staticUsers[0]
      : staticUsers[0];
    setCurrentUser(user);
  }, []);

  return (
    <VeltProvider apiKey={apiKey}>
      <VeltComments textMode={true} darkMode={true} />
      <AppContent
        currentUser={currentUser}
        staticUsers={staticUsers}
        onSwitchUser={switchUser}
      />
    </VeltProvider>
  );
}

export default App;

Enabling Live Sync

Comments make the document collaborative, but the content itself is still local. To enable true multi-user editing, we replaced local React state with Velt’s shared live state.

Instead of managing markdown with useState, we switched to useLiveState. This hook stores the document content in a shared real time layer managed by Velt.

Every update to the markdown now propagates instantly across connected users. No WebSockets, no manual sync logic, no conflict resolution setup.

The rest of the component structure remains unchanged. Only the state source is replaced.

Multi user editing — Multiple users can type in the same document simultaneously.
Instant shared updates — Changes appear in real time across all active sessions.

This is the moment where the editor becomes fully collaborative.

import React from 'react';
import { useLiveState } from '@veltdev/react';
import Header from './Header';
import Editor from './Editor';
import Preview from './Preview';
import StatusBar from './StatusBar';
import type { VeltUser } from '../types/veltUser';
import { defaultMarkdown } from '../constants/defaultTemplate';

interface LayoutProps {
    currentUser: VeltUser;
    staticUsers: VeltUser[];
    onSwitchUser: (user: VeltUser) => void;
}

const Layout: React.FC<LayoutProps> = ({ currentUser, staticUsers, onSwitchUser }) => {
    const [markdown, setMarkdown] = useLiveState<string>('hackmd-clone-markdown', defaultMarkdown);

    return (
        <div style={{
            display: 'flex',
            flexDirection: 'column',
            height: '100vh',
            width: '100vw',
            overflow: 'hidden'
        }}>
            <Header currentUser={currentUser} staticUsers={staticUsers} onSwitchUser={onSwitchUser} />
            <div style={{
                display: 'flex',
                flex: 1,
                overflow: 'hidden',
                position: 'relative'
            }}>
                <Editor value={markdown} onChange={setMarkdown} />
                <div style={{
                    width: '1px',
                    backgroundColor: '#000',
                    opacity: 0.3,
                    zIndex: 10
                }} />
                <Preview content={markdown} />
            </div>
            <StatusBar />
        </div>
    );
};

export default Layout;

Presence Awareness

Editing and commenting are core collaboration features, but presence adds awareness. It lets users see who else is currently active inside the document.

With Velt, presence is automatically tracked once the provider is configured. Active users can be identified in the session, enabling visual indicators such as avatars or active participant signals.

This creates a collaborative awareness layer. Users know when others are viewing or editing the same document, which reduces overlap and improves coordination.

import React, { useState } from 'react';
import { VeltPresence } from '@veltdev/react';
import type { VeltUser } from '../types/veltUser';

interface HeaderProps {
    currentUser: VeltUser;
    staticUsers: VeltUser[];
    onSwitchUser: (user: VeltUser) => void;
}

const Header: React.FC<HeaderProps> = ({ currentUser, staticUsers, onSwitchUser }) => {
    const [showUserMenu, setShowUserMenu] = useState(false);
    return (
        <div style={{
            height: 'var(--toolbar-height)',
            backgroundColor: '#2f3136', // Darker gray for toolbar
            display: 'flex',
            alignItems: 'center',
            justifyContent: 'space-between',
            padding: '0 16px',
            borderBottom: '1px solid #111',
            fontSize: '14px',
            color: '#b9bbbe'
        }}>
            {/* Left Section */}
            <div style={{ display: 'flex', alignItems: 'center', gap: '8px' }}>
                <div style={{ 
                    display: 'flex', 
                    alignItems: 'center', 
                    gap: '8px', 
                    color: '#fff', 
                    fontWeight: 600,
                    marginRight: '12px'
                }}>
                    <div style={{
                        width: '24px', 
                        height: '24px', 
                        borderRadius: '50%', 
                        background: '#3370b7', 
                        display: 'flex', 
                        alignItems: 'center', 
                        justifyContent: 'center'
                    }}>
                        <Power size={14} color="white" />
                    </div>
                    <span>My workspace</span>
                </div>

                <div style={{ width: '1px', height: '20px', background: '#4f545c', margin: '0 4px' }}></div>

                {/* Editor Mode Buttons */}
                <div style={{ display: 'flex', background: '#333', borderRadius: '4px', padding: '2px' }}>
                    <button style={{ padding: '4px 8px', background: '#444', borderRadius: '3px', color: '#fff' }}>
                        <Pencil size={14} />
                    </button>
                    <button style={{ padding: '4px 8px', color: '#888' }}>
                        <Columns size={14} />
                    </button>
                    <button style={{ padding: '4px 8px', color: '#888' }}>
                        <Eye size={14} />
                    </button>
                </div>

                <button style={{ padding: '4px' }}><Plus size={18} /></button>
                <button style={{ padding: '4px' }}><HelpCircle size={18} /></button>
                <button style={{ padding: '4px' }}><Search size={18} /></button>
            </div>

...

export default Header;

What We Didn’t Have to Build

Using Velt removed the need to build and maintain a complex collaboration infrastructure.

No WebSocket layer for managing real-time connections
No CRDT or conflict resolution system for concurrent edits
No custom backend service for syncing document state
No notification engine for mentions and updates
No database layer for storing and anchoring comments

This allowed us to ship faster, reduce engineering overhead, and keep the codebase focused on core product functionality rather than infrastructure.

Try It Yourself

You can run the full demo locally and explore the collaborative features in action.

Clone the repository
Install dependencies
Add your Velt API key
Start the development server

Once running, open the app in two different browsers or devices. You will see live sync, comments, and presence working in real time.

Key Takeaways

Modern tooling changes how fast we can ship collaborative software. AI can drastically accelerate UI replication, allowing you to move from design to production-ready components in minutes. At the same time, collaboration infrastructure no longer needs to be built from scratch. By layering Velt on top of a clean React architecture, you can enable live sync, comments, and presence without managing real-time systems yourself.

If you’re building collaborative features into your product, explore Velt and see how quickly you can turn a single user interface into a shared workspace.

Resources

Velt Documentation
GitHub Repository
Live Demo: Try the application yourself

Production-Aware AI: Giving LLMs Real Debugging Context

Arindam Majumder — Thu, 09 Apr 2026 05:22:32 +0000

TL;DR

Large language models struggle with production debugging because they do not have visibility into how code actually executes at runtime.
Inputs such as logs, stack traces, and metrics provide incomplete signals, which often cause confident but incorrect conclusions about root causes.
When AI reasoning is grounded in function-level runtime data collected from production systems, debugging becomes accurate, explainable, and reliable.

Introduction

Large language models are increasingly used by developers to understand code, analyze failures, and assist during incident response. In controlled environments, they are effective at explaining logic and suggesting fixes. In production systems, however, their usefulness often drops sharply.

A recent survey of developers found that a quarter of developers spend more time debugging than writing code each week. The same survey reported that bugs and tooling failures cost teams nearly 20 working days per year in lost productivity. These numbers reflect a reality most engineering teams already experience.

Production debugging takes time because failures depend on runtime factors such as traffic patterns, concurrency, queue depth, and system state that are absent in non-production environments. Most AI systems do not observe these execution conditions. They analyze code structure and reported symptoms, rather than the runtime behavior that caused the failure.

In this article, we will discuss why production context is critical for AI debugging, what production-aware AI really means, and how runtime intelligence enables more accurate and trustworthy debugging outcomes.

Why Production Issues Cannot Be Understood from Code Alone

Code defines control flow and data handling, but production behavior is determined by runtime conditions such as traffic volume, concurrency, and system state.

In production, requests arrive concurrently and compete for shared resources. As traffic increases, queues begin to accumulate work, caches evolve, and external dependencies respond with variable latency or partial failures. Together, these factors influence execution order, timing, and resource contention in ways that are not visible when reading code or running isolated tests.

Many production failures arise only when specific runtime conditions are met. Race conditions appear under concurrent access. Performance regressions surface under sustained or uneven load. Retry mechanisms can magnify transient upstream failures into system-wide impact. In each case, the logic itself may be correct, while the observed failure is a result of how that logic behaves under real execution pressure.

This leads to a common outcome during incident response. The code appears correct because the failure is not caused by a logical error. The root cause exists in how the code executes under real production conditions, not in how it reads in isolation.

How LLMs Debug Today: Strengths and Structural Limits

Large language models assist debugging by analyzing text. They infer intent, recognize common patterns, and map symptoms to known classes of problems. This makes them effective for code review, error explanation, and reasoning about familiar failure modes.

However, their understanding is entirely constrained by the inputs they receive. Without access to runtime execution data, their conclusions are based on probability rather than evidence.

Aspect	What LLMs Do Well	Structural Limitation
Code understanding	Explain logic, control flow, and common anti patterns	Cannot observe how code executes under real load
Input analysis	Reason over logs, stack traces, and snippets	Inputs represent symptoms, not full execution context
Pattern matching	Identify known bug patterns and typical fixes	Fails when failures are novel or environment specific
Root cause analysis	Propose plausible explanations	Cannot validate causality without runtime signals
Decision making	Rank likely fixes based on training data	Relies on probabilistic inference when facts are missing

Without visibility into execution order, timing, frequency, and state, LLMs are forced to guess. The results may sound correct, but they are not grounded in how the system actually behaved.

Hallucinations Are Caused by Missing Runtime Evidence

Hallucinations in AI-assisted debugging usually appear when the system does not have enough information about what actually happened during execution. This is common in production, where AI is asked to explain failures using logs, stack traces, or small pieces of code that describe symptoms but not runtime behavior.

Recent research on AI reliability shows that incorrect answers increase when important contextual details are missing. In debugging scenarios, these details include execution order, timing, system state, and how frequently specific code paths were executed. Without this information, AI systems infer causes based on likelihood rather than evidence.

The same pattern appears in studies on AI-driven debugging and code repair. When models are given execution traces or feedback from real runs, fault localization and fix accuracy improve. When this runtime information is absent, models often produce explanations and fixes that appear reasonable but fail to address the real cause of the issue.

Prompt refinement does not address this limitation. Clearer prompts help structure responses, but they do not introduce new facts. If execution data is missing, the model still reasons without evidence about how the system behaved.

In production debugging, hallucinations are therefore expected. They occur when AI systems are asked to explain failures they cannot observe, not because the reasoning process is flawed, but because the necessary runtime evidence is absent.

The Missing Context in AI Debugging Workflows

Most AI debugging workflows rely on the same signals engineers have used for years. These signals are useful, but they describe outcomes, not execution, which creates a gap between what failed and why it failed.

What AI usually receives today

Logs: Logs capture messages emitted by code paths that were explicitly instrumented. They are selective, often incomplete, and rarely reflect execution order, frequency, or timing across concurrent requests.
Stack traces: Stack traces show where an error surfaced, not how the system reached that state. They lack information about prior execution paths, state changes, and interactions with other components.
Metrics: Metrics summarize system behavior at an aggregate level. They indicate that something is slow or failing, but they do not identify which functions caused the issue or how behavior changed over time.

What is missing

Function level execution behavior: Which functions ran, how often they executed, and how long they took under real load conditions.
Runtime performance characteristics: Execution timing, concurrency effects, retries, and resource contention that emerge only during live operation.
Connection between user impact and code: Clear linkage between affected endpoints or workflows and the exact functions responsible for the observed behavior.

When AI reasons over incomplete signals, it cannot establish causality. Proposed fixes are derived from statistical patterns rather than observed execution, which often results in changes that compile or deploy successfully but do not resolve the underlying issue. Effective debugging requires visibility into execution behavior, not only error reports or surface-level symptoms.

Defining Production-Aware AI

Consider a common production incident. An API endpoint becomes slow after a deployment. Logs show no errors. Metrics show increased latency. The code itself looks unchanged or correct. An AI system reviewing this information can suggest several possible causes, such as a database query, a cache miss, or an external dependency. Each suggestion sounds reasonable, but none is confirmed.

This is where production awareness matters. A production-aware AI does not rely only on aggregated metrics or isolated log lines. It reasons using information about how the system actually executed under real traffic. It can see which functions ran more often than before, where execution time increased, and which code paths were exercised during the slowdown.

Production-aware AI is defined by the context it uses. It grounds reasoning in runtime behavior rather than static structure. It focuses on how functions are executed, how often they ran, and how their performance changes over time, instead of relying only on what the code looks like or what developers expect it to do.

This approach changes the quality of debugging. Instead of proposing likely explanations, the AI reasons from observed execution evidence.

Why Function-Level Runtime Intelligence Changes AI Debugging

Function-level runtime intelligence gives AI direct visibility into how software behaves while it is running. This visibility changes debugging from interpreting symptoms to analyzing execution.

Instead of inferring behavior from secondary signals, AI can reason using execution facts collected in real time.

Function-level data as the missing signal: Function-level data shows which functions executed, how frequently they ran, and how long they took under real load. This information allows AI to identify abnormal behavior at the exact point where performance or correctness changed.
Linking endpoints to execution paths: Runtime intelligence connects external symptoms to internal execution. When an HTTP endpoint slows down, or a queue backs up, AI can trace the issue to the specific functions involved, rather than reasoning only at the service or request level.
Temporal awareness across deployments: By comparing runtime behavior before and after a deployment, AI can identify which functions changed execution characteristics. This makes regressions visible without relying on alerts or manual comparison.

How Hud Enables Production-Aware AI

Hud captures function-level execution behavior directly from production systems. Instead of relying on aggregated metrics, sampled traces, or predefined alert rules, it observes how individual functions execute under real traffic, including errors and performance changes.

This execution data can be consumed directly by engineers and AI systems to reason about production behavior based on observed runtime evidence.

Below are the core capabilities that allow Hud to provide production-aware runtime context for AI-assisted debugging.

Runtime code sensing at the function level: Hud acts as a runtime code sensor. You get continuous function-level execution data from production, without manual instrumentation or ongoing maintenance. This data reflects how code actually runs under real traffic.
Automatic detection of errors and slowdowns: Hud automatically detects errors and performance degradations based on changes in runtime behavior, not static rules.
Linking user impact to code: When an endpoint slows down, or a queue backs up, Hud connects that business-level symptom directly to the functions responsible. You can see which parts of the code caused the impact, not just where it surfaced.
Post-deployment behavior comparison: Hud automatically detects deployments and compares function behavior across versions. You can see what changed in production after a release and identify regressions without manual diffing.
Runtime context for AI debugging: Hud provides a full forensic runtime context that you can use inside the IDE or pass to AI agents through its MCP server. This allows AI to reason from execution evidence instead of guessing from partial signals.

Key Takeaways

Without visibility into how code actually ran in production, AI systems reason over symptoms instead of causes, which leads to incorrect or incomplete fixes. Production systems demand runtime grounded reasoning, where function-level behavior, execution timing, and real traffic conditions are first-class inputs.

When AI is given this level of visibility, hallucination decreases, and confidence aligns with correctness. Production-aware AI is therefore not an optimization, but a requirement for reliable debugging.

Hud gives you function-level runtime visibility directly from production, with no configuration and no maintenance. Explore how Hud works, read the documentation, or book a demo to see how production-aware debugging changes the way you and your AI systems understand failures.

Build a Semantic Movie Discovery App with Claude Code and Weaviate Agent Skills

Arindam Majumder — Fri, 27 Mar 2026 20:45:45 +0000

Introduction

Versatility in agentic coding is increasing as new tools such as Model Context Protocol (MCP) servers and Agent Skills become more common. At the same time, many developers ask the same question when building AI applications: should they use MCP servers or Agent Skills? The important thing is understanding what each approach does well and choosing the one that fits your use case.

In this post, we’ll explain what MCP servers and Agent Skills are and how they differ, including architecture diagrams and technical details. In the later sections, we’ll also walk through how to use Weaviate Agent Skills with Claude Code to build a “Semantic Movie Discovery” application with several useful features.

Let’s get started!

Understanding MCP

The Model Context Protocol (MCP) is an open standard introduced by Anthropic that enables Large Language Models (LLMs) to interact with external systems such as data sources, APIs and services. MCP provides a structured way for an AI agent to connect to compliant tools through a single interface instead of requiring custom integrations for each service.

MCP Architecture

The MCP system operates on a client–server model and consists of three main components.

Host: the application that runs the AI model and provides the environment where the agent operates.
Client: the protocol connector inside the host that handles communication between the model and MCP servers.
Server: an external service that exposes tools, resources, or prompts that the agent can access.

MCP and Agentic Coding

Before MCP, each AI tool required custom integrations for every external service it wanted to connect to. MCP simplifies this process by introducing a shared protocol that multiple agents and tools can use.

Developers can now expose capabilities through an MCP server once and allow any compatible agent to access them without building separate integrations for each system.

Understanding Agent Skills

Agent Skills, also introduced by Anthropic, provide developers with a simple way to extend AI coding agents without running MCP servers. An Agent Skill is a structured configuration file, usually written as markdown files with YAML metadata that defines capabilities, parameter schemas and natural-language instructions describing how the agent should use those capabilities.

AI tools such as Claude Code read these files at session start and load the skills directly into the agent's working context without requiring an additional runtime.

How Agent Skills Work

When Claude Code detects a skill file in the project directory (typically under .claude/skills/), it loads the manifest into the agent's context at the beginning of the session.
The skill definition describes available capabilities, how to invoke them correctly and when to prefer one approach over another. Because the instructions are written in natural language alongside parameter schemas, the agent can reason about how to use the skill.
Skills are portable across repositories. If a developer commits a skill file to a repository, any collaborator who clones the project and opens it in Claude Code automatically gains access to the same capabilities without additional setup.

MCP and Agent Skills solve different problems in agent systems. MCP provides a standardized way for AI agents to connect to external tools, APIs, databases and services through a client–server architecture with structured schemas. Agent Skills extend the agent’s capabilities through configuration files that define workflows, instructions and parameter schemas without requiring a running server.

In simple terms, MCP enables agents to access external systems, while Agent Skills define how agents perform tasks or workflows within their environment.

Weaviate Agent Skills

Weaviate has released an official set of Agent Skills designed for use with Claude Code and other compatible agent-based development environments like Cursor, Antigravity, Windsurf and more. These skills provide structured access to Weaviate vector databases, allowing agents to perform common operations such as search, querying, schema inspection, data exploration and collection management.

The repository includes ready-to-use skill definitions for tasks like semantic, hybrid and keyword search, along with natural language querying through the Query Agent. It also supports workflows such as creating collections, importing data and fetching filtered results, and cookbooks. This enables agents to interact/build with Weaviate and perform multi-step retrieval and agentic tasks more effectively.

Agent Skills and Vector Databases

AI coding agents face difficulties when working with vector databases. Vector database APIs provide extensive capabilities, including basic “key–value” retrieval, single-vector near-text searches, multimodal near-image searches, hybrid BM25-plus-vector search, generative modules and multi-tenant system support. Without structured guidance, even a capable coding agent may produce suboptimal queries: correct syntax but the wrong search strategy, missing parameters or failure to use powerful features like the Weaviate Query Agent.

Weaviate Agent Skills address this by providing correct usage patterns, parameter recommendations and decision logic, enabling coding agents to generate production-ready code from their initial attempts.

The Weaviate Agent Skills repository is organized into two main parts:

Weaviate 𝗦𝗸𝗶𝗹𝗹 (skills/weaviate): Focused scripts for tasks such as schema inspection, data ingestion and vector search. Agents use these while writing application logic or backend code.
𝗖𝗼𝗼𝗸𝗯𝗼𝗼𝗸𝘀 Skill (skills/weaviate-cookbooks): End-to-end project examples that combine tools such as FastAPI, Next.js and Weaviate to demonstrate full application workflows.

Weaviate Agent Skills work with several development environments, including Claude Code, Cursor, GitHub Copilot, VS Code and Gemini CLI. When connected to a Weaviate Cloud instance, agents can directly interact with database modules and perform search, data management and retrieval tasks.

To evaluate how effective Weaviate Agent Skills really are, let’s build a small project and see how they accelerate RAG and agentic application development with Claude Code.

Building the Semantic Movie Discovery Application

We will build a Movie Discovery App that takes a natural-language description and returns the most semantically similar movies from a Weaviate collection. In the process, we will explore Weaviate capabilities such as multimodal storage, named vector search, generative AI (RAG) and the Query Agent in action with Claude Code, showing how these Agentic tools help you build applications faster.

Prerequisites

Python 3.10 or higher
Weaviate Cloud – Create a free cluster and obtain an API key.
TMDB API key – Used to fetch movie metadata
OpenAI API key – Required for RAG features.
Access to Claude Code
Node.js 18+ and npm – Required to run the Next.js frontend

Step 1: Project Setup

Create a movie-discovery-app folder

mkdir movie-discovery-app

Create and activate a Python virtual environment in the folder

cd movie-discovery-app py -m venv venv && source venv\Scripts\activate.bat

Install Python dependencies

pip install weaviate-client==4.20.1 fastapi uvicorn[standard] openai weaviate-agents>=1.3.0 requests python-dotenv

Install Node.js dependencies for the frontend

cd frontend && npm install

Now, create a .env file at the project root. Add the following parameters to configure Weaviate Agent Skills with Claude Code, along with your OpenAI API key and TMDB API key.

WEAVIATE_URL=your-cluster-host-without-https
WEAVIATE_API_KEY=your-api-key
OPENAI_API_KEY=your-openai-key
TMDB API key=your-tmdb-api-key

After signing up for Weaviate, click the Create Cluster button to start a new cluster for your use.

Click “How to Connect” to view the required Weaviate connection parameters.

Now that everything is set up, we can connect Weaviate Cloud with Claude Code by running claude in your project terminal:

Use the following prompt in your Claude terminal:

Write and run `check_modules.py` that connects using `weaviate.connect_to_weaviate_cloud`with `skip_init_checks=True`, loads credentials from `.env` with `python-dotenv`,
and prints the full JSON list of enabled Weaviate modules.
Run it with `venv/Scripts/python check_modules.py`."

Step 2: Create A Weaviate Collection and Import Sample Movie Data

In this step, we create a Weaviate collection and import the movie dataset into Weaviate. The dataset contains movie metadata sourced from the TMDB API. Each entry includes: title, overview, release_date, poster_url, popularity, and other important movie fields. You can import a JSON or CSV dataset directly into Weaviate.

Run this prompt to retrieve the dataset from the TMDB API and save it to a file named movies.json.

Create a TMDB dataset JSON file, movies.json, that contains 100 movie metadata and poster URLs directly from the TMDB API.

Afterwards, Weaviate Import Skills creates a Weaviate collection and imports the data from movies.json into the Weaviate database. Claude code activates Weaviate to perform this action when prompted with:

Import `movie.json` into a new Weaviate collection called Movie

Then the data is imported

Step 3: Building the FastAPI Backend and Next.js Frontend with Weaviate Cookbooks

Weaviate cookbooks enable the app to use a two-layer architecture: a FastAPI backend that exposes REST endpoints and a Next.js frontend that renders the UI. The backend connects directly to Weaviate Cloud and the Weaviate Query Agent. Weaviate cookbooks also include some frontend guidelines to communicate with the Weaviate backend over HTTP.

The app is organized into two views accessed via a collapsible sidebar:

Search view: performs semantic search and RAG using Weaviate named vectors.
Chat view: handles multi-turn conversations through the Weaviate Query Agent.

Our app includes the following features:

Layer	Component	Role
Backend	backend.py (FastAPI) - REST API on port 8000/docs	Routes: GET /health, GET /search, POST /ai/explain, POST /ai/plan, POST /chat
Frontend	Next.js + TypeScript (port 3000)	Single-page app with sidebar navigation
	SearchView.tsx	Semantic search (near_text), AI explanations (single_prompt), Movie Night Planner (grouped_task)
	MovieCard.tsx	Renders base64 poster inline, watchlist add/remove button
	ChatView.tsx	Multi-turn Query AI Agent chat
	AppSidebar.tsx	Navigation (Search/Chat), Weaviate logo + feature summary, watchlist manager with ‘.txt’ export

Use the following prompts with Claude Code to generate the backend and frontend:

Backend Prompt:

/weaviate cookbooks 

Create `backend.py`: a FastAPI app with CORS enabled for localhost:3000.
Connect to Weaviate Cloud using credentials from .env with skip_init_checks=True.
The /search endpoint should return genre and vote_average alongside title, description, release_year, and poster.
Implement these routes:  

- GET  /health                  → {"status": "ok"}  
- GET  /search?q=...&limit=3    → near_text on text_vector, return title/description/release_year/poster  
- POST /ai/explain              → generate.near_text with single_prompt  
- POST /ai/plan                 → generate.near_text with grouped_task  
- POST /chat                    → QueryAgent.ask() with full message history

Frontend Prompt:

Using Weaviate cookbooks frontend reference, create a Next.js TypeScript app in the frontend/ folder.
MovieCard.tsx should display a star rating (vote_average) and genre tag beneath the movie title. 

Components needed:  

- page.tsx        — SidebarProvider layout, view state (search | chat)  
- SearchView.tsx  — search input, MovieCard grid, AI explain and plan buttons  
- MovieCard.tsx   — poster image, title, year, description, watchlist button  
- ChatView.tsx    — message bubbles, source citations, clear chat  
- AppSidebar.tsx  — navigation, Weaviate logo + feature list, watchlist + exportBackend base URL from NEXT_PUBLIC_BACKEND_HOST env var (default localhost:8000)

Run backend and frontend servers with: uvicorn backend:app --reload --port 800 and npm run dev

After this, Claude Code will automatically build the app by adding relevant files and start both servers. You can start using the application immediately.

The FastAPI backend runs at http://localhost:8000/docswhile the frontend app is available at http://localhost:3000

You can also manually start both processes in separate terminals:

# Terminal 1 — Backend 
uvicorn backend:app --reload --port 8000
# Terminal 2 — Frontend
cd frontend && npm run dev

Congratulations! You’ve completed the project without needing to do much manual configuration or coding. 🔥

Demo

So far, we have used Weaviate Agent Skills with Claude Code to build a Semantic Movie Discovery Application powered by an OpenAI API key, a TMDB API key, and Weaviate.

The Movie Discovery app we built includes the following features:

Semantic search: Describe a mood or theme and retrieve matching movies using vector-based search (near_text).
AI explanations: Generate per-movie summaries using RAG with single_prompt.
Movie Night Planner: Create a viewing order, snack pairings and a theme summary using grouped_task.
Conversational chat: Ask questions about the movie collection through a chat interface powered by the Weaviate Query Agent, with source citations.
Watchlist: Save movies during your session and export the list as a .txt file.

What’s Next?

You could add image-based search to find similar movies and better meet your movie choices. You could also include a hybrid search feature that incorporates keyword-heavy queries and image search.

You can take your app even further by getting up to speed with Weaviate’s latest releases and becoming familiar with features such as server-side batching, async replication improvements, Object TTL and many more.

To explore further, check out the latest Weaviate releases and join the discussion on the community forum.

Weaviant Agent Skills in Action

Weaviate modules were used in the application:

Text2vec-weaviate: Responsible for text embeddings.
Multi2multivec-weaviate: Responsible for embedding images.
Generative-openai: Integrates GPT directly into the query workflow.
Weaviate Skill: Creates a collection and imports data.
Weaviate Cookbooks Skill: For defining the app’s logic.
Weaviate Query Agent: A higher-level abstraction that accepts natural language queries, decides the best query method, executes queries, synthesizes results and returns answers.

Weaviate Agent Skills help in shipping faster and more accurate RAG applications. Backend development tasks such as schema inspection, data ingestion and search operations are automated and optimized. Ultimately, this helps developers save valuable development time.

Conclusion

Both MCP servers and Agent Skills provide useful patterns for building AI-powered applications. MCP servers are well-suited for exposing external tools and services through a standardized interface, while Agent Skills focus on guiding coding agents with structured workflows and best practices.

In this tutorial, we demonstrated how Weaviate Agent Skills can simplify development by helping Claude Code generate correct database queries, ingestion pipelines and search logic. By combining vector search, multimodal storage and generative capabilities, we built a semantic movie discovery application with minimal manual setup.

As agentic development environments continue to evolve, tools like MCP servers and Agent Skills will likely be used together. The key is understanding where each approach fits and selecting the one that best supports your application architecture.

Happy building.

Resources

Model Context Protocol
Weaviate Agent Skills
Claude Code
GitHub Repository for the Movie Discovery App

We Cut Our MCP Token Spend in Half. Here's the Architecture

Arindam Majumder — Wed, 25 Mar 2026 19:04:52 +0000

When we started scaling our MCP workflows, token usage was something we barely tracked. The system worked well, responses were accurate, and adding more tools felt like the right next step. Over time, the cost began rising in ways that did not align with how much the system was actually used.

At first, we assumed this was due to higher usage or more complex queries. The data showed something else. Even simple requests were using more tokens than expected. This led us to ask a basic question. What exactly are we sending to the LLM on every call?

A closer look made things clearer. The issue came from how the system was built. We handled context, tool definitions, and execution flow by adding extra tokens at every step.

This article explains how we found the root cause and redesigned the architecture to fix it. The changes cut our MCP token usage by nearly half and gave us better control over how the system behaves.

Understanding Token Usage in MCP Systems

Once we started examining token usage, a clear pattern showed up. The LLM was receiving far more context than most requests actually needed. A large part of this came from tool definitions being sent repeatedly on every call.

Each request included the full list of tools, even when only one or two were needed. On top of that, earlier outputs and intermediate results were passed back into the model. The context kept growing, even for simple queries.

The execution flow added to the problem. The LLM would choose a tool, call it, process the result, and then repeat the same cycle if another step was needed. Each step added more tokens, and the same data often appeared many times across calls.

This setup worked at a small scale. As the number of tools increased, the cost grew quickly. More tools meant more context. More steps meant repeated processing. The system was doing extra work without adding real value. At this point, the cause was clear. Token usage came from how the system handled context and execution. The design itself was driving the overhead.

Introducing Bifrost

We started looking for a way to change how the system handled tool execution. The goal was simple. Reduce the amount of context sent to the LLM and avoid repeated processing across steps.

During this process, we came across Bifrost, an open source MCP gateway. It works between the application, the model, and the tools. It brings structure for how tools are discovered and executed, so the LLM receives only what is needed on each call.

This changed how we thought about the system. Tool access became more controlled. Context stayed limited to what was required for each request. The overall flow of execution became easier to follow and reason about.

These changes directly addressed the issues we were seeing. Tool definitions were sent only when required. Repeated decision loops were reduced. The system handled execution in a more controlled and predictable way.

From here, the focus moved away from adjusting prompts and toward changing how the system runs end-to-end.

Architectural Changes with Bifrost Code Mode

The main change came from how execution was handled inside Bifrost. Code Mode is a Bifrost feature that changes how the LLM interacts with MCP tools. Earlier, the LLM handled both planning and step-by-step tool interaction. Each step required another call, and each call carried a growing context.

Code Mode separates these responsibilities. The LLM focuses on planning. It generates executable code that defines the full workflow for a task.

Code Mode works best when multiple MCP servers are involved, workflows have several steps, or tools need to share data. For simpler setups with one or two tools, Classic MCP works well.

A mixed setup also works. Use Code Mode for heavier workflows like search or databases, and keep simple tools as direct calls.

This includes:

Selecting the right tools
Passing data between tools
Defining how the final output is produced

The system exposes a minimal interface to the LLM. It can list available tools, read tool details, and, when required, understand how each tool works. Tool definitions are accessed on demand, which keeps the initial context small.

Once the plan is generated, execution moves to a runtime environment. The code runs in a sandbox and interacts directly with tools. All intermediate steps, tool responses, and data transformations stay within this layer.

This removes the need for repeated LLM calls during execution. The workflow runs in one pass, guided by the generated code. The LLM is involved mainly at the planning stage and for producing the final response if required.

The flow becomes more structured. A request comes in, relevant tools are identified, code is generated, and execution happens in a controlled environment. The system handles state and intermediate data outside the LLM.

This approach improves clarity in how tasks are executed. The generated code can be inspected, debugged, and understood directly. Each request follows a defined path, which makes behavior easier to track and reason about.

Using Bifrost CLI in Our Workflow

Getting started required two commands. First, start the gateway:

npx -y @maximhq/bifrost

Then launch the CLI from a separate terminal:

npx -y @maximhq/bifrost-cli

MCP servers are registered once through the API. The key flag is is_code_mode_client, which tells Bifrost to handle that server through Code Mode instead of sending its tool definitions on every request:

curl -X POST http://localhost:8080/api/mcp/client \
  -H "Content-Type: application/json" \
  -d '{
    "name": "youtube",
    "connection_type": "http",
    "connection_string": "http://localhost:3001/mcp",
    "tools_to_execute": ["*"],
    "is_code_mode_client": true
  }'

Once registered, the LLM discovers tools on demand using listToolFiles and readToolFile, then submits a full execution plan through executeToolCode. A workflow that previously took six LLM turns now completes in three to four.

Bifrost organizes tool definitions using two binding levels. Server-level (default) groups all tools from a server into one .pyi file. Tool-level gives each tool its own file — better for servers with 30+ tools. Set it once in config.json:

{
  "mcp": {
    "tool_manager_config": {
      "code_mode_binding_level": "server"
    }
  }
}

Debugging became simpler because the generated code is the execution plan. When something went wrong, the issue was visible directly in the code rather than buried in prompt chains. This setup also made execution easier to inspect.

results = youtube.search(query="AI infrastructure", maxResults=5)
titles = [item["snippet"]["title"] for item in results["items"]]
result = {"titles": titles, "count": len(titles)}

The execution runs in a Starlark interpreter, a restricted subset of Python. A few constraints to keep in mind:

No import statements, file I/O, or network access
Classes are not supported, use dictionaries
Tool calls run synchronously; async handling is not required
Each tool call has a default timeout of 30 seconds

Code Mode also works with Agent Mode for automated workflows. The listToolFiles and readToolFile tools are always auto-executable since they are read-only.

The executeToolCode tool only auto-executes if every tool call within the generated code is on the approved list. If any call falls outside that list, Bifrost returns it to the user for approval before running.

Impact on Token Usage and System Efficiency

The reduction in token usage came from four specific changes:

Tool schemas were sent only when required
Intermediate outputs stayed within the execution layer
Repeated context across steps was removed
Fewer LLM calls were needed, since execution moved to a sandbox and ran in a single flow

These changes had a clear effect. Token usage dropped by nearly half. Latency reduced along with it. Execution became more predictable, since each request followed a defined path with fewer moving parts.

The broader takeaway is clear. Token cost comes from system design. Small changes in prompts or outputs help at the edges. The main overhead comes from the system's structure.

LLMs work best when they focus on planning. Managing execution through repeated loops adds cost and introduces variability. A separate execution layer keeps the flow stable and easier to understand. Context also needs careful control. It should be built for each request with only the required information. Letting it grow across steps results in unnecessary overhead and increased token usage.

Conclusion

Token inefficiency in MCP workflows comes from system design. Bifrost and Code Mode introduced a clear separation between planning and execution. The LLM handles planning, and the runtime handles execution. This brought immediate and measurable improvements in both cost and system behavior.

If you are working with MCP workflows at scale, Bifrost is worth exploring. The documentation provides a good starting point to set up the gateway, connect servers, and run workflows using Code Mode.

Managing Multi Provider AI Workflows in the Terminal with Bifrost CLI

Astrodevil — Sat, 21 Mar 2026 10:52:18 +0000

Command-line tools are still a common way to work with AI. They give better control and fit naturally into everyday workflows, which is why many people continue to use them.

A common issue with CLI-based tools is that they are often tied to a single provider. Switching between options usually means updating configs and handling multiple API keys. In some cases, it may even involve changing tools. This can slow things down and make everyday work feel a bit frustrating.

Bifrost CLI aims to simplify this setup. It provides a single way to connect CLI tools to multiple providers, without changing how the tools are used. In this article, let us look at how it works and how to get started.

What is Bifrost CLI

Bifrost is an open-source AI gateway that works between applications and model providers. It offers provider-compatible endpoints such as OpenAI, Anthropic, and Gemini formats. It manages request routing, API keys, and response formatting in one place, so separate setups for each provider are not required.

Bifrost CLI was recently released to extend this setup to command-line workflows. It allows existing CLI tools to connect through the Bifrost gateway in place of calling providers directly. The CLI tool continues to work in the same way, with only the endpoint updated.

The CLI tool is configured with Bifrost as the base URL. After this, all requests go through the gateway. Bifrost routes each request to the selected provider, converts it into the required API format, and returns a compatible response. The CLI workflow stays the same, with support for multiple providers through a single endpoint.

Key Features of Bifrost CLI

Bifrost CLI brings several practical features that improve how CLI-based workflows are set up and managed:

Automatic Setup for CLI Tools: Configures base URLs, API keys, and model settings for each agent. This reduces manual steps and keeps the environment ready to use.
Model Discovery from Gateway: Fetches available models directly from the Bifrost gateway using the /v1/models endpoint. This ensures the CLI always reflects the current set of available options.
MCP Integration for Tool Access: Attaches Bifrost’s MCP server to tools like Claude Code. This allows access to external tools and extended capabilities from within the CLI.
Session Activity Indicators: Displays activity badges for each tab. It becomes easy to see if a session is running, idle, or has triggered an alert.
Secure Credential Storage: Stores selections and keys securely. Virtual keys are saved in the OS keyring and are not written in plain text on disk.

Getting Started

Bifrost CLI is quick to set up and runs directly from the terminal. The flow includes starting the gateway, launching the CLI, and selecting the agent and model through a guided setup.

Step 1: Start the Bifrost Gateway

Make sure the gateway is running locally (default: http://localhost:8080):

npx -y @maximhq/bifrost

Step 2: Install and Launch Bifrost CLI

In a new terminal, run:

npx -y @maximhq/bifrost-cli

If installed, you can run:

bifrost

Step 3: Enter Gateway Details

Provide the Bifrost endpoint URL.

For local setup, this is usually: http://localhost:8080

If authentication is enabled, you can also enter a virtual key at this stage.

Step 4: Choose a CLI Agent

Select the CLI agent you want to use, such as:

Codex CLI
Claude Code
Gemini CLI
Opencode

The CLI shows which agents are available and can install missing ones during setup.

Step 5: Select a Model

The CLI fetches available models from the gateway and shows them in a searchable list.

You can choose one directly or enter a model name manually.

Step 6: Launch the Session

Review the configuration and start the session. The selected agent runs with the chosen model and setup.

Step 7: Work with Sessions

After launch, the CLI stays open in a tabbed interface.

You can open new sessions, switch between them, or close them without restarting the CLI. Each tab shows the current activity state.

Understanding the Bifrost CLI Session Flow

Bifrost CLI is built for repeated, session-based use in the terminal. You can switch between runs, update settings, and continue your work without having to go through the full setup again each time.

Here are the key steps in the session flow:

Launch: Select the agent and model, then start the session.
Work: Use the agent as usual. All requests go through Bifrost.
Switch Sessions: Press Ctrl + B to open the tab bar, switch between sessions, or start a new one.
Return: When a session ends, the CLI returns to the setup screen with the previous configuration.
Relaunch: Change the agent or model, or rerun the same setup.
Persistence: The last configuration is saved and shown the next time the CLI starts.

Working with Multiple Models

Bifrost CLI makes it easy to work with different models from the same setup. You do not need to change configurations or restart the tool each time you want to try a different option.

During setup, the CLI fetches available models from the Bifrost gateway and shows them in a list. You can select one directly or enter a model name if you already know what you want to use.

If you want to try another model, you can start a new session and choose a different one. Each session runs separately, so you can compare outputs or test different setups side by side.

All requests go through Bifrost, so differences between providers are handled in the background. The CLI experience stays the same across models.

When to Use Bifrost CLI

Bifrost CLI is useful when working with multiple providers or running repeated sessions from the terminal. Since it is built on top of Bifrost, it also brings the benefits of a central gateway into CLI workflows.

Testing Different Models: Try different models across providers from the same setup.
Running Iterative Sessions: Start, stop, and relaunch sessions with minor configuration changes.
Working from the Terminal: Keep the entire workflow inside the CLI, with Bifrost handling routing in the background.
Comparing Outputs: Run multiple sessions side by side and observe how different models respond.
Managing Multiple Providers: Use Bifrost as a single entry point to work across providers in one place.
Centralized Control with Bifrost: Route all requests through Bifrost for consistent handling of API keys, requests, and responses.

This setup helps keep workflows consistent and organized across different providers and sessions.

Conclusion

Bifrost CLI brings multi-provider access into the terminal through a single setup. It keeps existing workflows intact and reduces the need to manage separate configurations.

You can run sessions, switch agents, and try different models from the same interface, with Bifrost handling routing and integration in the background.

To get started or explore more details, check the Bifrost CLI documentation.

Why Your OpenClaw Agent Gets Slower and More Expensive Over Time

Astrodevil — Fri, 20 Mar 2026 21:00:34 +0000

Introduction

OpenClaw feels fast in the first week. You send a message, the agent responds, and the workflow makes sense. Then gradually, without any obvious change, responses take a little longer, and the API bill at the end of the month is higher than it was two weeks ago, with no single thing you can point to as the cause.

That is not a coincidence, and it is not bad luck. It is what happens when three separate problems compound on each other quietly, over time, without any of them being obvious on its own.

Context bloating, static content being reprocessed on every call, and every request hitting the same model regardless of what it actually needs, these are not dramatic failures. They are the kind of inefficiencies that feel invisible until they are not, and by the time the invoice makes them obvious, they have been running for weeks.

In this post, we will break down what is driving each of them and why routing, not prompt tuning or model switching, is the fix that addresses all three at the layer where they actually live.

Why the Default Setup Works Against You Over Time

OpenClaw's default configuration is built to get you started. It is not designed to remain efficient as your usage grows, and the gap between the two becomes apparent faster than most people expect. Three things are responsible for most of it.

Context grows faster than you think

Before you type a single message, your agent has already loaded a significant amount into the context window. SOUL.md, AGENTS.md, bootstrap files, the results of a memory search against everything you have accumulated, all of it lands in the prompt before your request even starts.

That base footprint is manageable in week one. By week three, the memory graph has grown, the search results are broader, and the conversation history from your previous sessions is traveling with every new request. The agent is not selectively pulling relevant data; it loads everything it has access to every time.

The result is a base token cost per request that is meaningfully higher than it was when you started, without any deliberate change on your part.

Static tokens are processed fresh every time

A large portion of what is loaded into every request consists of content that has not changed since last week, system instructions, bootstrap files, and agent configuration. Provider-side caching exists specifically to avoid paying full price for static content on repeat calls, but the default OpenClaw setup does not use it.

The same unchanged content, reprocessed from scratch on every heartbeat call.

Every call processes that unchanged content from scratch. For a setup running a 30-minute heartbeat, that means a full API call with no caching, hitting the configured model, every half hour, regardless of whether anything meaningful is happening in the session. Most users never think of the heartbeat as a cost source, but over a full month, it adds up to a figure worth paying attention to.

Every request hits the same model

OpenClaw routes all requests to a single globally configured model. There is no built-in distinction among task types: a status check, a memory lookup, a formatting task, and a multi-step reasoning problem all map to the same endpoint at the same price.

In practice, the majority of what an agent handles day-to-day is simple work. Summaries, lookups, structured output, short responses. None of it requires a frontier model, but all of it gets one anyway. That is not a usage problem; it is a configuration gap, and it is the highest-leverage thing to fix.

The Structural Fix: Routing

The problem with every approach people try first, switching to a cheaper model, trimming prompts, and reducing heartbeat frequency, is that they address one variable at a time. The bill declines slightly, then rises again. What is needed is a layer that sits between OpenClaw and the provider, evaluates each request before it is sent, and determines which model to route it to. That is what routing is, and that is why it is a structural fix rather than a configuration tweak.

That layer is Manifest, an open-source OpenClaw plugin built specifically to solve this. It sits between your agent and the provider, and the original OpenClaw configuration remains unchanged.

Manifest intercepts every request before it reaches the LLM. The routing decision takes under 2 ms with zero external calls, after which the request is forwarded to the appropriate model. During that interval, five distinct mechanisms run before the request moves anywhere, starting with how the scoring algorithm decides which tier a request belongs to.

How the scoring algorithm works

Before any request leaves your setup, Manifest runs a scoring pass across 23 dimensions. These dimensions fall into two groups:

13 keyword-based checks that scan the prompt for patterns like "prove", "write function", or "what is", and
10 structural checks that evaluate token count, nesting depth, code-to-prose ratio, tool count, and conversation depth, among others.

Each dimension carries a weight. The weighted sum maps to one of four tiers through threshold boundaries. Alongside the tier assignment, Manifest produces a confidence score between 0 and 1 that reflects how clearly the request fits that tier.

How Manifest scores a request across 23 dimensions and assigns it a tier in under 2 ms.

One edge case worth knowing: short follow-up messages like "yes" or "do it" do not get scored in isolation. Manifest tracks the last 5 tier assignments within a 30-minute window and uses that session momentum to keep follow-ups at the right tier, rather than dropping them to simple because they contain almost no content.

Certain signals also force a minimum tier regardless of score. Detected tools push the floor to the standard. Context above 50,000 tokens forces complex. Formal logic keywords move the request directly to reasoning.

The four tiers and what they route

The tier system is where the cost reduction actually happens. Manifest defines four tiers, each mapped to a different class of model:

Simple: greetings, definitions, short factual questions. Routed to the cheapest model.
Standard: general coding help, moderate questions. Good quality at low cost.
Complex: multi-step tasks, large context, code generation. Best quality models.
Reasoning: formal logic, proofs, math, multi-constraint problems. Reasoning-capable models only.

In a typical active session, most requests fall into the simple or standard category. Routing those away from frontier models, while sending only what genuinely needs it to complex or reasoning, is where the up to 70% cost reduction reported by users comes from.

How Manifest maps each request type to the cheapest model that can handle it.

Every routed response returns three headers you can inspect: X-Manifest-Tier, X-Manifest-Model, and X-Manifest-Confidence. If a request was routed differently than you expected, those headers tell you exactly what the algorithm saw.

OAuth and provider auth

Manifest lets users authenticate with their own Anthropic or OpenAI credentials directly through OAuth. If OAuth is unavailable or a session is inactive, it falls back to an API key.

Manifest lets users authenticate with their own Anthropic or OpenAI credentials

This keeps your model access under your own account, which matters for rate limits, spend visibility, and not routing your traffic through a third-party proxy. More providers are being added.

Fallbacks and what they protect

Each tier supports up to 5 fallback models. If the primary model for a tier is unavailable or rate-limited, Manifest automatically moves to the fallback chain. The request still resolves, just against the next available model in that tier's list. This is particularly relevant for the reasoning tier, where model availability can be less predictable during high-traffic periods, and losing a request entirely is more costly than a slight capability downgrade.

Spend limits without manual monitoring

Manifest lets you set rules per agent against two metrics: tokens and cost. Each rule has a period (hourly, daily, weekly, or monthly), a threshold, and an action. Notify sends an email alert when the threshold is crossed. Block returns HTTP 429 and stops requests until the period resets.

Rules that block are evaluated on every ingest, while rules that notify run on an hourly cron and fire once per rule per period to avoid repeated alerts for the same breach. For a setup with a 30-minute heartbeat running continuously, a daily cost block is the most direct way to prevent a runaway spend event from compounding overnight without any manual check.

The Rest Is Worth Knowing

Routing is the core of what Manifest does, but it ships with a few other things that are worth understanding before you use it in production.

Manifest provides a dashboard that gives a full view of each call: input tokens, output tokens, cache-read tokens, cost, latency, model, and routing tier. Cost is calculated against a live pricing table covering 600+ models, so nothing is estimated. The message log stores all requests and is filterable by agent, model, and time range.

Manifest dashboard

In local mode, nothing leaves your machine. In cloud mode, only OpenTelemetry metadata is sent: model name, token counts, and latency. Message content never moves. The full codebase is open source and self-hostable at github.com/mnfst/manifest, and the routing logic is fully documented.

A quick note before we move on.

Everything in this post reflects how Manifest works at the time of writing, and the space is moving fast enough that some details may already look different by the time you read it. The OAuth providers, the supported models, the scoring thresholds, and the team were shipping changes even while this article was being written. For anything that has moved since, the docs are the right place to check.

With that said, back to the article. Here is how all of it fits together.

Putting It Together

The three problems do not take turns. They compound on the same request, every time.

Three problems converging into every single request, all at once.

A heartbeat call on a 30-minute cycle loads accumulated context, reprocesses unchanged system files, and hits a frontier model for a task that needed none of that. Week one is a small number. In week three, it is a pattern you cannot see until the invoice lands.

Routing is the layer that addresses all three at once, not because it solves context or caching directly, but because it changes the cost of every request before it leaves your setup, and once that layer is in place, the three problems no longer have room to compound.

Where to Start

The order matters here. Do not start by switching models or trimming prompts.

Install Manifest and let it run for a few days without changing anything else. The dashboard will show you where the cost is actually coming from.
Check the model distribution. If simple and standard requests are hitting your highest-tier model, routing is the first thing to configure.
Set a daily cost block rule to prevent a runaway session from compounding overnight.
Once routing is active, the cache read token metric indicates how much static content was served from cache versus processed fresh. That number is worth watching.
Add per-tier fallbacks to prevent availability gaps from interrupting the session.

The Manifest docs cover installation, routing configuration, and limit setup in full. If you want the broader context on what makes OpenClaw production-ready, this post is a good place to start.

Running LLM Applications Across Providers with Bifrost

Arindam Majumder — Tue, 17 Mar 2026 16:15:23 +0000

Many modern applications include AI features that rely on large language models accessed through APIs. When an application sends a prompt to a model and receives a response, that request usually goes through an external service.

Getting access to different LLM models is easier today. Providers such as OpenAI and Anthropic provide model APIs, and platforms like Amazon Bedrock and Google Vertex AI give access to several models from one place. Because of this, many applications connect to more than one provider to compare models, manage cost, or keep a backup option if one service fails.

But each provider works a little differently. Authentication methods, rate limits, and request formats are not the same. Managing these differences inside an application can slowly add complexity to the system. In this article, let us explore Bifrost, an open-source LLM gateway that provides a single layer to route requests and manage interactions with multiple model providers.

The Hidden Cost of Provider Integrations

Connecting to several LLM providers may look simple at the start. Adding another provider can feel like just integrating one more API.

That situation changes once the application runs in production. Requests may need to go to different models based on cost, response quality, or latency. If a provider slows down or becomes unavailable, the system must redirect requests to another provider and keep the service running.

Handling these situations introduces additional logic into the codebase. The application needs to manage how requests are routed between models. It must also include retry logic for failed calls, fallback providers during outages, and tracking for how requests are distributed across models.

Each of these responsibilities adds extra work to the system. Over time, operational logic becomes part of the application and increases maintenance effort. This overhead becomes the hidden cost of working directly with multiple model providers.

Introducing Bifrost: A Gateway for LLM Infrastructure

Bifrost is an open-source LLM and MCP gateway designed to manage interactions between applications and model providers. It sits between the application and the LLM services and acts as a central layer that controls how requests move between systems.

Applications often connect directly to each provider they use. Bifrost adds a gateway layer between the application and the providers, so requests pass through a single entry point before reaching the model services.

This structure separates provider management from the application. The application sends requests to one endpoint, and the gateway manages communication with different model providers. Provider configuration and request handling stay inside the gateway layer, reducing provider-specific logic in the application code.

Core Infrastructure Capabilities

Bifrost provides several infrastructure capabilities for managing LLM interactions across providers. These capabilities move provider-specific handling out of the application and into the gateway layer.

Multi-provider routing: Bifrost supports multiple AI providers through a single API interface. Applications send requests to one endpoint, and the gateway routes each request to the configured provider or model.
Load balancing: When multiple providers or API keys are configured, Bifrost distributes requests across them based on defined rules. Traffic spreads across providers and reduces the chance of hitting rate limits on a single service.
Automatic fallback: When a provider returns an error or becomes unavailable, Bifrost sends the request to another configured provider.
Semantic caching: Bifrost stores responses and returns them for similar prompts. Prompt comparison uses semantic similarity. This reduces repeated API calls and improves response time.

Platform Support and Integrations

Bifrost fits environments where applications use multiple models and providers. The gateway exposes an OpenAI-compatible API, so applications that already use OpenAI SDKs can connect with minimal changes and send requests through a single endpoint.

Bifrost works with several LLM providers, such as OpenAI, Anthropic, Amazon Bedrock, Google Vertex AI, Cohere, and Mistral. Applications can reach these providers through the same gateway interface.

The gateway also supports the Model Context Protocol (MCP). Systems that use MCP can connect tools and external services through the same layer used for model requests. Bifrost also includes a plugin system for adding custom behavior such as request validation, logging, or request transformation.

Bifrost can run using tools such as NPX or Docker and can operate in local setups or production environments. The project is open source under the MIT license and can run across different infrastructure environments.

Gateway Performance and Benchmark

A gateway processes every request sent to a model provider. The performance of this layer becomes important in systems that handle a large number of AI requests.

Bifrost is written in Go, a language often used for backend services that process many requests simultaneously. The system focuses on keeping the extra processing time very small.

Benchmark tests show that Bifrost adds about 11 microseconds of latency at 5,000 requests per second. One microsecond equals 0.001 milliseconds, so 11 microseconds equals 0.011 milliseconds, which means the delay introduced by the gateway remains extremely small.

The published benchmarks were executed on AWS EC2 t3.medium and t3.large instances. These are cloud virtual machines with moderate CPU and memory resources that are commonly used to run backend services and APIs.

Bifrost also provides a public benchmarking repository with the scripts and setup used in the tests. Anyone can run the same tests or perform custom benchmarking based on their own infrastructure, traffic patterns, or model providers.

Getting Started with Bifrost

Bifrost is designed for quick setup and can run locally or in a server environment. The gateway can start in a few steps and begin routing LLM requests through a single endpoint.

One way to start Bifrost is by using NPX:

npx -y @maximhq/bifrost

Bifrost can also run using Docker, which allows the gateway to start inside a container environment:

docker run -p 8080:8080 maximhq/bifrost

After the gateway starts, applications can send LLM requests to the Bifrost endpoint. The gateway then routes the requests to the configured model providers.

Configuration options allow the gateway to define providers, API keys, routing rules, caching behavior, and fallback settings. These configurations control how requests move between different LLM providers.

Closing

Managing several LLM providers inside an application can introduce extra operational logic and maintenance effort. A gateway layer offers a cleaner structure for handling these interactions.

Bifrost provides this layer by placing a gateway between applications and model providers. Requests go through one endpoint, and the gateway manages routing and provider communication.

This approach keeps provider integrations outside the core application code and places request management in a separate infrastructure layer.

To explore configuration options, deployment steps, and additional features, refer to the official Bifrost documentation.

Create Your Custom WSL from Any Linux Distribution (Part - 2)

Debajyati Dey — Tue, 10 Dec 2024 14:00:00 +0000

In the previous part of this two-part blog series, we discussed how to install and set up Void Linux in WSL. In this article we'll cover how to do the same for Arch Linux! Hell Yeah!!

In the former blog, we went through how to obtain the tar of the desired Distro using a docker container. Here we will see how to obtain the tar if we don't have access to a working docker container.

Summary of the Content Prior to Reading the Article

In this article, I'll guide you through installing Arch Linux on WSL without using Docker, instead opting for a VirtualBox VM to create the necessary tar file. We'll cover generating the tar archive, transferring it to your host machine, and importing it into WSL. Additionally, we'll discuss fixing the common automounting error post-installation. This method ensures you can enjoy Arch Linux on WSL, leveraging the flexibility of VM-based installation.

Installing Arch Linux in WSL

Unfortunately, the official docker image of archlinux is not usable now (I think so). Because I faced a lot of issues when inside the container running. The commands didn't ever execute and always getting a newline whenever pressed enter. Very weird. Whatever...

So docker is NOT going to get our job done. Instead, we can use virtualbox to create a VM instance of Arch Linux. I am not going to give a complete archlinux installation tutorial here. That will be too much for this article. There are plenty of tutorials available in YouTube of archlinux installation on virtualbox. And if you are a REAL NERD Linux fanboy (like me!!!) you may want to install Arch "The Arch Way" (without archinstall script!).

Assuming You Have Already Done A Base Installation inside VBox

Getting the Tar File

Run this command(assuming you are the root user and currently in the /root directory), inside the virtual machine to generate the needed whole system archive -

tar -cpvzf archlinux.tar.gz --exclude=/proc --exclude=/sys --exclude=/root/archlinux.tar.gz --one-file-system /

Let me break down the command into understandable parts.

here the --one-file-system flag means -

We have excluded the /proc and /sys directories to make the tar file comparatively less bulky in size. It is safe because, they will be generated anyway while importing in WSL.

And finally, you may already understand why we excluded the tar file itself.

Because if we did include the tar, there's a 99% probability that, 2 TERRIFYING things could happen!

Self-Referencing File: The tar command would attempt to include the archlinux.tar.gz file in the archive. This can lead to recursive inclusion, where the tar process continually adds the same file over and over, causing an infinite loop of inclusion until disk space runs out or the process is forcibly terminated.
Import Issues: Even If the tar command does manage to execute without crashing, when you attempt to import this archive on WSL, including the archlinux.tar.gz file can cause confusion. The import process (which involves extraction) might attempt to re-extract the tar file recursively, complicating the extraction process and potentially leading to errors like system overload or any kind of unexpected results.

It will take a certain amount of time to complete the making of the full system backup.

So now you have the tar file in your current directory. Check that with ls command.

Transferring the tar file

In order to get the tar file outside the VM. You have many options. The most preferable one would be to install an advanced DE(Desktop Environment) like, XFCE or GNOME and then install the the virtualbox guest additions.
After you have the guest additions installed, shutdown the VM and run it again. When guest additions are installed in the system, we can use the shared folders feature of virtualbox.
- Quickly setup a shared folder and then transfer the tar into the host machine from the guest through the shared folder.

Hurray! Now you have the tar file in your host machine ready to be imported.

Importing it into WSL

Provide the appropriate directory for where the virtual hard disk image(vhdx) file will be created, in the command below. ( in place of E:\VMs\WSLs\Arch\ )

wsl --import Arch E:\VMs\WSLs\Arch\ .\archlinux.tar.gz

I keep all the custom WSL installations (vhdx files) inside this directory E:\VMs\WSLs\. This is the way I keep them organised.

For reference you can also watch this YouTube Tutorial by AgileDevArt -

Post Installation

From the previous part, you already know how to create and set user accounts and passwords in a linux command line. So, no need to fill this section with the same Instructions. Knowledge is easily transferable among similar Operating Systems.

But, still there will be a problem that you will face which must be fixed!

Fixing the Automounting Error

When entering in your archlinux WSL with the command wsl -d Arch after a fresh install, it shall first print 'Processing fstab with mount -a failed.' in the console and then shall enter the bash shell of the arch distribution.

What is fstab?

/etc/fstab is the configuration file which contains the information about all available partitions and indicates how and where they are mounted.

What is the reason behind this problem?

From your base installation method (in the VM of course), you should know that the original base install in VirtualBox had some separate filesystem (partitions). Those partitions don't exist in WSL or now has a different filesystem UUID.

What to do now?

Comment out or delete all the uncommented lines in /etc/fstab because their corresponding filesystem partitions no longer exist in WSL.

After a full system reboot (I mean reboot your windows machine), the errors should disappear.

Final Thoughts

So at last you have the complete knowledge.

From now on you can have any Linux distro you want inside WSL.

You just basically need a tar, which you can obtain using a container or VM. (depending on the situation)

If you found it useful, please consider to share this article with your other developer friends.

Feel free to connect with me :)

Thanks for reading! 🙏🏻 Written with 💚 by Debajyati Dey

Linux users are hackers! Happy Hacking! 🐱‍💻

Create Your Custom WSL from any Linux Distribution (Part-1)

Debajyati Dey — Sun, 08 Dec 2024 14:11:29 +0000

Summary of the Content Prior to Reading the Article

Ever wanted Arch or Void Linux as your WSL distro for Windows? Do you know that you can actually (YES ACTUALLY!!!) install any Linux distribution as your WSL distro? This guide covers how to import any Linux distro to WSL2 using a tar file. We'll use a Docker container to get the tar file, import it to WSL, and set up Void Linux as an example. Follow the steps to download the Docker image, export it to a tar file, and import it to WSL. We'll walk through post-installation configurations like creating user accounts, setting up default shell and user, updating the system, and making WSL accessible as a Windows desktop app. By the end, you'll have a fully functional, custom Linux distro on your Windows machine.

Initial Discussion

If you run wsl -l -o in your windows terminal (cmd or PowerShell), you'll see an output like this -

This list is really disappointing. I mean there are many more distros out there with different package management systems and different useful features. Like - fedora, Arch, Void, Artix, Alpine, etc.

Now you may also think that the options are very limited in case of WSL. Which is not actually that true.

If you even search at MS Store, You'll see some third party linux distributions that are specifically developed for WSL (WSL Only Linux Distributions).

While the ArchWSL and Fedora WSL at MS Store may seem great at first before installing, these distros have often showed compatibility issues and sometimes very weird bugs; even conflicts with scoop or chocolatey apps.

How to install other distros then?

WSL2 provides us a way to import any linux distro as a WSL instance from a tar file(or say backup) of the Linux OS in a machine.

For example you have a laptop where you have fully installed some linux distribution as the operating system. Then you can use the tar command to make a compressed tar file replicating your whole OS starting from the / (root) directory point as one file system.

Now you can transfer the tar file in your windows machine using a USB drive. Next, you use the --import flag of the wsl command and, a new WSL instance with a filesystem (virtual hard drive) & a provided name gets registered within the subsystem.

Let's walk through a complete tutorial to get you covered.

Installing Void Linux in WSL

Well by far the easiest way to get a tar file of an OS is to use a docker container. Follow the steps I describe below.

Obtaining The TAR file (Archive)

First of first, pull the official docker image of void linux from GitHub Container Registry. Make sure you already have a WSL instance (Ubuntu or openSUSE or any other) installed and setup for docker.
```
docker pull ghcr.io/void-linux/void-glibc-full
```
After pulling it should show up in the images list-
Now run the container using -
```
docker run -it ghcr.io/void-linux/void-glibc-full sh
```
It should successfully enter the voidlinux container. You can run any command to check if the shell is really working or not. like shown below -
Next keeping this terminal instance alive without exiting the container, open another WSL terminal instance and run this command.

You should be able to see the container included in running containers list like in the image.

Run the following commands below -

We've got our running container ID. Yoohooo!
Now run this final command to obtain the tar file. Change the path of the tar file based on your choice. In my case it is the VMs folder in my E: drive. (You should read first before running any command)
```
docker export $dockerContainerID > /mnt/e/VMs/voidlinux.tar
```
You can stop the container and exit the WSL terminal afterwards.

Using the TAR file

If you go to the path where the tar file was created in windows explorer, you will see the tar file there.

Now let's create a new folder 'WSLs'. Move the tar file in there. In there create a new folder Void.

Now open the folder 'WSLs' in your terminal (cmd or pwsh), and run this command -

wsl --import Void E:\VMs\WSLs\Void\ .\voidlinux.tar

Here,

The first argument after wsl --import is the one you choose to be the name of the distribution that is going to be imported.
The 2nd argument is the absolute path of the directory where the virtual hard disk image file(.vhdx) is going to created.
The 3rd argument is the path of the tar file.

After this command is successfully executed, you'll see the filesystem of Void in the Linux subsystem (open file explorer to see).

Yeah! Cool!

💡Caution!
Don't delete the (.vhdx) file just got created. If you think now you can delete the virtual hard disk image file because Void is now imported and currently exist in your Linux subsystem, then you are totally wrong. The filesystem you can see (just as in the above image) is only accessible when you have the vhdx file and!!! exists in the same path.

You've successfully installed Void Linux in your Linux Subsystem. Huge Congratulations! ✨✨✨

Things to Do After Installing a Custom Distribution

Now if you are thinking, you're all done, then you're again making a mistake.

How?

Because the way we installed the OS in the subsystem, it was not like how automated we get the online available WSL distros. Generally, when you install ubuntu, kali or OpenSUSE via the commandline or MS Store, it automatically creates a user account (makes it the default one) for you with a password, with a bunch of configurations behind the scenes.

Now because we installed the OS in our subsystem with a bare import, we got nothing of setups out of the box. There's only one user account, which is the root user.

We will create a user account, provide it a password and add it in the sudoers file.

Prerequisites

Open the void linux shell with wsl -d Void in your powershell/cmd terminal. (Don't worry we will make a convenient way at the end for launching the app or say, shell)

Once it opens, you'll run the following commands.

But before creating user accounts let's perform some prerequisites.

First, run -

clear

you'll be shocked!

Because our OS is imported from a base install of void linux container. We don't have all important tools yet. Clearly, the clear command shouldn't work because we don't have 'ncurses" installed. "ncurses" is a library that provides terminal handling and user interface functions for C programs. Void Linux uses the xbps package manager to install, update and remove apps/softwares.

xbps-install -S ncurses

Now the clear command should work successfully.

The second step should be updating the system. As Void Linux is a rolling release distribution, it gets frequent updates. You should update the system as often as you can (like every day if possible). Update it with -

xbps-install -Syu

The -S flag means sync, -y means yes and -u means update. We combine these flags together and write it as -Syu . The system update may take some time if you are having poor internet connection.

After the full system upgrade, install less and bash. Bash (Bourne Again Shell) is not currently installed in your system. It is not provided by default. The shell you are using to run commands is sh (Bourne Shell), the predecessor of Bash.

xbps-install -S less bash

💯 😎 Pro Tip
Pipe those commands into less which take more height in their STDOUT than your terminal height. Thus, you would be able to scroll through the output with arrow keys and search words with a forward slash (/). Particularly useful when viewing help texts of a program. No need to touch the mouse!

Now change the default shell from SH to BASH. -

chsh -s $(which bash)

Now if you exit the shell and re-enter in it, you'll see that the prompt string of the commandline has been changed indicating that, - the terminal is running in its default shell has been changed to bash from sh.

And it should look something like above.

Creating a user account

Now it is the time to create a user account which will be the default user.

xbps-install -S sudo

This will install sudo in your voidlinux system.

Now run this command below, to create a user account with home directory.

useradd -m <your-username>

replace <your-username> with the username you want.

Run the below command to list all available groups.

cat /etc/group

Now append the groups you want, to your user using the usermod command.

This is the syntax -

usermod -aG <group_name> <username>

Heh! BTW you can place multiple groups in place of <group_name> by keeping them separated by commas. For example, I think I would do this -

usermod -aG wheel,audio,video,kvm,tty,storage,plugdev,lp,dialout,users <username>

For satisfaction, you can run -

groups <username>

to see that the user actually got access to the groups we specified in the command.

In case you don't know, -

Groups	Their Meaning ( Usecase )
wheel	to grant users the ability to execute commands as the superuser (root) using `sudo`.
tty	related to terminal devices. Group for access to terminal devices if needed.
storage	This group is typically used for users who need access to storage devices, like - external drives.
audio	Grants access to audio devices.
video	Grants access to video devices.
dialout	Provides access to serial ports.
lp	Grants access to printer devices.
kvm	for users who need to manage virtual machines using KVM (Kernel-based Virtual Machine).
plugdev	Allows access to removable devices like USB drives.
users	This is a general group for regular users.

Neat! We're almost done creating a regular user account. But the last crucial step is not done yet!

We need to add the user to the sudoers file so that it can get superuser access (admin privilege) using the sudo command. Also, for that we should first set a password to both the regular and root user. You can set the same password for both users if the WSL we installed is not going to be used/touched by anyone else whom you don't trust, and you also tend to forget things.

Setting A Password

To add or change passwords for a user we need to have the passwd utility installed in the system. If not already available, install it -

xbps-install -S passwd -y

type passwd and you'll be prompted for setting the password for the root user.

Next, -

passwd <username>

Replace <username> with the user we just created now.

Adding user to Sudoers File

Now we need to edit the sudoers file to properly grant superuser access to our user. For that we need an editor. If you are comfortable with nvim install it otherwise install nano for text editing.

xbps-install -S neovim

xbps-install -S nano

Now do sudo visudo to edit the sudoers file.

EDITOR=nvim sudo -E visudo

Replace nvim with nano if you want to use nano.

Find the uncommented line shown in the image, in your file. And write this line (look below) under there.

<username> ALL=(ALL) ALL

It would look like this -

Now save the file and exit.

Finally set the new user as the default user.

myUsername=<username>
echo -e "[user]\ndefault=$myUsername" >> /etc/wsl.conf

Now we are all set. Exit the Linux shell and terminate the distro by running -

wsl --terminate Void

Then open it again with -

wsl -d Void

So, yeah! We did it!

Optional Extra Configurations

As of now without any configuration, the prompt string (PS1) would look like this -

which is really reallly ugly to me. I mean, I would need to enter pwd every time whenever I need to check which directory I am currently in, which absolutely sucks!

So, let's change the prompt string.

open your .bashrc file ( assuming you are using nvim )

nvim ~/.bashrc

Go to the last line.

Add this line of code to the next line to change the prompt string.

PS1="\[\e[32m\][\[\e[m\]\[\e[31m\]\u\[\e[m\]\[\e[33m\]@\[\e[m\]\[\e[32m\]\h\[\e[m\]:\[\e[36m\]\w\[\e[m\]\[\e[32m\]]\[\e[m\]\[\e[30;46m\]\\$\[\e[m\] "

I know the string may seem unreadable obfuscated some nonsensical gibberish, but it is only because of the ANSI Escape codes heavily used here.

Save the file, exit and do

source ~/.bashrc

to make the changes take effect and HURRAYY! Now you have an elegant and useful prompt string before the commandline.

Also if you read man pages a lot you will need to have the MANPATH environment variable set on the startup.

Add this line (as shown in the image above)-

export MANPATH=/usr/share/man

Now if you have man and man-db installed, you can successfully access man pages from the commandline with the man command.

Keeping this WSL as a Desktop App

As we all clearly understand, launching the WSL instance by opening any cmd shell and running wsl -d Void is not a very convenient approach.

Most probably after a reboot of your PC, you'll see a new terminal profile has been automatically added, in which our void linux shell resides. If not, then create a new terminal profile in your windows terminal for void.

Now go through the following steps:

Right click in your Desktop Background and select the option of new desktop shortcut. You'll see a popup like below.

You have to write the appropriate command in the empty input area provided, that will open the newly created terminal profile of void linux in its $HOME directory of the default user.
Now, type - C:\Users\user\AppData\Local\Microsoft\WindowsApps\wt.exe nt -p Void --tabColor #27e336 in there and click next. (make sure the path of wt.exe is correct. If you have wt which is windows terminal in a different path, then type that one)
Now type the name of the shortcut as you want. I'm giving it the name - void Now click Finish.

Congrats! Another milestone achieved! Now you can access this WSL from your windows search bar directly by typing void. How amazing!

I suggest you change the icon of the app to anything else that will enable it to get easily caught in sight (you may need to download an image and convert it to an ico file if you want it to have the void linux logo as the icon).

This is how my desktop app look like -

And, that's all! Really! You finally got a new custom Linux distribution in your subsystem which is not readily available in the MS Store or online WSL registries, configured and ready to use.

Final Thoughts

Soo, how are you feeling! Tell me in the comments!

If must feel chaotic good to get access to most of the needed cutting edge development softwares/tools right within the terminal. One Package Manager to rule them all. If you are tired of getting old or outdated packages in Debian/Ubuntu then this is going to be an overwhelming experience.

In case you couldn't follow the steps to produce the tar file, or somehow faced any kind of trouble and thus not getting the tar (archive). Don't worry.

I am attaching the mega link of the Void Linux tar file I created for you so that you can at least try it out! ;)

This is the decrypt key of the mega file - 3uMXrmDWP6WUb6kKjzb5B0Zc-Qh1w5oLE2LbZ4lOzhA

Thank you for giving this article a read!

Mega Link

If you found it useful, please consider to share this article with your other developer friends.

Feel free to connect with me :)

Thanks for reading! 🙏🏻 Written with 💚 by Debajyati Dey

Linux users are hackers! Happy Hacking! 🐱‍💻

How to Create a Number-Guessing Game in Python

Sophia Iroegbu — Tue, 26 Nov 2024 12:59:21 +0000

Hello there! 👋

In this guide, you will learn how to build a number-guessing game using basic Python concepts, such as loops, if-else statements, handling inputs, and more. This is inspired by the Number guessing game project in the Roadmap projects section.

Let’s get started! 😎

Creating the game’s function

We would need to create a function to handle the generation of the random number for the user to guess using Python’s random module.

Start by importing the module then create the function using random.randint() within the module to generate a random number between 1 - 100, this is the number the player has to guess and it will be stored in a variable, number_to_guess.

Next, set guessed_correctly var to false to stop the game once the player guessed the right number and set an attempts_limit to make the game more difficult.

import random

def number_guessing_game(attempts_limit=7):
  number_to_guess = random.randint(1, 100)
  guessed_correctly = False
  attempts = 0

Next, use a print statement to welcome your player with some messages and instructions on how to play the game. You can customize this to your preference.

NB: This should be within the function.

  print("Welcome to Number guessing game")
  print("I have selected a number from 1-100, can you guess it?")

Creating the guessing loop

Next, we need to create the core of the game. This will manage the loop that continues until the player guesses the number correctly or reaches their guess limit.

Start by using a while loop to increase the attempt count with each try. If the player doesn't guess correctly, they should be given another turn to guess as long as they haven't exceeded their limit.

  while attempts < attempts_limit and not guessed_correctly:
     guess = int(input("Please add your guess: "))
     attempts +=1

Next, use an if-else statement to compare the player's guess with the random number and provide feedback. If the guess is lower than the correct number, print "too low." If it's too high, print "too high." If it matches the correct number, set guessed_correctly to True, break the loop, and print a congratulations message.

if guess < number_to_guess:
    print("Too low!")
elif guess > number_to_guess:
    print("Too high")
else:
    guessed_correctly = True
    print(f"Congratulations, you guessed the number in {attempts} attempts")

Next, let's add an extra layer of error handling. Users can be unpredictable, and many might try to break your program. For example, if a player decides to use a letter or a decimal number to guess, the program will stop unexpectedly. That's why we need this extra layer.

Using the try-except block, we can catch such an error. It should only accept whole numbers and should send an error message and stop if the player decides to use something else.

  while attempts < attempts_limit and not guessed_correctly:
    try:
      guess = int(input("Please add your guess: "))
      attempts +=1

      if guess < number_to_guess:
        print("Too low!")
      elif guess > number_to_guess:
        print("Too high")
      else:
        guessed_correctly = True
        print(f"Congratulations, you guessed the number in {attempts} attempts")

    except ValueError:
      print("Oops! This is not a valid number, please a whole number")

Once that's done, we move on to the final step. If the player runs out of guesses and hasn't guessed the correct number, display a message saying "Game over" and inform them that they are out of guesses.

  if not guessed_correctly:
    print(f"You are out of guesses, the correct guess was {number_to_guess}")

  print("Game over, Thanks for playing!")

Testing your game

Now that’s all done! Let’s test our game and see if it works. Also, remember to call your function at the bottom of your program to run your program.

number_guessing_game()

Conclusion

And there you have it! A simple yet fun game in Python that you can create as a beginner. I hope this helps you become more comfortable with key programming concepts like loops, conditionals, and random numbers.

The source code can be found here. If you prefer the video version of this guide, check it out:

DEV Community: Studio1

Building a Zulip Style Collaborative Chat App with Next.js and Velt

Why Zulip’s Approach Works

What We’re Building With

Prerequisites

Project Setup

Clone the Repository

Install Dependencies

Configure the Velt API Key

Run the App Locally

Project Setup

Understanding the App Router and Layout Setup

Building the Chat Interface with Components

Chat Area:

Layout Components:

Reusable UI Components

User Management for Collaboration Testing

Theme Management with useTheme

Utility Functions

Running the App and Testing the Output

How It Works

Collaborative Features You Get Automatically

Things to Note Before Shipping to Production

Demo Video

Conclusion

Scaling Karpathy’s AutoResearch Using Nebius Token Factory

Introduction

Pitfalls in the Original AutoResearch Implementation

What We Are Going to Build

Why Nebius Token Factory

Tutorial: Building an AutoResearch-like System with Nebius Token Factory

Step 1 — Clone the Project Repository

Step 2 — Configure Nebius Token Factory

Step 3 — Understanding the AutoResearch Loop

Step 4 — Run the System

Key Takeaways

Build a HackMD-Style Collaborative Markdown Editor with React, Antigravity IDE & Velt

TL;DR

What We’re Building

Tech Stack

Step 1: Generating a Pixel-Perfect UI with Antigravity

Why Use Antigravity?

The Prompt Strategy

Generated Component Structure

Implementing Markdown + Live Preview

Step 2: Making the Editor Collaborative with Velt

Why Use Velt?

Installing & Setting Up Velt

Adding Text Comments

Enabling Live Sync

Presence Awareness

What We Didn’t Have to Build

Try It Yourself

Key Takeaways

Production-Aware AI: Giving LLMs Real Debugging Context

TL;DR

Introduction

Why Production Issues Cannot Be Understood from Code Alone

How LLMs Debug Today: Strengths and Structural Limits

Hallucinations Are Caused by Missing Runtime Evidence

The Missing Context in AI Debugging Workflows

Defining Production-Aware AI

Why Function-Level Runtime Intelligence Changes AI Debugging

How Hud Enables Production-Aware AI

Key Takeaways

Build a Semantic Movie Discovery App with Claude Code and Weaviate Agent Skills

Introduction

Understanding MCP

MCP Architecture

MCP and Agentic Coding

Understanding Agent Skills

How Agent Skills Work

Weaviate Agent Skills

Agent Skills and Vector Databases

Building the Semantic Movie Discovery Application

Prerequisites

Step 1: Project Setup

Step 2: Create A Weaviate Collection and Import Sample Movie Data

Step 3: Building the FastAPI Backend and Next.js Frontend with Weaviate Cookbooks

Demo

Theme Management with `useTheme`