DEV Community: DigitalOcean

Tutorial: Build an AI-Powered GPU Fleet Optimizer

DigitalOcean — Fri, 17 Apr 2026 19:00:00 +0000

This article was originally written by Shamim Raashid (Senior Solutions Architect) and Anish Singh Walla (Senior Technical Content Strategist and Team Lead)

Key Takeaways

Deploy a serverless LangGraph agent on the DigitalOcean Gradient AI Platform that monitors your GPU fleet using natural language queries.
Scrape real-time NVIDIA DCGM metrics (temperature, power, VRAM, engine utilization) from GPU Droplets over Prometheus-style endpoints on port 9400.
Detect idle and underutilized GPUs automatically by defining configurable threshold dictionaries that compare live metrics against your baseline workload patterns.
Customize the blueprint to your needs: Change target Droplet types, adjust idle detection thresholds, enrich the data payload with additional metrics, and add actionable tools like automated power-off commands.
Reduce GPU cloud costs by replacing reactive dashboard monitoring with a proactive AI agent that identifies waste the moment it starts.

Managing a GPU fleet in the cloud is a constant balancing act between performance and cost. A single idle GPU Droplet left running overnight can add hundreds of dollars to your monthly bill. Traditional monitoring dashboards surface raw metrics, but they still require a human to interpret whether a machine is “working” or “wasting money.”

This tutorial walks you through building an AI-powered GPU fleet optimizer using the DigitalOcean Gradient AI Platform and the Agent Development Kit (ADK). You will deploy a serverless, natural-language AI agent that audits your GPU infrastructure in real time, scrapes NVIDIA DCGM (Data Center GPU Manager) metrics like temperature, power draw, VRAM usage, and engine utilization, and flags idle resources before they inflate your cloud bill.

This blueprint is designed to be forked and customized. By the end of this guide, you will know how to tune the agent's personality and efficiency thresholds, add new monitoring tools, and deploy the agent as a production-ready serverless endpoint.

Reference repository

You can view the complete blueprint code here: dosraashid/do-adk-gpu-monitor.

Prerequisites

DigitalOcean Account: With at least one active GPU Droplet running.
DigitalOcean API Token: A Personal Access Token with read permissions and GenAI scopes.
Gradient Model Access Key: Generated from the Gradient AI Dashboard.
Python 3.12: Recommended for the latest LangGraph and asyncio features.
Familiarity with Python, REST APIs, and Linux command-line basics.

The challenge: “Invisible” cloud waste

When scaling AI workloads, engineering teams often spin up expensive, specialized GPU Droplets (like NVIDIA H100s or H200s) for training or inference tasks.

The Problem: Hidden costs and wasted resources

Once a training script finishes or a model endpoint stops receiving traffic, the Droplet itself remains online and billing by the hour. This creates two compounding issues:

Generic monitoring falls short: Standard cloud dashboards typically show host-level metrics like CPU and RAM. A machine learning node might report 1% CPU utilization, but those monitors do not reveal whether the GPU's VRAM is empty or whether the compute engine is completely idle.
Dashboard fatigue: Even if you install specialized tools like Grafana to track NVIDIA DCGM metrics, an engineer still has to remember to log in, interpret the charts, and manually map the IP address of an idle node back to a specific cloud resource to shut it down.

The Solution: A proactive AI fleet analyst

Instead of waiting for an engineer to check a dashboard, you can build an AI agent that acts as an autonomous infrastructure analyst.

Using the DigitalOcean Gradient ADK, you will deploy a Large Language Model (LLM) equipped with custom Python tools. When you ask the agent a question like, “Are any of my GPUs wasting money right now?”, it executes a multi-step reasoning loop:

Discovery: Calls the DigitalOcean API to get a live inventory of your Droplets.
Interrogation: Pings the NVIDIA DCGM exporter on each node's public IP to read VRAM, temperature, and engine load.
Analysis: Runs those raw metrics against a threshold dictionary you define (e.g., “If VRAM usage is below 5% and engine utilization is below 2%, mark this GPU as IDLE”).
Actionable Output: Replies in plain English, naming the specific node, its current hourly cost, and the exact metrics proving it is idle.

Understanding NVIDIA DCGM metrics for GPU monitoring

NVIDIA Data Center GPU Manager (DCGM) exposes hardware telemetry through a Prometheus-compatible exporter that runs on port 9400.

Metric	What It Measures	Why It Matters
`DCGM_FI_DEV_GPU_TEMP`	GPU die temperature in Celsius	High temperatures indicate active computation.
`DCGM_FI_DEV_POWER_USAGE`	Current power draw in watts	Idle GPUs draw significantly less power than busy ones.
`DCGM_FI_DEV_FB_USED`	Framebuffer (VRAM) memory in use	Empty VRAM means no models are loaded.
`DCGM_FI_DEV_GPU_UTIL`	GPU engine utilization percentage	The most direct indicator of compute work.

You can query these metrics directly:

curl -s http://<DROPLET_PUBLIC_IP>:9400/metrics | grep -E "DCGM_FI_DEV_GPU_TEMP|DCGM_FI_DEV_POWER_USAGE|DCGM_FI_DEV_FB_USED|DCGM_FI_DEV_GPU_UTIL"

The AI agent in this blueprint automates this scraping across your entire fleet, parses the Prometheus text format, and feeds the structured data into the LLM for analysis. If DCGM is not available on a particular node (for example, because the exporter is not installed or port 9400 is blocked by a firewall), the agent falls back to standard CPU and RAM metrics and reports “DCGM Missing” for that node.

For production deployments, consider pairing DCGM data collection with a full Prometheus and Grafana monitoring stack for historical trend analysis alongside the AI agent’s real-time assessments.

Step 1: Clone the blueprint and set up your environment

Start with the foundational repository rather than writing everything from scratch.

Clone the repo and set up your Python environment:

git clone https://github.com/dosraashid/do-adk-gpu-monitor
cd do-adk-gpu-monitor
python3.12 -m venv venv
source venv/bin/activate
pip install -r requirements.txt

Configure your secrets by creating a .env file in the root directory:

DIGITALOCEAN_API_TOKEN="your_do_token"
GRADIENT_MODEL_ACCESS_KEY="your_gradient_key"

Security note: Never commit .env files to version control. The repository’s .gitignore already excludes this file.

Step 2: How it works (the architecture)

Before you customize the blueprint, it helps to understand the data flow inside the code:

User Prompt: You ask the agent a question via the /run endpoint.
LangGraph State: The agent checks its conversation memory (thread_id) via MemorySaver, which enables multi-turn follow-up questions within the same session.
Tool Execution: The LLM decides to call @tool def analyze_gpu_fleet() defined in main.py.
Parallel Scraping: analyzer.py uses Python’s ThreadPoolExecutor to query the DigitalOcean API and each Droplet’s DCGM endpoint (metrics.py) concurrently. This parallel approach prevents network bottlenecks when monitoring dozens of nodes.
The Omniscient Payload: The analyzer packages all raw data (temperature, power, VRAM, RAM, CPU, cost) into a structured JSON dictionary that the LLM can reason about.
Synthesis: The LLM reads the JSON payload and responds in natural language with specific node names, costs, and actionable recommendations.

If you want to learn more about building stateful AI agents with LangGraph, follow the Getting Started with Agentic AI Using LangGraph tutorial.

Step 3: Customizing the blueprint to your needs

This repository is built to be forked and modified. Here are the four main areas you should adjust to match your organization’s requirements.

Customization 1: Tuning the logic (config.py)

Open config.py. This is the control center for your agent’s behavior.

The Persona: Edit AGENT_SYSTEM_PROMPT to change how the AI communicates. For a highly technical DevOps assistant, remove the emojis and instruct it to output raw bullet points. For a management-facing report, tell it to summarize in cost terms.
The Thresholds: The blueprint considers a GPU “Idle” when utilization falls below 2% by default. If your baseline workloads idle at a higher percentage, adjust the THRESHOLDS dictionary:

THRESHOLDS = {
    "gpu": {
        "max_temp_c": 82.0,
        "max_util_percent": 95.0,
        "max_vram_percent": 95.0,

        "idle_util_percent": 2.0,
        "idle_vram_percent": 5.0,

        "optimized_util_percent": 40.0,
        "optimized_vram_percent": 50.0,
    },
    "system": {
        "idle_cpu_percent": 3.0,
        "idle_ram_percent": 15.0,
        "idle_load_15": 0.5,

        "starved_cpu_percent": 85.0,
        "starved_ram_percent": 90.0,

        "optimized_cpu_percent": 40.0,
        "optimized_ram_percent": 50.0,
    },
}

For example, if your inference servers typically idle at 8% GPU utilization between request bursts, set idle_util_percent to 10.0 to avoid false positives.

Customization 2: Changing the target infrastructure (analyzer.py)

By default, the blueprint only scans Droplets with "gpu" in the size_slug to reduce unnecessary API calls. Open analyzer.py and locate the slug filter. If you want the agent to monitor CPU-optimized or standard Droplets, modify this line:

# Change "gpu" to "c-" for CPU-Optimized, or remove the filter entirely to scan all Droplets.
target_droplets = [d for d in all_droplets if "gpu" in d.get("size_slug", "").lower()]

Customization 3: Enriching the omniscient payload (analyzer.py and metrics.py)

The LLM only knows what you explicitly pass to it. The default payload includes temperature, power, and VRAM data. If you install Prometheus Node Exporter on your instances and want the AI to also analyze disk space, you would:

Update metrics.py to scrape disk metrics from Node Exporter on port 9100.
Update the return dictionary at the bottom of process_single_droplet in analyzer.py to include the new field:

return {
    "droplet_id": droplet_id,
    "gpu_temp": temp_val,
    "gpu_power": power_val,
    "vram_used": vram_val,
    "disk_space_free_gb": disk_val,  # New metric
}

Customization 4: Adding actionable tools (main.py)

The default blueprint is read-only. The most powerful upgrade is giving the AI permission to act on your infrastructure. In main.py, you can add a new function with the @tool decorator that uses the DigitalOcean API to power off a specific Droplet:

@tool
def power_off_droplet(droplet_id: str) -> str:
    """Power off a Droplet by ID. Use only when the user explicitly asks to stop an idle node."""
    import requests
    import os

    token = os.getenv("DIGITALOCEAN_API_TOKEN")
    response = requests.post(
        f"https://api.digitalocean.com/v2/droplets/{droplet_id}/actions",
        headers={
            "Authorization": f"Bearer {token}",
            "Content-Type": "application/json",
        },
        json={"type": "power_off"},
    )
    if response.status_code == 201:
        return f"Successfully sent power-off command to Droplet {droplet_id}."
    return f"Failed to power off Droplet {droplet_id}: {response.status_code} {response.text}"

After adding any new tools, bind them to the LLM so the agent can invoke them:

llm_with_tools = llm.bind_tools([analyze_gpu_fleet, power_off_droplet])

Warning: Giving an AI agent write access to your infrastructure requires careful guardrails. Consider adding confirmation prompts, restricting which Droplet tags the agent can act on, and logging all actions for audit purposes.

Step 4: Testing your custom agent

Once you have tailored the code, test it locally before deploying. Start the local development server:

gradient agent run

In a separate terminal, simulate user requests using curl.

Example 1: Deep diagnostic

curl -X POST http://localhost:8080/run \
     -H "Content-Type: application/json" \
     -d '{
           "prompt": "Give me a full diagnostic on my GPU nodes including temperature and power.",
           "thread_id": "audit-session-1"
         }'

Expected Output: The AI uses the Omniscient Payload to report exact temperatures, wattage, and RAM utilization for each GPU Droplet, alongside cost-saving recommendations for any idle nodes.

Example 2: Contextual memory

Because you are passing thread_id: "audit-session-1", the agent retains conversation context. You can ask follow-up questions without triggering a full re-scan of your infrastructure:

curl -X POST http://localhost:8080/run \
     -H "Content-Type: application/json" \
     -d '{
           "prompt": "Which of those nodes was the most expensive?",
           "thread_id": "audit-session-1"
         }'

Example 3: Thread isolation

The memory is strictly scoped by thread_id. A request with a different thread ID sees no prior history and starts a fresh conversation:

curl -X POST http://localhost:8080/run \
     -H "Content-Type: application/json" \
     -d '{
           "prompt": "What was the second question I asked you?",
           "thread_id": "audit-session-2"
         }'

Expected Output: The agent responds that it has no record of previous questions in this session, confirming that thread isolation is working correctly.

Step 5: Cloud deployment:

Once you are satisfied with your customizations, deploy the agent as a serverless endpoint on the DigitalOcean Gradient AI Platform:

gradient agent deploy

You will receive a public endpoint URL that you can integrate into Slack bots, internal dashboards, CI/CD pipelines, or any HTTP client. The Gradient platform handles scaling, so your agent can serve multiple concurrent users without manual infrastructure management.

For more details on building and deploying agents with the ADK, see How to Build Agents Using ADK.

GPU fleet cost optimization: When to use an AI agent vs. static dashboards

One of the most common questions teams face when setting up GPU monitoring is whether to build a custom AI agent or rely on traditional dashboard tooling. The right choice depends on your fleet size, the complexity of your workloads, and how quickly you need to act on idle resources.

Factor	Static Dashboards (Grafana + Prometheus)	AI Agent (This Blueprint)
Setup complexity	Moderate: requires Prometheus server, Grafana, and DCGM exporter configuration	Low: clone the repo, set env vars, deploy with `gradient agent deploy`
Real-time alerting	Rule-based alerts with fixed thresholds	Natural language queries with adaptive reasoning
Multi-metric correlation	Manual: you visually compare multiple charts	Automatic: the LLM correlates temperature, power, VRAM, and cost in a single response
Actionability	Read-only dashboards; separate automation needed	Extensible with `@tool` decorator for direct API actions
Conversational follow-ups	Not supported	Built-in via LangGraph `MemorySaver` and `thread_id` scoping
Best for	Large teams with dedicated SRE/DevOps staff and historical trend analysis	Small-to-mid teams that need fast, conversational GPU auditing without building dashboard infrastructure

For teams running fewer than 20 GPU Droplets, the AI agent approach eliminates the overhead of maintaining a full monitoring stack while still providing actionable insights. For larger fleets, consider running both: use Prometheus and Grafana for long-term trend storage and the AI agent for on-demand, conversational diagnostics.

Advantages and tradeoffs

When adapting this blueprint for production, keep these architectural considerations in mind:

Contextual intelligence: LangGraph’s MemorySaver gives the agent conversation history, allowing natural drill-down investigations. You can ask “Which node is idle?” followed by “How much is it costing me per hour?” without repeating context.
Parallel processing: The analyzer uses Python’s ThreadPoolExecutor to scan dozens of Droplets concurrently, preventing the LLM from timing out while waiting for sequential network calls.
Cost justification: If the AI agent spots a single idle $500/month GPU instance, it pays for itself many times over. The inference cost of running a single diagnostic query on the Gradient platform is negligible compared to the savings.
Graceful degradation: If the DCGM metric scraper cannot reach port 9400 (for example, because of firewall rules or the exporter not being installed), the agent reports “DCGM Missing” for that node and falls back to standard CPU and RAM metrics rather than failing entirely.
Security considerations: The agent requires a DigitalOcean API token with read permissions. If you add write tools (like the power_off_droplet example), scope the token’s permissions carefully and implement audit logging.

Conclusion

You have successfully deployed a multi-tool AI agent using the DigitalOcean Gradient AI Platform that transforms raw infrastructure metrics into conversational, actionable intelligence. By combining DigitalOcean API data with real-time NVIDIA DCGM telemetry and an LLM reasoning engine, you have built a system that addresses three major operational challenges:

1. Stopping the silent budget drain

The most immediate value this agent delivers is catching “forgotten resources.” When engineers spin up GPU Droplets for experiments or temporary training runs, those instances often continue billing long after the work is done. Standard CPU monitors might show background processes at 1%, making the instance look active.

By querying the NVIDIA DCGM exporter directly for engine and VRAM utilization, the AI agent cuts through that noise. It identifies premium GPU nodes that are doing no meaningful compute work, letting you stop the financial drain before it compounds.

2. Eliminating dashboard fatigue

In a traditional workflow, diagnosing a cloud infrastructure issue means opening the DigitalOcean Control Panel to check Droplet status, switching to Grafana to review DCGM metrics, and consulting an architecture diagram to remember what each node is responsible for.

This agent consolidates that entire workflow. Using LangGraph’s conversational memory and the Omniscient Payload, you ask a single question and receive a complete summary of host details, GPU temperature, power usage, and cost impact in one response.

3. Bridging observability and action

Traditional dashboards are read-only. They can alert you that a resource is idle, but they do not provide the tools to act on that information.

Because this blueprint is built on the Gradient ADK, the agent is inherently extensible. By adding a few lines of Python using the @tool decorator, you can upgrade this agent from a passive monitor into an active operator that executes API commands to power off idle nodes, resize underutilized instances, or trigger scaling events automatically.

The do-adk-gpu-monitor repository is your starting point. Clone the code, adjust the efficiency thresholds to match your specific workloads, and start having conversations with your infrastructure today.

Reference and resources

Ready to take your GPU fleet management and AI agent development further? Explore these resources:

DigitalOcean Gradient AI Platform Documentation: Full reference for deploying and managing AI agents, models, and inference endpoints.
How to Build Agents Using ADK: Step-by-step guide to creating custom agents with the Agent Development Kit.
Getting Started with Agentic AI Using LangGraph: Learn the fundamentals of building stateful, multi-step AI agents with LangGraph.
Stable Diffusion on DigitalOcean GPU Droplets: Run GPU-accelerated AI workloads on DigitalOcean GPU Droplets.
Scaling Gradient with GPU Droplets and Networking: Architect production GenAI deployments with GPU Droplets, global load balancers, and VPC networking.

March 2026 DigitalOcean Tutorials: GPT-5.4 and Nemotron 3

DigitalOcean — Mon, 06 Apr 2026 16:00:00 +0000

AI development continues to change with the consistent release of new models, standards, and system architectures. It can often be a lot to keep track of and learn. But DigitalOcean has you covered with our community tutorials and resources.

These 10 tutorials from last month cover both practical, hands-on topics (such as building a game with GPT-5.4) and explanatory concepts (like migrating to multi-agent systems). Take a look and try them out—or bookmark them for some weekend coding!

Getting Started with Qwen3.5 Vision-Language Models

This tutorial walks through how to run and experiment with Qwen 3.5, an open-source multimodal model family that handles text, images, and even video. It breaks down the model’s architecture and demonstrates how to deploy it on GPU infrastructure so you can build apps like coding assistants or document analyzers on your own stack. You’ll see how high-performing multimodal AI is becoming accessible without relying on proprietary APIs.

A2A vs MCP: How These AI Agent Protocols Actually Differ

Read about the difference between two emerging standards for agent-based systems: agent-to-agent communication (A2A) and model context protocol (MCP). You’ll learn when to use each—A2A for coordinating multiple agents and MCP for structured tool integration—and why most production systems combine both. It’s a practical breakdown of the protocols shaping how agentic AI systems are actually built.

Nemotron 3 Helped Me Find the Perfect Dish Rack?

Get insight into how NVIDIA’s Nemotron 3 model pairs with NemoFinder to improve retrieval and reasoning workflows. This tutorial demonstrates how combining LLMs with optimized search and ranking pipelines can yield more accurate results, especially in enterprise or knowledge-intensive applications. You’ll also learn more about how retrieval-augmented generation (RAG) systems are evolving with tighter model–tool integration.

Train YOLO26 for Retail Object Detection on DigitalOcean GPUs

This hands-on guide shows how to train a YOLOv26 model for retail use cases such as shelf monitoring and product detection on GPU infrastructure. It walks through dataset prep, training, and deployment so you can build real-world computer vision pipelines. You’ll gain a better understanding of how to move from raw image data to a production-ready detection model.

Building Long-Term Memory in AI Agents with LangGraph and Mem0

If you’re curious about how to add persistent memory to agent workflows using LangGraph and Mem0, check out this tutorial. It shows how agents can retain context across sessions, enabling more personalized and stateful interactions over time. Its key takeaway is how long-term memory transforms agents from stateless responders into systems that can learn and adapt.

Crafting a Game from Scratch with GPT-5.4

This article breaks down GPT-5.4’s capabilities, improvements, and practical use cases. It highlights advancements in reasoning, efficiency, and multimodal performance, and shows how developers can integrate the model into real applications. You’ll see how this frontier model integrates into modern AI stacks and the steps involved in creating a 3D badminton game from the ground up.

What are Text Diffusion Models? An Overview

This guide introduces diffusion models for text generation and explains how they differ from traditional autoregressive LLMs. It walks through how diffusion-based approaches iteratively refine outputs and where they may outperform standard models. You’ll get a conceptual and practical understanding of an emerging alternative to transformers.

LLM Tool Calling with Gradient™ AI Platform and Databases

Discover how to connect LLMs to external tools—like databases—using structured tool calling. It walks through building workflows in which models query, retrieve, and act on real data rather than relying solely on prompts. You’ll get to see that tool integration makes LLMs more reliable and production-ready.

How to Generate Videos with LTX-2.3 on DigitalOcean GPU Droplets

This tutorial explores how to generate videos using LTX 2.3, covering setup, prompts, and rendering workflows. It demonstrates how generative AI is expanding beyond text and images into video creation. After this article, you’ll know how to experiment with video generation pipelines and integrate them into creative or product workflows.

From Single to Multi-Agent Systems: Key Infrastructure Needs

Get an overview of what changes when you move from a single AI agent to a multi-agent system. This tutorial goes through the full infrastructure stack—covering orchestration patterns, communication protocols, memory, and observability—so you can design systems where multiple agents collaborate reliably. Ultimately, multi-agent setups unlock scalability and specialization but require significantly more coordination, state management, and fault tolerance to work in production.

Build an End-to-End RAG Pipeline for LLM Applications

DigitalOcean — Wed, 01 Apr 2026 01:06:34 +0000

This article was originally written by Shaoni Mukherjee (Technical Writer)

Large language models have transformed the way we build intelligent applications. Generative AI Models can summarize documents, generate code, and answer complex questions. However, they still face a major limitation: they cannot access private or continuously changing knowledge unless that information is incorporated into their training data.

Retrieval-Augmented Generation (RAG) addresses this limitation by combining information retrieval systems with generative AI models. Instead of relying entirely on the knowledge embedded in model weights, a RAG system retrieves relevant information from external sources and provides it to the language model during inference. The model then generates a response grounded in this retrieved context.

An end-to-end RAG pipeline refers to the full system that manages this process from beginning to end. It includes ingesting documents, transforming them into embeddings, storing them in a vector database, retrieving relevant information for a user query, and generating an answer using a large language model.

This architecture is increasingly used in modern AI systems such as enterprise knowledge assistants, internal documentation search engines, developer copilots, and AI customer support tools. Organizations adopt RAG because it allows models to remain lightweight while still accessing large knowledge bases that change frequently.

In this tutorial, we will walk through how to design and build a complete RAG pipeline. Along the way, we will explore architectural considerations, optimization strategies, and production challenges developers encounter when deploying retrieval-based AI systems.

Key Takeaways

RAG combines retrieval and generation for more accurate AI systems: Retrieval-Augmented Generation (RAG) bridges the gap between static language models and dynamic, real-world data. Instead of relying only on pre-trained knowledge, it fetches relevant information at runtime and uses it to generate answers. This makes responses more accurate, up-to-date, and context-aware. It is especially useful for applications like chatbots, internal knowledge assistants, and search systems. Overall, RAG helps reduce hallucinations and improves trust in AI-generated outputs.
Vector embeddings are the foundation of semantic search in RAG: Embeddings convert text into numerical vectors that capture meaning rather than exact wording. This allows the system to understand similarity between queries and documents even if they use different phrasing. As a result, retrieval becomes more intelligent and context-driven instead of keyword-based. High-quality embedding models like text-embedding-3-large or bge-large-en can significantly improve retrieval performance. Choosing the right embedding model directly impacts the overall quality of your RAG system.
Each component of the pipeline plays a critical role: A RAG system is made up of multiple steps, including ingestion, chunking, embedding, storage, retrieval, and generation. If any one component is poorly optimized, it can affect the entire pipeline’s performance. For example, bad chunking can lead to irrelevant retrieval, even if your embedding model is strong. Similarly, weak retrieval will result in poor answers, no matter how powerful the language model is. This is why building an end-to-end RAG system requires careful design and tuning at every stage.
Evaluation is essential for building reliable RAG applications: It is not enough to just build a RAG pipeline, but you must also evaluate how well it performs. This includes checking whether the system retrieves the correct documents and whether the generated answers are accurate and grounded. Metrics like precision and recall help measure retrieval quality, while human evaluation helps assess answer correctness. Creating benchmark datasets with known questions and answers makes it easier to track improvements over time. Continuous evaluation ensures your system remains reliable in production.

Understanding the RAG System Architecture

Before implementing the pipeline, it is important to understand how the different components interact. A typical RAG system architecture can be divided into two major workflows: the indexing pipeline and the retrieval pipeline.

The indexing pipeline prepares the knowledge base so that it can be searched efficiently. During this stage, documents are ingested, cleaned, split into chunks, converted into embeddings, and stored in a vector database. This process is usually executed offline or periodically when new data becomes available.

The retrieval pipeline operates during inference. When a user asks a question, the system converts that query into an embedding, searches the vector database for semantically similar chunks, and provides those retrieved passages to the language model. The model then generates a response using both the query and the contextual information.

A simplified representation of the pipeline looks like this:

Document Sources
       (PDFs, Docs, APIs, Knowledge Base)
                        |
                        v
               Document Processing
                        |
                        v
                  Text Chunking
                        |
                        v
               Embedding Generation
                        |
                        v
               Vector Database Index
                        |
                        v
User Query → Query Embedding → Similarity Search
                        |
                        v
             Retrieved Context Chunks
                        |
                        v
                  LLM Generation
                        |
                        v
                  Final Response

This architecture enables the system to retrieve information dynamically rather than relying solely on model training.

Data Ingestion in a RAG Pipeline

The first stage of the pipeline involves gathering the data that the AI system will use as its knowledge source. In many real-world applications, this information is distributed across multiple systems. Organizations may store documentation in internal knowledge bases, PDFs, wikis, product manuals, or database records.

The ingestion stage extracts textual information from these sources and prepares it for processing. Depending on the data format, ingestion may involve parsing HTML pages, converting PDFs to text, or querying APIs to retrieve structured records.

At this stage, developers often implement preprocessing steps such as removing redundant formatting, normalizing whitespace, and filtering irrelevant sections. These steps are important because retrieval performance strongly depends on the quality of the text data stored in the system.

For enterprise knowledge retrieval systems, ingestion pipelines are usually automated and scheduled. For example, an internal documentation chatbot might update its knowledge base daily by ingesting the latest documentation changes from a repository.

Text Chunking: Preparing Documents for Retrieval

After ingestion, documents must be divided into smaller pieces before they can be embedded. This step, known as text chunking, plays a critical role in the overall performance of the RAG pipeline.

Large documents cannot be embedded effectively because embedding models have token limits and because large chunks reduce retrieval precision. Instead, documents are broken into manageable segments that capture a coherent piece of information.

Chunk size is typically chosen between 200 and 500 tokens. Smaller chunks provide more precise retrieval results, while larger chunks preserve more contextual information. Many production pipelines use overlapping chunks to prevent important sentences from being split across boundaries.

The following diagram illustrates how a long document is transformed into multiple overlapping chunks:

Original Document
-------------------------------------------------------
| Paragraph 1 | Paragraph 2 | Paragraph 3 | Paragraph 4 |
-------------------------------------------------------

After Chunking
-------------------------------------------------------
| Chunk 1 | Chunk 2 | Chunk 3 | Chunk 4 | Chunk 5 |
-------------------------------------------------------

Chunk Example
Chunk 1: Paragraph 1 + part of Paragraph 2
Chunk 2: Paragraph 2 + part of Paragraph 3

Choosing an effective chunking strategy significantly improves retrieval accuracy because each chunk represents a focused semantic concept.

Embedding Generation

Once documents are divided into chunks, each chunk must be converted into a numerical representation called an embedding. Embeddings transform text into high-dimensional vectors that capture semantic meaning.

For example, two sentences that express similar ideas will produce vectors that are close to each other in vector space. This property allows vector databases to retrieve semantically related text even when the wording differs.

Embedding models are trained using large datasets and transformer architectures. When a chunk is processed, the model generates a vector with hundreds or thousands of dimensions. These vectors serve as the foundation for similarity search.

Embedding generation occurs during both indexing and retrieval. During indexing, embeddings are generated for each document chunk. During retrieval, the user’s query is also converted into an embedding so that it can be compared against stored vectors.

This mechanism allows the RAG system to perform semantic search, which is far more powerful than traditional keyword matching.

Vector Embedding

Vector embeddings are dense numerical representations of data, which can be text, images, or audio. Vector embeddings are used to capture the semantic meaning of the data in a high-dimensional vector space. In an end-to-end RAG pipeline, embeddings are used to convert both documents and user queries into vectors so that similarity between them can be measured using metrics like cosine similarity. This allows the system to retrieve context based on meaning rather than exact keyword matches, making responses more accurate and relevant.

For example, even if a query doesn’t contain the same words as a document, embeddings can still identify it as relevant if the underlying intent is similar. Popular embedding models used in RAG systems include text-embedding-3-large, all-MiniLM-L6-v2, bge-large-en, and e5-large-v2, each offering different trade-offs in performance, cost, and deployment flexibility.

Storing Vectors in a Database

After embeddings are created, they must be stored in a specialized database capable of performing fast similarity searches. These systems are known as vector databases and form the core of the RAG retrieval infrastructure.

Unlike traditional databases that index numeric or textual fields, vector databases are optimized to search across high-dimensional vectors. They use approximate nearest neighbor algorithms to identify vectors that are closest to a query embedding.

The structure of a stored vector typically includes the embedding itself, the original text chunk, and metadata describing the source of the information. Metadata can include document identifiers, timestamps, or categories that allow filtering during retrieval.

A simplified representation of vector storage looks like this:

Vector Database

ID     Vector Embedding        Text Chunk
---------------------------------------------------------
1   [0.12, -0.44, 0.92...]   "RAG combines retrieval..."
2   [0.55, 0.33, -0.14...]   "Vector databases enable..."
3   [-0.77, 0.08, 0.62...]   "Embeddings represent..."

Popular vector database technologies include managed services and open-source platforms designed specifically for AI workloads. The choice often depends on scale, infrastructure preferences, and latency requirements.

Retrieval in a RAG Pipeline

When a user submits a question, the system begins the retrieval stage. The query is first converted into an embedding using the same embedding model used during indexing. Maintaining the same embedding model is important because similarity comparisons rely on consistent vector representations.

The query embedding is then sent to the vector database. The database performs a similarity search to find document chunks whose embeddings are closest to the query vector. These chunks represent the pieces of information most relevant to the user’s question.

The retrieved chunks are then combined and passed to the language model as contextual input. The model uses this context to generate a response grounded in actual documents rather than relying solely on its training data.

This process ensures that answers are based on real knowledge sources and can be updated whenever the underlying documents change.

Generation with a Large Language Model

The final stage of the pipeline involves generating a response using a language model. At this point, the system already has two pieces of information: the user’s question and the retrieved context.

These elements are combined into a prompt that instructs the model to answer the question using the provided information. Because the context is derived from authoritative documents, the model’s output becomes significantly more reliable and factual.

This stage also allows developers to control how responses are generated. Prompts may instruct the model to summarize information, provide citations, or answer in a specific format. Some systems also include guardrails that prevent hallucinations or restrict responses to retrieved information.

For example, if a user asks a question, the system first pulls the most relevant text from your knowledge base, then the LLM rewrites that content into a helpful answer, making it more conversational, structured, and easy to understand. This step is what makes RAG powerful, because it combines accurate, up-to-date information with fluent natural language generation, reducing hallucinations and improving answer quality.

Code Demo: Building a Simple End-to-End RAG Pipeline

The following example demonstrates how a basic RAG pipeline for LLM applications can be implemented in Python. The example uses document loading, chunking, embeddings, and a vector database to create a minimal working pipeline.

Install dependencies

pip install langchain chromadb sentence-transformers openai

Load documents

from langchain.document_loaders import TextLoader

loader = TextLoader("knowledge_base.txt")
documents = loader.load()

Split documents into chunks

from langchain.text_splitter import RecursiveCharacterTextSplitter

splitter = RecursiveCharacterTextSplitter(
   chunk_size=500,
   chunk_overlap=100
)

chunks = splitter.split_documents(documents)

Generate embeddings

from langchain.embeddings import HuggingFaceEmbeddings

embeddings = HuggingFaceEmbeddings(
   model_name="sentence-transformers/all-MiniLM-L6-v2"
)

Store vectors

from langchain.vectorstores import Chroma

vector_db = Chroma.from_documents(
   documents=chunks,
   embedding=embeddings
)

Retrieval and generation

from langchain.chains import RetrievalQA
from langchain.chat_models import ChatOpenAI

llm = ChatOpenAI()

qa_chain = RetrievalQA.from_chain_type(
   llm=llm,
   retriever=vector_db.as_retriever()
)

response = qa_chain.run(
   "What is retrieval augmented generation?"
)

print(response)

This simple implementation demonstrates how document retrieval and language models can be combined into a working RAG system.

Evaluating RAG System Performance

Evaluating a RAG system is important because you need to be sure that it is not only retrieving the right information but also generating correct and useful answers from it. In simple terms, a good RAG pipeline should find the right content and then explain it correctly.

First, let’s look at retrieval evaluation. This checks whether the system is pulling the right documents from your database. Imagine you have a knowledge base about cloud services, and a user asks, “How can I run AI models on GPUs?”. If your system retrieves documents about GPU Droplets or AI infrastructure, that’s a good sign. But if it returns unrelated content like pricing pages or networking docs, retrieval quality is poor. Metrics like recall (did we find all relevant documents?) and precision (were the retrieved documents actually relevant?) help measure this. For example, if 5 documents are relevant but your system only retrieves 2, recall is low.

Next is generation evaluation, which focuses on the answer produced by the language model. Even if retrieval is correct, the model (like GPT-4 or Llama 3) might still generate incomplete or incorrect responses. For instance, if the retrieved document clearly says “GPU droplets support CUDA workloads”, but the model responds with “GPU support is limited”, that’s a problem. This is why human evaluation is often needed to check if the answer is factually correct, complete, and grounded in the provided context. Automated metrics struggle to detect things like s or subtle inaccuracies.

To make evaluation consistent, teams usually create an evaluation dataset. This is a collection of sample questions along with their correct answers and sometimes the expected source documents. For example:

Question: “What are GPU droplets used for?”
Expected answer: “They are used for AI/ML workloads, training models, and high-performance computing.”

You can then run your RAG system on this dataset and compare its answers against the expected ones. Over time, this helps you track improvements, catch errors, and tune your system (for example, by improving chunking, choosing a better embedding model, or adjusting prompts).

In practice, strong RAG evaluation combines:

Retrieval checks: Did we fetch the right information?
Answer checks: Did we explain it correctly?
Continuous testing: Are we improving over time?

This ensures your RAG pipeline is reliable, accurate, and ready for real-world use.

Scaling and Production Considerations

Prototype RAG pipelines often work well with small datasets, but production deployments introduce additional challenges. Large organizations may store millions of document chunks, requiring scalable infrastructure for indexing and retrieval.

Latency also becomes an important concern. Vector searches, embedding generation, and LLM inference all contribute to response time. Developers must carefully optimize these components to ensure interactive performance.

Production systems frequently incorporate caching layers, query batching, and efficient indexing strategies. Monitoring tools are also used to track retrieval accuracy, system latency, and cost per query.

Cost and Latency Optimization

Operating a RAG pipeline at scale can become expensive if not carefully optimized. Each query may require embedding generation, vector search, and language model inference.

Several strategies help reduce these costs. Caching responses for frequently asked questions prevents repeated model inference. Limiting the number of retrieved chunks also reduces token usage and speeds up generation.

Another important technique is re-ranking. Instead of sending many retrieved documents to the language model, a re-ranking model selects the most relevant passages before generation. This improves response quality while reducing computational overhead.

RAG vs Fine-Tuning

A common question among developers is whether to use retrieval-augmented generation or fine-tuning.

Fine-tuning changes a model’s internal weights by training it on additional datasets. This approach works well for teaching models specific styles or behaviors. However, it is less effective for continuously changing knowledge because retraining the model is expensive and time-consuming.

RAG systems take a different approach by keeping the model unchanged while retrieving knowledge dynamically. This makes them ideal for applications where information changes frequently, such as product documentation or customer support knowledge bases.

For most knowledge-intensive applications, RAG provides a more flexible and maintainable solution.

Conclusion

Building an end-to-end RAG pipeline is about combining the strengths of retrieval systems and large language models to create applications that are both accurate and context-aware. Instead of relying only on pre-trained knowledge, a RAG system can fetch relevant information in real time and use models like GPT-4 or Llama 3 to generate clear, human-like responses grounded in that data. In this article, we understood each of the steps used to create the RAG pipeline from data ingestion and chunking to vector embeddings, retrieval, and response generation. Each component plays a critical role, and even small improvements (like better chunking strategies or choosing the right embedding model) can significantly impact overall performance. As organizations continue to build AI-powered applications, RAG stands out as a practical and scalable approach for use cases like chatbots, knowledge assistants, and document search. By continuously evaluating and refining your pipeline, you can create systems that are not only intelligent but also reliable and production-ready.

Resources

Tutorial: Deploy NVIDIA's NemoClaw in One Click

DigitalOcean — Mon, 23 Mar 2026 18:28:14 +0000

This article was originally written by Amit Jotwani (Staff Developer Advocate at DigitalOcean)

Key Takeaways

NemoClaw is an open-source stack from NVIDIA designed to help developers run OpenClaw securely.
DigitalOcean offers NemoClaw 1-Click Droplets that enable you to set up this stack on a CPU-optimized virtual machine and run NemoClaw.
This tutorial illustrates how to SSH into your Droplet, configure inference settings and policies, connect to NemoClaw, and effectively reconnect after the initial setup.

At GTC 2026, NVIDIA announced NemoClaw, an open-source stack that makes it easy to run OpenClaw autonomous agents securely. OpenClaw is an open-source agent platform that Jensen Huang called “the operating system for personal AI.” We covered how to run OpenClaw on a Droplet in an earlier tutorial. NemoClaw takes a different approach — it wraps OpenClaw with sandboxing, security policies, and inference routing through NVIDIA’s cloud.

NemoClaw is still in alpha, so expect rough edges. Interfaces may change, features might be incomplete, and things could break. But if you’re curious to try it out or just want to see what NVIDIA’s vision for agents looks like, this tutorial will get you up and running on a DigitalOcean Droplet in under 10 minutes.

Prerequisites

Before you begin, you’ll need:

A DigitalOcean account (sign up here if you don’t have one)
An NVIDIA account to generate an API key at build.nvidia.com

Step 1 - Create a Droplet from the Marketplace

Head to the NemoClaw 1-Click Droplet on the DigitalOcean Marketplace. Click Create NemoClaw Droplet.

When configuring the Droplet, select the CPU-Optimized plan with Premium Intel. You’ll want the option with 32 GB of RAM and 16 CPUs. NemoClaw runs Docker containers, a Kubernetes cluster (k3s), and the OpenShell gateway, so it needs the headroom.

Pick a data center region near you, add your SSH key, and hit Create Droplet.

Heads up: This Droplet costs $336/mo, so make sure to destroy it when you’re done experimenting. It adds up fast if you forget about it.

Step 2 - SSH into the Droplet

Once your Droplet is ready, SSH in:

ssh root@your_server_ip

You’ll see the usual Ubuntu login banner, and then the NemoClaw onboarding wizard will kick off automatically. It runs through a series of preflight checks, making sure Docker is running, installing the OpenShell CLI, and spinning up the gateway. You’ll see checkmarks fly by as each step completes.

Step 3 - Walk Through the Onboard Wizard

The onboarding wizard will ask you a few things. Here’s what to do at each prompt:

Sandbox Name

The first prompt asks for a sandbox name. Just press Enter to accept the default (my-assistant). The wizard will then create the sandbox, build the container image, and push it to the gateway. This takes a couple of minutes, and you’ll see it run through about 20 steps as it builds and uploads everything.

NVIDIA API Key

Once the sandbox is ready, the wizard asks for your NVIDIA API key. In this setup, inference is routed through NVIDIA’s cloud using the nvidia/nemotron-3-super-120b-a12b model, so it needs a key to authenticate.

To get your key, head to build.nvidia.com/settings/api-keys, sign in, and click Generate API Key. Give it a name, pick an expiration, and hit Generate Key.

Copy the key (it starts with nvapi-), paste it into the terminal prompt, and press Enter.

The wizard saves the key to ~/.nemoclaw/credentials.json and sets up the inference provider. You’ll see it confirm the model and create an inference route.

Policy Presets

After the inference setup, NemoClaw sets up OpenClaw inside the sandbox and then asks about policy presets. You’ll see a list of available presets including Discord, Docker Hub, Hugging Face, Jira, npm, PyPI, Slack, and more. These control what external services the agent is allowed to reach.

At the bottom, the wizard asks:

Apply suggested presets (pypi, npm)? [Y/n/list]:

Type n and press Enter. These presets grant the sandbox network access to package registries, which you don’t need for a basic setup. You can always add them later if your agent needs to install packages.

Once onboarding finishes, you’ll see a clean summary with your sandbox details and the commands you’ll need going forward:

Sandbox    my-assistant (Landlock + seccomp + netns)
Model      nvidia/nemotron-3-super-120b-a12b (NVIDIA Cloud API)
NIM        not running

Run:       nemoclaw my-assistant connect
Status:    nemoclaw my-assistant status
Logs:      nemoclaw my-assistant logs --follow

Step 4 - Connect to NemoClaw

Now for the fun part. Connect to your sandbox.

nemoclaw my-assistant connect

This drops you into a shell inside the sandboxed environment. From here, launch the OpenClaw TUI (terminal user interface):

openclaw tui

That’s it. You should see the OpenClaw chat interface come up. The agent will greet you and introduce itself, ready to chat.

Type a message and hit Enter. You’re now talking to an AI agent running inside a secure, sandboxed environment on your own Droplet.

Reconnecting After a New SSH Session

If you close your terminal and SSH back into the Droplet later, you’ll find that nemoclaw and related commands aren’t available. That’s because the onboarding script installed everything through nvm in a separate shell, and that doesn’t carry over to new sessions.

Run this once to fix it permanently. It adds nvm to your .bashrc so it loads automatically on every login:

echo 'export NVM_DIR="$HOME/.nvm"' >> ~/.bashrc && echo '[ -s "$NVM_DIR/nvm.sh" ] && \. "$NVM_DIR/nvm.sh"' >> ~/.bashrc && echo '[ -s "$NVM_DIR/bash_completion" ] && \. "$NVM_DIR/bash_completion"' >> ~/.bashrc && source ~/.bashrc

Then reconnect to your sandbox and launch the TUI the same way as before:

nemoclaw my-assistant connect

openclaw tui

Everything picks up right where you left off. Your sandbox and agent are still running.

What's Next

By default, the sandbox has limited network access, so the agent can’t reach external services out of the box. To unlock more capabilities - like connecting to Slack, GitHub, or pulling packages from PyPI - you’ll want to configure policy presets. Check the NemoClaw documentation for the full list of available integrations and how to set them up.

NemoClaw is still very early, so expect things to be rough around the edges. But if you want to get a feel for where always-on agents are headed, this is a good way to start poking around.

Resources

GPT 5.3 Codex is the Next Level for Agentic Coding

DigitalOcean — Thu, 19 Mar 2026 20:00:00 +0000

Agentic Coding models are one of the obvious and most impressive applications of LLM technologies, and their development has gone hand in hand with massive impacts to markets and job growth. There are numerous players vying to create the best new LLM for all sorts of applications, and many would argue no company and their products in this space have more of a significant impact than OpenAI.

GPT‑5.3‑Codex is a truly impressive installment in this quest to create the best model. OpenAI promises that GPT-5.3-Codex is their most capable Codex model yet, advancing both coding performance and professional reasoning beyond GPT-5.2-Codex. Benchmark results show state-of-the-art performance on coding and agentic benchmarks like SWE-Bench Pro and Terminal-Bench, reflecting stronger multi-language and real-world task ability. Furthermore, the model is ~25% faster than GPT-5.2-Codex for Codex users thanks to infrastructure and inference improvements. Overall, GPT‑5.3‑Codex might be the most powerful agentic coding model ever released (Source).

So let’s see what it can do. Now available on the DigitalOcean GradientTM AI Platform and all OpenAI ChatGPT and Codex resources, we can test the model to see how it performs. In this tutorial, we will show how to use Codex to write a completely new project from scratch. We are going to make a Z-Image-Turbo Real-Time image-to-image application using GPT‑5.3‑Codex, without any user coding! Follow along to learn what GPT‑5.3‑Codex has to offer, how to use GPT‑5.3‑Codex for yourself, and a guide to vibe coding new web applications from scratch!

Key Takeaways

State-of-the-Art Agentic Performance: GPT-5.3-Codex delivers impressive results across software engineering and agentic tasks, outperforming GPT-5.2-Codex in reasoning, multi-language capability, and real-world coding evaluations like SWE-Bench Pro and Terminal-Bench 2.0.
Getting Started with GPT-5.3-Codex on GradientTM AI Platform is easy: All you need is access to the DigitalOcean Platform to begin integrating your LLM’s calls seamlessly into your workflows at scale.
From Prototype to Production in Record Time: With roughly 25% improved speed and real-time interactive steering, GPT-5.3-Codex feels less like a static generator and more like a responsive engineering partner capable of iterating, debugging, and refining projects alongside you. By handling scaffolding, architecture decisions, edge cases, and deployment-ready details, GPT-5.3-Codex can dramatically compress development timelines, making it possible to ship fully functional applications from scratch more quickly than ever (Source).

GPT‑5.3‑Codex Overview

GPT-5.3-Codex is a major agentic coding model upgrade that combines stronger reasoning and professional knowledge with enhanced coding performance, runs about 25 % faster than GPT-5.2-Codex, and excels on real-world and multi-language benchmarks like SWE-Bench Pro and Terminal-Bench. It’s designed to go beyond simple code generation to support full software lifecycle tasks (e.g., debugging, deployment, documentation) and lets you interact and steer it in real time while it’s working, making it feel more like a collaborative partner than a generator. It also has expanded capabilities for long-running work and improved responsiveness, with broader availability across IDEs, CLI, and apps for paid plans. (Source)

As we can see from the table above, GPT‑5.3‑Codex is a major step forward over GPT‑5.2‑Codex across software engineering, agentic, and computer use benchmarks. This, paired with the marked improvement in efficiency, make for an incredible indicator of how great this model is. We think this is a significant upgrade to previous GPT Codex model users, as well as new users looking for a powerful agentic coding tool to aid their process.

Getting Started with GPT-5.3-Codex

There are two ways to get started with GPT-5.3-Codex that we recommend to developers. First, is accessing the model with Serverless Inference through the GradientTM AI Platform. With Serverless Inference, we can Pythonically integrate the LLM generations into any pipeline. All you need to do is create a model access key, and begin generating! For more information on getting started, check out the official documentation.

The other way to get started quickly is the official OpenAI Codex application. It’s easy to get started with Codex on your local machine. Simply download the application onto your computer, and launch it. You will then be prompted to log in to your account. From there, simply choose which project you wish to work in, and you’re ready to get started!

Vibe Coding a Z-Image-Turbo Web Application with GPT‑5.3‑Codex

So now that we have heard about how GPT‑5.3‑Codex performs, let’s see it in action. For this experiment, we sought to see how the model performed on a relatively novel assignment that has a basis in past applications. In this case, we asked it to create a real-time image-to-image pipeline for Z-Image-Turbo that uses webcam footage as image input.

To do this, we created a blank new directory/project space to work in. We then asked the model to create a skeleton of the project to begin, and then iteratively added in the missing features on subsequent queries. Overall, we were able to create a full working version of the application with just 5 prompts and 30 minutes of testing. This extreme speed made it possible to ship the project in less than a day, from inspiration to completion. Now let’s take a closer look at the application project itself.

This project, which can be found here, is a real-time webcam-driven image-to-image generation application built in Python around a Gradio interface and a dedicated Z-Image-Turbo inference engine, where the UI in app.py presents side-by-side live input and generated output panes, parameter controls, and explicit Start/Stop gating so inference only runs when requested, while the backend in inference.py loads Tongyi-MAI/Z-Image-Turbo via ZImageImg2ImgPipeline, introspects the pipeline signature to bind the correct image-conditioning argument, enforces true img2img semantics instead of prompt-only generation, and executes inference in torch.inference_mode() with dynamic argument wiring so behavior adapts to the installed diffusers API. Critically, it can compute per-frame target resolution from webcam aspect ratio, snapping dimensions to a model-friendly multiple (default 16), and caps both sides below 1024, then applies post-generation safeguards that made the app stable in practice: dtype strategy (auto preferring bf16 then fp32, avoiding fp16 black-frame failure modes), degenerate-output detection with automatic float32 recovery, robust PIL/NumPy/Tensor output decoding and normalization, effective-strength clamping to preserve source structure, frame-hash seed mixing so scene changes influence results, and configurable structure-preserving input blending, all parameterized in config.py and documented in the README.md, with runtime status reporting latency plus internal diagnostics (pipe, dtype, size, effective strength, blend, seed, warnings) so you can observe exactly how each frame is being processed.

Closing Thoughts

GPT-5.3-Codex feels less like an incremental update and more like a meaningful shift in how developers interact with code. The combination of stronger reasoning, benchmark gains seen in testing, and a noticeable speed improvement makes it clear that agentic coding is maturing into something even more production-ready. What once required hours of boilerplate, debugging, and manual wiring can now be orchestrated through iterative prompts and high-level direction. As we demonstrated with the Z-Image-Turbo real-time application, a fully functional project can move from blank directory to working prototype in much less time traditionally required. While the actual results and performance benefits you experience will vary based on specific project requirements, complexity, and individual developer workflows, we are confident that GPT-5.3-Codex provides a substantial upgrade and a meaningful step forward in agentic coding capability, as evidenced by its stronger reasoning and measurable benchmark gains.

We recommend trying out GPT-5.3-Codex in all contexts, especially with DigitalOcean’s GradientTM AI Platform!

Getting Started with Qwen3.5 Vision-Language Models

DigitalOcean — Tue, 17 Mar 2026 16:00:00 +0000

This article was originally written by James Skelton (Senior AI/ML Technical Content Strategist II)

Vision Language models are one of the most powerful and highest potential applications of deep learning technologies. The reasoning behind such a strong assertion lies in the versatility of VL modeling: from document understanding to object tracking to image captioning, vision language models are likely going to be the building blocks of the incipient, physical AI future. This is because everything that we can interact with that will be powered by AI - from robots to driverless vehicles to medical assistants - will likely have a VL model in its pipeline.

This is why the power of open-source development is so important to all of these disciplines and applications of AI, and why we are so excited about the release of Qwen3.5 from Qwen Team. This suite of completely open source VL models, ranging in size from .8B to 397B (with activated 17B) parameters, is the clear next step forward for VL modeling. They excel at bench marks for everything from agentic coding to computer use to document understanding, and nearly match closed source rivals in terms of capabilities.

In this tutorial, we will examine and show how to make the best use of Qwen3.5 using a Gradienttm GPU Droplet. Follow along for explicit instructions on how to setup and run your GPU Droplet to power Qwen3.5 to power applications like Claude Code and Codex using your own resources.

Key Takeaways

Qwen3.5 VL demonstrates the growing power of open multimodal AI. The fully open-source model suite spans from 0.8B to 397B parameters and achieves strong benchmark performance across tasks like coding, document understanding, and computer interaction, approaching the capabilities of leading proprietary models.
Its architecture enables efficient large-scale multimodal training. By decoupling vision and language parallelism strategies, using sparse activations, and employing an FP8 training pipeline, Qwen3.5 improves hardware utilization, reduces memory usage, and maintains high throughput even when training on mixed text, image, and video data.
Developers can deploy Qwen3.5 on their own infrastructure. With tools like Ollama and GPU Droplets, it is possible to run large Qwen3.5 models locally or in the cloud to power applications such as coding assistants, computer-use agents, and custom AI tools without relying on proprietary APIs.

Qwen3.5: Overview

Qwen3.5 is a fascinating model suite with a unique architecture. It “enables efficient native multimodal training via a heterogeneous infrastructure that decouples parallelism strategies across vision and language components” (Source). This helps to make it avoid uniform approaches’ inefficiencies, such as over-allocating compute to lighter modalities, synchronization bottlenecks between vision and language towers, memory imbalance across devices, and reduced scaling efficiency when both modalities are forced into the same parallelism strategy.

By leveraging sparse activations to enable overlapping computation across model components, the system reaches nearly the same training throughput as pure text-only baselines even when trained on mixed text, image, and video datasets. Alongside this, a native FP8 training pipeline applies low-precision computation to activations, Mixture-of-Experts (MoE) routing, and GEMM operations. Runtime monitoring dynamically preserves BF16 precision in numerically sensitive layers, reducing activation memory usage by roughly 50% and delivering more than a 10% training speed improvement while maintaining stable scaling to tens of trillions of tokens.

To further leverage reinforcement learning at scale, the team developed an asynchronous RL framework capable of training Qwen3.5 models across all sizes, supporting text-only, multimodal, and multi-turn interaction settings. The system uses a fully disaggregated training–inference architecture, allowing training and rollout generation to run independently while improving hardware utilization, enabling dynamic load balancing, and supporting fine-grained fault recovery. Through techniques such as end-to-end FP8 training, rollout router replay, speculative decoding, and multi-turn rollout locking, the framework increases throughput while maintaining strong consistency between training and inference behavior.

This system–algorithm co-design also constrains gradient staleness and reduces data skew during asynchronous updates, preserving both training stability and model performance. In addition, the framework is built to support agentic workflows natively, enabling uninterrupted multi-turn interactions within complex environments. Its decoupled architecture can scale to millions of concurrent agent scaffolds and environments, which helps improve generalization during training. Together, these optimizations produce a 3×–5× improvement in end-to-end training speed while maintaining strong stability, efficiency, and scalability (Source).

Qwen3.5 Demo

Getting started with Qwen3.5 is very simple. Thanks to the foresight of Qwen Team & their collaborators, their are numerous ways to access and run the Qwen3.5 model suite’s models from your own machine. Of course, running the larger models will require significantly more computational resources. We recommend at least an 8x NVIDIA H200 setup for the larger models in particular, though a single H200 is sufficient for this tutorial. We are going to use Ollama to power Qwen3.5-122B-A10B.

To get started, simply start up a GPU Droplet with an NVIDIA H200 with your SSH key attached, and SSH in using the terminal on your local machine. From there, navigate to the base directory of your choice. Create a new directory with mkdir to represent your new workspace, and change into the directory.

Creating a custom game with Qwen3.5 running on Ollama and Claude Code

For this demo, we are going to do something simple: create a Python based video game for one of the most popular Winter Olympics sports: curling. To get started, paste the following code into the remote terminal:

curl -fsSL https://ollama.com/install.sh | sh
ollama launch claude --model qwen3.5:122b

This will launch Claude Code. If everything worked, it should look like above. From here, we can begin giving instructions to our model to begin generating code!

For this demo, provide it with a base set of instructions. Try customizing the following input:

“I want to create a simple game of curling in python code. i want it to be playable on my computer. Please create a sample Python program.

Packages: pygame”

This will give you, if your model ran predictably, a python file named something like “curling_game.py” with a full game’s code inside. Simply download this file onto your local computer, open the terminal and run it with python3.11 curling_game.py. Our game looks like this:

But looks are deceiving: this game is far from playable in the one-shot state. It requires serious work to amend the code to make the game playable, especially for two players. We can either use Claude Code with Qwen3.5 to make those adjustments, switch to an Anthropic Model like Sonnet 4.6 or Opus 4.6, or make the changes manually. From this base state, it took Qwen3.5 over an hour and at least 10 requests to make the game playable. Time was notably constrained by the single H200 GPU deployment we used for this demo, but the code output leaves significant room for improvement nonetheless. We expect that Opus 4.6 could accomplish the same task in a much quicker time frame, given its optimization for Claude Code, relatively superior benchmark scores, and more optimized infrastructure for inference.

If you want to try it out, this file can be found on Github Gist.

Closing Thoughts

Qwen3.5 VL represents an important step forward for open-source multimodal AI, demonstrating that publicly available models can increasingly rival proprietary systems in capability while offering far greater flexibility for developers. With its scalable architecture, efficient training infrastructure, and strong performance across tasks like coding, document understanding, and computer use, the Qwen3.5 suite highlights the growing maturity of the open AI ecosystem. As tools like GPU Droplets and frameworks such as Ollama make deploying large models easier than ever, vision-language systems like Qwen3.5 are poised to become foundational components in the next generation of AI-powered applications and physical AI systems.

7 OpenClaw Security Challenges to Watch for in 2026

DigitalOcean — Thu, 12 Mar 2026 16:00:00 +0000

This article was originally written by Fadeke Adegbuyi (Manager, Content Marketing)

OpenClaw isn’t just another chatbot wrapper. It executes shell commands, controls your browser, manages your calendar, reads and writes files, and remembers everything across sessions. The project runs locally on your machine and connects to WhatsApp, Telegram, iMessage, Discord, Slack, and over a dozen other platforms via pre-built integrations. It functions as a truly connected personal assistant. As a result, the use cases people have dreamed up for OpenClaw are wild.

One user showed an OpenClaw agent making money on Polymarket by monitoring news feeds and executing trades automatically. Another gave their bot access to home surveillance cameras. Someone else unleashed subagents to apply for UpWork freelancing jobs on their behalf.

// Detect dark theme var iframe = document.getElementById('tweet-2019044317621567811-81'); if (document.body.className.includes('dark-theme')) { iframe.src = "https://platform.twitter.com/embed/Tweet.html?id=2019044317621567811&theme=dark" }

But this kind of access to your digital life comes with real consequences when things go wrong. And things have gone wrong. Security researchers found that the agent shipped with serious flaws that made it possible for attackers to hijack machines with a single malicious link. Meanwhile, Moltbook, a Reddit-style platform with over 2.8 million AI agents, had its database completely exposed, so anyone could take control of any AI agent on the platform.

None of this means you should avoid OpenClaw entirely. It means you should understand OpenClaw security challenges and take precautions before spinning up an agent with root access to your laptop. Running OpenClaw in an isolated cloud environment can help neutralize some of these risks—DigitalOcean's 1-Click Deploy for OpenClaw, for example, handles authentication, firewall rules, and container isolation out of the box so your personal machine stays out of the equation.

What are OpenClaw security challenges?

OpenClaw security challenges boil down to a design tension: the tool needs broad system permissions to be useful, but those permissions create a massive attack surface when something goes wrong. The agent runs with whatever privileges your user account has—full disk, terminal, and network access—by design.

It's also agentic and self-improving, meaning it can modify its own behavior, update its memory, and install new skills autonomously. This is impressive from a capability standpoint, but another vector that can cause things to spiral when guardrails are missing. Pair that with defaults that skip authentication, an unvetted skill marketplace, and persistent memory storing weeks of context, and trouble follows. The takeaway: approach with caution, isolate from production systems, and carefully scrutinize the defaults.

To his credit, OpenClaw creator Peter Steinberger has been openly vocal about these risks and actively encourages running OpenClaw in a sandboxed environment, which isolates tool execution inside Docker containers to limit filesystem and process access when the model misbehaves. DigitalOcean's one-click deployment does exactly this out of the box, giving you that isolation without the manual setup.

7 OpenClaw security challenges to watch out for

We've already seen a security audit uncover 512 vulnerabilities (eight critical) and malicious ClawHub skills stealing cryptocurrency wallets. None of these challenges are theoretical. They're all based on incidents that have already played out within weeks of OpenClaw’s launch.

These are the challenges you need to have on your radar if you're experimenting with OpenClaw:

1. One-click remote code execution through WebSocket hijacking

One of the most alarming OpenClaw vulnerabilities discovered so far is CVE-2026-25253, a one-click remote code execution flaw that Mav Levin, a founding researcher at DepthFirst, disclosed in late January 2026. The attack worked because OpenClaw's local server didn’t validate the WebSocket origin header—so any website you visited could silently connect to your running agent. An attacker just needed you to click one link. From there, they chained a cross-site WebSocket hijack into full code execution on your machine. The compromise happened in milliseconds. This is the core danger of running an agent locally on the same machine you're browsing the web with—one careless click and an attacker is already inside.

Levin's proof-of-concept showed that visiting a single malicious webpage was enough to steal authentication tokens and gain operator-level access to the gateway API—giving an attacker access to change your config, read your files, and run commands.

Security checks: In this instance, the fix landed in version 2026.1.29, so update immediately if you’re a version behind. Beyond that, best practices include avoiding running OpenClaw while browsing untrusted sites and considering putting the agent behind a reverse proxy with proper origin validation for an additional layer of protection.

2. Tens of thousands of unprotected OpenClaw instances sitting open on the internet

Here's the thing about OpenClaw's early defaults: the agent trusted any connection from localhost without asking for a password. That sounded fine until the gateway sits behind a misconfigured reverse proxy—at which point every external request got forwarded to 127.0.0.1, and your agent thought the whole internet was a trusted local user. SecurityScorecard's STRIKE team found over 30,000 internet-exposed OpenClaw instances.

Security researcher Jamieson O'Reilly showed just how bad this gets. He accessed Anthropic API keys, Telegram bot tokens, Slack accounts, and complete chat histories from exposed instances, even sending messages on behalf of users and running commands with full admin privileges. No authentication required.

This has since been addressed—gateway auth is now required by default, and the onboarding wizard auto-generates a token even for localhost.

// Detect dark theme var iframe = document.getElementById('tweet-2015401219746128322-801'); if (document.body.className.includes('dark-theme')) { iframe.src = "https://platform.twitter.com/embed/Tweet.html?id=2015401219746128322&theme=dark" }

Security checks: At a minimum, check whether your instance is reachable from the public internet. Use a firewall to restrict access, enable gateway token authentication, and never expose the control plane without a VPN or SSH tunnel in front of it. This is a case where a managed cloud deployment can solve the problem outright—because your personal API keys, chat histories, and credentials aren’t sitting on an exposed local machine in the first place.

3. Malicious skills on ClawHub are poisoning the supply chain

ClawHub, OpenClaw's public skill marketplace, lets anyone publish an extension—the only requirement is a GitHub account older than one week. That low bar has unfortunately turned the marketplace into a target. Koi Security audited all 2,857 skills on ClawHub and found 341 that were outright malicious. Bitdefender's independent scan put the number closer to 900 malicious skills, roughly 20% of all packages. A single account—"hightower6eu"—uploaded 354 malicious packages by itself.

The attack is clever. You install what looks like a useful skill and the documentation looks professional. But buried in a "Prerequisites" section, it asks you to install something first—and that something is Atomic Stealer (AMOS), a macOS credential-stealing malware.

Security checks: OpenClaw has since partnered with VirusTotal to scan new skill uploads, but Steinberger himself admitted this isn't a silver bullet. At a minimum, before installing any skill, read its source code. Check the publisher's account age and history. Put simply, treat every skill as untrusted code running with your agent's full permissions. Unlike some exposure risks, malicious skills are a threat regardless of where OpenClaw runs—a poisoned skill executes the same way on a cloud server as it does on your laptop.

4. Credential storage in plaintext and API key leakage

One of the less glamorous but more dangerous issues is how OpenClaw handles secrets. The platform stores credentials in plaintext—including API keys for your LLM provider and tokens for every messaging platform your agent connects to—and those become targets the moment your instance is accessible to anyone other than you. Prompt injection attacks can also trick the agent into exfiltrating credentials by embedding hidden instructions in content the agent processes.

Cisco's team tested a skill called "What Would Elon Do?" and surfaced nine security findings, two of them critical. The skill instructed the bot to execute a curl command sending data to an external server controlled by the skill's author. Functionally, it was malware hiding behind a joke name.

Security check: At a minimum, rotate your API keys regularly and store secrets using environment variables or a dedicated secrets manager rather than config files. It's also worth setting spending limits on your LLM provider accounts. That way, even if a key is compromised, it can't rack up thousands in charges.

5. Prompt injection attacks amplified by persistent memory

What makes prompt injection in OpenClaw worse than in a typical chatbot is the persistent memory. The agent retains long-term context, preferences, and conversation history across sessions—which is one of its best features. But it also means a malicious instruction embedded in a website, email, or document doesn't have to execute immediately. Palo Alto Networks warned that these become "stateful, delayed-execution attacks". A hidden prompt in a PDF you opened last Tuesday could sit dormant in the agent's memory until a future task triggers it days later.

Security check: There's no perfect fix for prompt injection right now; it's an unresolved problem in agentic AI. But you can reduce the blast radius by limiting what tools and permissions your agent has access to, segmenting its access to sensitive systems, and reviewing its memory and context periodically for anything unexpected.

6. Shadow AI spreading through enterprise networks

This one's for anyone working at a company where developers tinker on their work machines. Token Security found that 22% of their enterprise customers have employees running OpenClaw as shadow AI without IT approval. Bitdefender confirmed the same, showing employees deploying agents on corporate machines connected to internal networks. An OpenClaw agent on a developer's laptop with VPN access to production means every vulnerability above is now a business problem.

Security check: If you're on a security team, you should scan your network for OpenClaw instances now. Set up detection for its WebSocket traffic patterns, and mandate that any approved use runs in an isolated environment—a VM or cloud server—rather than on laptops with internal access. Giving teams an approved, isolated deployment path is the fastest way to get ahead of shadow AI—it's much easier to enforce guardrails when the alternative isn't 'don't use it at all.'

7. The Moltbook database breach exposing millions of agent credentials

The security mess isn't limited to OpenClaw itself. Moltbook, the social network for AI agents built by Matt Schlicht, suffered a database exposure that cybersecurity firm Wiz discovered in early February. The database had zero access controls. Anyone who found it could view 1.5 million API tokens, 35,000 email addresses, and private messages between agents—enough to take control of any agent on the platform. China's Ministry of Industry and Information Technology issued a formal warning about OpenClaw security risks, citing incidents like this breach.

Security check: If you've used Moltbook, rotate every API key and token associated with your agent. Treat third-party platforms in the OpenClaw ecosystem with the same skepticism you'd apply to any new service asking for your credentials and consider additional security checks.

Any references to third-party companies, trademarks, or logos in this document are for informational purposes only and do not imply any affiliation with, sponsorship by, or endorsement of those third parties.

Pricing and product information accurate as of February 2026.

[Boost]

DigitalOcean — Tue, 10 Mar 2026 18:07:19 +0000

DigitalOcean for DigitalOcean

Mar 10

GPU Programming for Beginners: ROCm + AMD Setup to Edge Detection

#gpu #amd #programming #ai

Comments

2 min read

GPU Programming for Beginners: ROCm + AMD Setup to Edge Detection

DigitalOcean — Tue, 10 Mar 2026 16:00:00 +0000

In this hands-on tutorial, we demystify GPU computation and show you how to write your own GPU programs from scratch. Understanding GPU programming is essential for anyone looking to grasp why AI models depend on this specialized hardware.

We'll use ROCm and HIP (AMD's version of CUDA) to take you from zero to running real GPU code, culminating in a computer vision edge detector that processes images in parallel.

You can find the code in the project repository: https://github.com/oconnoob/intro_to_rocm_hip/blob/main/README.md

👇 WHAT YOU'LL LEARN IN THIS VIDEO 👇

🔧 Getting Set Up with ROCm Two ways to get started: spin up a GPU Droplet on DigitalOcean with ROCm pre-installed, or install ROCm yourself on an Ubuntu system with an AMD GPU. We cover both methods step-by-step.

➕ Example 1: Vector Addition (The Basics) Learn the fundamental structure of GPU programs—kernels, threads, blocks, and memory management. We'll add one million elements in parallel and verify our results.

⚡ Example 2: Matrix Multiplication (Why Libraries Matter) Discover why optimized libraries like rocBLAS dramatically outperform naive implementations. This is the operation powering most AI models you use daily.

👁️ Example 3: Edge Detection with Sobel Filter (The Cool Stuff) Apply your GPU programming skills to a real computer vision problem—detecting edges in images using a classic Sobel filter, all running massively parallel on the GPU.

Whether you're an AI enthusiast wanting to understand the hardware layer or a developer looking to harness GPU compute power, this tutorial gives you the foundation to start writing efficient parallel programs.

In case you haven't heard, we're back! Follow the DigitalOcean organization for updates, tutorials, and hands-on AI learning.

DigitalOcean — Fri, 06 Mar 2026 22:25:26 +0000

Jess Lulka for DigitalOcean

Feb 2

DigitalOcean on Dev.to: Practical AI Insights for Builders

#ai #digitalocean #machinelearning #learning

Comments 1

2 min read

We're DigitalOcean and we're excited to be here with you!

DigitalOcean — Thu, 23 Jul 2020 11:47:06 +0000

Hey everyone! We're so excited to be here at CodeLand:Distributed.

DigitalOcean offers the most easy-to-use and developer-friendly cloud platform. We help you manage and scale apps with an intuitive API, multiple storage options, integrated firewalls load balancers, and more. We're on a mission to simplify cloud computing so developers and businesses can spend more time creating software that changes the world!

Got Questions?! Let's chat!

Sammy the shark and our team members are here to connect with any questions you might have. We'll be at our DEV Connect channel all day, so stop by and say hello!

We're also happy to respond to any comments down below. 👇

Digital Swag

Today, we're offering all CodeLand attendees a $100 USD free trial. Sign up below and we'll follow up with all the details:

Sign up for a $100 DigitalOcean Promo!

You must first sign in to DEV Community.

We also have some fun wallpapers for anyone looking to spruce up their desktop or virtual backgrounds. ✨

Job Opportunities at DigitalOcean

DigitalOcean is a values-driven organization. Here is what we believe in:

Our community is bigger than just us
Simplicity in all we DO
We speak up when we have something to say and listen when others DO
e are accountable to deliver on our commitments
Love is at our core

Come swim with us: https://do.co/careers

How to Code in Go eBook

DigitalOcean — Mon, 22 Jun 2020 15:46:31 +0000

Introduction to the eBook

This book is designed to introduce you to writing programs with the Go programming language. You’ll learn how to write useful tools and applications that can run on remote servers, or local Windows, macOS, and Linux systems for development.

This book is based on the How To Code in Go tutorial series found on DigitalOcean Community. The topics that it covers include how to:

Install and set up a local Go development environment on Windows, macOS, and Linux systems
Design your programs with conditional logic, including switch statements to control program flow
Define your own data structures and create interfaces to them for reusable code
Write custom error handling functions
Building and installing your Go programs so that they can run on different operating systems and different CPU architectures
Using flags to pass arguments to your programs, to override default options

Each chapter can be read on its own or used as a reference, or you can follow the chapters from beginning to end. Feel free to jump to the chapter or chapters that best suits your purpose as you are learning Go with this book.

Download the eBook

You can download the eBook in either the EPUB or PDF format by following the links below.

After you’re finished this book, if you’d like to learn more about how to build tools and applications with Go, visit the DigitalOcean Community’s Go section.