DEV Community: Boris Barac

LLM Finetunings, how and why

Boris Barac — Wed, 22 Jul 2026 10:32:29 +0000

You have a 560-page neurosurgery textbook. Or a decade of internal wikis. Or a folder of PDFs that your team swears someone read once. You want an AI that actually knows this material, not a generic chatbot that makes up answers instead of admitting it does not know.

There are two ways to give a language model domain knowledge. Retrieval-augmented generation (RAG) keeps the documents external and looks them up at query time. The model reads the retrieved chunks and answers from them. Fine-tuning changes the model itself. You train it on examples so it learns to produce certain outputs without looking anything up. The first teaches the model where to look. The second teaches it how to respond. This post is about the second approach. It shows a pattern for generating synthetic training data from documents and fine-tuning a compact model on the results.

The pattern has three stages. Document conversion extracts structured content from raw files, preserving headers, tables, and lists as typed sections. Synthetic QA generation uses a large "teacher" model to generate question-answer pairs from each section, and the teacher does the work of understanding the material. Fine-tuning trains a small "student" model on those pairs using LoRA, which modifies a fraction of the model's parameters while keeping the rest frozen. The student memorizes what the teacher extracted. You get a compact model that runs locally, works offline, and responds like a domain expert, without a retrieval step.

The pattern generalizes. You can swap the document converter, swap the teacher model, or swap the student, and the loop stays the same. The rest of this post walks through each stage with code from a working implementation. The closing section covers when fine-tuning is the right tool and when RAG, or a combination of the two, is the better choice.

Technology stack

Metaflow — Workflow orchestration, parallel processing, resource management
Unsloth — GPU-accelerated LoRA fine-tuning (CUDA)
mlx-tune — Apple Silicon LoRA fine-tuning (MPS)
Docling — Document parsing (PDF, DOCX, PPTX, TXT)
OpenAI API — Teacher model for synthetic QA generation
Polars — Fast data processing

Turning documents into data

The pipeline's first job is turning a pile of varied files into something a language model can learn from. This is where most fine-tuning tutorials say "just put your data in JSONL" and move on, and where most practitioners give up.

We use Docling, IBM's open-source document converter. It runs a layout model that recognizes visual structure, including headings, paragraphs, tables, and code blocks, and preserves it as typed sections:

LABEL_TO_SECTION_TYPE = {
    DocItemLabel.TITLE: "title",
    DocItemLabel.SECTION_HEADER: "section_header",
    DocItemLabel.PARAGRAPH: "text",
    DocItemLabel.TEXT: "text",
    DocItemLabel.TABLE: "table",
    DocItemLabel.LIST_ITEM: "list_item",
    DocItemLabel.CODE: "code",
    DocItemLabel.FORMULA: "formula",
}

def _extract_items(doc, source_filename: str) -> list[dict]:
    rows = []
    for item, _level in doc.iterate_items():
        label = item.label
        section_type = LABEL_TO_SECTION_TYPE.get(label, "text")

        text_content = ""
        if hasattr(item, "text"):
            text_content = item.text or ""
        elif hasattr(item, "export_to_markdown"):
            text_content = item.export_to_markdown(doc) or ""

        if not text_content.strip():
            continue

        page_number = None
        if hasattr(item, "prov") and item.prov:
            page_number = item.prov[0].page_no

        rows.append({
            "source_file": source_filename,
            "page_number": page_number,
            "section_type": section_type,
            "text_content": text_content,
        })
    return rows

Every section becomes a row that is typed, queryable, and written to Parquet. Tables export as markdown. Section headers are tagged as headers, not just bold text. Docling also writes a full markdown rendering of each document alongside the Parquet, so you can check what the parser understood before spending money on QA generation.

The converter also caches its work. It checks whether the output Parquet is newer than all source files, and if nothing changed, it skips reconversion:

def _should_skip_conversion(source_dir: Path, output_dir: Path) -> bool:
    output_parquet = output_dir / OUTPUT_PARQUET_NAME
    if not output_parquet.exists():
        return False
    output_mtime = output_parquet.stat().st_mtime
    return not any(
        f.stat().st_mtime > output_mtime
        for f in source_dir.iterdir()
        if f.is_file() and f.suffix.lower() in SUPPORTED_EXTENSIONS
    )

Documents are converted in parallel with ProcessPoolExecutor. If a single file fails, the error is logged and the pipeline continues. It only raises an error if zero documents convert successfully.

Teaching a teacher to quiz itself

The pipeline now needs real training data, meaning question-answer pairs rather than raw text. The approach is to feed document chunks to a large language model and ask it to generate QA pairs. The prompt is constrained:

TEACHER_SYSTEM_PROMPT = (
    "You are a helpful assistant that generates question-answer pairs "
    "based on the provided document content. Generate questions that are "
    "answerable only from the given content. Each question should be clear "
    "and specific. Each answer should be comprehensive and accurate based "
    "on the content provided."
)

The key part of the prompt is the rule that questions must be answerable only from the given content. Without it, the teacher generates questions that sound reasonable but cannot be answered from the source. The student would learn to guess rather than to know.

The converted Parquet is split into chunks before it is sent to the API. Chunking respects document structure. It splits at section headers first, and falls back to a limit of 2000 words by default. Each chunk is prefixed with metadata like [Source: filename.pdf, Page: 42]. Chunks below 50 words are discarded. The --num-examples budget is distributed by word count.

The API call forces structured JSON output:

response = client.chat.completions.create(
    model=model,
    messages=[
        {"role": "system", "content": TEACHER_SYSTEM_PROMPT},
        {"role": "user", "content": prompt},
    ],
    response_format={"type": "json_object"},
)

Every generated pair is checked. Questions must be at least 5 words, and answers at least 20:

def filter_qa_pairs(examples, min_question_words=5, min_answer_words=20):
    return [
        ex for ex in examples
        if len(ex.get("question", "").split()) >= min_question_words
        and len(ex.get("answer", "").split()) >= min_answer_words
    ]

If fewer than half survive filtering, the pipeline errors out. Something is wrong with the source material, and training on bad data is worse than not training at all:

if len(filtered) < num_examples * 0.5:
    raise RuntimeError(
        f"Only {len(filtered)}/{num_examples} QA pairs generated "
        f"({len(filtered) / num_examples * 100:.0f}%). "
        f"Training data insufficient — check source documents or reduce --num-examples."
    )

The cost is low. With DeepSeek V3 via OpenRouter, 200 QA pairs from a 560-page textbook costs about $0.03. Any OpenAI-compatible endpoint works, including GPT-4o, Together, or a locally hosted model for work that needs privacy.

The output is another Parquet file with instruction and output columns, ready for training.

Fine-tuning with LoRA

A 1.6 billion parameter model, Gemma 3 1B, gets fine-tuned on your synthetic QA data. No GPU cluster is needed.

LoRA, which stands for Low-Rank Adaptation, freezes the base model entirely and injects small trainable matrices into the attention layers. Only these matrices are updated. In our case, LoRA trains 4.06 million parameters out of 1,631.75 million, or 0.249% of the model. Because so few parameters are trained, memory stays low at 2.7 GB, since you are not storing gradients for 1.6 billion parameters. The output is also a set of small adapter files rather than a full model copy, so you can keep multiple adapters for multiple domains and swap them without re-downloading the base model.

The model is loaded in 4-bit quantization, cutting base model memory from ~3.2 GB to ~0.8 GB:

model, tokenizer = FastLanguageModel.from_pretrained(
    model_name="unsloth/gemma-3-1b-it",
    max_seq_length=2048,
    load_in_4bit=True,
)

model = FastLanguageModel.get_peft_model(
    model,
    r=64,
    target_modules=["q_proj", "k_proj", "v_proj", "o_proj"],
    lora_alpha=64,
)

The training code works the same way on Apple Silicon and NVIDIA GPUs. It auto-detects the hardware, checking for CUDA first and then MPS, and dispatches to the right library:

def _detect_backend():
    try:
        import torch
        if torch.cuda.is_available():
            return "cuda"
        if hasattr(torch.backends, "mps") and torch.backends.mps.is_available():
            return "mlx"
    except ImportError:
        pass
    raise RuntimeError(
        "No supported accelerator found. "
        "Requires Apple Silicon (MPS) or NVIDIA GPU (CUDA)."
    )

def _get_training_imports(backend):
    if backend == "cuda":
        from unsloth import FastLanguageModel
        from unsloth.chat_templates import get_chat_template
        from trl import SFTConfig, SFTTrainer
    else:
        from mlx_tune import FastLanguageModel, SFTConfig, SFTTrainer, get_chat_template
    return FastLanguageModel, SFTTrainer, SFTConfig, get_chat_template

Both backends expose the same functions: FastLanguageModel, SFTTrainer, and SFTConfig. The rest of the training code doesn't know which one it is running on. There is no if apple_silicon: check scattered through the training loop.

On an M-series Mac, Gemma 3 1B trained on 481 QA pairs for 100 iterations with these results:

Metric	Value
Training loss	2.198 → 0.503
Validation loss	2.699 → 0.922
Peak memory	2.7 GB
Trainable params	4.06M / 1,631.75M (0.249%)

Training loss drops from 2.2 to 0.5, which shows the model is learning. Validation drops from 2.7 to 0.9, which shows it is generalizing. The output is a directory of LoRA adapter files, loaded alongside the untouched base model for inference.

One command

The three stages are orchestrated by Metaflow as a directed graph of steps connected by self.next():

start → convert_documents → generate_qa → process_chunks → merge_results → train_model → export_model_step → end

Each step declares its own resource requirements. Conversion, which is CPU-heavy, gets memory=4000, cpu=8. Training, which is GPU-heavy, gets memory=16000, gpu=1. On a local machine, Metaflow uses these as documentation. On a cluster such as AWS Batch or Kubernetes, it schedules steps on appropriate hardware:

@resources(memory=4000, cpu=8)
@step
def convert_documents(self):
    self.converted_parquet_path = convert_documents(
        self.docs_path_str, self.docs_output_str,
        num_threads=int(self.num_threads),
    )
    self.next(self.generate_qa)

@resources(memory=16000, cpu=4, gpu=1)
@step
def train_model(self):
    config = TrainingConfig(
        data=self.output_dir_str,
        iters=int(self.training_iters),
        batch_size=int(self.batch_size),
        backend=str(self.backend),
        model=str(self.model_name),
        chat_template=str(self.chat_template) if self.chat_template else None,
        learning_rate=float(self.learning_rate),
        lora_rank=int(self.lora_rank),
        max_seq_length=int(self.max_seq_length),
    )
    self.adapter_dir = train_model(
        data=config.data,
        iters=config.iters,
        batch_size=config.batch_size,
        backend=config.backend,
        model=config.model,
        chat_template=config.chat_template,
        learning_rate=config.learning_rate,
        lora_rank=config.lora_rank,
        max_seq_length=config.max_seq_length,
    )
    self.next(self.export_model_step)

Every configurable option, including teacher model, chunk size, LoRA rank, learning rate, and backend, is a @Parameter with a sensible default. The entire pipeline runs with one command:

uv run python ./src/pipeline.py run \
  --docs-path ./my-documents \
  --teacher-model deepseek-chat \
  --num-examples 200 \
  --training-iters 100 \
  --model unsloth/gemma-3-1b-it

The full pipeline inherits from a standalone data preparation flow, so you can run just conversion and QA generation without training, or just training on data you already have.

The transformation step uses Metaflow's parallel_map to split the QA Parquet into JSONL chunks across processes, then merges and splits into train and validation sets. The validation ratio adapts to the sample count. It uses 15% validation below 300 samples, 10% below 1000, and 5% above:

chunk_results = parallel_map(process_chunk, chunk_inputs)
train_path, valid_path = split_dataset(combined_path, output_dir)

What fine-tuning gives you

There is a clean way to think about the choice between fine-tuning and retrieval-augmented generation. RAG supplies knowledge. It retrieves relevant documents at query time and injects them into the prompt, so the model answers using current information rather than what was in its weights. Fine-tuning shapes behavior. It trains the model on examples so it learns patterns in outputs, not dynamic facts. A useful shortcut is that RAG teaches the model where to look, and fine-tuning teaches it how to respond.

You should pick RAG when the problem sounds like "answer from my data." It covers private documents that change often, use cases that need source citations, and knowledge bases that live across PDFs, code, tickets, and databases. RAG is faster to update because you refresh the index without retraining. It can expose citations. Its weak point is that answer quality depends heavily on retrieval quality, including chunking, embeddings, reranking, and context assembly. Get any of those wrong and the model answers from garbage.

You should pick fine-tuning when the problem sounds like "answer in my format." It covers repeatable output structure, task-specific behavior, and a certain tone or style, for example classification, structured JSON extraction, domain-specific response formatting, and brand voice control. Fine-tuning produces more consistent outputs with smaller prompts, but it takes training data and experimentation. It also doesn't naturally cite sources, and it is a poor way to store large, evolving knowledge bases.

The two approaches also work well together. Use RAG for live knowledge retrieval. Fine-tune a smaller model for response style, schema adherence, or better use of retrieved context. For a domain-specific assistant, keep RAG over the document corpus for accuracy and citations, and fine-tune a compact model for the common queries where speed and autonomy matter more than exact sourcing.

The pipeline described in this post sits in a middle ground. It generates synthetic QA pairs from documents using a teacher model, then fine-tunes a student to reproduce those answers. The result behaves like a domain expert, since it is fast, works offline, and needs no retrieval step. But it is closer to shaping behavior than to injecting knowledge. The student hasn't memorized the source documents. It has learned to answer questions about them as the teacher would, which means the model is wrong in the same ways the teacher is wrong, and it doesn't know anything the teacher didn't extract.

The pipeline is a reusable pattern. You can swap the document converter for a database connector, swap the teacher model for a different API, or swap the student for a larger model. The core loop stays the same.

Agentic Loops: Quick Guide

Boris Barac — Mon, 13 Jul 2026 18:19:10 +0000

The coding agent is usually not the hard part of an agentic loop. It is everything around it: task selection, the sandbox, branch rules, review, permissions, and what happens when a run gets weird.

The loop itself is almost boring. Take a task from a queue, run it through a coding harness, validate, commit, then grab the next item. Basically a for loop with an agent inside.

Making that loop useful, and safe enough to leave running, is the real work. A few things I learned while building mine.

Keep the agents contained

I run the agents inside Docker containers, with Codex or OpenCode in headless mode. Not because Docker makes the agent smarter. It just keeps the environment boring and limits what the agent can touch.

It also makes failures less mysterious. I know what was installed and which files were available. I am not guessing what the agent found somewhere on my machine.

Implementation is not the last step

Once the code exists, the task is not done. I run a separate review agent after implementation. It either approves the change or sends it back with feedback.

Keeping those steps separate matters. The agent that wrote the code already made a bunch of assumptions. Asking it to review itself mostly gives those same assumptions another pass.

Interactive and headless modes need different rules

Take another look at AGENTS.md before putting it inside a loop. Instructions that work nicely in an interactive session can be wrong in headless mode.

An interactive agent can just stop and ask me something. A headless one cannot. It needs to know when to skip work, when to open a ticket, how to report a block, maybe when to give up entirely.

Same agent, different rules.

Use different loops for different jobs

I prefer small, specialized loops over one big loop doing everything. I have different ones for review and cleanup, plus e2e tests and security work.

They do not all end the same way. Some produce pull requests. Some only create tickets. A test loop might just report what broke. That is enough.

The predictable loops can move to cloud workflows. I do this for cleanup work. The workflow only picks up tasks with the right label, does its narrow job, and leaves everything else alone.

Design around branches and labels

Branching is part of the loop design, not something to decide later. Some tasks get one branch each. Related cleanup tasks might need to stay together on a shared branch.

If the loop does not know this upfront, you get conflicting changes. Or work that belongs together gets split across several PRs for no good reason.

And use labels a lot. Each loop should only see tasks carrying its label. That label is the boundary. Without it, sooner or later, the wrong loop picks up the wrong work.

The loop can stay simple. The system around it cannot be vague. Give the agent a narrow job, a controlled place to run, and a clear way out when the task does not fit.

What did I miss? If you are building similar loops, share what worked for you. Especially the ugly parts. Those are usually the useful ones.

Serverless GPU Inference: Deploy Any Hugging Face Model on Google Cloud Run

Boris Barac — Mon, 15 Jun 2026 12:53:43 +0000

curl https://vllm-endpoint-xxxxx-ew4.a.run.app/v1/chat/completions \
  -H "Content-Type: application/json" \
  -d '{
    "model": "Qwen/Qwen3.5-4B",
    "messages": [{"role": "user", "content": "Say hello in one word."}],
    "max_tokens": 32
  }'

That endpoint is real. An NVIDIA L4 GPU on Cloud Run, running vLLM (best open source way to serve LLMs) OpenAI-compatible API.

Four commands to get there:

make tf-init        # initialize Terraform
make build-cloud    # build the Docker image remotely
make deploy         # provision infrastructure + deploy
make smoke-test     # verify it works end-to-end

No Kubernetes, no Helm charts, no 47-step tutorials. One Makefile, a Dockerfile, a few Terraform files. Here's how the pieces fit together:

HuggingFace
     │
     ▼   docker build (downloads weights into image)
 Baked Image ───► Artifact Registry
                        │
                        ▼   terraform apply
                  Cloud Run (GPU)
                        │
                        ▼
              OpenAI-compatible Endpoint

Four pieces, one responsibility each.

The Dockerfile takes a model name and a HuggingFace token, downloads the weights at build time, and bundles them into a vLLM image. The result is a baked image — self-contained, no internet at runtime.

Artifact Registry holds that image. Terraform creates the repo with a cleanup policy that keeps only the latest tag. Old images get deleted. No surprise bills.

Cloud Run runs the image on an NVIDIA L4 GPU. Serverless — no VM, no cluster, just a URL.

The Makefile ties it together. build-cloud triggers a remote build on Cloud Build, so your laptop never touches 15 GB of weights. deploy runs Terraform. smoke-test hits the endpoint with three requests and tells you if anything's broken.

First piece: the Dockerfile.

The Dockerfile

FROM vllm/vllm-openai:latest-cu129-ubuntu2404

ARG MODEL_NAME
ARG HF_TOKEN

ENV HF_HOME=/model-cache
ENV MODEL_NAME=${MODEL_NAME}
ENV PORT=8080

RUN hf auth login --token ${HF_TOKEN} && \
    hf download ${MODEL_NAME}

ENV HF_HUB_OFFLINE=1

COPY entrypoint.sh /entrypoint.sh
RUN chmod +x /entrypoint.sh

ENTRYPOINT ["/entrypoint.sh"]

Seven instructions. The base image is vllm/vllm-openai — vLLM preconfigured to speak the OpenAI API. No server to write, nothing to install. Give it a model and it serves.

The RUN line is the important one. It logs into HuggingFace and downloads the model during the image build, not at startup. The weights end up baked into the image layer.

When Cloud Run pulls this image, it gets the model too. No runtime download, no waiting on HuggingFace, no cold start that depends on someone else's network.

Then HF_HUB_OFFLINE=1. This tells the HuggingFace libraries: don't try the internet. If the model isn't already local, fail immediately. If you built the image without weights, you find out at startup — not after ten minutes of confusing errors.

The entrypoint script runs vLLM with bfloat16 precision, 90% GPU memory utilization, and 4096 max context tokens. Reasonable defaults for a single L4. Tune per model.

#!/bin/bash
exec vllm serve "${MODEL_NAME}" \
    --host 0.0.0.0 \
    --port "${PORT:-8080}" \
    --dtype bfloat16 \
    --gpu-memory-utilization 0.9 \
    --tensor-parallel-size 1 \
    --reasoning-parser qwen3 \
    --max-model-len 4096

exec replaces the shell with the vLLM process so SIGTERM passes through. Cloud Run sends SIGTERM to shut down the container. Without exec, the signal gets swallowed and Cloud Run force-kills after the timeout.

Build it, push it, Cloud Run serves the model. But how does Cloud Run know to attach a GPU?

The Terraform

Cloud Run doesn't attach a GPU by default. You have to ask explicitly, and in the right way. Here's the Cloud Run service resource — the core of the deployment:

resource "google_cloud_run_v2_service" "vllm" {
  name     = var.service_name
  location = var.region
  project  = var.project_id

  template {
    execution_environment = "EXECUTION_ENVIRONMENT_GEN2"

    scaling {
      min_instance_count = 1
      max_instance_count = var.max_instances
    }

    containers {
      image = local.image_url
      ports {
        container_port = 8080
      }

      resources {
        limits = {
          cpu              = "8"
          memory           = "24Gi"
          "nvidia.com/gpu" = "1"
        }
        cpu_idle = false
      }

      startup_probe {
        tcp_socket {
          port = 8080
        }
        initial_delay_seconds = 240
        timeout_seconds       = 240
        failure_threshold     = 2
        period_seconds        = 240
      }
    }

    node_selector {
      accelerator = "nvidia-l4"
    }
  }
}

Three things make the GPU work. Miss any one and it won't.

"nvidia.com/gpu" = "1" in the resource limits. A Kubernetes-style resource request that Cloud Run adopted. "1" means one GPU.

node_selector { accelerator = "nvidia-l4" }. Tells Cloud Run which GPU. nvidia-l4 is the sweet spot — 24 GB VRAM, relatively cheap, available in most regions.

execution_environment = "EXECUTION_ENVIRONMENT_GEN2". GPUs require Gen2. Gen1 doesn't support them. Forget this and Terraform will error, but the message won't tell you why.

Then the startup probe. vLLM needs 60–90 seconds to load a 4B-parameter model into VRAM. The probe waits 240 seconds — generous, but it covers larger models too.

Cloud Run won't route traffic until the probe passes (TCP connect on port 8080). Until then, the service shows "not ready" and requests get a 503.

The rest is plumbing. A service account, an Artifact Registry repo with a one-image cleanup policy, API enablement for Cloud Run and IAM. All boilerplate, all in the repo.

Image built, infrastructure defined. What does deploying actually look like?

Deploying, start to finish

Prerequisites: a GCP project, gcloud CLI, Terraform >= 1.5, a HuggingFace token, and make.

Configure HF_TOKEN in .env and project_id in terraform/terraform.tfvars, then run:

make tf-init        # first time only
make build-cloud    # 5–10 min, builds remotely on Cloud Build
make deploy         # terraform apply, outputs the endpoint URL
make smoke-test     # three requests against the live endpoint

build-cloud submits to Cloud Build so your laptop never touches the weights. For local builds, make build && make push works too.

Clone to working endpoint: ~15 minutes, mostly the build. The endpoint is OpenAI-compatible — set base_url and go:

from openai import OpenAI

client = OpenAI(
    base_url="https://vllm-endpoint-xxxxx-ew4.a.run.app/v1",
    api_key="not-needed",  # public endpoint, no auth yet
)

response = client.chat.completions.create(
    model="Qwen/Qwen3.5-4B",
    messages=[{"role": "user", "content": "Explain GPUs in one sentence."}],
)
print(response.choices[0].message.content)

The takeaway

vLLM is built for throughput under concurrency — PagedAttention, continuous batching, and scaling knobs for tensor parallelism, pipeline parallelism, data parallelism, and chunked prefill. Controls like max_num_batched_tokens, max_num_seqs, and gpu_memory_utilization let you tune the latency-throughput tradeoff. Cloud Run's GPU support means you get all of that without managing infrastructure.

Bake the model, push the image, run. Change MODEL_NAME to swap models — the infrastructure stays the same.

What's not covered yet: authentication (the endpoint is public), monitoring alerts, multi-model routing, and cost control at scale. Not hard problems — decisions for when you need them.

When JavaScript Isn't Fast Enough

Boris Barac — Fri, 12 Jun 2026 16:41:52 +0000

500 requests per second. One minute. An Express endpoint computing BTC technical indicators over 3.6 million data points.

Node.js folded. Average latency: 3,318 ms. p95: 8.4 seconds. 15.5% of requests failed. The median response crawled in at 1.6 seconds. The JavaScript computation pipeline couldn't keep up.

Same server. Same load. Same data. One change: the /price-rust endpoint, backed by a Rust addon compiled to a .node binary via NAPI.RS. Average latency: 660 ms. p95: 2 seconds. Zero errors. Peak RSS: 283 MB — less memory than the JS path.

Same Express process, same port, same payload. The only difference was who did the math.

What the API BENCHMARK actually does

The benchmark is a BTC price analysis endpoint. We synthetically generate 360 000 datapoints.
On that dataset the server computes SMA (25/50/100/200-day windows), RSI (14-period), MACD (12/26/9), Bollinger Bands (20-period), and a composite trading signal built from golden/death crosses, RSI divergence, MACD crossovers, and Bollinger squeeze detection.

Branching, windowing, stateful accumulation, multi-indicator correlation. The kind of workload that makes a runtime earn its keep — or not.

The full picture

The HTTP benchmark shows what happens under real load. The function benchmark isolates raw computation from HTTP overhead, request queuing, and GC pressure.

HTTP — k6, 500 req/s, 1 minute

Runtime	Endpoint	Avg latency	p95	Throughput	Errors	Peak RSS
Node.js	`/price` (JS)	3,318 ms	8,437 ms	62.4 rps	15.5%	292 MB
Node.js	`/price-rust` (N-API)	660 ms	2,061 ms	35.6 rps	0%	283 MB
Bun	`/price` (JS)	1,845 ms	5,146 ms	33.4 rps	0%	470 MB
Bun	`/price-rust` (N-API)	642 ms	2,412 ms	32.1 rps	0%	259 MB

Node.js on JS is the only combination that failed requests. Every other configuration held — including Bun running the exact same JavaScript. Bun's JS path is 1.8x faster than Node's and dropped zero requests.

The Rust paths are nearly identical across runtimes: 660 ms on Node, 642 ms on Bun. The runtime barely matters when Rust is doing the work.

Function — mitata, 366K data points

Variant	Runtime	ops/sec	Avg (ms)	Speedup vs JS
JS	Node.js	20	49.11	1x
Native (N-API)	Node.js	63	15.95	3.1x
JS	Bun	28	36.10	1x
Native (N-API)	Bun	69	14.51	2.5x
JS	Browser (Chromium)	26	38.04	1x
Native (WASM)	Browser (Chromium)	56	17.71	2.2x

Three runtimes, three JS baselines, all within a narrow band: 20–28 ops/sec. The Rust speedup is consistent — ~3x on Node, ~2.5x on Bun, ~2.2x on WASM.

WASM in the browser hits 56 ops/sec — closer to native N-API (63–69) than to any JS baseline. That's worth sitting with for a moment.

Sync vs async N-API made no measurable difference (63 vs 63 on Node, 69 vs 68 on Bun). The async variant offloads to a Tokio thread pool via spawn_blocking, but the bottleneck is the math, not the calling convention.

How a Rust function becomes a Node endpoint

The binding layer is thinner than you'd expect. napi-rs is the bridge: annotate a Rust function with #[napi], compile to a .node binary, and Node can require() it like any other module.

Here's the JS moving average function — the one that runs in every request to /price:

export function calculateMovingAverages(prices, smaWindows, cutoffYears = 9) {
  const cutoffIndex = Math.max(0, prices.length - cutoffYears * 365);
  const movingAverages = [];
  const runningSums = new Array(smaWindows.length).fill(0);

  for (let i = 0; i < prices.length; i++) {
    const price = prices[i][1];
    for (let wi = 0; wi < smaWindows.length; wi++) {
      const w = smaWindows[wi];
      runningSums[wi] += price;
      if (i >= w) runningSums[wi] -= prices[i - w][1];
    }
    if (i < cutoffIndex) continue;
    // ... build entry with date, price, SMA values
  }
  return movingAverages;
}

And here's the Rust equivalent, exposed to Node:

#[napi]
pub fn calculate_moving_averages(
    prices: Float64Array,
    sma_windows: Vec<u32>,
    cutoff_years: Option<u32>,
) -> Buffer {
    let cutoff_years = cutoff_years.unwrap_or(9);
    let dates = crate::utils::precompute_dates(&prices);
    let result = calc_ma(&prices, &sma_windows, cutoff_years, &dates);
    Buffer::from(serde_json::to_vec(&result).unwrap())
}

That #[napi] macro generates the C FFI glue Node's N-API runtime expects. It handles type conversion between V8 values and Rust types, and registers the function as a module export.

Float64Array comes in as a borrowed &[f64] slice with no copy. Buffer goes out as raw bytes, skipping V8's UTF-16 string conversion entirely.

On the JS side, loading the addon is one line:

const native = require("./napibench-native.node");

Then native.calculateMovingAverages(prices, [25, 50, 100, 200]) calls directly into compiled Rust. No HTTP, no IPC, no child process. Same call stack, same thread.

Need async? The async variant parks the work on a Tokio thread pool and returns a Promise. Same function, different concurrency model.

The server wires it into Express the same way the JS endpoint is wired. Same req → compute → res flow. The handler just calls a different function.

Same Rust, different planet

The Cargo.toml has a feature gate:

[features]
default = ["napi", "napi-derive", "rayon", "napi-build"]
wasm = ["wasm-bindgen", "js-sys"]

Build with the default features, you get napi_impl.rs — N-API bindings, rayon parallelism, tokio async. Build with --no-default-features --features wasm, you get wasm.rs — the same indicator logic compiled to WebAssembly via wasm-bindgen.

The WASM build strips away some optimizations. No rayon, no async thread pool, no Buffer returns — the WASM function returns a JSON string. It's the simpler, less-tuned version of the Rust code.

And it still hits 56 ops/sec in headless Chromium. That's 2.2x faster than Chromium's own JS engine running the same algorithms, and it's closer to native N-API performance (63–69 ops/sec) than to any JS baseline (20–28 ops/sec).

The bulk of the speedup comes from Rust's compiler — LLVM optimizing tight loops, stack-allocated structs, no GC pauses, no hidden type checks — not from rayon or async thread pools. Those help, but the floor is already high.

The browser benchmark ran in headless Chromium via Playwright: the WASM module and JS indicator code, same 366K-data-point pipeline, same 5-second window. The WASM binary is produced by wasm-pack build --target web.

Because napi-rs makes Rust functions callable like JS functions, the existing JS test suite doubles as a parity test for Rust. The test imports the JS indicator functions alongside the native addon, runs the same input through both, and asserts the outputs match field by field:

const jsMa = calculateMovingAverages(pricesJs, [25, 50, 100, 200], 1);
const rustMa = decode(native.calculateMovingAverages(prices, [25, 50, 100, 200], 1));
expect(rustMa).toHaveLength(jsMa.length);
for (let i = 0; i < jsMa.length; i++) {
  expect(rustMa[i].price).toBe(jsMa[i].price);
}

The JS implementation becomes the test oracle for the Rust implementation. The test runner (vitest) doesn't know or care that one side is native.

When to reach for Rust

Three runtimes. Two languages. One workload. Here's what the numbers actually say.

Reach for Rust when you're CPU-bound. This workload is arithmetic-heavy, loop-heavy, and largely sequential — the kind of thing V8 and JavaScriptCore optimize well but can't match against LLVM.

The 2.2–3x speedup appeared across every runtime, calling convention, and optimization level. If your server spends most of its time computing — parsing, transforming, aggregating, encrypting — a native addon pays for itself.

But N-API has a floor. The FFI boundary between V8 and native code adds fixed overhead per call. On small datasets — a few hundred data points — that overhead erases the compute savings. Node/Bun JS outperforms Node/Bun + Rust N-API at that scale. The native path only wins once the dataset is large enough to amortize the crossing cost.

Don't reach for Rust for I/O. The 3x raw compute speedup became a 5x latency reduction under HTTP load because the JS endpoint was past its breaking point. But the Rust endpoints on Node and Bun had nearly identical latencies (660 ms vs 642 ms).

The runtime's HTTP stack, event loop, and memory management dominate when computation is fast. Rust made the compute a non-factor; then the runtime didn't matter.

Bun is a cheaper upgrade than Rust. If you're on Node.js and your JS code is too slow, switching to Bun gives you a 40% speedup with zero code changes. Same JavaScript, same endpoints, different binary.

In this benchmark, Bun's JS path avoided errors entirely where Node.js failed 15.5% of requests. That might be enough. Rust is the bigger hammer, but Bun is the one you don't have to think about.

WASM is viable. Not a curiosity — genuinely competitive. 56 ops/sec in a browser tab, 2.2x faster than the browser's own JS, built from the same Rust codebase with a feature flag.

Unlike NODE/BUN, WASM's speedup holds even at small workloads. The calling convention is lighter — no FFI boundary to cross, the module runs inside the same engine. Where N-API needs a large dataset to amortize its overhead, WASM delivers from the start.

If you're running heavy computation in the browser — image processing, data analysis, simulations — WASM is the answer, and you don't need a separate codebase to get there.

The binding layer is not the bottleneck. napi-rs adds almost nothing to call overhead. Sync and async variants performed identically. The FFI boundary — V8 to Rust and back — is cheap enough that it doesn't appear in the numbers.

What matters is what happens on the other side of that boundary.

#rust #nodejs #bun #napi #webassembly #performance #benchmark

You Might Not Need Docker: The Case for Lightweight, Headless VMs

Boris Barac — Fri, 12 Jun 2026 16:38:04 +0000

Lately, whenever I need to run a quick experiment or spin up a localized environment with multiple moving pieces, I've completely stopped using Docker. Instead, I've been running full, headless Ubuntu virtual machines directly from my host terminal using Canonical's Multipass. And I love it.

It is using your native hypervisor (KVM on Linux, Hyper-V on Windows, or QEMU on macOS), it drops a clean Ubuntu environment onto your laptop with next to zero performance overhead.

The Zero-Friction Sandbox

Instead of wrestling with installation mirrors and configuration screens, Multipass treats virtual machines as disposable command-line primitives.

To spin up a basic machine (we'll name it devbox) and jump inside, the standard baseline looks like this:

# Spin up a fresh Ubuntu instance instantly
multipass launch --name devbox

# Drop straight into the VM's interactive bash shell
multipass shell devbox

Running Headless Commands Directly

You don't actually have to open an interactive shell session just to execute a single task. If you want to fire off commands and capture the output straight back to your host terminal, you can leverage exec:

# Check the VM's kernel version without entering the shell
multipass exec devbox -- uname -a

# Remotely trigger an update and install packages
multipass exec devbox -- sudo apt update
multipass exec devbox -- sudo apt install -y nginx

# this is usually needed to run custom tools out of the shell
multipass exec devbox -- bash -c "source ~/.profile && command"

Moving Files Between Host and VM

When you are experimenting with local scripts or configuration files, moving data in and out of the isolated environment shouldn't require setting up network shares or SSH keys. Multipass handles this natively with a built-in transfer primitive:

# Copy a local configuration script from your host into the VM
multipass transfer ./setup.sh devbox:/home/ubuntu/setup.sh

# Pull a generated log file or benchmark report back out to your host
multipass transfer devbox:/home/ubuntu/output.log ./output.log

If a testing session gets messy or you corrupt a system-level networking rule, you don't spend hours debugging. You simply nuke the instance and wipe the slate clean:

# Obliterate the broken VM and purge its allocated storage
multipass delete devbox
multipass purge

Infrastructure as Code: Local Automation via Cloud-Init

For more advanced experiments requiring multiple pre-installed binaries or specific user permissions, you can bypass manual setup entirely using cloud-init. This is the exact same industry-standard engine used by major cloud providers to bootstrap instances at launch.

If we need a repeatable baseline containing Redis and Git, we can define the required environment state inside a standard cloud-config.yaml file:

#cloud-config
users:
  - name: devuser
    sudo: ALL=(ALL) NOPASSWD:ALL
    shell: /bin/bash
packages:
  - redis-server
  - git
runcmd:
  - systemctl enable redis-server
  - systemctl start redis-server

To parse this configuration block directly during the VM's hardware initialization phase, pass the file using the --cloud-init flag:

# Launch a fully customized, automated instance using the configuration file
multipass launch --name cache-layer --cloud-init cloud-config.yaml

Instead of spending ten minutes manually running configuration commands every time you need a new sandbox, dropping this YAML into a directory gives you an identical baseline whenever you need it. If the configuration breaks or the environment gets contaminated, you just destroy the VM and re-run the exact same launch command.

Architecture Breakdown: Multipass vs. Docker

A common question that comes up is why use Multipass when you could just run a Docker container. The core distinction lies entirely in where the isolation boundaries are drawn across the system stack.

+-----------------------------------+     +-----------------------------------+
|          DOCKER ARCHITECTURE      |     |       MULTIPASS ARCHITECTURE      |
|                                   |     |                                   |
|  [App A]   [App B]   (Containers) |     |  [Systemd] [Kernel]  (Full VM)    |
|  +-----------------------------+  |     |  +-----------------------------+  |
|  |    Shared Host Kernel       |  |     |  |     Native Hypervisor       |  |
+--+-----------------------------+--+     +--+-----------------------------+--+

Docker isolates user-space processes. The containers sit directly on top of your host machine's operating system kernel. This makes them incredibly fast and resource-efficient for deploying standardized applications or microservices, but they don't have true system autonomy.

Multipass isolates the entire operating system block. It boots a completely independent Linux kernel, initializes a standalone virtual network loop, and runs its own init system (systemd).

If your experiment requires testing firewall structures (iptables), adjusting low-level network parameters, running background daemons orchestrated by systemd, or managing separate kernel modules, a Docker container requires fragile workarounds. Multipass gives you a true server abstraction directly on your laptop. Or even if you just want to spin up 5 things.

Hope it was interesting, I hope you learned something new. Thx for reading.

Linkloom - AIWebReader

Boris Barac — Fri, 12 Jun 2026 16:24:17 +0000

LinkLoom

A web scraping and content extraction toolkit for TypeScript/Bun.

Pass a URL, get clean markdown. That's the core. But LinkLoom also handles the cases that break simple scrapers: JavaScript-heavy pages rendered through a stealth browser, PDFs parsed into structured text, iframes pulled from nested frames, HTML tables converted to markdown tables, links extracted and classified. It exposes a library API, a CLI, and an MCP server — so you can use it from code, from the terminal, or from an AI client like Claude Desktop or Cursor.

The full list: URL-to-markdown conversion, HTML-to-markdown via Readability + Turndown, PDF-to-markdown via pdf.js, headless browser rendering through Camoufox (stealth Firefox on Playwright), iframe extraction with configurable wait strategies, link extraction and classification, table scraping, text embeddings via OpenAI or Gemini, a CLI for every feature, and an MCP server for AI tool-use workflows.

Built with Bun, Camoufox, JSDOM, Readability, Turndown, and pdf.js-extract. Optional embedding support through LangChain.

import { convertLinkToMarkdown } from "linkloom";

const markdown = await convertLinkToMarkdown("https://example.com");

That's it. One import, one call. The function auto-detects whether the URL points to an HTML page or a PDF and routes it to the right converter. You get back a string of clean markdown — no boilerplate, no configuration objects, no setup ceremony.

The CLI equivalent:

bunx @boris.barac/linkloom scrape https://example.com

Same result, different interface. Pipe it, redirect it, pass -o output.md to write to a file.

But plenty of pages don't hand you their content on the first request. They render everything with JavaScript — SPAs, dashboards, dynamically loaded articles. A simple fetch returns an empty shell. LinkLoom handles this through headless browser rendering via Camoufox, a stealth Firefox build on Playwright that avoids bot detection.

import { renderers } from "linkloom";

const browser = await renderers.puppeterRendered.initialize();
const result = await renderers.puppeterRendered.renderPage(browser, url, {
  timeout: 15000,
  waitUntil: "networkidle",
  viewport: { width: 1920, height: 1080 },
  frames: { enabled: true, timeout: 5000 },
});
await browser.close();

The renderPage function loads the URL in a real browser, waits for the network to settle (or for a specific event), and returns the rendered HTML. The frames option tells it to also extract content from nested iframes — with its own timeout, because iframes load on their own schedule and you don't want one slow frame to block everything.

The CLI version:

bunx @boris.barac/linkloom render https://example.com --wait-until networkidle --timeout 15000

Add --selector "table.stats" to extract only a specific element instead of the full page. Useful when you know exactly what you're after.

Then there are PDFs. Research papers, technical reports, product documentation — a surprising amount of the web's useful content lives in PDFs, not HTML pages. The same convertLinkToMarkdown call handles both, but you can also convert PDFs directly:

import { pdfConverter } from "linkloom";
import { readFile } from "node:fs/promises";

const buffer = await readFile("document.pdf");
const markdown = await pdfConverter.convertPdfToMarkdown(buffer);
const text = await pdfConverter.convertPdfToText(buffer);

Two output modes: convertPdfToMarkdown preserves structure (headings, lists, formatting), while convertPdfToText strips everything down to plain text. Pick whichever fits your pipeline.

The CLI:

bunx @boris.barac/linkloom pdf document.pdf -o output.md

Under the hood it uses pdf.js-extract to parse the binary, so there's no external dependency on system tools like pdftotext. It works out of the box.

Content conversion is half the job. The other half is pulling structured data out of pages — links, tables, the things that aren't prose.

Link extraction finds and classifies URLs from plain text or HTML. Feed it a string and it returns every link, tagged as a PDF or a regular page:

import { linkExtraction } from "linkloom";

const links = linkExtraction.extractLinks("check https://example.com/doc.pdf");
const pdfLinks = await linkExtraction.extractDownloadLinksFromHtml(htmlContent);

extractLinks works on raw text — it finds URLs and classifies them. extractDownloadLinksFromHtml parses an HTML document and pulls out links that point to downloadable files (PDFs, mostly). Useful when you're crawling a page and want to know which links lead to documents worth converting.

Table extraction renders a page in the headless browser and pulls out HTML tables as structured data:

import { tableExtraction, renderers } from "linkloom";

const browser = await renderers.puppeterRendered.initialize();
const data = await tableExtraction.extractTableData(browser, url, "table");
const md = tableExtraction.tableDataToMarkdownTable(data);
await browser.close();

The third argument is a CSS selector — pass "table" for all tables, or "table.stats" for a specific one. The output is a markdown table string, ready to drop into a document.

The CLI shortcuts:

bunx @boris.barac/linkloom links https://example.com
bunx @boris.barac/linkloom tables https://example.com/data --selector "table.stats"

All of this is also available as an MCP server. If you use Claude Desktop, Cursor, or any MCP-compatible client, you can expose LinkLoom's tools without writing code — the AI calls them directly.

Six tools: scrape, html_to_markdown, pdf_to_markdown, render_page, extract_links, extract_tables. Same capabilities as the library and CLI, but surfaced as tool calls an AI agent can use autonomously.

Configuration is a few lines of JSON. For Claude Desktop, edit ~/Library/Application Support/Claude/claude_desktop_config.json:

{
  "mcpServers": {
    "linkloom": {
      "command": "bun",
      "args": ["x", "@boris.barac/linkloom", "mcp"]
    }
  }
}

For Cursor, add the same block to .cursor/mcp.json in your project or ~/.cursor/mcp.json globally. Any MCP client — point it at bun x @boris.barac/linkloom mcp and it works.

The server communicates over stdio. It reads JSON-RPC from stdin and writes responses to stdout. You don't run it directly; MCP clients spawn it as a child process. If you want to test it interactively, there's the MCP Inspector:

bunx @modelcontextprotocol/inspector bunx @boris.barac/linkloom mcp

That opens a web UI where you can browse the available tools, call them with custom parameters, and inspect the JSON-RPC messages going back and forth.

Get started

bun add @boris.barac/linkloom

Or skip the install and use it directly:

bunx @boris.barac/linkloom scrape https://example.com

No API keys needed for the core scraping pipeline. Only the optional text embedding feature requires an OpenAI or Gemini key.

A Practical Guide to Local Lambda Debugging: Using SAM CLI with Terraform and VS Code

Boris Barac — Fri, 12 Jun 2026 16:21:04 +0000

A practical guide to debugging Lambda functions locally using SAM CLI and VS Code — no more deploying just to test a print() statement.

What You'll Learn

How to invoke a Lambda function locally from the command line
How to spin up a local API that mirrors your API Gateway
How to attach a debugger in VS Code and step through your code
How to hit a deployed Lambda from your machine

The Project

Github repo

This repo deploys a simple serverless stack:

Resource	Purpose
Lambda Function	Python function that writes a timestamp to S3
S3 Bucket	Stores timestamp files
API Gateway (HTTP)	Exposes the Lambda at `GET /timestamp`

All infrastructure is defined in Terraform. The Lambda source lives in lambda_function/lambda_function.py:

import time
import boto3
import os


def lambda_handler(event, context):
    """
    AWS Lambda function that stores current timestamp in S3.
    """
    current_timestamp = int(time.time())

    s3 = boto3.client("s3")
    bucket_name = os.environ["S3_BUCKET_NAME"]
    file_name = f"timestamp-{current_timestamp}.txt"

    s3.put_object(Bucket=bucket_name, Key=file_name, Body=str(current_timestamp))

    return {
        "statusCode": 200,
        "body": {
            "message": "Timestamp saved successfully",
            "timestamp": current_timestamp,
            "file": file_name,
        },
    }

Prerequisites

Make sure these are installed and working before you start: Terraform, AWS SAM CLI, Docker, Python, VSCode. Verify everything is in place:

terraform version
sam --version
docker ps
python3 --version

Note: docker ps should run without error. If Docker isn't running, SAM CLI will fail when trying to build or invoke locally.

Project File Map

Here's what matters for debugging:

.
├── main.tf                         # All Terraform infra (Lambda, S3, API Gateway, IAM)
├── lambda_function/
│   ├── lambda_function.py          # Your Lambda handler
│   └── requirements.txt            # Python dependencies
├── ev.json                         # Sample event payload for local invoke
├── .vscode/
│   └── launch.json                 # VS Code debug configurations
└── cli_helper.txt                  # Quick reference CLI commands

Method 1: Debugging with SAM CLI (Command Line)

SAM CLI can read your Terraform project directly using the --hook-name terraform flag. No SAM template needed.

Step 1 — Build

SAM needs to prepare a local build of your function (it resolves dependencies and packages the code):

sam build --hook-name terraform

You'll see output like:

Building codeuri: /Users/you/tf-sam-play-master/lambda_function runtime: python3.13
metadata: {} functions: my_lambda
Running PythonPipBuilder:ResolveDependencies
Running PythonPipBuilder:CopySource

Build Succeeded

Built Artifacts  : .aws-sam-iacs/build
Built Template   : .aws-sam-iacs/build/template.yaml

Tip: Re-run this every time you change your Python code or dependencies.

Step 2 — Invoke Locally with a Custom Event

The repo includes ev.json — a sample S3 event payload. You can use it to simulate an invocation:

sam local invoke --event ./ev.json --hook-name terraform --debug

--event ./ev.json — feeds the event payload to your handler
--hook-name terraform — tells SAM to read from Terraform, not a SAM template
--debug — prints verbose logs (Docker commands, environment, etc.)

Expected output:

{
  "statusCode": 200,
  "body": {
    "message": "Timestamp saved successfully",
    "timestamp": 1745232000,
    "file": "timestamp-1745232000.txt"
  }
}

Important: The s3 put_object call will fail locally unless you have real AWS credentials that can reach the S3 bucket. For offline-only testing, you'd mock boto3 (covered in the Troubleshooting section).

Step 3 — Run a Local API

If you want to test through the API Gateway path (not just direct invoke):

sam local start-api --hook-name terraform --debug

SAM starts a local HTTP server. You'll see:

Mounting my_lambda at http://127.0.0.1:3000/{path+}
You can now browse to the above endpoints to invoke your functions.
Running on http://127.0.0.1:3000 (Press CTRL+C to quit)

Now hit it from another terminal:

curl http://127.0.0.1:3000/timestamp

This is useful for verifying that the API Gateway integration is wired correctly and that event routing works.

Step 4 — Remote Invoke (Hit the Deployed Lambda)

Once you've deployed with terraform apply, you can invoke the live Lambda from your machine:

sam remote invoke my_lambda --stack-name terraform-2025 --event '{"test": "data"}'

This sends a real event to the deployed function in AWS. Useful for comparing local vs. remote behavior.

Method 2: Debugging in VS Code

Step 1 — Install the AWS Toolkit Extension

Open VS Code
Go to Extensions (Cmd+Shift+X)
Search for AWS Toolkit and install it

This extension understands SAM projects and provides the aws-sam debug type.

Step 2 — Understand the Launch Config

The repo ships with .vscode/launch.json containing three configurations:

{
    "version": "0.2.0",
    "configurations": [
        {
            "type": "aws-sam",
            "request": "direct-invoke",
            "name": "lambda_function:lambda_function.lambda_handler (python3.13)",
            "invokeTarget": {
                "target": "code",
                "projectRoot": "${workspaceFolder}/lambda_function",
                "lambdaHandler": "lambda_function.lambda_handler"
            },
            "lambda": {
                "runtime": "python3.13",
                "payload": {
                    "json": {
                        "fake": "event"
                    }
                },
                "environmentVariables": {}
            }
        },
        {
            "name": "Attach to SAM Local",
            "type": "debugpy",
            "request": "attach",
            "listen": {
                "host": "127.0.0.1",
                "port": 3000
            },
            "pathMappings": [
                {
                    "localRoot": "${workspaceFolder}/lambda_function/lambda_function",
                    "remoteRoot": "/timestamp "
                }
            ]
        },
        {
            "name": "Python: Current File",
            "type": "debugpy",
            "request": "launch",
            "program": "${file}",
            "console": "integratedTerminal",
            "justMyCode": true
        }
    ]
}

Here's what each one does:

Config Name	When to Use
Direct Invoke (`aws-sam`)	Quick one-click invoke. Builds and runs in Docker.
Attach to SAM Local (`debugpy`)	Start API from CLI, then attach debugger. Full breakpoints.
Python: Current File	Run the Python file directly (no Lambda context).

Step 3 — Set a Breakpoint and Debug

Option A — Direct Invoke (quickest)

Open lambda_function/lambda_function.py
Click the gutter to the left of line 11 (current_timestamp = ...) to set a breakpoint (red dot)
Press F5 or go to Run → Start Debugging
Select "lambda_function:lambda_function.lambda_handler (python3.13)"
VS Code builds a Docker container, invokes the handler, and pauses at your breakpoint

You can now inspect event, context, and all local variables in the Variables panel.

Option B — Attach to a Running Local API

This is the most realistic debugging setup because it mirrors the full request flow:

Terminal 1 — Start SAM's local API with the debug port open:

sam local start-api --hook-name terraform --debug-port 3000

VS Code — Attach the debugger:

Set your breakpoints in lambda_function.py
Press F5 and select "Attach to SAM Local"
The debugger connects and waits

Terminal 2 — Trigger a request:

curl http://127.0.0.1:3000/timestamp

VS Code pauses execution at your breakpoints. Step through the handler line by line.

Step 4 — Customize the Event Payload

In the Direct Invoke config, change the payload to match a real event:

"payload": {
    "json": {
        "queryStringParameters": {
            "key": "value"
        },
        "pathParameters": {},
        "body": null
    }
}

Or point it to a file:

"payload": {
    "jsonPath": "${workspaceFolder}/ev.json"
}

Working with Events

Generate Sample Events with SAM

Don't hand-write event payloads. SAM can generate them for you:

# Generate an S3 Put event
sam local generate-event s3 put --bucket my-bucket --key my-key

# Generate an API Gateway HTTP API event (proxy integration)
sam local generate-event apigateway http-api-proxy --method GET --path timestamp

# Save to a file
sam local generate-event s3 put > my-event.json

Then use it:

sam local invoke --event my-event.json --hook-name terraform --debug

5 AI Agents, 0 System Crashes: Secure Sandboxing for Free

Boris Barac — Fri, 12 Jun 2026 16:13:51 +0000

Everyone's hyped about running five agents at once, but hardly anyone talks about how to keep them secure without crashing your system. Daytona is a cool option, but it's paid. Here's a way to do it for free while leveling up your existing dev skills.

Docker Sandboxes can run AI agents or code inside isolated microVMs with their own Docker daemon, while mounting your local project into the sandbox at the same absolute path as on your host. That gives the agent real access to your code without exposing your host Docker environment directly.

Main points

Isolation: each sandbox has its own filesystem, network, and private Docker daemon.
Local machine link: your project is mounted directly, and host services are reachable from the sandbox via host.docker.internal.
Networking: outbound traffic is routed through host-controlled proxy/policy layers, and services inside the sandbox must be explicitly published to be reachable from your browser (easy to control and change).
Persistence: installed packages, images, and config changes stay until you remove the sandbox.
Customization: you can extend an agent base template, add tools like Bun, push the image to an OCI registry, and run the sandbox from that template.

Minimal workflow

Note this is not gonna pick up the global config but just the local one

brew install docker/tap/sbx
sbx login
cd ~/my-project
sbx run claude

Shell connect to Sandbox

# Agent sandbox
sbx exec -it <sandbox-name> bash

Run sandbox

sbx run shell

Run command in sandbox without connecting

sbx exec -it <sandbox-name> <your-command>

Template example (Bun)

Can not be used with local templates

You can also use the opencode keyword instead of Claude Code.

FROM docker/sandbox-templates:opencode

USER root
RUN apt-get update && apt-get install -y --no-install-recommends curl \
    && rm -rf /var/lib/apt/lists/*

USER agent
RUN curl -fsSL https://bun.sh/install | bash

ENV PATH="/home/agent/.bun/bin:${PATH}"

WORKDIR /app

RUN bun --version

# 1. Log in to Docker Hub (if you haven't already)
docker login docker.io

# 2. Build the image locally
docker build -t docker.io/my-org/my-bun-template:v1 .

# 3. Push the image to your registry
docker push docker.io/my-org/my-bun-template:v1

# 4. Run your sandbox environment
# add -name variable if you do not want to use the name of the folder
sbx run --template docker.io/my-org/my-bun-template:v1 claude

This is how to use a custom template. You can also just install stuff while in the sandbox — the sandbox has root access, and the changes are gonna stay in it.

More docs at: https://docs.docker.com/ai/sandboxes/

#AIAgents #Docker #CyberSecurity #SoftwareEngineering #DevOps #OpenSource #AIInfrastructure #LLMs #ClaudeCode #Sandboxing