DEV Community: Shrijith Venkatramana

Prefill-Decode Disaggregation: Why LLM Inference Is Starting to Look Like a Distributed System

Shrijith Venkatramana — Sun, 09 Aug 2026 18:11:41 +0000

Hello, I'm Shrijith Venkatramana. I'm building git-lrc, an AI code reviewer that runs on every commit. Star Us to help devs discover the project. Do give it a try and share your feedback for improving the product.

Prefill-Decode Disaggregation: Why LLM Inference Is Starting to Look Like a Distributed System

There is a strange thing about serving an LLM.

A user sends you 4,000 tokens and asks for 200 more.

The GPU spends one kind of computation processing those 4,000 tokens—and then spends the next 200 steps doing something fundamentally different.

Yet, in a conventional serving system, we often put both workloads on exactly the same GPUs, under the same scheduler, fighting for the same resources.

That is beginning to look increasingly unreasonable.

The idea of prefill-decode disaggregation is simple:

Run the part of inference that reads the prompt separately from the part that generates the answer.

This sounds like an obvious optimization once you see it. But getting it right involves KV caches, GPU memory, network bandwidth, batching, latency SLOs, scheduling, and some surprisingly interesting economics.

The idea was explored systematically in Microsoft's Splitwise work, published at ISCA 2024, and then pushed further for latency-sensitive serving by DistServe, published at OSDI 2024.

Let's build the idea from first principles.

1. An LLM request actually contains two very different jobs

Suppose you send this to a model:

Explain how TCP congestion control works.
Assume I already understand IP routing.
Give me a detailed explanation with examples.

Imagine that becomes 25 input tokens and the model generates 300 output tokens.

Inference looks roughly like this:

                INPUT
                  |
                  v
        +-----------------+
        |     PREFILL     |
        |                 |
        | Process all     |
        | input tokens    |
        +--------+--------+
                 |
                 | KV cache
                 v
        +-----------------+
        |      DECODE     |
        |                 |
        | token -> token  |
        | -> token -> ... |
        +--------+--------+
                 |
                 v
               OUTPUT

Prefill

The model processes the entire prompt.

If the prompt contains 4,000 tokens, the transformer can process those tokens largely in parallel.

This is a large, dense computation.

GPUs are extremely good at this.

Decode

Now the model generates the answer one token at a time:

token 1
   |
   v
token 2
   |
   v
token 3
   |
   v
token 4
   |
   v
...

There is an unavoidable autoregressive dependency:

P(token_n | token_1 ... token_(n-1))

You cannot generally generate token 400 before knowing token 399.

So decode consists of many relatively small GPU operations.

This creates an important asymmetry:

Prefill wants compute throughput. Decode wants memory bandwidth and predictable latency.

Microsoft researchers Pratyush Patel, Esha Choukse, Chaojie Zhang and colleagues characterized exactly this difference in their Splitwise work. They found prompt processing to be compute-intensive while token generation was much more memory-intensive and underutilized expensive accelerator resources.

This is the fundamental observation behind disaggregation.

2. The surprising part: the same GPU is a bad compromise

Imagine a restaurant with two kinds of customers.

One customer arrives with 100 people and wants to place a large order.

Another customer arrives alone and wants one sandwich every 30 seconds for the next ten minutes.

If you force both groups through the same kitchen workflow, optimizing for one will hurt the other.

LLM serving has a similar problem.

Consider a decode batch:

request A -> generate next token
request B -> generate next token
request C -> generate next token
request D -> generate next token
...

Batching is extremely useful here.

The GPU can reuse the model weights across many requests, so the cost of reading the weights gets amortized.

Now a new request arrives with a 20,000-token prompt.

Its prefill is a large compute-heavy operation.

If the scheduler inserts that prefill into the same GPU workload, it can temporarily consume enormous amounts of compute and memory bandwidth.

The users already generating tokens don't care that the new request has a giant prompt.

They just experience:

token
token
token
     <- giant prefill
...
token

Their inter-token latency spikes.

This is why TTFT and TPOT/ITL matter.

TTFT — Time To First Token: how long the user waits before seeing anything.
ITL — Inter-Token Latency: how long the user waits between generated tokens.
TPOT — Time Per Output Token: closely related to ITL in serving measurements.

A system can have excellent average throughput while feeling terrible interactively.

DistServe's central argument was that prefill and decode interfere with one another strongly enough that treating them as one homogeneous workload makes it difficult to simultaneously satisfy TTFT and TPOT requirements.

3. What is actually being "disaggregated"?

Here is where the implementation gets interesting.

You don't split the neural network into:

GPU A:
  layers 1-40

GPU B:
  layers 41-80

That's ordinary model parallelism.

Instead, you run essentially the same model in two different pools:

                  Request
                     |
                     v
            +----------------+
            | Prefill Pool   |
            |                |
            | GPU GPU GPU    |
            +-------+--------+
                    |
                    | KV cache
                    |
                    v
            +----------------+
            |  Decode Pool   |
            |                |
            | GPU GPU GPU    |
            +-------+--------+
                    |
                    v
                 Tokens

The prefill workers process the prompt.

They produce the intermediate state needed by the decoder: primarily the key/value (KV) cache.

That cache is transferred to a decode worker.

The decode worker then continues generation.

So the architecture becomes:

             network / interconnect
        ----------------------------->

PREFILL GPU                         DECODE GPU
----------                         ----------
prompt
  |
  v
transformer
  |
  v
KV cache ------------------------> KV cache
                                       |
                                       v
                                   generate
                                   token 1
                                     |
                                   token 2
                                     |
                                   token 3

This is why the technique is much more interesting than simply running two queues.

The KV cache becomes a distributed-system object.

And that changes everything.

4. The KV cache is the thing holding the whole system together

During attention, the model needs information about previous tokens.

Instead of recomputing the entire history for every generated token, serving systems store the relevant keys and values in the KV cache.

A simplified picture is:

Prompt:
A B C D E

KV cache:
K(A) V(A)
K(B) V(B)
K(C) V(C)
K(D) V(D)
K(E) V(E)

When generating F, the model can attend to this existing state.

Then it adds:

K(F) V(F)

and continues.

This cache can become enormous.

A rough formula for KV-cache memory is:

M_KV ~= 2 * L * H_KV * D * T * B

where:

L     = number of transformer layers
H_KV  = number of KV heads
D     = head dimension
T     = number of tokens
B     = bytes per element

The factor 2 represents keys + values.

Consider a model with:

80 layers
8 KV heads
128-dimensional heads
BF16 KV cache = 2 bytes
4,000 tokens

Then:

M_KV
~= 2 * 80 * 8 * 128 * 4000 * 2
~= 1.31 billion bytes

or roughly 1.2 GiB per request.

At 32K tokens, the same request is roughly:

9.8 GiB of KV cache.

Now the distributed-systems problem becomes obvious.

Suppose you have to move 1.2 GiB from a prefill GPU to a decode GPU.

With a theoretical 400 Gb/s interconnect:

400 Gb/s = 50 GB/s

so the absolute bandwidth floor is approximately:

1.2 GB / 50 GB/s
~= 24 ms

That isn't necessarily the actual latency—you have protocol overhead, topology, serialization, synchronization, GPU copies, contention, and so on.

But it gives you the right intuition:

Disaggregation replaces GPU interference with a network-transfer problem.

That trade is worthwhile only when the interference you eliminate is more expensive than the communication you introduce.

This is why DistServe explicitly considers cluster bandwidth and KV-cache transfer when deciding where to place prefill and decode workers.

5. The math explains why decode is such a strange workload

Here's a useful back-of-the-envelope calculation.

Suppose you have a 70B-parameter model running in BF16.

The weights alone occupy roughly:

70B parameters * 2 bytes
~= 140 GB

A rough rule for transformer inference is around:

2 * parameters

FLOPs per token for the dense model.

So:

70B * 2
~= 140 GFLOPs/token

For a 4,000-token prompt, that gives approximately:

140 GFLOPs/token * 4000 tokens
= 560 TFLOPs

of model computation, ignoring architectural details and attention-specific terms.

That is a substantial matrix-multiplication workload.

Now consider decoding one token.

It's only around:

140 GFLOPs

Yet the model still needs to use its enormous parameter set.

This is why decode benefits enormously from batching.

Imagine 32 requests decoding simultaneously.

The same model weights can participate in the computation for 32 tokens:

                    Model weights
                         |
        +----------------+----------------+
        v                v                v
      req A            req B            req C ...

Instead of paying the weight-access cost independently for each request.

This is one reason a decode GPU can look strangely underutilized in conventional compute-utilization metrics while still being the bottleneck for latency.

It isn't that the GPU has "nothing to do."

It is that its workload has a very different arithmetic intensity from prefill.

This distinction is central to Splitwise's economic argument: the latest, most expensive GPU is extraordinarily valuable for compute-heavy prefill, but its additional compute capability is much less valuable for decode.

6. Now the economics become interesting

Suppose you have two kinds of machines:

                    Prefill             Decode
                    -------             ------
Compute             extremely useful   less important
Memory bandwidth    useful             extremely important
Power               high               lower preferred
Cost                high               lower preferred

A conventional deployment might look like:

GPU GPU GPU GPU
|   |   |   |
+---+---+---+
    everything

Every GPU has to support both workloads.

But with disaggregation:

             PREFILL                  DECODE

          H100 H100 H100          A100 A100 A100
             |                       |
             +------- network -------+

you can independently scale the two pools.

Suppose your workload suddenly changes:

Before:
10,000 short prompts
500 output tokens each

After:
2,000 enormous prompts
500 output tokens each

The prefill workload has exploded.

You may want:

Prefill:
3 GPUs -> 8 GPUs

Decode:
4 GPUs -> 4 GPUs

With a monolithic serving architecture, you're effectively scaling both together.

With disaggregation, they become separate capacity-planning problems.

This is exactly the kind of optimization Splitwise explored.

The authors reported up to 1.4x higher throughput at 20% lower cost, or 2.35x more throughput under the same cost and power budgets, depending on the cluster configuration.

Those numbers are important because they demonstrate that this isn't merely a latency trick.

It can become a datacenter economics optimization.

7. But you don't always need disaggregation

This is perhaps the most important practical point.

If you're running:

7B model
8 concurrent users
2K context
single GPU

you probably shouldn't build a distributed prefill/decode architecture.

The complexity is ridiculous relative to the workload.

There are intermediate techniques.

One particularly important one is chunked prefill.

Instead of allowing a 20,000-token prompt to monopolize the GPU:

20K-token prefill
████████████████████████

you split it:

chunk 1
████

chunk 2
████

chunk 3
████

and schedule decode work between those chunks.

Some systems explored this approach, combining chunked prefills with decode-heavy batching. On their tested workloads, they reported substantial improvements, including up to 10x decode throughput for LLaMA-13B on an A6000 and 1.33x end-to-end throughput.

So there is a spectrum:

Simple
  |
  +-- Continuous batching
  |
  +-- Chunked prefill
  |
  +-- Better scheduling
  |
  +-- Prefill/decode disaggregation
       |
       v
Complex

The last step makes sense when the scale and latency requirements justify it.

Current vLLM documentation reflects exactly this distinction: its disaggregated-prefill implementation runs separate prefill and decode instances and transfers KV cache between them, while explicitly noting that disaggregation by itself does not necessarily improve raw throughput. Its major benefit is giving operators independent control over TTFT and inter-token latency and avoiding prefill-induced latency spikes.

That qualification matters.

Disaggregation isn't magic.

It gives you a better set of knobs.

Whether those knobs improve your system depends on your workload.

8. The deeper lesson: inference is becoming an operating-systems problem

The interesting thing about this research is that the model itself isn't changing.

The transformer is still:

attention
  |
MLP
  |
attention
  |
MLP
  |
...

What is changing is everything around it.

Once models become sufficiently expensive, inference starts looking less like:

"Run this neural network on a GPU."

and more like:

"Schedule heterogeneous computational phases across a distributed hardware system while managing a huge state object under latency SLOs."

That sounds remarkably like distributed systems.

You now have:

queues
admission control
scheduling
cache placement
memory management
network topology
batching
backpressure
load balancing
tail latency
capacity planning
heterogeneous hardware
cost optimization

And the KV cache is effectively distributed state.

Splitwise asked: why not put the phases on different machines?

DistServe went further and treated TTFT and TPOT as separate SLOs, optimizing resource allocation and parallelism for each phase.

And the idea has since moved into real inference software: vLLM now exposes experimental disaggregated-prefill machinery based around separate prefill/decode instances and KV-cache transfer.

The trajectory is revealing.

We started by optimizing the neural network.

Then we optimized the GPU kernels.

Then batching and memory management.

Now we're increasingly optimizing the entire inference system.

That is probably the more important shift.

Conclusion: Don't think of an LLM request as one computation

The useful mental model is no longer:

request
   |
   v
LLM
   |
   v
response

Think:

                    REQUEST
                       |
                       v
               +--------------+
               |    PREFILL   |
               |              |
               | compute-heavy|
               +------+-------+
                      |
                  KV CACHE
                      |
               -------+-------
                  network
               -------+-------
                      |
               +------+-------+
               |    DECODE    |
               |              |
               | memory-heavy |
               | autoregressive|
               +------+-------+
                      |
                      v
                   RESPONSE

Once you see those as two different workloads, a lot of otherwise strange behavior in LLM serving starts making sense.

Why does a long prompt suddenly destroy token latency?

Why does GPU utilization fail to tell you whether your inference system is healthy?

Why can adding GPUs fail to improve interactive latency?

Why might a cheaper GPU be perfectly adequate for part of an LLM workload?

Why does network bandwidth suddenly become an inference bottleneck?

And why are systems researchers talking about LLM serving in terms of queues, SLOs, cache placement and goodput rather than simply FLOPS?

Because the neural network is only one component of the system anymore.

The interesting question for developers is:

As LLM inference gets increasingly disaggregated, which other parts of the inference stack do you think will become independently schedulable next?

*AI agents write code fast. They also silently remove logic, change behavior, and introduce bugs -- without telling you. You often find out in production.

git-lrc fixes this. It hooks into git commit and reviews every diff before it lands. 60-second setup. Completely free.*

Any feedback or contributors are welcome! It's online, source-available, and ready for anyone to use.

HexmosTech / git-lrc

Free, Micro AI Code Reviews That Run on Git Commit

git-lrc

Free, Micro AI Code Reviews That Run on Commit

GenAI today is a race car without brakes. It accelerates fast -- you describe something, and large blocks of code appear instantly. But AI agents silently break things: they remove logic, relax constraints, introduce expensive cloud calls, leak credentials, and change behavior -- without telling you. You often find out in production.

git-lrc is your braking system. It hooks into git commit and runs an AI review on every diff before it lands. 60-second setup. Completely free.

In short, git-lrc helps Prevent Outages, Breaches, and Technical Debt Before They Happen

At a glance: 10 risk categories · 100+ failure patterns tracked · every commit…

View on GitHub

How FSRS Replaces 30-Year-Old Algorithms in Modern Memory Software

Shrijith Venkatramana — Thu, 06 Aug 2026 19:34:22 +0000

Why your spaced repetition app needs a scheduler that learns

Picture this: You are building a language learning app. Your users review flashcards daily. Some cards they remember easily. Others they forget repeatedly. Your scheduler must decide when to show each card again.

For decades, the industry standard was SM-2 — the algorithm behind SuperMemo and Anki. SM-2 uses a single number (an "ease factor") to adjust intervals. It works. But it does not learn from user behavior.

Then came Jarrett Ye. Working on MaiMemo, a language learning platform in China, Ye and his team asked a fundamental question: Can we build a scheduler that actually models how human memory works?

The result was FSRS — the Free Spaced Repetition Scheduler. It is now the default scheduler in Anki as of version 23.10. It is open source. And it is backed by serious research: two peer-reviewed papers, one at ACM SIGKDD 2022 and one in IEEE Transactions on Knowledge and Data Engineering.

This article explains FSRS from the ground up. We start with intuition. We go deeper into the math. We show code. By the end, you will know enough to implement FSRS in your own application.

1. The Problem with One-Number Scheduling

SM-2 uses a single parameter — the "ease factor" — to adjust intervals for all cards. If you rate a card "easy", the ease factor increases. If you rate it "hard", the ease factor decreases.

This works for simple cases. But memory is not simple.

Consider two cards you review today:

Card A: "Paris is the capital of France" — you know this well.
Card B: "The chemical symbol for Tungsten is W" — you always confuse this.

Both cards might have the same ease factor under SM-2. But they are not the same. One is easy. One is hard. One should have longer intervals. The other should have shorter intervals.

SM-2 cannot distinguish them. It uses one number for all cards.

FSRS solves this by using three numbers per card.

2. The DSR Model: Three Numbers Instead of One

FSRS models memory using three variables:

Difficulty (D) — How hard is this card? Range: 1.0 (easiest) to 10.0 (hardest). A card about "Paris" gets low difficulty. A card about "Tungsten" gets higher difficulty.

Stability (S) — How strong is the memory? Definition: the number of days for recall probability to drop from 100% to 90%. If S = 30 days, you have a 90% chance of recalling it after 30 days.

Retrievability (R) — The probability you will recall it right now. Range: 0 to 1. This changes daily as time passes.

Here is the key insight: Stability and Difficulty are properties of the card. Retrievability is a property of the card and the time since your last review.

When you review a card, FSRS updates D and S based on your rating. Between reviews, R decays over time.

The scheduler's job is simple: show the card when R drops to your target retention (usually 90%).

3. The Forgetting Curve: Why Power Beats Exponential

Most spaced repetition systems use an exponential forgetting curve:

R(t) = 0.9^(t/S)

FSRS uses a power-law forgetting curve:

R(t, S) = (1 + F * (t/S))^(-0.5)

Where F = 19/81 ~ 0.2346.

Why a power law? The research shows that a power function fits human memory data better than an exponential function.

Here is the intuition: Human memory is not a single process. It is a superposition of many memory traces with different decay rates. A power law emerges naturally when you average many exponential decays.

The practical difference: At long intervals, a power law predicts more forgetting than an exponential curve. This matches real data — we forget more over long periods than exponential models predict.

At short intervals (under 10 days), the two curves are nearly identical. The difference shows up over weeks and months.

For developers: this means FSRS gives longer intervals for well-known cards and shorter intervals for hard cards — with better accuracy than exponential models.

4. State Updates: The Math Behind Each Review

When a user rates a card, FSRS updates the memory state. Here is how it works.

Initial Review (New Card)

For a new card, FSRS sets initial stability based on the user's rating:

Rating	Initial Stability
Again (1)	0.4 days
Hard (2)	0.6 days
Good (3)	2.4 days
Easy (4)	5.8 days

These values come from default parameters — trained on 738 million reviews from 20,000 users.

Initial difficulty is calculated as:

D_0(G) = w_4 - e^(w_5 * (G-1)) + 1

Clamped to [1, 10]. Higher ratings yield lower difficulty.

Successful Review (Grade >= 2)

When you remember a card, difficulty updates first:

D_next = D_prev - w_6 * (G - 3)
D_new = w_7 * D_0(3) + (1 - w_7) * D_next

Rating > 3 -> difficulty decreases (card gets easier)
Rating < 3 -> difficulty increases (card gets harder)
Mean reversion (w_7) prevents extreme values

Then stability updates:

GrowthFactor = e^(w_8) * (11 - D) * S^(-w_9) * (e^(w_10*(1-R)) - 1) * h * b

Where:

h = w_15 if Grade = 2 (Hard penalty), else 1
b = w_16 if Grade = 4 (Easy bonus), else 1

The new stability is:

S_new = S_prev * (1 + GrowthFactor)

This looks complex. But the intuition is simple: The growth factor depends on difficulty, current stability, retrievability at review time, and the user's rating.

Higher difficulty -> smaller growth (hard cards improve more slowly)
Higher current stability -> smaller growth (already stable cards improve more slowly)
Lower retrievability -> larger growth (if you almost forgot it, reviewing it strengthens memory more)
"Easy" rating -> bonus growth
"Hard" rating -> penalty

Failed Review (Grade = 1)

When you forget a card, stability decreases:

S_new = w_17 * D^(-w_18) * (S_prev + 1)^(-w_19)

Difficulty increases:

D_new = D_prev + w_11

Clamped to [1, 10].

5. Interval Calculation: When to Show the Card

Once we have stability S and target retention R_req, we calculate the interval:

I(S, R_req) = S * (81/19) * (R_req^(-2) - 1)

For R_req = 0.9 (90% target):

I = S * (81/19) * (1/0.81 - 1) = S * (81/19) * (19/81) = S

At 90% retention, the optimal interval equals stability.

For other retention targets:

R_req = 0.85 -> I ~ 0.75 * S (shorter interval, higher retention)
R_req = 0.95 -> I ~ 1.38 * S (longer interval, lower retention)

This trade-off is the core economics of spaced repetition: Higher retention means more reviews. Lower retention means fewer reviews but more forgetting.

6. The Optimizer: Learning from Your Data

The scheduler uses 19 parameters (w_0 through w_19). Default parameters come from 738 million reviews across 20,000 users.

But FSRS can do better: it can learn your personal memory patterns from your review history.

The optimizer uses two techniques:

Maximum Likelihood Estimation (MLE) — Find parameters that make your observed review outcomes most probable
Backpropagation Through Time (BPTT) — Train the model on time-series review data

The optimizer takes your review logs — each with timestamp, card ID, and rating — and finds the parameters that best fit your data.

In practice, optimized FSRS beats default FSRS, which beats SM-2:

Default FSRS better than SM-2 for 92% of users
Optimized FSRS better than SM-2 for 99% of users

For developers: The FSRS Optimizer is available as a Python library. You can feed it review logs and get optimized parameters. The optimizer runs on the client side (in the browser via WebAssembly) or on a server.

7. Implementation: Code Examples

Python

from fsrs import Scheduler, Card, Rating

# Initialize with default parameters
scheduler = Scheduler()

# Create a new card
card = Card()

# Simulate a review with "Good" (rating 3)
review_log = scheduler.review_card(card, Rating.Good)

# Get the next review date
next_interval_days = review_log.scheduled_days

print(f"Review again in {next_interval_days} days")

Py-FSRS is available on PyPI.

TypeScript

import { createEmptyCard, fsrs, Rating } from 'ts-fsrs';

const card = createEmptyCard();
const scheduler = fsrs();

const { card: updatedCard, reviewLog } = scheduler.review(card, Rating.Good);

console.log(`Review again in ${updatedCard.due} days`);

ts-fsrs is available on npm.

Rust

use fsrs::{FSRS, Card, Rating};

let mut fsrs = FSRS::default();
let mut card = Card::default();

let (new_card, log) = fsrs.review(card, Rating::Good)?;

println!("Review again in {} days", new_card.due);

FSRS-rs provides a Rust implementation with full training support.

Optimizing Parameters

from fsrs_optimizer import optimize_parameters
from fsrs import FSRSItem

# Load your review logs
items = load_review_logs()  # List of FSRSItem

# Optimize parameters for your data
optimized_params = optimize_parameters(items)

# Use optimized parameters
scheduler = FSRS(parameters=optimized_params)

The FSRS Optimizer package handles this workflow.

9. Developer Considerations

Data Requirements

FSRS works from day one with default parameters. But optimization requires data:

At least 100 reviews for basic optimization
1,000+ reviews for reliable personalization

Implementation Options

FSRS has implementations in:

Python (py-fsrs)
TypeScript (ts-fsrs)
Rust (fsrs-rs)
Dart
PHP
C# (.NET)

Migration from SM-2

If you have existing users with SM-2 data, you can migrate. FSRS can estimate initial D and S from review history. Anki handles this automatically when you enable FSRS.

Free Scheduling

The "Free" in FSRS means you can review cards early or late. FSRS adapts. If you review a card late and still remember it, stability increases more than if you reviewed it on time. If you review early, the algorithm accounts for that too.

10. Conclusion

FSRS represents a fundamental improvement over traditional spaced repetition algorithms. It models memory with three variables instead of one. It uses a power-law forgetting curve that fits human memory data better. It learns from each user's review history.

The research is solid: two peer-reviewed papers, one at ACM SIGKDD 2022 and one in IEEE TKDE 2023. The data is massive: 738 million reviews from 20,000 users. The results are clear: FSRS beats SM-2 for 92% of users with default parameters, and for 99% with optimized parameters.

For developers, FSRS is ready to use. Pick an implementation. Import the library. Replace your scheduler. Your users will spend less time reviewing and remember more.

Now your turn: Have you implemented a spaced repetition system? What algorithm did you use? Share your experience in the comments — including the data size, the user base, and what you learned.

*AI agents write code fast. They also silently remove logic, change behavior, and introduce bugs -- without telling you. You often find out in production.

git-lrc fixes this. It hooks into git commit and reviews every diff before it lands. 60-second setup. Completely free.*

Any feedback or contributors are welcome! It's online, source-available, and ready for anyone to use.

HexmosTech / git-lrc

Free, Micro AI Code Reviews That Run on Git Commit

git-lrc

Free, Micro AI Code Reviews That Run on Commit

git-lrc is your braking system. It hooks into git commit and runs an AI review on every diff before it lands. 60-second setup. Completely free.

In short, git-lrc helps Prevent Outages, Breaches, and Technical Debt Before They Happen

At a glance: 10 risk categories · 100+ failure patterns tracked · every commit…

View on GitHub

SGLang Explained: Why LLM Inference Needed Its Own Programming Language

Shrijith Venkatramana — Wed, 05 Aug 2026 19:29:06 +0000

Every LLM demo looks fast.

A user sends one prompt. The model replies. Everybody is happy.

Production is a different world.

One GPU serves hundreds of users. Many prompts start with the same system prompt. Some users generate 20 tokens. Others generate 20,000. Agents branch into multiple reasoning paths. Memory fills up. GPU utilization falls. Costs rise.

This is the problem that SGLang tries to solve.

It is easy to think of SGLang as "another inference framework." That misses the main idea.

SGLang is really a co-design of a programming language and a runtime. The language describes how an LLM application behaves. The runtime uses that information to execute the application much more efficiently.

Let's see why that matters.

The Story Behind SGLang

SGLang came from researchers at UC Berkeley, Stanford, and the LMSYS project. Several of the authors, including Lianmin Zheng, Ion Stoica, Joseph Gonzalez, and Christos Kozyrakis, have worked on distributed systems, machine learning infrastructure, and high-performance computing for many years. ([Stanford MAST Lab][1])

Their observation was simple.

Large language models no longer answer only one prompt.

Modern applications perform:

multiple LLM calls
branching logic
loops
retrieval
tool calls
structured outputs

A simple customer support agent may execute ten or more model generations before it produces a final answer.

Traditional inference engines treat every generation almost independently.

SGLang treats the entire workflow as one program.

That small shift changes everything.

The High-Level Intuition: Think Like a CPU Compiler

Suppose a C compiler sees this:

for (int i = 0; i < 1000; i++) {
    sum += a[i];
}

The compiler does not execute one instruction at a time.

It optimizes the whole loop.

It removes unnecessary work.

It improves memory access.

It uses vector instructions.

The programmer still writes simple code.

The compiler makes it fast.

SGLang applies the same idea to LLM applications.

Instead of optimizing machine instructions, it optimizes language generation programs.

The runtime already knows:

which prompts share prefixes
which branches reuse earlier work
where generations start and stop
which outputs have fixed structure

That extra information lets the runtime remove a large amount of repeated computation.

The Economics: Why Recomputing Tokens Is Expensive

Every generated token depends on every earlier token.

That means the model creates a growing KV cache.

For a long conversation, this cache becomes much larger than the generated text itself.

Imagine 1,000 users.

Each request starts with exactly the same system prompt.

System prompt
↓

"You are an expert software architect..."

Suppose the prompt contains 4,000 tokens.

Without reuse:

1000 users × 4000 tokens

=

4 million prompt tokens

The GPU performs the same computation one thousand times.

That is wasted work.

If the shared prefix is computed once and reused, almost all of that repeated computation disappears.

This is exactly the kind of workload SGLang targets.

As context windows continue to grow into hundreds of thousands or even millions of tokens, prefix reuse becomes even more valuable because the expensive part of inference moves from generation to prompt processing. ([Stanford MAST Lab][1])

RadixAttention: The Main Technical Idea

The key innovation inside SGLang is called RadixAttention.

The name comes from the radix tree, also called a prefix tree.

A radix tree stores strings by sharing common prefixes.

For example:

apple
application
apply

All three words share:

appl

Instead of storing the prefix three times, the tree stores it once.

SGLang applies the same idea to KV caches.

Many prompts begin with identical text:

System Prompt
↓

Few-shot examples
↓

User Question

Only the user question changes.

Instead of creating three independent KV caches:

Prompt A
Prompt B
Prompt C

the runtime stores one shared prefix and lets all requests reference it.

The result is:

less GPU memory
fewer repeated computations
higher throughput
lower latency

This is why RadixAttention becomes especially useful for:

chat applications
coding assistants
retrieval systems
agent frameworks
few-shot prompting

Programming LLM Applications Instead of Chaining API Calls

Many LLM applications today look like this:

response1 = llm(...)
response2 = llm(...)
response3 = llm(...)

The application logic lives in Python.

The inference engine sees only isolated requests.

SGLang introduces a higher-level way to describe generation.

A program can contain:

variables
loops
branches
structured generation
tool calls

Conceptually:

Generate summary

If confidence is low:
    Search documents
    Generate again

Return JSON

Instead of many disconnected API calls, the runtime sees one complete generation program.

That visibility lets it schedule requests better, reuse cached prefixes, batch work more effectively, and avoid unnecessary memory movement.

The result resembles the evolution from assembly language to modern compilers.

Developers describe intent.

The runtime performs optimization.

Why This Matters for the Next Generation of AI Systems

The first generation of LLM software focused on model quality.

The second generation focuses on systems engineering.

A frontier model may cost tens of thousands of dollars per GPU.

Large deployments may run thousands of GPUs continuously.

Even a small increase in utilization can save millions of dollars each year.

This is why inference has become a major research area.

Recent work includes:

continuous batching
speculative decoding
paged attention
prefix caching
quantization
expert parallelism

SGLang combines many of these ideas into one production runtime while also providing a programming model that exposes more optimization opportunities than a simple HTTP API. Today it powers production deployments across hundreds of thousands of GPUs and generates trillions of tokens each day. ([SGLang Documentation][2])

Final Thoughts

For many years, faster software came from better compilers.

SGLang applies the same philosophy to LLM applications.

Instead of treating every prompt as an isolated request, it treats the entire workflow as a program that can be analyzed and optimized.

That change sounds small.

In large production systems, it changes memory use, throughput, latency, and ultimately cost.

As AI systems become more agentic and long-context models become common, this style of runtime optimization will likely become as important as model quality itself.

Question for developers: Do you think the future of LLM infrastructure belongs to general-purpose serving engines, or will specialized systems like SGLang become the standard for production AI?

*AI agents write code fast. They also silently remove logic, change behavior, and introduce bugs -- without telling you. You often find out in production.

git-lrc fixes this. It hooks into git commit and reviews every diff before it lands. 60-second setup. Completely free.*

Any feedback or contributors are welcome! It's online, source-available, and ready for anyone to use.

HexmosTech / git-lrc

Free, Micro AI Code Reviews That Run on Git Commit

git-lrc

Free, Micro AI Code Reviews That Run on Commit

git-lrc is your braking system. It hooks into git commit and runs an AI review on every diff before it lands. 60-second setup. Completely free.

In short, git-lrc helps Prevent Outages, Breaches, and Technical Debt Before They Happen

At a glance: 10 risk categories · 100+ failure patterns tracked · every commit…

View on GitHub

vLLM Explained: Ideas That Made LLM Serving Practical

Shrijith Venkatramana — Tue, 04 Aug 2026 19:49:06 +0000

Many AI demonstrations look impressive.

One developer sends one prompt. One GPU generates one answer.

A production system is different.

One GPU serves hundreds of users at the same time. One user sends a short question. Another user uploads a 50-page PDF. One request finishes in 100 ms. Another request generates thousands of tokens. GPU memory fills. Latency increases. Throughput decreases.

In 2023, researchers at UC Berkeley studied this problem. The researchers found that the main bottleneck was no longer matrix multiplication.

The bottleneck was memory management.

The researchers built vLLM to solve this problem. Today, vLLM is one of the most widely used open-source systems for LLM serving.

The key idea did not come from machine learning.

The key idea came from operating systems.

1. Running One Model Is Easy. Serving Many Users Is Hard.

Many developers first use an LLM with a small Python program.

from transformers import AutoModelForCausalLM

output = model.generate(...)

This program works well during development.

A production service has different requirements.

Consider a customer support chatbot.

User A sends 20 tokens.
User B sends 5,000 tokens.
User C receives streamed output for 30 seconds.
User D cancels the request before completion.

All users share one GPU.

A modern GPU is expensive. An NVIDIA H100 can cost tens of thousands of dollars. Cloud providers also charge several dollars per hour for one GPU.

Suppose one GPU has only 20% utilization.

Then about 80% of the GPU investment produces no useful work.

The engineering question changes.

Instead of asking,

Can the model generate the correct answer?

the engineer asks,

Can the system keep the GPU busy every millisecond?

vLLM focuses on this second question.

2. The Hidden Memory Cost

Many developers think that model weights use most GPU memory.

During inference, another data structure also uses a large amount of memory.

This data structure is the KV cache.

Each generated token creates key tensors and value tensors.

Future attention operations reuse these tensors.

Without the KV cache, the model must process the complete prompt again for every generated token.

The KV cache removes this repeated work.

Consider a book.

If the writer has no notes, the writer must read the complete manuscript before writing the next paragraph.

The KV cache acts like these notes.

The KV cache grows during generation.

Suppose one request contains:

8,000 prompt tokens
1,000 generated tokens

The KV cache stores information for about 9,000 tokens.

Now multiply this memory by hundreds of active requests.

The available GPU memory decreases quickly.

Each request also grows at a different rate.

Some conversations finish after 50 tokens.

Some conversations continue for thousands of tokens.

Memory becomes fragmented.

This fragmentation became one of the main limits for high-throughput LLM serving.

3. The Operating System Idea

The Berkeley team saw a familiar problem.

The KV cache behaves like computer memory.

A process starts.

A process grows.

A process ends.

Operating systems solved this problem many years ago.

A modern operating system does not require one continuous block of RAM for every process.

Instead, the operating system divides memory into fixed-size pages.

Virtual memory maps logical addresses to physical pages.

An application does not know the physical location of each page.

The vLLM team applied the same idea to the KV cache.

This design became PagedAttention.

The KV cache is divided into many small blocks.

The blocks can exist anywhere in GPU memory.

A lookup table records the physical location of every block.

Traditional

Request
-------------------------
| Continuous KV Cache   |
-------------------------

PagedAttention

Logical Cache

Block 1
Block 2
Block 3
Block 4

↓

GPU Memory

Page 7
Page 2
Page 19
Page 5

The model sees one logical KV cache.

The GPU stores many separate physical blocks.

This design is the same general idea that operating systems use for virtual memory.

4. Continuous Batching

PagedAttention solves one problem.

Scheduling solves another problem.

Traditional batching works like a bus.

The bus waits for all passengers.

Then the bus starts.

LLM inference originally behaved in a similar way.

The system created one batch.

The system waited.

The system completed the batch.

Then the system created the next batch.

Continuous batching behaves more like an elevator.

When one request finishes, a new request immediately enters the batch.

The GPU does not wait for the remaining requests.

Time →

Request A ███████████████

Request B █████

Request C        ███████████

Request B finishes first.

The scheduler immediately inserts another request.

GPU utilization stays high.

Latency decreases.

Throughput increases.

This scheduling method is one reason why vLLM achieves much higher throughput than earlier serving systems.

5. Small Performance Gains Have Large Financial Value

Suppose one GPU serves 80 requests every second.

An improved serving system increases throughput to 120 requests every second.

This change increases throughput by 50%.

Now consider a service that processes 12 million requests every day.

Before optimization:

120 GPUs

After optimization:

80 GPUs

The service removes 40 GPUs.

Suppose one GPU costs $3 per hour.

40 × $3 × 24

≈ $2,880 each day

≈ $1 million each year

A small increase in utilization can produce a large reduction in infrastructure cost.

For many AI companies, inference now costs more than model training.

6. Why vLLM Matters

vLLM was developed by researchers including Woosuk Kwon, Zhuohan Li, Siyuan Zhuang, Lianmin Zheng, Joseph Gonzalez, Hao Zhang, and Ion Stoica at UC Berkeley.

The work grew from the LMSYS research group, which also created Chatbot Arena and Vicuna.

The researchers operated public LLM services.

The researchers saw that model quality improved quickly, but serving systems improved more slowly.

The team presented the paper Efficient Memory Management for Large Language Model Serving with PagedAttention at SOSP 2023, one of the leading conferences in computer systems.

This detail is important.

The paper is a systems paper.

The paper does not change the transformer architecture.

The paper changes resource management.

Today, many LLM serving systems use similar ideas.

7. The Bigger Lesson

Many people expect the next breakthrough to come from a larger model.

vLLM shows another path.

The transformer stayed the same.

The model weights stayed the same.

The prompts stayed the same.

The improvement came from better memory management, better scheduling, and better resource use.

This pattern appears many times in computer science.

Hardware improves.

Algorithms improve.

Then systems engineering becomes the next bottleneck.

LLM serving has reached this stage.

Future progress will depend on both better models and better systems.

What do you think?

Most discussion about AI focuses on larger models, better prompts, and fine-tuning.

Will the next 10× improvement come from better models, or will it come from better systems such as memory management, scheduling, caching, and GPU utilization?

*AI agents write code fast. They also silently remove logic, change behavior, and introduce bugs -- without telling you. You often find out in production.

git-lrc fixes this. It hooks into git commit and reviews every diff before it lands. 60-second setup. Completely free.*

Any feedback or contributors are welcome! It's online, source-available, and ready for anyone to use.

HexmosTech / git-lrc

Free, Micro AI Code Reviews That Run on Git Commit

git-lrc

Free, Micro AI Code Reviews That Run on Commit

git-lrc is your braking system. It hooks into git commit and runs an AI review on every diff before it lands. 60-second setup. Completely free.

In short, git-lrc helps Prevent Outages, Breaches, and Technical Debt Before They Happen

At a glance: 10 risk categories · 100+ failure patterns tracked · every commit…

View on GitHub

A Developer's Guide to LLM Model Serving - Part 2

Shrijith Venkatramana — Mon, 03 Aug 2026 19:25:36 +0000

There are LLM inference enthusiasts who talk a lot about LLM serving.

People mention terms like prefill, decode, KV cache, continuous batching, and paged attention. After reading several articles, I could recognize the words, but I still could not answer a basic question:

What actually happens after I send a prompt to an LLM?

As software engineers, we usually learn systems as a sequence of events. A web request reaches a server, middleware runs, business logic executes, and a response returns.

I wanted the same mental model for LLMs.

This article covers the second part of that journey.

1. The Request Begins

A simplified request flow looks like this:

Client
    ↓
Load Balancer
    ↓
LLM Server
    ↓
Tokenizer
    ↓
Prefill
    ↓
Decode
    ↓
Streaming Response

The two important stages are:

Prefill
Decode

Once these two ideas become clear, many later topics fit into place.

2. What Is Prefill?

Suppose the prompt is:

Explain quantum computing like I'm ten.

My first question was:

Does the model start generating the response immediately?

The answer is simpler than I expected.

The model first processes the entire prompt.

This stage is called prefill.

A useful mental model is:

kvCache, firstToken := Prefill(prompt)

Prefill performs two tasks:

Build the initial KV cache.
Generate the first output token.

The model now has enough information to continue generating the response.

3. Decode Is a Simple Loop

After prefill, the rest of the response comes from a repeated operation.

cache, token := Prefill(prompt)

for token != EOS {
    emit(token)
    cache, token = Decode(cache, token)
}

Each call to Decode() performs three actions:

Generate the next token.
Update the KV cache.
Return the new cache.

That loop continues until the model produces an end-of-sequence token.

This was the first point where LLM inference started to feel like ordinary software instead of a black box.

4. What Is the KV Cache?

The name made me think of a cache like Redis or a Go map.

The actual idea is much simpler.

The KV cache is data stored in GPU memory.

During prefill, the model computes internal data for every prompt token and stores it.

During decode, the model adds one new entry for every generated token.

A simple mental model looks like this:

type KVCache struct {
    Layer0 [...]
    Layer1 [...]
    Layer2 [...]
    ...
}

The cache grows during the entire conversation.

The purpose is straightforward:

Save expensive computation so future tokens can reuse it.

5. Why Doesn't the Model Process the Prompt Again?

Imagine a prompt with 2,000 tokens.

Suppose the model generates 500 output tokens.

One approach would be:

Read the 2,000 prompt tokens.
Generate one token.
Read the same 2,000 prompt tokens again.
Generate the next token.
Repeat.

That would repeat a large amount of computation.

The KV cache changes the process.

The model reads the prompt once during prefill.

During decode, it reuses the cached information and adds data for each newly generated token.

The expensive work happens once.

6. Wait... How Large Does the KV Cache Become?

This was the next question I asked.

If every prompt token adds data to the cache...

What happens with a one million token prompt?

The answer follows directly from the mental model.

The cache grows with every token.

A rough estimate for a 7B model gives memory usage on the order of hundreds of kilobytes per token.

That leads to numbers like these:

Prompt Length	Approximate KV Cache Size
1,000 tokens	Hundreds of MB
100,000 tokens	Tens of GB
1,000,000 tokens	Hundreds of GB

Modern models reduce this memory with techniques such as Grouped Query Attention (GQA). Several query heads share the same Keys and Values, so the cache becomes much smaller.

The important idea is simple.

Long context windows require large amounts of memory because the KV cache keeps growing.

This explains why modern serving systems spend so much effort on KV cache management.

7. A Simple Mental Model

After working through these ideas, I arrived at one picture that I keep in my head.

Prompt
    ↓
Prefill(prompt)
    ↓
KV Cache + First Token
    ↓
Decode(...)
    ↓
Next Token
    ↓
Decode(...)
    ↓
Next Token
    ↓
...

This model answers many basic questions.

Why does the model read the prompt before generating text?
Why does GPU memory increase with context length?
Why does every serving framework discuss the KV cache?
Why do serving optimizations focus on prefill and decode?

I still have many questions about attention, batching, scheduling, and parallelism.

Now I have a place to attach those ideas as I learn them.

What part of LLM inference took the longest to click for you? Was it attention, the KV cache, batching, or something else?

*AI agents write code fast. They also silently remove logic, change behavior, and introduce bugs -- without telling you. You often find out in production.

git-lrc fixes this. It hooks into git commit and reviews every diff before it lands. 60-second setup. Completely free.*

Any feedback or contributors are welcome! It's online, source-available, and ready for anyone to use.

HexmosTech / git-lrc

Free, Micro AI Code Reviews That Run on Git Commit

git-lrc

Free, Micro AI Code Reviews That Run on Commit

git-lrc is your braking system. It hooks into git commit and runs an AI review on every diff before it lands. 60-second setup. Completely free.

In short, git-lrc helps Prevent Outages, Breaches, and Technical Debt Before They Happen

At a glance: 10 risk categories · 100+ failure patterns tracked · every commit…

View on GitHub

A Developer's Guide to LLM Model Serving - Part 1

Shrijith Venkatramana — Sun, 02 Aug 2026 19:59:02 +0000

Training a model and deploying it are two very different problems.

After completing a PyTorch tutorial, it's easy to think of the resulting .pt file as the artifact that matters. In production, however, the challenge shifts from training to serving: accepting requests, loading models, scheduling inference, managing GPUs, and returning responses with predictable latency.

As LLMs have grown larger, serving infrastructure has become a significant engineering problem in its own right. John Hennessy remarked in 2023 that running an LLM request could be around ten times more expensive than a traditional keyword search, making inference costs an important consideration for production systems.

This article walks through the fundamentals of model serving, from what a deployed model actually consists of to why frameworks like vLLM and Triton exist.

1. A Model Is More Than a Weight File

It's common to think of a trained model as simply a large collection of learned weights.

In practice, a deployed model consists of three distinct parts:

Model architecture – the code that defines the structure of the neural network.
Model data – the learned weights, biases, and configuration.
Execution code – the runtime responsible for loading the model and executing inference.

The book describes models as executable programs rather than passive data files. The weights alone are not enough—they need an architecture and an execution runtime before they can produce predictions.

One practical consequence is that many projects store the architecture and weights separately. This makes it possible to evolve the architecture while partially loading compatible weights during deployment, rather than retraining the model from scratch.

2. Training and Serving Have Different Goals

Training and serving optimize for different workloads.

During training, the objective is to update model parameters using backpropagation. This usually involves very large batches running across multiple GPUs to maximize throughput.

Serving only performs forward propagation. Instead of optimizing for training speed, it focuses on:

low latency
high availability
scalability
predictable operational costs

The serving stack is expected to handle anything from a handful of requests to millions while maintaining consistent response times. Cost-to-serve becomes one of the primary metrics when evaluating different deployment approaches.

3. What Model Serving Actually Looks Like

Production model serving involves considerably more than exposing a generate() endpoint.

A typical request flows through several components before it reaches the GPU:

Client
    ↓
API Gateway
    ↓
Load Balancer
    ↓
Inference Server
    ↓
Continuous Batch Scheduler
    ↓
GPU

Traditional web infrastructure handles authentication, routing, and load balancing. ML-specific infrastructure is responsible for tokenization, scheduling, batching requests together, executing inference, and streaming responses back to clients. The architecture shown in the book separates these responsibilities explicitly.

4. Containerization Is the Foundation

Most production inference happens inside containers.

A serving container typically includes:

the serving API
model management
the inference backend
access to model storage

The container exposes an HTTP or gRPC interface while handling model initialization, execution, and resource management internally. This allows the same deployment to run consistently across development, testing, and production environments.

A simplified request flow looks like this:

Request
    ↓
Serving API
    ↓
Inference Backend
    ↓
Model Runtime
    ↓
Prediction

5. Why Specialized Serving Frameworks Exist

General-purpose ML libraries are designed primarily for experimentation and training. Serving large language models efficiently requires additional optimizations.

Frameworks such as vLLM, TensorRT-LLM, and SGLang are built specifically for inference workloads.

For example, a typical vLLM deployment exposes tuning parameters such as GPU memory utilization, maximum concurrent sequences, tensor parallelism, and maximum batched tokens:

python -m vllm.entrypoints.openai.api_server \
  --model openai/gpt-oss-20b \
  --dtype bf16 \
  --gpu-memory-utilization 0.9 \
  --max-num-seqs 16 \
  --max-num-batched-tokens 16384 \
  --tensor-parallel-size 2

These settings control how requests are scheduled and distributed across available hardware.

The benchmark reproduced in Hands-On LLM Serving and Optimization shows substantially higher serving throughput for vLLM than standard Hugging Face serving under the tested configurations.

6. Serving More Than One Model

Many production applications rely on multiple models rather than a single LLM.

A typical application might combine:

an embedding model for semantic search
a reranker
a vision model
a language model for generation

Managing each model independently quickly becomes operationally complex.

NVIDIA Triton Inference Server addresses this by providing a unified serving platform capable of loading multiple models in different formats, including PyTorch, ONNX, TensorRT, and TensorFlow. It also manages model loading, caching, and routing through a common API.

7. Why Serving Economics Matter

The cost of serving an LLM is determined by more than GPU prices.

A large part of the problem is utilization.

GPUs that spend significant time idle still incur the same infrastructure cost. Shared inference platforms can aggregate requests from many customers, keeping hardware utilization high and reducing the average cost per request.

The notes summarize the resulting cost hierarchy as:

Model labs
General-purpose inference providers
Enterprise self-hosted deployments

The more effectively GPU capacity is utilized, the lower the cost of serving each token tends to be.

Closing Thoughts

Model serving sits at the intersection of machine learning and distributed systems.

The model itself is only one component of a production deployment. The serving stack is responsible for scheduling requests, managing hardware, exposing APIs, handling failures, and keeping inference costs under control.

Understanding these pieces makes it easier to evaluate serving frameworks, deployment architectures, and the trade-offs between latency, throughput, and operational cost.

What serving framework or deployment architecture are you using today, and what trade-offs led you to that choice?

*AI agents write code fast. They also silently remove logic, change behavior, and introduce bugs -- without telling you. You often find out in production.

git-lrc fixes this. It hooks into git commit and reviews every diff before it lands. 60-second setup. Completely free.*

Any feedback or contributors are welcome! It's online, source-available, and ready for anyone to use.

HexmosTech / git-lrc

Free, Micro AI Code Reviews That Run on Git Commit

git-lrc

Free, Micro AI Code Reviews That Run on Commit

git-lrc is your braking system. It hooks into git commit and runs an AI review on every diff before it lands. 60-second setup. Completely free.

In short, git-lrc helps Prevent Outages, Breaches, and Technical Debt Before They Happen

At a glance: 10 risk categories · 100+ failure patterns tracked · every commit…

View on GitHub

Figuring Out Math Concepts with Ask Gemini

Shrijith Venkatramana — Wed, 29 Jul 2026 18:45:53 +0000

I've always wanted to learn more math.

I have made efforts here and there, but the overwhelming challenge, and my lack of time due to effort intensive work such as at Hexmos has kept my overall progress limited.

These days, with LLMs, I have a renewed hope that -- perhaps, even a time poor person like me could get a decent grasp of mathematical ideas -- with a fraction of time investment compared to what it used to take before.

My goal here is "Exposure", as Sean McClure tends to call it, "exposure":

Anyway, how does one go about familiarize with Mathematical concepts with the help of LLMs?

Alright, let's get started with something practical.

I remember studying "eigen vectors" and such long back in my college days, and I'd like to get a refresher on it.

And I already know that I like going through problem/solution sets or rather worked out examples, rather than going through tons of theory first.

I am a "problem-first" explorer of topics.

So I also happen to know that "Schaum's outlines" are good for this sort of stuff.

And I google "Schaum's linear algebra" and get this

The first link seems to have a PDF of some sort, let's check it out:

Then, in the text, I search for "Eigen", and I get the following. Also I hit "Ask Gemini" on the sidebar, and ask it a basic question to check whether it has context on the presently active page (page 299) -- it does!

What catches my attention now is -- "two points of view", and that catches my interest.

So I open a skill in Gemini "Break it down", and ask it "What are these two points of views"

Gemini has these skills and one can add many other useful skills (idea - perhaps I can get it to quiz me to check my understanding -- something to check later!)

A good start here -- the perspectives are "formula and numbers" vs "geometry and spaces"

OK -- noticed there are quizzes and flashcards too, pretty cool:

I'm reading this second paragraph, and I happen not to immediately have in my memory what is "diagonalization" or what is "invertible matrix".

I think dealing with "unknown terms" is a key operation in any kind of mathematical activity.

Now, I can proceed in two ways:

Ask Gemini to find the relevant part of the text
Go the Ctrl-F route I went earlier.

Also, as a side thought, I am starting to realize, reading a mathematical text with AI requires these sorts of "bookmarks" to switch between a session for various concepts.

Having a free-flowing chat is fine, but then moving between references of the book can be way more powerful.

Perhaps something like Google's NotebookLM can do a better job at this, but now, we will make do with Ask Gemini, and see if it can handle a question about finding content in the book itself.

I also check whether the sidebar can take actions on my behalf (scroll to the page where concept x is introduced)

Gemini responds -- gives me the section number, but it cannot scroll for me (how disappointing!). The blue link you see just links to the PDF top - no page number reference. This makes me think "Ask Gemini" doesn't have page number context at all or in a usable way.

But the information is legit, I was able to Ctrl-F the piece of text, and find exactly what Gemini promised me:

Gemini gave an answer of its own along with the previous response, but I found a better example in the book itself -- essentially multiply A x B, or B x A, you get a singular matrix -- and here A and B are inverses:

Now I am ready to go back to diagonalization, I wish the AI could help me navigate across pages as I need them, but there is no such facility as of now. So I find my own way with Ctrl-F for now.

I wanted an ELI5 skill for this "Ask Gemini", but all I see is a "simplify" one:

The whole skill is available for inspection:

Objective: Transform complex content into a simple explanation a 5 year old could understand

1. **Identify the central point of the content**

    * Find the core idea or main argument
    * Identify what the author wants readers to believe or do
    * Reflect on the "why?" of the content
    * Note the scope and limitations of the content

2. **Assess complexity**

    * Identify technical terms, jargon, or specialized vocabulary
    * Note assumed knowledge or prerequisites

3. **Make abstract concepts concrete with analogies**

    * Connect to everyday experiences (for example, playground, kitchen, family)
    * Use physical objects to represent abstract ideas
    * Create simple cause-and-effect relationships

4. **Rewrite using language accessible to a kindergarten student**

    * Replace jargon with common words
    * Shorten sentences to one idea each
    * Use active voice and present tense
    * Avoid hedging words like "might", "perhaps", and "arguably"

5. **Handling exceptions**

    Prioritize excellent content in your response. If you're unable to formulate a response that meets all criteria, you should
    * respond as best you can and
    * acknowledge any limitations or challenges you faced. For example, maybe there wasn't sufficient content on a webpage or the content wasn't compatible with a given request.

    Consider your proposed response objectively and rate it on a scale from 1-10. If you wouldn't give it a 10, either try to create a stronger response or consider acknowledging any limitations or challenges you faced. The score is just for your own purposes; don't share it with the user.

6. **Final response**

    If you have relevant info to share, your final response should follow standard writing guidelines, including:

    * Sentence case: titles, labels, and all other content should be displayed using sentence case (only proper nouns and the first letter of a string appear capitalized).
    * Favor simple sentences that use common words

    Your final response should also be:
    * 3-5 short paragraphs
    * Under 200 words total
    * No jargon or technical terms
    * At least one concrete analogy

7. **Follow-up questions**

    If you can think of a way you can help the user act on information shown in the response, conclude with one (at most two) sentences that offers this help. Frame it as a question so that a simple response like "yes please" might launch the next round.

Let's trigger an explanation with this:

This is the explanation I got, and it doesn't make me feel better. It seems to be giving some toy analogy which made it worse (for me). It's like you can either spin the tray to arrange toys or you can look at it from another perspective (!). The robot teacher is leading me astray here.

Need something better. Let me try a hand-rolled prompt:

Alright, much better - because it combines both technical analogy and ELI5 analogy:

I am starting to get an intuitive feel for what the whole thing is about. The matrix view is not very meaningful to me. I can imagine points in space. I can also imagine them being "crooked" or "messy". Then we change our default point of view, using change of basis. I know what basis vectors are from history, so I have a sense of what these are. Basis vectors are the "core direction givers" for any object we put in a vector space. With these basis vectors and a multiplier matrix or such, we can regenerate the object. So these basis vectors are "compression objects" of sorts. Eigen vectors are something of the sort.

I like the ELI5 example much better here -- it actually helps.

You have a crooked paper hanging in front, it's awkward to draw on. You want a straight line (90 degree like). So you change your chair position so that from your perspective it is crooked anymore, then draw, then put the chair back in position. This explains it best:

The formula $B = P^{-1}AP$ says: Change your perspective ($P$), do the action ($A$), then switch back ($P^{-1}$). The final result $B$ is the exact same action, but expressed in the easiest way possible!

Now I have an intuitive sense of what is going on here. Now let's see where this is used most in terms of applications.

PageRank, PCA, phew -- this is everywhere, something truly important.

Concluding thoughts

Overall, this experiment left me fairly optimistic.

I don't think LLMs replace textbooks.

They make textbooks much easier to read.

A few things stood out to me:

The textbook remained the source of truth. Gemini was most useful when explaining, connecting ideas, or helping me locate information.
Unknown terms stopped being roadblocks. They became quick detours before returning to the main flow.
Prompt quality mattered a lot. The built-in "Simplify" skill wasn't particularly helpful here. A hand-written prompt asking for both intuition and the technical explanation produced much better results.
Applications made concepts stick. Once diagonalization clicked, seeing PageRank and PCA immediately answered the question: "Why should I care?"

There are still some missing pieces.

Navigation is the biggest one. Gemini could tell me where something was, but it couldn't take me there.
Page references would help enormously. "Jump to page 284" feels like an obvious capability.
Bookmarks and multiple threads would be even better. Mathematical reading naturally branches into prerequisites and side quests. A chat interface isn't ideal for keeping track of all of them.

I'm curious to see where these tools go over the next year or two.

For now, though, this already feels like a much more enjoyable way to work through technical books than constantly bouncing between PDFs, Google searches, and Stack Overflow.

Zig's Incremental Compilation: Why It Could Change the Way We Build Software

Shrijith Venkatramana — Tue, 28 Jul 2026 17:14:25 +0000

What if changing a single function in a million-line codebase took **50 milliseconds* to rebuild instead of 30 seconds?*

For decades, we've accepted a strange reality of software engineering.

You change one line of code, yet the compiler often behaves as though you've rewritten the world.

Build systems like Make, Ninja, Bazel, Cargo, and Gradle have become remarkably sophisticated at figuring out which files need rebuilding. But most of that intelligence lives outside the compiler. Once the compiler is invoked, it frequently has to redo far more work than seems necessary.

The Zig project takes a different approach.

Instead of asking:

Which files changed?

it asks:

Which declarations actually changed, and what truly depends on them?

That subtle shift in perspective opens the door to dramatically faster feedback loops.

Let's explore why incremental compilation matters, how Zig approaches the problem, and why compiler researchers have been pursuing this idea for decades.

The hidden tax of waiting

Every developer knows this loop.

Edit

↓

Build

↓

Wait...

↓

Run

↓

Repeat

On a toy project, the delay is barely noticeable.

On a large codebase, it becomes part of your day.

A few seconds here. Ten seconds there. Hundreds of builds every week.

The cost isn't simply time.

Long feedback loops discourage experimentation.

Instead of testing an idea immediately, developers begin batching unrelated changes together because "it's not worth waiting for another rebuild."

Kent Beck has often argued that software development is fundamentally governed by the speed of feedback. Faster feedback encourages experimentation, smaller commits, and more confident refactoring.

Compiler engineers have known this for decades as well. Improving compile times isn't just about benchmarks—it changes how programmers think.

Traditional compilation thinks in files

Imagine you're renovating a city.

One house changes.

A traditional compiler often behaves like a city inspector who says:

"Let's inspect every building on this street, just to be safe."

Modern build systems improve this considerably.

main.c
├── parser.c
├── lexer.c
└── util.c

If parser.c changes, the build system avoids recompiling completely unrelated files.

That's a huge improvement.

But files are still a fairly crude unit of change.

Suppose this file contains twenty functions.

Only one function changes.

The compiler still ends up reprocessing the entire translation unit because, historically, C and C++ compilers fundamentally operate on source files and headers rather than individual declarations.

This design made perfect sense in the 1970s.

Today's software is rather larger.

Zig thinks in declarations instead

Andrew Kelley, Zig's creator, has long argued that much of the compiler's work should be understood in terms of semantic dependencies rather than files. His article, :contentReference[oaicite:0]{index=0}, outlines many of the design goals that eventually led to Zig's modern compiler architecture.

Consider this Zig source file.

const PI = 3.14159;

fn area(r: f64) f64 {
    return PI * r * r;
}

fn perimeter(r: f64) f64 {
    return 2 * PI * r;
}

fn fibonacci(n: u32) u32 {
    ...
}

Suppose you modify only area().

Should the compiler revisit perimeter()?

Probably not.

Should it revisit fibonacci()?

Certainly not.

Instead of treating the entire file as one indivisible object, Zig tracks dependencies between individual declarations.

Only the declarations affected by your edit—and anything depending on them—need to be reconsidered.

That's much closer to how humans naturally reason about software.

The compiler becomes a dependency graph

The key insight is that software is really a graph.

Not a graph of files.

A graph of relationships.

Function A
      │
      ▼
Type B
      │
      ▼
Constant C
      │
      ▼
Function D

Now imagine changing Constant C.

Only the reachable portion of the graph needs updating.

Everything else remains valid.

A useful analogy is Microsoft Excel.

When you change one spreadsheet cell, Excel doesn't recompute the entire workbook.

Only the formulas depending on that cell are recalculated.

Incremental compilation follows exactly the same philosophy.

The difference is that software dependency graphs are vastly more complicated than spreadsheets.

Functions call functions.

Types depend on other types.

Compile-time evaluation generates additional dependencies.

Generic instantiations create entirely new ones.

Keeping track of all this correctly is one of the hardest problems in compiler engineering.

The real challenge is invalidation

People often imagine incremental compilation as simply "saving previous work."

That's actually the easy part.

The difficult question is:

How does the compiler know exactly what became invalid?

Phil Karlton famously joked:

"There are only two hard things in Computer Science: cache invalidation and naming things."

Incremental compilation is essentially one enormous cache invalidation problem.

Imagine changing

const BufferSize = 4096;

Now what?

Should the compiler revisit functions using it?

Generic instantiations?

Compile-time evaluated expressions?

Type layouts?

Generated machine code?

Getting this wrong either produces incorrect binaries—or wastes time rebuilding things unnecessarily.

Much of Zig's recent compiler work has focused precisely on this problem.

Compiler engineer Matthew Lugg explains this journey in the excellent Zig engineering article :contentReference[oaicite:1]{index=1}, describing how the compiler's type resolution and semantic analysis were redesigned so dependency tracking becomes far more precise instead of repeatedly "over-analyzing" the program.

This isn't flashy work.

But it's exactly the kind of engineering that determines whether incremental compilation actually feels instantaneous.

Why compiler researchers have chased this for decades

Incremental compilation didn't begin with Zig.

Researchers have explored it since the 1980s, driven first by interactive programming environments and later by IDEs, language servers, and ever-larger software systems.

One particularly approachable modern paper is :contentReference[oaicite:2]{index=2}, which argues that dependency management should increasingly become the compiler's responsibility rather than being handled exclusively by external build systems.

Zig is interesting because it was designed with this direction in mind.

Rather than bolting incremental compilation onto an existing decades-old compiler architecture, the project has steadily evolved its semantic analysis, dependency tracking, and self-hosted compiler so that incremental rebuilding becomes a natural consequence of the design.

That's a much harder path.

It may also prove to be the more scalable one.

More than faster builds

It's tempting to think incremental compilation is simply about shaving seconds off compile times.

The deeper benefit is psychological.

Imagine your compiler remaining alive between edits.

It already knows your program.

It remembers previous analysis.

You save a file.

Only a tiny portion of the dependency graph changes.

Errors appear almost immediately.

That changes how you work.

Instead of thinking,

"I'll make five edits before rebuilding,"

you naturally start thinking,

"Let's verify every tiny change."

Smaller feedback loops encourage smaller experiments.

Smaller experiments usually lead to cleaner designs, easier debugging, and greater confidence while refactoring.

The fastest compiler isn't necessarily the one that produces binaries the quickest.

It's the one that tells you whether your last idea worked before you've lost the thread of your thinking.

Final thoughts

Programming languages often compete on syntax.

Or performance.

Or memory safety.

Or ecosystems.

Zig is making another bet.

That developer feedback latency deserves to be treated as a first-class design goal.

Instead of asking,

"How can we compile faster?"

the Zig compiler increasingly asks,

"How little work is actually necessary after this edit?"

Those aren't the same question.

One optimizes the compiler.

The other optimizes for the programmer.

As our codebases continue growing into millions of lines, that distinction may become one of the more important innovations in modern compiler design.

*AI agents write code fast. They also silently remove logic, change behavior, and introduce bugs -- without telling you. You often find out in production.

git-lrc fixes this. It hooks into git commit and reviews every diff before it lands. 60-second setup. Completely free.*

Any feedback or contributors are welcome! It's online, source-available, and ready for anyone to use.

HexmosTech / git-lrc

Free, Micro AI Code Reviews That Run on Git Commit

git-lrc

Free, Micro AI Code Reviews That Run on Commit

git-lrc is your braking system. It hooks into git commit and runs an AI review on every diff before it lands. 60-second setup. Completely free.

In short, git-lrc helps Prevent Outages, Breaches, and Technical Debt Before They Happen

At a glance: 10 risk categories · 100+ failure patterns tracked · every commit…

View on GitHub

Beyond Prompts: 10 Installable Superpowers for Coding Agents

Shrijith Venkatramana — Mon, 27 Jul 2026 18:22:52 +0000

Hi there! I'm Shrijith Venkatramana, and I'm building something I think you'll find useful—git-lrc, an AI code reviewer that checks every commit before it lands in your repo. If that sounds interesting, give us a star on GitHub to help other devs discover it. And please, try it out—I'd genuinely love your feedback.

AI coding agents are becoming more capable every month. But one of the biggest shifts isn't a better model—it's teaching your agent new skills.

Instead of writing the same prompt over and over ("review my architecture", "design a better UI", "help me write tests"), you can install reusable skills that encode proven workflows and best practices.

The open skills.sh ecosystem has already crossed 1 million+ installs, with hundreds of reusable skills available for Claude Code, Cursor, GitHub Copilot, Cline, and other compatible agents. :contentReference[oaicite:0]{index=0}

Let's look at the current Top 10 skills, what each one does, and when you should actually use it.

Installing Skills

First, install the Skills CLI.

npx skills add <owner/repository>

If the repository contains multiple skills:

npx skills add <owner/repository> --skill <skill-name>

Useful commands:

# Search for skills
npx skills find react

# Check for updates
npx skills check

# Update installed skills
npx skills update

The CLI works with many popular AI coding agents including Claude Code, Cursor, GitHub Copilot, and Cline. :contentReference[oaicite:1]{index=1}

1. find-skills (vercel-labs/skills)

What it does

Think of this as the package manager for AI skills.

Whenever your agent doesn't know the best workflow for something, this skill teaches it how to search the Skills ecosystem and recommend high-quality skills.

Install:

npx skills add vercel-labs/skills --skill find-skills

Use it by asking:

"Find me a skill for Docker deployments."

"Is there a skill for writing better documentation?"

It searches the ecosystem, recommends trustworthy skills, and even suggests install commands.

Best for

Discovering new capabilities
Exploring the ecosystem
Finding community skills

2. frontend-design (anthropics/skills)

What it does

Makes your AI much better at UI and frontend design.

Instead of generating generic HTML, it starts thinking more like an experienced product designer by considering:

visual hierarchy
spacing
typography
color usage
responsive layouts
accessibility
modern UI patterns

Install:

npx skills add anthropics/skills --skill frontend-design

Try prompts like:

"Redesign this dashboard."

"Make this login page feel premium."

"Improve the visual hierarchy."

You'll usually get noticeably better design suggestions than with a vanilla model.

Best for

React apps
SaaS dashboards
Landing pages
Internal tools

3. grill-me (mattpocock/skills)

What it does

This is one of my favorites.

Instead of agreeing with everything you say, the AI becomes a skeptical reviewer.

It starts asking difficult questions like:

Why?
Have you considered X?
What if this fails?
What assumptions are you making?

Install:

npx skills add mattpocock/skills --skill grill-me

Example prompt:

"Grill my API design."

"Challenge my startup idea."

It feels like discussing your work with a strong senior engineer.

Best for

Architecture reviews
Design reviews
Startup ideas
Technical proposals

4. agent-browser (vercel-labs/agent-browser)

What it does

Teaches your agent browser automation workflows.

Instead of just talking about web pages, the agent can reason about interacting with websites and browser-driven tasks.

Install:

npx skills add vercel-labs/agent-browser

Example requests:

"Test this signup flow."

"Navigate the application and summarize the UX."

"Check whether this responsive layout breaks."

Best for

QA
Browser automation
End-to-end testing
UI inspection

5. vercel-react-best-practices

What it does

A huge collection of React and Next.js best practices from Vercel.

It teaches your AI things like:

rendering patterns
performance
component design
React Server Components
data fetching
caching
composition
maintainability

Install:

npx skills add vercel-labs/agent-skills --skill vercel-react-best-practices

Now ask:

"Refactor this React component."

"Improve the performance of this page."

The suggestions become much closer to what experienced React developers would recommend.

Best for

React
Next.js
Performance optimization
Code reviews

6. grill-with-docs (mattpocock/skills)

What it does

Similar to grill-me, but instead of challenging your ideas from experience alone, it uses documentation as evidence.

Rather than saying:

"I think this is wrong."

it says:

"According to the documentation, this pattern isn't recommended."

Install:

npx skills add mattpocock/skills --skill grill-with-docs

Great prompts:

"Review this API usage."

"Challenge this implementation against the framework docs."

Best for

Learning frameworks
Avoiding outdated APIs
Documentation-driven reviews

7. improve-codebase-architecture

What it does

Looks beyond individual files.

Instead of suggesting variable renames, it examines:

project structure
module boundaries
coupling
abstractions
maintainability
scalability

Install:

npx skills add mattpocock/skills --skill improve-codebase-architecture

Example:

"Review this repository's architecture."

"How would you reorganize this project?"

This is particularly useful on mature codebases.

Best for

Refactoring
Large repositories
Technical debt
Long-term maintainability

8. tdd (mattpocock/skills)

What it does

Encourages the agent to work using Test-Driven Development.

Instead of immediately generating implementation code, it tends to:

write tests
think through edge cases
implement only what's needed
refactor afterwards

Install:

npx skills add mattpocock/skills --skill tdd

Example prompt:

"Build a URL parser using TDD."

"Let's implement this feature test-first."

Excellent if you're trying to build more reliable software.

Best for

Library development
Backend code
Complex business logic
Learning TDD

9. web-design-guidelines

What it does

Focuses specifically on good web design principles.

Unlike frontend-design, which improves overall UI generation, this skill emphasizes:

usability
accessibility
spacing
responsive behavior
layout consistency
navigation
readability

Install:

npx skills add vercel-labs/agent-skills --skill web-design-guidelines

Try:

"Audit my landing page."

"Improve accessibility."

"Review this homepage."

It's particularly useful before shipping a public website.

Best for

Marketing sites
Documentation
Landing pages
Design reviews

10. microsoft-foundry

What it does

Provides guidance for building AI applications using Microsoft's AI Foundry ecosystem.

If you're working with Azure AI services, this skill helps the agent recommend the right workflows, architectures, and platform capabilities.

Install:

npx skills add microsoft/azure-skills --skill microsoft-foundry

Example prompts:

"Help me build an AI application on Azure."

"Recommend an AI Foundry architecture."

"How should I deploy this AI workload?"

Best for

Azure users
Enterprise AI
Microsoft AI services

Which Skills Should You Install First?

If you're a general software engineer, I'd start with these five:

Skill	Why install it?
find-skills	Discover everything else
grill-me	Better technical decisions
improve-codebase-architecture	Higher-quality code reviews
tdd	More reliable implementations
vercel-react-best-practices	Essential if you write React

Frontend developers should also add:

frontend-design
web-design-guidelines

Azure users should definitely install:

microsoft-foundry

Skills Are Becoming the New Prompt Library

One thing stands out after looking through the leaderboard: these skills aren't replacing AI models—they're teaching them repeatable workflows.

Instead of repeatedly explaining how you want your AI to think, you install that expertise once and reuse it across every conversation.

As the ecosystem grows, I expect most developers to have a personal toolkit of skills for architecture, testing, design, documentation, DevOps, and domain-specific workflows—much like we already rely on packages and libraries today.

Which AI skill has saved you the most time? Have you published one yourself? I'd love to hear what's in your toolbox.

AI agents write code fast. They also silently remove logic, change behavior, and introduce bugs—without telling you. You often find out in production.

git-lrc fixes this. It hooks into git commit and reviews every diff before it lands. 60-second setup. Completely free.

Any feedback or contributors are welcome! It's online, source-available, and ready for anyone to use.

HexmosTech / git-lrc

Free, Micro AI Code Reviews That Run on Git Commit

git-lrc

Free, Micro AI Code Reviews That Run on Commit

git-lrc is your braking system. It hooks into git commit and runs an AI review on every diff before it lands. 60-second setup. Completely free.

In short, git-lrc helps Prevent Outages, Breaches, and Technical Debt Before They Happen

At a glance: 10 risk categories · 100+ failure patterns tracked · every commit…

View on GitHub

Building Enterprise AI Applications: It's Not About the Model. It's About the System

Shrijith Venkatramana — Sun, 26 Jul 2026 18:47:32 +0000

Most AI demos fail in production for a surprisingly simple reason.

The model isn't the product.

A chatbot that summarizes documents or answers questions is impressive for a demo. But enterprises don't buy demos. They buy reliability, traceability, governance, and predictable outcomes.

The moment AI is asked to review contracts, approve financial decisions, analyze medical records, or assist engineers, the question changes from:

"Can the model do this?"

"Can we trust the entire system?"

That's the difference between consumer AI and enterprise AI.

In this article we'll explore what actually makes an enterprise AI application, moving from high-level intuition down to the technical architecture that makes these systems dependable.

1. AI Applications Are Knowledge Systems, Not Chatbots

Many teams start with a language model and ask:

"What can we build with GPT?"

Successful teams usually ask the opposite question:

"What knowledge does our organization already possess, and how can AI leverage it?"

Every company has accumulated years of institutional knowledge:

Internal documentation
Policies
Standard operating procedures
Contracts
Engineering playbooks
Historical decisions
Regulatory requirements

This knowledge—not the LLM—is the company's competitive advantage.

The language model is simply the reasoning engine that learns to navigate this knowledge.

Think of the LLM as the CPU.

Your enterprise knowledge is the operating system.

Without the operating system, the CPU is remarkably limited.

2. Configuration Is Really Knowledge Engineering

A common misconception is that building enterprise AI means fine-tuning models.

In reality, much of the work involves structuring organizational knowledge.

Instead of retraining the model, organizations typically:

organize documents
build retrieval pipelines
encode business rules
define workflows
establish permissions
create reusable prompts
connect internal systems

Imagine building an AI assistant for legal teams.

Instead of saying:

"Review this contract."

you teach it things like:

preferred contract language
company negotiation policies
escalation rules
jurisdiction-specific requirements
acceptable risk thresholds

The AI is no longer answering generic questions.

It is reasoning inside your organization's operating model.

This process is less about machine learning and more about knowledge engineering.

3. Enterprise AI Needs Verification, Not Just Evaluation

Consumer AI is often evaluated with questions like:

Does the answer sound good?
Is the response fluent?
Does it seem helpful?

Enterprise AI demands much stronger guarantees.

Imagine an AI reviewing a million-dollar contract.

A response that is "mostly correct" is still a failure.

Instead, organizations create evaluation suites that resemble software testing.

For example:

edge cases
ambiguous wording
conflicting policies
historical mistakes
adversarial prompts
regulatory scenarios

Rather than asking:

"Can the model answer questions?"

they ask:

"Can the system consistently produce acceptable decisions across thousands of difficult situations?"

In many organizations, evaluation becomes a continuous engineering discipline rather than a one-time benchmark.

4. Every AI Decision Should Be Explainable

One of the biggest differences between enterprise software and consumer AI is accountability.

Imagine these two answers.

Consumer AI

"The answer is 17."

Done.

Now compare that with an enterprise AI assistant.

"The answer is 17 because:

Policy section 4.2 applies.

Customer qualifies under rule B.

Historical precedent X was matched.

Confidence score: 96%.

Human approval required."

The second response is much more valuable.

Not because it sounds smarter.

Because someone can verify it.

Enterprise users don't just want answers.

They want reasoning they can inspect.

This is especially important in finance, healthcare, insurance, legal services, and government.

Explainability turns AI from an opinion generator into a decision-support system.

5. Governance Is an Engineering Problem

People often think AI governance means lawyers writing policies.

In practice, much of it becomes software engineering.

Questions include:

Which model generated this answer?
Which prompt version was used?
Which documents were retrieved?
Which user approved the decision?
Which workflow executed?
Which policies were active?

Every decision should leave behind a reproducible trail.

This resembles version control systems.

Developers already expect every code change to have:

a commit
an author
a timestamp
a review history

Enterprise AI requires similar discipline.

Every AI-generated outcome should have provenance.

Not because regulations demand it.

Because debugging AI without provenance is nearly impossible.

6. AI Systems Need Operational Safety

Even highly capable models occasionally hallucinate.

Enterprise systems assume this will happen.

Instead of trying to eliminate all mistakes, they build systems that detect and contain them.

Examples include:

confidence thresholds
human approval workflows
policy validation
structured outputs
retrieval verification
anomaly detection
rollback mechanisms
emergency disable switches

Consider an AI processing financial documents.

If confidence drops below a threshold, the workflow can automatically:

stop execution,
notify a reviewer,
attach supporting evidence,
wait for approval before proceeding.

This philosophy mirrors how modern distributed systems are built.

We don't assume servers never fail.

We build systems that fail safely.

Enterprise AI should follow the same principle.

7. The Architecture Shift: From Models to Platforms

As organizations mature, they stop thinking about individual AI agents.

Instead, they build AI platforms.

A mature platform typically includes:

                Enterprise Knowledge
      Documents • Policies • Databases • APIs
                     │
                     ▼
             Retrieval & Context Layer
                     │
                     ▼
            Prompt & Workflow Engine
                     │
                     ▼
               Foundation Models
                     │
                     ▼
          Evaluation & Policy Checks
                     │
                     ▼
         Human Review (when necessary)
                     │
                     ▼
      Logging • Audit Trail • Monitoring

Notice something interesting.

The language model is only one component.

Most of the engineering effort goes into everything surrounding it:

context management
orchestration
observability
evaluation
governance
integrations
safety

This is why successful enterprise AI projects increasingly resemble distributed systems rather than chatbot applications.

Final Thoughts

The first generation of AI applications focused on making models smarter.

The next generation is focused on making AI systems trustworthy.

Organizations are discovering that the real challenge isn't generating text.

It's building systems that can justify their decisions, operate safely, adapt to organizational knowledge, and remain auditable months or even years later.

The future of enterprise AI won't be won by the company with the largest model.

It will be won by the teams that build the most reliable systems around those models.

What do you think?

When building enterprise AI, where do you believe the hardest engineering challenge lies today?

Knowledge management?
Evaluation?
Governance?
Safety?
Or something entirely different?

I'd love to hear your experiences in the comments.

*AI agents write code fast. They also silently remove logic, change behavior, and introduce bugs -- without telling you. You often find out in production.

git-lrc fixes this. It hooks into git commit and reviews every diff before it lands. 60-second setup. Completely free.*

Any feedback or contributors are welcome! It's online, source-available, and ready for anyone to use.

HexmosTech / git-lrc

Free, Micro AI Code Reviews That Run on Git Commit

git-lrc

Free, Micro AI Code Reviews That Run on Commit

git-lrc is your braking system. It hooks into git commit and runs an AI review on every diff before it lands. 60-second setup. Completely free.

In short, git-lrc helps Prevent Outages, Breaches, and Technical Debt Before They Happen

At a glance: 10 risk categories · 100+ failure patterns tracked · every commit…

View on GitHub

SVGBench: Measuring LLM Spatial Intelligence Through Vector Code

Shrijith Venkatramana — Sat, 25 Jul 2026 16:59:27 +0000

Large language models (LLMs) can draft Python scripts, pass medical board exams, and write poetry. But can they actually visualize the physical world, or are they simply echoing text patterns from their training data?

Standard coding benchmarks like HumanEval or MBPP test syntactic correctness and procedural logic. However, they tell us very little about an AI’s internal spatial awareness. When an LLM claims to understand concepts like "next to," "overlapping," or "below," is it truly reasoning about visual space, or is it just predicting token sequences?

To answer this, we need a test that bridges text generation and visual reality.

The Problem: The Visual Blindspot in LLM Benchmarks

Traditional text-based code evaluation suffers from a major visual blindspot:

Syntax ≠ Spatial Understanding: A model can produce syntactically valid Python or HTML while remaining completely blind to the visual outcome.
Pixel Matching Is Too Rigid: Evaluating image generation models using metrics like FID (Fréchet Inception Distance) or exact pixel matching fails for vector graphics. A red circle shifted 2 pixels to the left isn't a failure, but rigid pixel-based evaluation penalizes it as one.
Multimodal Blindspots: Vision-Language Models (VLMs) can describe an image, but asking a model to build a scene from scratch requires true 2D spatial arrangement, layering order, perspective, and domain knowledge.

Without a way to evaluate how code renders into visual spatial logic, we cannot accurately measure an AI's mental model of the physical world.

Objectives of SVGBench

SVGBench was built to turn raw vector code into a quantitative benchmark for spatial reasoning. Its primary goals include:

Test Visual Spatial Logic: Measure whether models understand anatomical placement, relative positioning, and physical interactions (e.g., placing a saddle on a horse, or floating an object in water).
Evaluate Precision Instruction Following: Force models to satisfy complex prompts containing 10–15 distinct visual sub-requirements simultaneously.
Establish Objective Multimodal Scoring: Move away from binary pass/fail syntax tests or fuzzy global image scores by using requirement-based evaluation.
Provide an End-to-End Automated Pipeline: Offer an automated workflow that generates SVG code, renders it via headless browsers, and evaluates the visual output.

The Conceptual Solution: Code, Render, Judge

Instead of asking a model to describe a scene, SVGBench forces the model to express the scene programmatically using Scalable Vector Graphics (.svg).

┌──────────────┐     ┌──────────────┐     ┌──────────────┐     ┌──────────────┐
│  Prompt &    │ ──> │ Target Model │ ──> │ Headless     │ ──> │ Evaluator    │
│ Requirements │     │ Generates    │     │ Browser      │     │ Vision LLM   │
│ (JSON)       │     │ SVG Code     │     │ Renders PNG  │     │ Scores Match │
└──────────────┘     └──────────────┘     └──────────────┘     └──────────────┘

Conceptually, the evaluation works in three distinct steps:

The Code Phase: The target model receives a prompt along with detailed requirements (e.g., "Draw a toy rocket with a blue body, red nose cone, and three symmetrical base fins"). It must output valid SVG code.
The Render Phase: The raw SVG string is rendered into a high-resolution PNG image inside an isolated headless Chrome browser. This turns abstract code into actual visual pixels.
The Judgment Phase: A multimodal Evaluator LLM (e.g., GPT-4o) inspects both the rendered PNG image and the raw SVG source code against every single requirement. The final score represents the percentage of visual and structural requirements successfully met ($0.0$ to $1.0$).

Strengths & Weaknesses

Like any benchmark architecture, SVGBench makes intentional trade-offs.

Strengths

Granular Evaluation: Scoring individual requirements produces a nuanced percentage score rather than a misleading pass/fail metric.
Resilient to Formatting Differences: SVG allows infinite ways to draw the same object. By evaluating rendered images alongside code, models aren't penalized for using different drawing techniques (e.g., <path> vs. <rect>).
Tests Multimodal Synthesis: It evaluates code synthesis, visual aesthetic arrangement, coordinate math, and domain knowledge simultaneously.

Weaknesses

Judge LLM Dependency: Evaluation quality depends on the vision model's ability to accurately spot fine visual details in the rendered PNG.
2D Vector Limitation: SVGBench evaluates 2D layouts and pseudo-3D projections; it does not measure native 3D mesh or spatial depth generation.
Execution Overhead: Running headless browser screenshots and calling multimodal evaluator APIs makes evaluation slower and more resource-intensive than running standard unit tests.

Under the Hood: System Architecture & Execution

Let's look at how SVGBench handles this pipeline under the hood.

SVGBench Architecture
├── questions/
│   ├── questions.json        # Main benchmark (100+ complex prompts)
│   └── test_questions.json   # 3-question smoke test dataset
├── src/
│   ├── run.py                # Main CLI controller
│   ├── benchmark/
│   │   └── benchmark.py      # Multi-threaded execution engine
│   └── utils/
│       ├── llm.py            # API client with vision support
│       └── svg_renderer.py   # Selenium & Headless Chrome renderer
└── results/                  # Generated artifacts (.svg, .png, JSON)

1. Headless Browser Rendering (`src/utils/svg_renderer.py`)

To render SVG code accurately, SVGRenderer uses Selenium with Google Chrome:

Calculates bounding boxes if explicit viewBox dimensions are missing.
Wraps raw SVG code into an inline HTML5 document.
Captures a high-resolution screenshot to convert the vector graphics into a PNG raster image.

2. Parallel Benchmark Engine (`src/benchmark/benchmark.py`)

To process datasets efficiently, Benchmark uses Python's ThreadPoolExecutor to evaluate multiple questions in parallel. It saves progress automatically, allowing you to resume interrupted benchmark runs without re-evaluating completed items.

3. Quick Testing vs. Full Evaluation

SVGBench includes two test sets:

questions/questions.json: The standard dataset containing over 100 complex prompts (covering UI layouts, physical scenes, and spatial positioning).
questions/test_questions.json: A lightweight 3-question dataset for sanity-checking your setup without burning API credits.

How to Run SVGBench

Setup

# Clone and prepare the environment
git clone https://github.com/johnbean393/SVGBench.git
cd SVGBench

conda create -n svgbench python=3.13
conda activate svgbench
pip install -r requirements.txt

Note: Ensure Google Chrome and chromedriver are installed and added to your system PATH.

Execution

Configure your .env file with your API key:

OPENROUTER_API_KEY=your_key_here

Run the evaluation across one or multiple models:

# Evaluate a single model
python src/run.py --model "anthropic/claude-sonnet-4"

# Benchmark multiple models sequentially
python src/run.py --model "google/gemini-2.5-pro;anthropic/claude-sonnet-4;openai/gpt-4.1"

After the benchmark completes, launch the built-in web UI to inspect the results:

python -m http.server 8000 --directory results

Leaderboard Results

Here is how leading AI models perform on SVGBench:

Rank	Model	Score
1	Claude Opus 4.6 (Medium)	75.6%
2	GPT-5.2 (X-High)	74.4%
3	Claude Opus 4.5	72.0%
4	GLM-5	70.3%
5	Gemini 3 Pro Preview	68.7%
6	Kimi K2.5	67.5%
7	GPT-5.1 (High)	67.5%
8	MiniMax M2.5	65.0%
9	Claude Sonnet 4.5	62.2%
10	Gemini 2.5 Pro	61.4%
11	o3	56.7%
12	DeepSeek V3.1	53.1%

(Source: SVGBench GitHub Repository)

Key Takeaways

Frontier Models Lead Visual Reasoning: Top reasoning and frontier models like Claude Opus 4.6 and GPT-5.2 score around 75%, demonstrating strong spatial composition skills.
Significant Room for Improvement: Even top-tier models miss roughly 25% of granular visual instructions, often struggling with precise overlapping elements, complex alignment, or coordinate math.
Code Intelligence ≠ Visual Intelligence: Strong text coding performance does not always translate to high visual fidelity, proving that spatial vector reasoning requires unique capabilities beyond standard code generation.

By combining raw code generation with automated visual evaluation, SVGBench provides a clear window into how well AI models truly understand the visual and spatial world.

*AI agents write code fast. They also silently remove logic, change behavior, and introduce bugs -- without telling you. You often find out in production.

git-lrc fixes this. It hooks into git commit and reviews every diff before it lands. 60-second setup. Completely free.*

Any feedback or contributors are welcome! It's online, source-available, and ready for anyone to use.

HexmosTech / git-lrc

Free, Micro AI Code Reviews That Run on Git Commit

git-lrc

Free, Micro AI Code Reviews That Run on Commit

git-lrc is your braking system. It hooks into git commit and runs an AI review on every diff before it lands. 60-second setup. Completely free.

In short, git-lrc helps Prevent Outages, Breaches, and Technical Debt Before They Happen

At a glance: 10 risk categories · 100+ failure patterns tracked · every commit…

View on GitHub

Beyond Code Generation: How OmniSVG Rethinks Vector Graphics with Vision-Language Models

Shrijith Venkatramana — Fri, 24 Jul 2026 19:36:05 +0000

Vector graphics are the holy grail of modern UI/UX design, CAD systems, and digital illustration. Unlike raster images (JPEGs, PNGs) composed of fixed pixel grids, Scalable Vector Graphics (SVGs) use geometric shapes—Bézier curves, polygons, and fill attributes. They render crisply at 8K resolution and weigh only a few kilobytes.

However, automating SVG creation has always been a major pain point for developers:

Optimization methods (like DiffVG) treat vectorization as an iterative guessing game. They generate cluttered, uneditable SVG code with thousands of overlapping anchor points, consuming massive GPU power for every single image.
LLM code-generation methods (like Chat2SVG or StarVector) try to write raw XML tags directly. Because standard AI tokenizers split XML tags and coordinate numbers haphazardly, models suffer from "coordinate hallucination" and hit strict character limits whenever designs get complex.

A team of researchers from Fudan University and StepFun introduced OmniSVG—an end-to-end framework built on native Vision-Language Models (Qwen2.5-VL) that treats vector generation not as raw text writing, but as structured token sequence prediction.

1. High-Level Intuition: Tokenizing Geometry Instead of Text

Standard Large Language Models (LLMs) view an SVG as a long string of raw text: <path d="M 10 20 C 30 40 ..."/>. To a typical AI tokenizer, <path, d=", and 10 are disconnected sub-word chunks. The AI has to learn XML syntax rules, floating-point math, and spatial geometry all at the same time.

OmniSVG changes the paradigm:

Flatten the Structure: All complex grouping tags, transformations, and inline styles are stripped away into simple, flat drawing instructions.
Atomic Commands: Any vector graphic is simplified into five core path commands:
M (Move To)
L (Line To)
C (Cubic Curve)
A (Arc)
Z (Close Path)
Explicit Fill Token (F): Color information is mapped directly into hex fill commands (F) right inside the drawing order, treating color as just another drawing step rather than a messy CSS attribute.

+-----------------------------------------------------------------------+
|                            OmniSVG Pipeline                           |
|                                                                       |
|  [ Input Instruction ]                                                |
|  "A cute red panda icon"                                              |
|          |                                                            |
|          v                                                            |
|  +-----------------------+     +-----------------------------------+  |
|  |   Qwen2.5-VL VLM      | --> | Flattened & Parameterized Tokens  |  |
|  |   Autoregressive Core |     | <SOP> M [2000] C [2045] F #FF0000 |  |
|  +-----------------------+     +-----------------------------------+  |
|                                                  |                    |
|                                                  v                    |
|                                        [ Clean, Editable SVG ]        |
+-----------------------------------------------------------------------+

By separating spatial geometry from low-level XML syntax, the underlying model can focus entirely on visual layout and drawing order.

2. The Math Explained in Plain English: Coordinate Compression & Economics

Spatial Token Compression

Traditional methods tokenized horizontal ($X$) and vertical ($Y$) coordinates as separate numbers. If an SVG path has 100 points, it takes at least 200 tokens just to specify where those points go.

OmniSVG fits every SVG onto a standardized 200 by 200 pixel canvas grid. Instead of sending two separate numbers for every point, OmniSVG merges the two coordinates into a single number index using simple grid math:

Combined Token Index = (X Coordinate * Canvas Width) + Y Coordinate

For a canvas that is 200 pixels wide and 200 pixels tall, this formula translates every possible $(X, Y)$ spot on the canvas into a single unique ID number between 0 and 39,999.

To convert that single ID number back into 2D coordinates on the fly:

X Coordinate: Divide the Token ID by the Canvas Width (200) and round down.
Y Coordinate: Take the remainder after dividing the Token ID by the Canvas Width (200).

Back-of-the-Envelope Token Savings

Imagine writing a single smooth curve segment with a start point, two control handles, an end point, and a red fill color:

Raw XML Text String: <path d="M 120 45 C 130 50 140 85 150 90 Z" fill="#FF5733"/>
Standard Tokenizer Count: 22 to 28 tokens.
OmniSVG Approach:
Commands: M, C, Z, F (4 tokens)
Start point (Combined 120 and 45): 1 token
Curve handle points and end point: 3 tokens
Red Color Code (#FF5733): 1 token
Total Token Count: 9 tokens (A 65% reduction in total tokens!).

System Economics & Operational Efficiency

When AI models generate content, their computational work and latency scale rapidly with the total length of the token sequence. Shorter sequences mean significantly faster rendering and lower server costs.

       Token Sequence Length vs. Inference Latency (Conceptual)
   Latency
      ^
      |                                              / (Raw XML: ~32k Tokens)
      |                                             /
      |                                            /
      |                                           /
      |                                          /
      |                  /----------------------/ (OmniSVG: ~4.8k Tokens)
      |_________________/________________________________________>
                                                              Sequence Length

Metric	DiffVG / Live Optimization	Raw XML LLM Generation	OmniSVG (4B/8B Models)
Method Type	Repeated Gradient Guesswork	Standard Text LLM	Structured Vision-Language Model
Tokens / Steps	50,000 to 300,000 steps	~18,500 raw text tokens	~3,800 to 5,800 tokens
Time per Image	45 to 180 seconds	12 to 25 seconds	1.5 to 4 seconds
Output Structure	Messy, overlapping anchor points	Clean, but prone to syntax errors	Hierarchical, clean, fully editable

3. How OmniSVG Learns to Draw

OmniSVG fine-tunes the Qwen2.5-VL Vision-Language Model using standard "next-token prediction"—the exact same technique LLMs use to write text, but applied to vector commands.

Input Context: The model receives the user's instructions (such as a text prompt like "a blue bird icon" or a reference image).
Sequential Prediction: Based on what it has drawn so far, the model predicts the single most likely next drawing command or coordinate token.
Training Objective: During training, the model is shown millions of completed vector graphs and adjusts its internal weights to minimize the difference between its predicted tokens and the actual target tokens.

4. Scalability: MMSVG-2M & MMSVG-Bench

To train OmniSVG, the authors created MMSVG-2M, a dataset containing 2 million high-quality SVG assets across three core categories:

MMSVG-Icon (1.1 Million Assets): Web icons, interface symbols, UI elements.
MMSVG-Illustration (0.5 Million Assets): Richly colored flat graphics, web banners, vector artwork.
MMSVG-Character (0.4 Million Assets): Detailed vector anime art and character designs.

   +-------------------------------------------------------------------+
   |                     MMSVG-2M Dataset Breakdown                    |
   |                                                                   |
   |   [ MMSVG-Icon ]          [ MMSVG-Illustration ]  [ Character ]   |
   |   1.1M Assets             0.5M Assets             0.4M Assets     |
   |   (Monochrome/UI)         (Rich Flat Graphics)    (Complex Anime) |
   +-------------------------------------------------------------------+

Benchmark Results (MMSVG-Bench)

OmniSVG was tested against existing tools across two main tasks: generating SVGs from text descriptions and converting raster images to SVGs.

Text-to-SVG Task

Visual Quality (Lower FID is better): OmniSVG scored 137.40, vastly outperforming older text models like VectorFusion (250.77) and Chat2SVG (190.87).
Text Alignment: OmniSVG maintained high prompt accuracy while generating outputs in less than 4,000 tokens, compared to optimization tools that required over 60,000 iterations.

Image-to-SVG Vectorization Task

Visual Similarity (Higher SSIM is better): OmniSVG achieved a structural similarity score of 0.950, matching expensive optimization tools like DiffVG (0.954) while generating clean, organized vector paths instead of messy shapes.
Accuracy vs. Large Models: OmniSVG drastically outperformed GPT-4o's direct SVG output (0.792 SSIM) and StarVector (0.881 SSIM) in vectorizing detailed images.

5. Tokenization Ablation Insights

To prove that combining coordinates and colors into custom tokens was the secret to success, the authors compared four model variants:

Configuration	Visual Error (Lower is better)	Output Accuracy (Higher is better)	Average Token Count
Raw XML Text (No Tokenization)	High Error (218.76 FID)	Low Accuracy (0.718 SSIM)	18,500 tokens
Without Coordinate Tokens	Medium Error (193.42 FID)	Medium Accuracy (0.809 SSIM)	10,200 tokens
Without Color Tokens	Low-Medium Error (167.28 FID)	Good Accuracy (0.879 SSIM)	6,300 tokens
OmniSVG (Full Tokenization)	Lowest Error (145.89 FID)	Highest Accuracy (0.928 SSIM)	4,800 tokens

Key Takeaway: Combining $X$ and $Y$ coordinates into single scalar tokens reduced file size and sequence lengths by 74% while making the resulting graphics twice as visually accurate.

6. Hands-On Example: Tokenizer Implementation

Below is a Python script showing how OmniSVG compresses 2D coordinates and drawing commands into discrete tokens:

import re

class OmniSVGTokenizer:
    def __init__(self, canvas_width=200, canvas_height=200):
        self.w = canvas_width
        self.h = canvas_height

        # Command mapped tokens
        self.cmd2token = {'M': '<M>', 'L': '<L>', 'C': '<C>', 'A': '<A>', 'Z': '<Z>', 'F': '<F>'}
        self.token2cmd = {v: k for k, v in self.cmd2token.items()}

    def encode_point(self, x: int, y: int) -> int:
        """Flattens 2D coordinates into a single 1D scalar token ID."""
        x = max(0, min(self.w - 1, x))
        y = max(0, min(self.h - 1, y))
        return (x * self.w) + y

    def decode_point(self, token_id: int) -> tuple[int, int]:
        """Restores 2D coordinates from a single 1D scalar token ID."""
        x = token_id // self.w
        y = token_id % self.w
        return x, y

    def tokenize_path(self, path_str: str, fill_hex: str = None) -> list:
        """Converts raw SVG path string into OmniSVG discrete tokens."""
        tokens = ["<SOP>"]

        if fill_hex:
            tokens.extend([self.cmd2token['F'], fill_hex])

        # Parse commands and coordinates using regex
        pattern = r'([MLCAZ])|(-?\d+\.?\d*)'
        matches = re.findall(pattern, path_str)

        current_cmd = None
        coords_buffer = []

        for cmd, num in matches:
            if cmd:
                if current_cmd and coords_buffer:
                    tokens.extend(self._process_coords(coords_buffer))
                    coords_buffer = []
                current_cmd = cmd
                tokens.append(self.cmd2token[cmd])
            elif num:
                coords_buffer.append(int(float(num)))

        if coords_buffer:
            tokens.extend(self._process_coords(coords_buffer))

        tokens.append("<EOS>")
        return tokens

    def _process_coords(self, coords: list) -> list:
        """Helper to pair raw coordinates into scalar tokens."""
        encoded = []
        for i in range(0, len(coords), 2):
            if i + 1 < len(coords):
                encoded.append(f"<COORD_{self.encode_point(coords[i], coords[i+1])}>")
        return encoded

# Example Usage
tokenizer = OmniSVGTokenizer(canvas_width=200, canvas_height=200)

raw_path = "M 10 20 C 30 40 50 60 70 80 Z"
fill_color = "#FF0000"

token_sequence = tokenizer.tokenize_path(raw_path, fill_hex=fill_color)
print("Tokenized Sequence:")
print(token_sequence)

# Demonstrate Coordinate Reconstruction
sample_token_id = tokenizer.encode_point(120, 45)
decoded_x, decoded_y = tokenizer.decode_point(sample_token_id)
print(f"\nCoordinate Check: Original (120, 45) -> Combined ID: {sample_token_id} -> Reconstructed: ({decoded_x}, {decoded_y})")

7. What's Next for Vector Graphics?

OmniSVG proves that AI vector generation works best when we treat geometry as a structured sequence of drawing actions rather than raw code text. By combining Vision-Language Models with smart spatial tokenization, we can generate clean, editable vector graphics in seconds instead of minutes.

How do you handle SVG vectorization in your current development workflows—do you rely on traditional image processing libraries, optimization scripts, or manual design tools? What challenges do you run into most with AI-generated SVG code? Share your thoughts in the comments below!

*AI agents write code fast. They also silently remove logic, change behavior, and introduce bugs -- without telling you. You often find out in production.

git-lrc fixes this. It hooks into git commit and reviews every diff before it lands. 60-second setup. Completely free.*

Any feedback or contributors are welcome! It's online, source-available, and ready for anyone to use.

HexmosTech / git-lrc

Free, Micro AI Code Reviews That Run on Git Commit

git-lrc

Free, Micro AI Code Reviews That Run on Commit

git-lrc is your braking system. It hooks into git commit and runs an AI review on every diff before it lands. 60-second setup. Completely free.

In short, git-lrc helps Prevent Outages, Breaches, and Technical Debt Before They Happen

At a glance: 10 risk categories · 100+ failure patterns tracked · every commit…

View on GitHub