DEV Community: Bharath Adithya

For anyone curious how Jev works, How LLMs Can Generate Structured JSON Without Generating It Token by Token

Bharath Adithya — Tue, 22 Sep 2026 10:27:39 +0000

{
  "risk_level": "HIGH",
  "requires_review": true,
  "action_tier": "TIER_2"
}

Looks simple.

But a standard autoregressive LLM typically generates that response token by token.

{
   ↓
"risk_level"
   ↓
:
   ↓
"HIGH"
   ↓
,
   ↓
"requires_review"
   ↓
:
   ↓
true
   ↓
...

Every new token depends on the previous context.

But what if the schema is already known?

What if we already know that:

risk_level     → HIGH | MEDIUM | LOW | NONE

requires_review → true | false

action_tier    → TIER_1 | TIER_2 | TIER_3

Then generating every token of the JSON from scratch starts to look unnecessary.

This is where an interesting approach to structured LLM inference comes in.

🧠 The Core Idea

Instead of asking the LLM to generate the complete JSON sequence, we can treat the problem more like classification over constrained choices.

The model receives:

Context + JSON Schema

and the schema tells us what values are actually allowed.

For example:

risk_level
├── HIGH
├── MEDIUM
├── LOW
└── NONE

requires_review
├── true
└── false

action_tier
├── TIER_1
├── TIER_2
└── TIER_3

Now the model doesn't have to invent the structure.

It only needs to determine:

Which valid value is most likely for each field?

⚡ Step 1: Prefill the Context and Schema

The first step is to process the context and JSON schema through the Transformer.

Conceptually:

Context + Schema
       ↓
   Transformer
       ↓
    KV Cache

This is the prefill phase.

During this phase, the Transformer processes the input tokens and stores their keys and values in the KV cache.

The important part is that this information can be reused later.

We don't want to repeatedly process the same context if we don't have to.

🧩 Step 2: Reuse the KV Cache

Now consider the individual fields.

For example:

risk_level
requires_review
action_tier

For each field, we can reuse the cached context and process the field-specific suffix.

Conceptually:

             KV Cache
                 │
       ┌─────────┼─────────┐
       ↓         ↓         ↓
  risk_level  requires_  action_tier
              review
       ↓         ↓         ↓
     Scores    Scores    Scores
       ↓         ↓         ↓
     HIGH      true      TIER_2

Instead of repeatedly starting from the entire context, the model can reuse information that has already been computed.

🎯 Step 3: Look Only at Valid Candidates

A language model normally produces logits across its entire vocabulary.

That could mean tens of thousands of possible tokens.

But for:

risk_level

we only care about:

HIGH
MEDIUM
LOW
NONE

So why consider everything else?

We can focus on the candidate tokens relevant to that field.

For example:

Full vocabulary
      ↓
Filter candidates
      ↓
HIGH
MEDIUM
LOW
NONE

Then the candidate scores can be converted into probabilities.

Example:

HIGH      → 0.9924
MEDIUM    → 0.0068
LOW       → 0.0006
NONE      → 0.0002

The highest-probability valid candidate becomes the prediction:

HIGH

🔥 From Token Generation to Structured Prediction

This changes the way we think about the task.

Traditional LLM generation:

Prompt
  ↓
Token
  ↓
Token
  ↓
Token
  ↓
Token
  ↓
Valid JSON

Structured prediction:

Context + Schema
       ↓
    KV Cache
       ↓
 ┌─────┼─────┐
 ↓     ↓     ↓
 F1    F2    F3
 ↓     ↓     ↓
Score Score Score
 ↓     ↓     ↓
Value Value Value
 └─────┼─────┘
       ↓
   Valid JSON

The model is no longer responsible for inventing every bracket, comma, quote, key, and value.

The application already knows the structure.

The model focuses on the semantic decision.

🛡️ Another Big Advantage: Valid Structure

One common problem with LLM-generated JSON is that the model can produce something like:

{
  "risk_level": "VERY_HIGH",
  "requires_review": "maybe"
}

But our schema might only allow:

risk_level
→ HIGH | MEDIUM | LOW | NONE

requires_review
→ true | false

With constrained structured prediction, the candidate space itself can be restricted.

That means the system isn't simply asking:

"Please generate valid JSON."

It is enforcing the allowed structure around the model's prediction.

Conceptually:

LLM Vocabulary
      ↓
Schema Constraints
      ↓
Valid Candidates
      ↓
Probability Scores
      ↓
Selected Values
      ↓
Structured JSON

This makes the output much more predictable.

🏗️ The Architecture

A simplified architecture looks like this:

                 Input Document
                       │
                       ▼
                Context + Schema
                       │
                       ▼
               Transformer Decoder
                       │
                       ▼
                    KV Cache
                       │
          ┌────────────┼────────────┐
          ▼            ▼            ▼
      Field 1       Field 2      Field 3
          │            │            │
          ▼            ▼            ▼
    Candidates    Candidates    Candidates
          │            │            │
          ▼            ▼            ▼
       Logits        Logits        Logits
          │            │            │
          ▼            ▼            ▼
     Probabilities Probabilities Probabilities
          │            │            │
          └────────────┼────────────┘
                       ▼
                 Structured JSON

The interesting part is the shared cached context.

🚀 Why KV Caching Matters

Imagine processing a large document.

The model might need to extract:

category
priority
risk_level
requires_review
action
confidence

If the same context is repeatedly processed from scratch, inference becomes unnecessarily expensive.

With KV caching:

Context
   ↓
Transformer
   ↓
KV Cache
   ↓
Reuse
 ┌─┼─┼─┼─┼─┐
 ↓ ↓ ↓ ↓ ↓ ↓
F1 F2 F3 F4 F5 F6

The expensive shared computation can be reused instead of recalculated for every field.

This is one of the reasons KV caching is such an important concept in LLM inference engineering.

📊 Where This Becomes Useful

This approach becomes especially interesting when your output schema is predictable.

Document Intelligence

{
  "document_type": "INVOICE",
  "priority": "HIGH",
  "requires_review": true
}

Resume Analysis

{
  "experience_level": "ENTRY",
  "skill_match": "HIGH",
  "recommendation": "SHORTLIST"
}

AI Agents

{
  "tool": "SEARCH",
  "priority": "HIGH",
  "requires_confirmation": false
}

Classification Pipelines

{
  "category": "TECHNICAL",
  "sentiment": "POSITIVE",
  "risk": "LOW"
}

In all of these cases, the possible values can be known before inference begins.

🧠 The Bigger AI Engineering Lesson

This idea made me think about something important.

When building LLM applications, we often ask:

How can I make the model generate better?

But another question is:

How can I make the model generate less?

Those are very different optimization strategies.

Instead of:

Generate everything
       ↓
Parse JSON
       ↓
Validate
       ↓
Fix errors

we can move toward:

Define schema
       ↓
Constrain candidates
       ↓
Reuse cached context
       ↓
Score valid choices
       ↓
Build structured output

The model handles the intelligence.

The system handles the structure.

⚙️ Traditional Generation vs Structured Inference

Traditional

Prompt
  ↓
LLM
  ↓
Token 1
  ↓
Token 2
  ↓
Token 3
  ↓
...
  ↓
JSON
  ↓
Validation

Structured

Context + Schema
       ↓
    Prefill
       ↓
    KV Cache
       ↓
Candidate Scoring
       ↓
Constrained Values
       ↓
Structured JSON

The second approach shifts more responsibility from free-form generation to the inference and application layer.

🔬 The Part I Find Most Interesting

The fascinating thing isn't just JSON.

It's the underlying idea:

A language model doesn't always need to behave like a text generator.

For some problems, it can behave more like a probabilistic decision engine operating inside a constrained space.

That opens up interesting possibilities around:

constrained decoding
structured generation
KV-cache reuse
inference optimization
efficient batching
model serving
structured prediction
AI agent orchestration

These are the kinds of details that become increasingly important when moving from simply using an LLM to actually engineering LLM systems.

💡 One Mental Model

The simplest way I'm thinking about it now is:

Traditional LLM

Understand
    ↓
Generate
    ↓
Generate
    ↓
Generate
    ↓
Validate

versus:

Structured LLM Inference

Understand
    ↓
Reuse Context
    ↓
Restrict Choices
    ↓
Score Candidates
    ↓
Select
    ↓
Build Valid Output

The model doesn't have to do everything.

Good AI engineering is often about deciding what the model should do — and what the system should handle instead.

🔥 Final Thought

LLM inference is much more than:

Prompt → Model → Response

Under the hood, concepts like:

Transformers → attention → KV cache → logits → softmax → constrained decoding → structured outputs

all contribute to how efficiently and reliably an AI system can operate.

The more I explore this layer, the more I realize that building AI applications isn't only about choosing a powerful model.

It's also about designing the inference process intelligently.

What other LLM inference optimization should I explore next? 👇

AI #LLM #MachineLearning #GenerativeAI

# 10 Git Commands That Made Me More Confident With Git

Bharath Adithya — Sat, 29 Aug 2026 20:06:16 +0000

If you've used Git for a while, you probably know the basics:

git add
git commit
git pull
git push

These commands are enough to get started.

But as your projects become bigger and you start working with branches, pull requests, conflicts, and other developers, Git starts getting a little more interesting.

Here are 10 Git commands and techniques that I found especially useful for moving beyond the basics.

1. `git merge`

Let's say you have a feature branch and the main branch has received some new changes.

You can bring those changes into your current branch using:

git merge main

A simplified history might look like:

A---B---C main
     \
      D---E feature

After merging:

A---B---C------M
     \        /
      D---E---

M is the merge commit.

When is merge useful?

Use merge when you want to preserve the branch history and clearly show that two lines of development were combined.

2. `git rebase`

Rebase takes a different approach.

Instead of creating a merge commit, Git moves your commits and reapplies them on top of another branch.

git rebase main

The previous history:

A---B---C main
     \
      D---E feature

can become:

A---B---C---D'---E'

The result is often easier to read.

But there's an important thing to remember:

Rebase rewrites commit history.

So a simple rule is:

Rebase your own local work carefully, especially before sharing it with others.

If you're rebasing a shared branch, you can create unnecessary problems for teammates.

3. `git commit --amend`

Ever made a commit and immediately realized you forgot one small change?

Instead of creating another unnecessary commit, you can add the change to the previous commit.

git add forgotten-file.js
git commit --amend --no-edit

The --no-edit option keeps your previous commit message.

If you also want to change the message:

git commit --amend

Important

Amending creates a new commit, which means the commit hash changes.

So this is safest when the commit hasn't already been shared with other people.

4. `git push --force-with-lease`

Sometimes you rewrite your local history using:

rebase
amend
interactive rebase
squash

Then a normal push may be rejected.

You might be tempted to use:

git push --force

But there's a safer option:

git push --force-with-lease

The difference is important.

--force basically says:

Replace the remote branch with my version.

--force-with-lease checks whether the remote branch has changed before replacing it.

That makes it a much safer choice when you actually need to force-push.

5. `git rebase -i`

This is one of my favorite Git commands for cleaning up commits.

git rebase -i HEAD~5

This lets you interact with your last five commits.

You'll see something similar to:

pick a111111 Add login page
pick b222222 Add validation
pick c333333 Fix typo
pick d444444 Fix validation
pick e555555 Remove debug code

You can change pick to commands such as:

`pick`

Keep the commit as it is.

`reword`

Change only the commit message.

`edit`

Pause at that commit so you can modify it.

`squash`

Combine the commit with the previous one while keeping its commit message available for editing.

`fixup`

Combine the commit with the previous one and discard its commit message.

`drop`

Remove the commit.

For example:

pick   a111111 Add login page
fixup  b222222 Fix typo
fixup  c333333 Remove debug code

can turn several messy commits into a much cleaner history.

6. `git stash`

Imagine you're working on a feature and suddenly need to switch to another task.

Your current code isn't ready to commit yet.

That's where git stash helps.

git stash

Git temporarily stores your uncommitted changes and gives you a clean working directory.

You can then switch branches:

git switch main

Later, you can bring your changes back.

Include untracked files

If you've also created new files:

git stash -u

You can even give your stash a useful name:

git stash push -u -m "WIP authentication feature"

To see your stashes:

git stash list

To restore one:

git stash apply stash@{0}

Or:

git stash pop

The simple difference:

apply = restore the changes
pop   = restore + remove the stash

7. `git cherry-pick`

Sometimes you don't need an entire branch.

You only need one specific commit.

That's where cherry-pick comes in:

git cherry-pick <commit-hash>

For example:

git cherry-pick a1b2c3d

Git takes the changes from that commit and applies them to your current branch.

This is useful when:

A bug fix exists on another branch
You need one specific feature
You don't want to merge the entire branch

One important detail:

Cherry-pick creates a new commit with a different hash.

8. `git reset`

git reset is useful when you want to move your branch backwards.

There are three common versions:

git reset --soft HEAD~1

git reset --mixed HEAD~1

git reset --hard HEAD~1

The easiest way to remember them:

--soft  → keep changes staged
--mixed → keep changes unstaged
--hard  → discard the changes

For example:

Soft reset

git reset --soft HEAD~1

Removes the last commit but keeps its changes staged.

Mixed reset

git reset HEAD~1

Removes the last commit and keeps the changes in your working directory.

Hard reset

git reset --hard HEAD~1

Moves the branch backwards and removes the changes from the working tree.

⚠️ Be careful with --hard.

Make sure you understand what you're about to discard before using it.

9. `git reflog`

This command can be extremely useful when you think you've lost a commit.

git reflog

While git log shows your reachable commit history, reflog records movements of references such as HEAD.

You might see:

e35fa12 HEAD@{0}: reset: moving to HEAD~2
821cd77 HEAD@{1}: commit: Add authentication
f992ab1 HEAD@{2}: commit: Add login page

That previously "lost" commit may still be accessible.

Instead of immediately resetting everything again, you can create a recovery branch:

git branch rescue 821cd77

Then inspect it safely.

Remember

reflog isn't a magic backup.

If your work was never committed or otherwise stored by Git, reflog cannot recover it.

10. `git revert`

Here's an important situation.

You pushed a commit to a shared branch, and that commit introduced a problem.

Should you rewrite the branch history?

Usually, no.

Instead, use:

git revert <commit-hash>

Suppose your history is:

A---B---C

And C introduced the problem.

Running:

git revert C

creates another commit:

A---B---C---D

D contains the opposite changes needed to undo C.

The original commit remains visible in history.

That's why revert is generally a better choice for undoing changes on shared branches.

Quick Cheat Sheet

Command	Main purpose
`git merge`	Combine branch histories
`git rebase`	Reapply commits on top of another branch
`git commit --amend`	Modify the previous commit
`git push --force-with-lease`	Safer force push
`git rebase -i`	Clean up recent commits
`git stash`	Temporarily save unfinished changes
`git cherry-pick`	Apply one specific commit
`git reset`	Move branch history backwards
`git reflog`	Find previous HEAD/reference states
`git revert`	Safely undo a shared commit

Final Thoughts

Git becomes much easier once you stop thinking of it as just a collection of commands and start understanding what happens to your repository history after each command.

If you're currently learning Git, I'd recommend becoming comfortable with these first:

git stash
git merge
git rebase
git revert
git reset
git reflog

Then gradually add:

git rebase -i
git cherry-pick
git commit --amend
git push --force-with-lease

The more you understand Git's history, the less scary those inevitable conflicts and mistakes become.

Which Git command has saved you the most time? 👇

git #beginners #programming #webdev

DEV Community: Bharath Adithya

For anyone curious how Jev works, How LLMs Can Generate Structured JSON Without Generating It Token by Token

🧠 The Core Idea

⚡ Step 1: Prefill the Context and Schema

🧩 Step 2: Reuse the KV Cache

🎯 Step 3: Look Only at Valid Candidates

🔥 From Token Generation to Structured Prediction

🛡️ Another Big Advantage: Valid Structure

🏗️ The Architecture

🚀 Why KV Caching Matters

📊 Where This Becomes Useful

Document Intelligence

Resume Analysis

AI Agents

Classification Pipelines

🧠 The Bigger AI Engineering Lesson

⚙️ Traditional Generation vs Structured Inference

Traditional

Structured

🔬 The Part I Find Most Interesting

💡 One Mental Model

🔥 Final Thought

AI #LLM #MachineLearning #GenerativeAI

# 10 Git Commands That Made Me More Confident With Git

1. git merge

When is merge useful?

2. git rebase

3. git commit --amend

Important

4. git push --force-with-lease

5. git rebase -i

pick

reword

edit

squash

fixup

drop

6. git stash

Include untracked files

7. git cherry-pick

8. git reset

Soft reset

Mixed reset

Hard reset

9. git reflog

Remember

10. git revert

Quick Cheat Sheet

Final Thoughts

git #beginners #programming #webdev

1. `git merge`

2. `git rebase`

3. `git commit --amend`

4. `git push --force-with-lease`

5. `git rebase -i`

`pick`

`reword`

`edit`

`squash`

`fixup`

`drop`

6. `git stash`

7. `git cherry-pick`

8. `git reset`

9. `git reflog`

10. `git revert`