DEV Community: Abhishek

Why AI Coding Tools Still Waste Tokens (And How Context Engineering Can Fix It)

Abhishek — Sat, 25 Jul 2026 05:16:29 +0000

Every AI coding assistant promises the same thing:

"Write code faster."

Yet after months of using Claude, ChatGPT, Gemini, Cursor, Windsurf, and other coding assistants, I noticed something frustrating.

The biggest bottleneck isn't model intelligence anymore.

It's context.

The Real Problem

Large language models don't understand your project.

They only understand the context you give them.

That means every prompt competes for a limited context window.

A typical workflow looks like this:

Paste the error.
Paste three files.
Paste another file because the first wasn't enough.
Explain the project structure.
Explain the API.
Explain previous attempts.
Finally ask the question.

By the time you're done, you've already spent thousands of tokens before the model even starts solving the problem.

Even worse, the model often asks:

"Can you also provide the implementation of..."

And the cycle repeats.

Bigger Models Aren't the Solution

Many people assume larger context windows solve this.

Not really.

A 1M-token context is impressive, but most projects don't need more context.

They need the right context.

Giving an AI your entire repository is like asking a developer to read every file before fixing a CSS bug.

It's possible.

It's just incredibly inefficient.

Context Is Becoming an Engineering Problem

We're entering an era where developers won't just write prompts.

They'll engineer context.

That means deciding:

Which files matter?
Which functions are actually related?
What dependencies should be included?
What history is still relevant?
What can safely be removed?

The AI should receive only the information required to solve the current task.

Nothing more.

Nothing less.

Imagine This Workflow

Instead of manually attaching files:

Fix authentication bug.

The tool automatically discovers:

✓ Relevant API routes

✓ Authentication middleware

✓ Token validation logic

✓ User model

✓ Environment configuration

✓ Recent Git changes affecting auth

The model receives a concise, structured package instead of hundreds of irrelevant files.

Same repository.

Much smaller prompt.

Better answers.

Lower cost.

Why This Matters

Token usage is becoming one of the hidden costs of AI development.

More context means:

Higher latency
More expensive requests
Lower signal-to-noise ratio
Greater chance of the model getting distracted

Reducing unnecessary context doesn't just save money.

It often improves output quality.

The Future

I think the next generation of AI developer tools won't compete primarily on which model they use.

They'll compete on how intelligently they build context.

The winning tools will know:

what to include,
what to exclude,
and why.

Model quality will always matter.

But context quality may become the bigger differentiator.

Final Thoughts

Prompt engineering helped us communicate with AI.

Context engineering will help AI understand our software.

And that shift could be one of the biggest productivity improvements we'll see in developer tooling over the next few years.

What do you think?

Is context the real bottleneck, or do larger models make this problem disappear?

I Built ChunkWiser: A Tool That Understands Large Codebases Without Hallucinating

Abhishek — Sun, 19 Jul 2026 07:24:14 +0000

I Built ChunkWiser: An AI That Understands Large Codebases Without Hallucinating

Large repositories break most AI coding assistants. I built ChunkWiser to solve that problem.

The Problem

Modern AI models are excellent at writing code, but they struggle when you ask them to understand an entire repository.

As projects grow larger, developers run into familiar issues:

Context windows are limited.
Important files are left out.
AI loses relationships between modules.
Responses become inconsistent or hallucinated.

I experienced this firsthand while working on larger codebases. Every time I asked an AI assistant about a project, it either forgot earlier files, invented functions that didn't exist, or completely misunderstood the architecture.

There had to be a better approach.

That's why I started building ChunkWiser.

What is ChunkWiser?

ChunkWiser is an AI-powered code understanding platform that indexes entire repositories into semantic knowledge.

Instead of sending thousands of files directly to an LLM, ChunkWiser first understands the repository, retrieves only the relevant context, and then asks the model to answer using that information.

The result is significantly better repository understanding while using fewer tokens.

How It Works

The workflow looks like this:

Git Repository
      │
      ▼
Repository Scanner
      │
      ▼
Smart Code Chunking
      │
      ▼
Generate Embeddings
      │
      ▼
Vector Database
      │
      ▼
Semantic Retrieval
      │
      ▼
LLM
      │
      ▼
Accurate Answer

Rather than asking the model to memorize everything, ChunkWiser retrieves only the parts of the repository that are actually relevant to the user's question.

Features

Current features include:

Repository indexing
Smart semantic chunking
Vector search
AI repository summaries
Cross-file understanding
Documentation generation
Code explanations
Bug analysis
Repository review
Fast semantic search

More features are planned as the project evolves.

Tech Stack

Frontend

React
Vite
Monaco Editor

Backend

FastAPI
PostgreSQL
Redis
Qdrant

AI

Embeddings
Retrieval-Augmented Generation (RAG)
Gemini
OpenRouter

Infrastructure

Docker
Google Cloud Run

Biggest Challenge

One of the hardest problems wasn't building the AI.

It was deciding how code should be chunked.

If chunks are too small:

AI loses context.

If chunks are too large:

Token usage increases.
Retrieval quality decreases.

Finding the right balance required experimenting with different chunk sizes, overlaps, and retrieval strategies.

That process taught me that retrieval quality often matters more than the language model itself.

Why Semantic Chunking Matters

Traditional chunking usually splits code by a fixed number of characters or lines.

That approach ignores the actual structure of the code.

ChunkWiser instead focuses on keeping meaningful sections together whenever possible, helping preserve context during retrieval.

Better chunks lead to better embeddings.

Better embeddings lead to better retrieval.

Better retrieval leads to better answers.

Current Roadmap

I'm actively working on:

Better semantic chunking
Dependency graph understanding
Multi-language support
Repository memory
Faster indexing
Smarter retrieval
Improved developer experience

Why I'm Building in Public

One of my goals is to document everything I learn while building ChunkWiser.

Over the coming weeks I'll be writing about:

Building RAG for source code
Embedding strategies
Vector databases
Chunking techniques
AI developer tools
Lessons learned
Scaling repository understanding

Hopefully these articles help others building similar tools.

I'd Love Your Feedback

ChunkWiser is still evolving, and there are many improvements ahead.

If you've built developer tools, worked with RAG systems, or have ideas for improving repository understanding, I'd love to hear your thoughts.

Feedback, suggestions, and discussions are always welcome.

Thanks for reading!

If you enjoyed this article, consider following me for more posts about:

AI Engineering
RAG
Developer Tools
FastAPI
React
LLM Applications
Open Source

More technical deep dives on ChunkWiser are coming soon.