DEV Community

Cover image for We Built an AI Coding Assistant That Refuses to Finish Your Code
10XScale profile image Mothilal M
Mothilal M for 10XScale

Posted on

We Built an AI Coding Assistant That Refuses to Finish Your Code

#ai #opensource #vscode #python

A student finished our twelve-week backend course with near-perfect assignment scores.

In a mock interview two days later, he couldn't write a loop that summed a list of numbers. Not "wrote it inefficiently" — couldn't start. He sat there waiting for something to autocomplete.

He hadn't cheated. He'd used ChatGPT and Copilot exactly the way we all do at work: describe the problem, accept the suggestion, move on. Twelve weeks of that. Every assignment passed. Nothing learned.

That's when it clicked for us. For a working developer, an AI that finishes your code is a productivity tool. For a student, it's a machine that removes the struggle — and the struggle was the entire point.

So we built the opposite.

The rule: do the plumbing, stop at the lesson

Every coding exercise has two kinds of code in it.

There's the part that is the lesson — the recursion, the reducer, the SQL join, the async coordination. Whatever the module is actually teaching.

And there's everything else. Imports. Reading the CSV. The if __name__ == "__main__" block. Wiring up the router. Error handling around the bit that matters. Boilerplate the student has already done fifteen times, or won't be tested on, or that's just noise between them and the concept.

A tutor who does the second kind for you is helpful. A tutor who does the first kind for you is stealing your education.

So that became the rule our assistant follows:

Write the scaffolding in full. Stop dead at the learning-critical part. Say what you did and what they must finish.

In practice a response looks like this:

import csv
from collections import defaultdict

def generate_report(csv_filepath: str):
    aggregated = defaultdict(int)

    try:
        with open(csv_filepath, newline="", encoding="utf-8") as f:
            reader = csv.DictReader(f)
            for row in reader:
                # TODO(student): Implement the core aggregation logic here.
                # Each `row` is a dict like {'Category': 'A', 'Value': '100'}.
                # Update `aggregated` based on what you're reporting.
                pass

        for key, value in aggregated.items():
            print(f"{key}: {value}")

    except FileNotFoundError:
        print(f"Error: '{csv_filepath}' not found.")
Enter fullscreen mode Exit fullscreen mode

I've scaffolded the CSV reader and the output formatting. You need to implement the aggregation inside the loop.

The file opens in their editor. It runs. It just doesn't do anything yet — and the one thing standing between "runs" and "works" is precisely the thing the module is teaching.

Building it on AgentFlow

We built this on AgentFlow, our own Python framework for AI agents. It's MIT licensed and on PyPI:

pip install 10xscale-agentflow
Enter fullscreen mode Exit fullscreen mode

Why not LangGraph? We started there. We like the graph mental model — that part AgentFlow keeps. What we kept rebuilding by hand was everything after the graph compiled: an API server, streaming that didn't fall over, persistence that survived a restart, a TypeScript client for the frontend. And we didn't want LangChain in the dependency tree.

So AgentFlow ships the production stack in the box: graph orchestration, a REST + SSE server, a TypeScript SDK, three-layer memory (Redis cache, Postgres, vector store), native MCP, and parallel tool execution. It's LLM-agnostic — OpenAI, Gemini, Anthropic, or any OpenAI-compatible endpoint.

The tutor itself is a small graph:

from agentflow.graph import StateGraph

graph = StateGraph(TutorState(), container=container)
graph.add_node("tutor", tutor_agent)
graph.add_node("tools", tool_node)
graph.set_entry_point("tutor")

graph.add_conditional_edges("tutor", should_continue, {"tools": "tools", END: END})
graph.add_conditional_edges("tools", route_after_tools, {"tutor": "tutor"})

app = graph.compile(checkpointer=checkpointer, store=store)
Enter fullscreen mode Exit fullscreen mode

The checkpointer is the part that matters more than it looks. Conversation threads live in Postgres, so a student can close the laptop mid-exercise, come back the next evening, and the tutor still knows which milestone they're stuck on and what they already tried. No re-explaining.

The interesting bit: the agent's hands are in your editor

Here's the architectural problem. The agent runs on our server. The student's code lives in their editor. How does a model on a server read a selection, or write a file, or highlight the three lines where the bug is?

AgentFlow solves this with remote tools, and it's my favourite feature in the framework.

You register a tool from the client — in our case, the VS Code extension:

client.registerTool({
  node: 'tools',
  name: 'read_code',
  description: "Read the student's code. Use want='selection' for what they've " +
               "highlighted, 'error_context' for the file plus last terminal output.",
  parameters: {
    type: 'object',
    properties: {
      want: { type: 'string', enum: ['file', 'selection', 'range', 'error_context'] },
    },
    required: ['want'],
  },
  handler: async (args) => {
    // runs inside VS Code, with the full editor API available
    const editor = vscode.window.activeTextEditor;
    return editor.document.getText(editor.selection);
  },
});

await client.setup();   // publishes the schemas to the server
Enter fullscreen mode Exit fullscreen mode

The model sees the tool schema and calls it like any other tool. But the server doesn't execute it — it returns a RemoteToolCallBlock, the SDK spots it, runs your handler in the editor, and sends the result back as a tool message. The graph picks up exactly where it left off.

The model gets eyes and hands inside VS Code without a single line of code leaving the student's machine unless the agent explicitly asks for it. No shipping the whole file on every message. No stale context.

The same mechanism drives the writing side — write_file, highlight, open_file, propose_edit. The agent decides what should happen; the editor decides how.

The pedagogy is a markdown file, not an if-statement

The thing I'd most want another team to steal: none of the teaching rules live in Python.

AgentFlow has a skills system — playbooks in markdown, loaded per turn based on what the student is doing. Ours reads like guidance to a human tutor:

## The Hardest-Part Boundary

Learning-critical core -> coach, never solve.
Boilerplate and plumbing -> write it freely.

Milestone is "fetch and display data"?
  - Stuck on the fetch -> that IS the lesson. Scaffold around it, hand it back.
  - Stuck on the loading spinner CSS -> not the lesson. Just write it.

## Understanding Gate

Before marking anything complete, ask WHY they chose their approach.
Ask what happens at [edge case]. Passing tests is not understanding.
Enter fullscreen mode Exit fullscreen mode

Our instructional designers edit that file. They don't open a Python file, they don't wait for a deploy, they don't file a ticket. When a cohort struggles with a module, the person who understands why changes the rules directly.

That separation — orchestration in code, pedagogy in markdown — has been worth more than any individual feature.

Measuring who actually wrote the code

Here's the part I think most AI-in-education tools are missing.

If your AI can write code into a student's file, you have created a brand new way to cheat, and you have an obligation to measure it. Otherwise "completed the assignment" becomes meaningless.

So every AI-originated edit in our extension goes through exactly one function, and that function records the event before it writes:

async function applyBlock(block) {
  record('ai_applied', block.file, block.code.length);   // ledger first
  await vscode.workspace.fs.writeFile(target, buffer);   // then the write
}
Enter fullscreen mode Exit fullscreen mode

Student keystrokes get counted from onDidChangeTextDocument. The ratio between them is displayed live in the panel header — a small typed 78% badge that both the student and their trainer can see.

Two hard-won lessons on this:

It has to be a single choke point. The moment one code path writes to a file without going through it, the number silently becomes fiction — and a fiction that looks like data is worse than no data.

Build it on day one. You cannot retrofit authorship tracking. The edits you didn't record are gone forever.

A high AI-ratio isn't automatically bad, by the way. It's a signal. It tells the tutor to probe harder before marking a milestone complete, and it tells a trainer which students to check on before the mock interview, not after.

What we've seen so far

[FILL IN — replace this section with your own numbers before publishing.]
Good things to include if you have them: how many students are using it, change in
mock-interview pass rate, average authorship ratio per module, or a direct student
quote. If you don't have hard data yet, say so plainly — "we're one cohort in and
still collecting data" is more credible than a number you can't defend, and readers
on dev.to will absolutely ask.

Try it

AgentFlow is open source and MIT licensed:

If you're building anything that puts an AI inside an editor, the remote-tools pattern is worth an hour of your time even if you use a different framework entirely. Being able to run a tool handler in the client while the graph stays on the server unlocks a category of product that's genuinely painful to build otherwise.

The question I keep going back and forth on: should a student be able to override the tutor and just get the full answer? There's a real argument that fighting a tool teaches nothing but resentment, and adults should get to choose. There's an equally real argument that the override becomes the default within a week.

We landed on "no override, but unlimited explanations." I'm genuinely not certain we got it right. What would you have done?

Top comments (0)