I developed an autonomous coding agent designed to solve challenging programming tasks in a competitive environment. Rather than relying on an LLM to generate code in a single pass, I built a complete workflow combining repository analysis, planning, implementation, execution, testing, and iterative refinement.
The agent’s workflow:
Task → Analyze → Plan → Implement → Execute → Test → Diagnose → Refine → Retest → Submit
The feedback loop is where the real improvement happens. When a test fails, the agent analyzes the error, identifies the likely cause, updates the code, and reruns the relevant tests. This helps catch logic errors and edge cases before submission instead of accepting the first plausible-looking solution.
Performance comparison
I evaluate the agent with and without iterative feedback using the same benchmark tasks and execution limits:
Task success rate: [baseline]% → [optimized]%
Tests passed per task: [baseline] → [optimized]
Average completion time: [baseline] → [optimized]
Compute cost per successful task: [baseline] → [optimized]
These measurements help show whether each optimization improves correctness and reliability without introducing excessive latency or compute overhead.
Beyond the feedback loop, I optimize model selection, prompt design, repository context, tool usage, inference settings, and failure recovery. The goal is to build an agent that doesn't just generate code, but uses execution results as evidence to improve its solutions systematically.
Top comments (11)
The feedback loop focus is spot on. Single-pass LLM generation is basically a trap, and active test-diagnose-refine passes are the only way to catch silent logic edge cases before shipping.
Building multimodal AGI architectures and autonomous agents from scratch at AGI Systems Directorate (where I'm Founder & CEO, working on our agent Project Lumina), the biggest headache with iterative loops like this is always context state corruption and compute latency stacking up during re-tests.
Quick question on your setup: are you running deterministic local execution sandboxes for the diagnostic phase, or letting the agent dynamically alter its tool suite based on error outputs? Would love to know what stack you're running this on.
Also AI agents hallucination is also one of the problems
Excellent point. AI hallucinations are another challenge and a problem that becomes a headache for anyone.
This is especially true when agents misinterpret error messages or assume a solution is correct without sufficient evidence. I believe combining sandbox execution, test-based feedback, and verification at each step can help reduce these issues while controlling context and latency.
When I said context, to be precise, I meant memory management.
While there may not be a silver bullet for these issues, in my experience, using a dedicated memory management product as an MCP server with a plugin also seems like a good option.
Sorry for the late reply here, completely got buried in CEO work at AGI Systems Directorate.
Spot on regarding memory management and error misinterpretation. When you have multi-turn loops where the agent starts gaslighting itself over a bad stack trace, hallucination spirals get out of hand real quick. We run into that exact friction building our multimodal agents and autonomous systems on Project Lumina.
On your verification phase: are you handling memory pruning dynamically per turn, or just dumping history into a structured context window before it hits the diagnostic stage?
Also mostly I work in artificial general intelligence but what are you working on right now
Pruning is a very important issue in memory management.
In my experience, I prune when a specific deliverable is being generated, especially when version 1.0 of the deliverable is complete and there's an intention to develop the context for moving to version 2.0 in parallel while serial production is ongoing.
Memory dumps are generally not used.
Recently, I have been focusing on automating AI systems.
Not gonna lie, using Dev.to usually feels like shouting into an empty void where new accounts get zero engagement and people just treat it like static documentation or a diary. Running into an actual technical back-and-forth here instead of total silence is a nice plot twist. Honestly might have to look into collaborating on some agent infra down the line if we keep trading notes like this.
Anyway, getting back to the technical side. That take on pruning right when version 1.0 locks down and spinning up context for version 2.0 in parallel is a solid approach. Handling that state separation without the background worker bleeding into the active execution path is usually where things go sideways. How are you isolating those branch states when version 2.0 starts spinning up while v1 is still chewing through tasks?
Also in a different note. I am working on project lumina autonomous AI agent was stop from scratch no existing frameworks . It is the large language model and autonomous AI agent with biological mechanism for artificial general intelligence for achieving it
We are adding safeguards to maintain stable synchronization. Each version has an independent state repository, and v2.0 starts from a consistent checkpoint instead of reading v1.0's live mutable state. We will prevent stale updates from overwriting the latest state by using concurrency control with checks. All state transitions are recorded in an append-only event log, which includes checkpoints for post-crash recovery.
I'm sorry, but are you also conducting LLM training research?
To answer your question directly: undoubtedly yes. We handle the whole pipeline from initial problem scoping all the way to deployment and beyond, and we are deep into LLM training, model development, and practical execution. When you're managing training runs alongside agentic loops, keeping infrastructure stable without gradient updates breaking the state machine is a whole separate headache.
Since we've been trading technical notes back and forth for a bit, let me finally give you my proper intro. I'm the Founder and CEO of AGI Systems Directorate and a full-stack AI developer. Most of my work is focused on artificial general intelligence, specifically integrating biological mechanisms with AI architectures. Right now, I'm building Project Lumina from scratch as an autonomous AI agent without using any existing off-the-shelf frameworks.
Not gonna lie, juggling deep research with practical execution as a founder gets pretty chaotic, but that's just part of the grind. Curious on your end—when you're pushing updates across your agent setups, how are you handling regression testing so a new patch doesn't silently break downstream inference?
I think later we can collaborate if you don't mind
Regression testing issues are a very important problem that exists everywhere in software development. I also have not found an effective method and am currently researching it. The method I tried is as follows.
It is to design regression tests by layer (unit test + integration test).
The evaluation benchmark is strictly fixed. Regression testing itself causes an increase in entropy, and if you change the benchmark as well, the results will be uncontrollable.
I am sorry that there is nothing more to elaborate on regarding this issue.
Although I haven't done much model training research, Project Lumina sounds very interesting! May I ask a few questions?
Beyond the research aspect, what practical applications are you initially targeting, and what are your plans for commercialization? Are you envisioning a paid API or an enterprise product? What are the fundamental differences compared to commercial models like GPT?
Ah, again I am late to reply. Not just late, actually very late, been buried deep in execution.
On your point about regression testing and entropy spikes, layering unit and integration tests is a solid baseline, but once agents start mutating state dynamically across multi-turn loops, static test suites start missing emergent failure modes. That friction is hard to escape.
To answer your questions on commercialization, we are actually doing both. We are deploying enterprise applications and products alongside robust developer APIs, featuring both free and paid tiers so it is accessible across the board.
As for how Lumina differs from standard commercial models like GPT, the fundamental gap comes down to the architecture. GPT models are just text predictors running on raw probability distributions and attention heads; they do not actually understand or reason. Lumina is a triple hybrid combining autonomous agents, large language models, and biological mechanisms built from scratch, replicating things like frontal lobe structures, fluid intelligence, synaptic plasticity, the hippocampal-entorhinal complex, vmPFC, and dlPFC. It targets true semantic understanding and reasoning, and its weights can actually adapt during inference in the forward pass rather than remaining completely static. It is designed as a generalist system for everyday users and developers.
There are thousands of biological mechanisms packed into the design so I will not turn this into a massive info dump, but I am going to publish a detailed breakdown about it on Hashnode soon. If you want to read up on the architecture later, just ask and I will drop the link.
But seriously been the founder and CEO is not very easy
I tried LLM research, but I stopped midway due to many obstacles.
I am impressed by your abilities as a business partner.
Excuse me for asking, but are you a woman?
If so, I am even more impressed. I respect women. Even more so for women who do things that are difficult for men.
Of course yes I am . But seriously let not turn this into gender talk but seriously do you want me to give you the link. When I published the article on Hashnode
Where I am going to do the long detailed description of the project lumina architecture