DEV Community: alex lee

Developing with AI right now: 11 field rules from a veteran

alex lee — Mon, 26 Oct 2026 09:29:02 +0000

Someone asked me what good approaches there are for developing with AI right now. I said it mainly depends on how far you are using AI, and whether it is a new project or maintaining an old project. If this were really expanded on, it would be a very large amount of content. But briefly, from my personal experience, there are a few points:

Requirements and detailed design can be done by AI, but the architecture must be reviewed and controlled by you personally.
There must be four core constraints. The first is to require documents and conversations to speak in plain human language. The second is that any creation or rewriting of documents must not compress any information, and must not be oriented toward saving tokens. The third is that code must have comments: any function, class, interface, and important logic must have comments. These three points, to put it plainly, solve the problem of whether you can understand your documents and whether you can understand your code. The last one is to reuse existing code whenever you can, and if there is a mature open-source project or third-party library, use it. Do not reinvent the wheel. This is also very important. Otherwise you will find that the AI has written a large amount of code, which in practice is not as useful as one mature library.
The first batch of documents written by AI must be read carefully. Even if you feel very impatient reading them, read them carefully anyway, and try to solve as many problems as possible at the document stage. Otherwise, as the AI writes more and more code and more and more documents, you will become less and less willing to read documents and less willing to read code. A lot of drift and scope expansion comes from this point.
In the requirements, architecture, and design stages, the interview style is very useful. You can have the agent use an interview-style approach to write together with you, or you can have the agent and sub-agents ask and answer questions among themselves as a substitute for you. But every step must be appended to a list. Do not trust that its context compression mechanism is good enough.
Unit tests and quality gates are needed, especially as the project gets bigger. But the rules must be set properly: one is the timing for writing and using unit tests and gates, and the other is not to run the full suite every time. This can decide the development speed and the waste of tokens, while also avoiding falling into the premature optimization swamp too early.
Whether one agent writes your project or multiple sub-agents develop in parallel, one very obvious problem appears: duplicate code and duplicate logic. This is currently hard to avoid, even if you have written constraints like in my point 2. The Ponytail skill library has a certain effect, but you cannot use the original version. It has to be modified, otherwise the effect will not be very good. Looking across the whole project, there will still be a large amount of code that could have been merged or abstracted, but in fact exists in multiple written versions. And if it is used improperly, it will cause problems with your code quality.
Review is a must, no matter the size of each task, and the review must be independent and strict. What does independent mean? It means the developer does not review its own work. Hand it to other sub-agents, and if conditions allow, even hand it to a different large model. Reviews generally need to go through multiple rounds, depending on complexity. For something simple, one agent reviews it. For something complex, multiple agents review from different angles, or are assigned different modules to review, and then the main agent is responsible for deduplication, cross-verification, and the final investigation. This step is very important, because many issues require cross-verification to find the root cause. Otherwise it becomes patch-style development over and over: the fix only solves the symptom, not the cause. You have it fix something, then review again and new problems appear. After multiple rounds you have fixed a big pile of things, but the root cause might actually be only one. Another point is to watch whether the review is over-reviewing. You can write prompts for this, or write a skill yourself, but in the end it still relies on your own ability. No prompt or skill can guarantee that over-reviewing will not happen. Finally, do not just say "review this for me" and be done with it. Point out the aspects to focus on. For example: review whether there are memory problems such as out-of-bounds access, overflow, or leaks; whether there is over-design; whether thread ownership and the use of async threads are correct, and so on.
On the use of skills, I do not recommend enabling strongly constraining skill packs like Superpowers on the current top-tier models. These do have an effect, but the benefit is not necessarily that large. Personally, the built-in skills plus a few I have modified myself are enough. For a team, one or two engineered skills are enough. Any more than that becomes a burden. To put it in plain words, even if you use TDD, DDD and all that stuff, the output is still reviewed by AI in practice. The bigger the project, the harder human review becomes.
The most important thing is that even if your project runs tens of thousands of gates and passes them all, it does not mean your project can go live. This itself is a false proposition. Because no matter how many gates there are, and no matter how complete the unit test coverage is, it does not mean the project itself will not have problems. At most, these guarantee that under limited, already verified preconditions, the project is fine. As for how to do it, every company has its own way. For example, on one small project of ours, the full gate suite has over 100,000 checks and takes 30 minutes to run once. But we still had another agent rewrite a set of attack programs based on the design documents, and added it as the final stage of CI.
There are some other techniques that many people online have already talked about. For example, AGENTS.md and CLAUDE.md should contain no more than 100 lines of core instructions, and everything else should only serve as an index. Internal documents should be stored in a structured directory, with an entry index in each directory. Tests should have a ledger, and every test run must be appended to the ledger. Document management should not rely on numbering, but on the git repository. And so on. I will not say more about these; they are everywhere online.
Oh, I almost forgot, there is also the selection of large models. I do not know domestic (Chinese) models very well, and I have basically never used them. Just from Claude and GPT, they are suitable for different fields. Claude's analytical ability is very strong. It performs very well in the architecture and design stages of a project, much better than GPT. Its only real downside is that its autonomy is too strong. If you do not constrain it well, it will do a lot of unnecessary things for you. GPT is indeed much worse than Claude at solving complex problems, but it is more obedient when it comes to doing the work, so it depends on how you use them. I generally give complex problems to Claude, and simple, local code writing to GPT. Also, a lot of the time I use Claude to direct GPT to do the work. This works quite well. In Claude Code, it supervises GPT doing the specific tasks, and corrects and adjusts in time. As for model switching, I do not recommend switching back and forth the way people online suggest, like using the strongest model for analysis first, and a second-tier model for doing the work. A continuous task is best not switched back and forth, unless it is inherently a main task and sub-tasks. This is not a context problem, it is a problem of the large model's own cache hits.

There is more that I either cannot recall at the moment, or that needs proper expansion to explain. In short, the above is a brief summary of some of my experience. If it were really expanded, this would be a very large body of content, not something a few points can make clear. You happened to ask, so I briefly shared my experience. Since this was off the cuff, some of it may be incomplete or incorrect. For reference only.

I tore down HumanLayer's source code: it leashes your coding agent, but sessions move to its cloud

alex lee — Mon, 26 Oct 2026 09:27:45 +0000

HumanLayer open-sourced its whole system the other day. 11.8k stars, Apache 2.0, and they say the released code is the same code their cloud runs. What it sells is something a lot of people want right now: let a coding agent work on its own, but make it stop and ask before it does anything dangerous.

The repo is about 22 megabytes of TypeScript. I read through it, then went back and checked the important files against the official docs. The engineering is real, the idea is right. It's also much heavier than it looks, and a few things won't show up in the launch post.

What it sells

Hand Claude Code or Codex a task and it will run commands, edit files, install packages on its own. Fast, genuinely fast. The catch is you can't watch it every second, and a wrong directory deleted or a wrong command run takes about a second.

HumanLayer sells an approval layer. Every time the agent wants to do something risky, it stops and pushes the request to your web dashboard or your phone. Approve, and it continues. Deny, and it stops, and your reason for denying is shown to the model. Not at your desk? You can watch it work and approve things from wherever you are.

Sounds simple. Underneath, there are three completely different ways it connects, and which one you're using changes what you actually get.

Way one: it drives

You create a task in its web app. A small program it installs on your machine picks the task up and starts the real Claude Code itself.

Because it started the process, it holds every gate. A handful of actions pass automatically (searching code, reading files, calling skills). Everything else first gets written to the cloud as "waiting for approval," and the program freezes on the spot until your answer comes back. Approve and it proceeds. Deny and it hands your exact words to the AI.

This one is a real gate. The steering wheel is in its hands.

Way two: you drive, it relays

If you'd rather start Claude Code yourself in a terminal, there's a plugin. When Claude Code stops to ask you something, the plugin mirrors that question to the web app, and answering there counts. The terminal prompt and the web card race each other; whoever answers first wins.

Clever. There's a ceiling, though, and to its credit the docs are honest about it: if your Claude Code is set to "don't ask me, just do it," it never produces a question, so the plugin has nothing to relay. You keep the recording and lose the gate. The plugin also leans on a fairly new internal interface inside Claude Code, so when Anthropic changes that interface, the plugin has to chase it.

Way three: the "Codex" in there is theirs

This is the one that stopped me while reading.

Pick "Codex" inside HumanLayer and what runs is a program HumanLayer wrote itself. It takes your ChatGPT login and calls OpenAI's API directly, the tools are its own implementations, and the conversation is stored in its own database. That line in its docs, "Codex sessions are stored by HumanLayer," is literal.

For completeness: OpenCode gets the relay treatment. The Pi plugin is relay only, no approvals at all. Its own README says it plainly: Pi doesn't ask, so Pi just runs.

Where your data actually lives

This is the part I'd want someone to tell me before installing.

The official install is cloud-first. The program on your machine does the work, but the ledger lives in HumanLayer's cloud. Every conversation, every action the agent took, every file it changed, your task files, how much you used. All of it is written into a database on their side, then synced back out to your devices.

Credit where due: their docs state plainly that they don't proxy your model traffic. The calls to the model go direct. But the session content itself sits on their cloud.

There is a self-hosted road. Build it from source, run it yourself, and they say it sends nothing back; the telemetry switches live in configuration and stay off when unset. I believe that part. But self-hosting right now is rough: several third-party service keys are required before the thing will even start, and the official guide is still "in progress." One more small thing: their two most core packages ship without a license field in the package file. The code is public, but anyone who's maintained open source knows that detail matters.

Now the ban risk, which almost nobody mentions

Using a subscription login inside a third-party tool to call the model: Anthropic drew that line at the start of 2026. Their docs say it isn't allowed, they blocked it technically, and accounts did get banned in a wave before some were reinstated. The logic is simple enough. A subscription is priced for the official client, and feeding it to third-party programs is not what that price was for.

Hold HumanLayer up against that. Its Claude path drives the real Claude through official tooling, which keeps it comparatively inside the lines. Its home-grown "Codex" takes your ChatGPT login and calls the API from its own program, which is exactly the pattern the vendors are tightening on. OpenAI hasn't run Anthropic's full playbook yet. The line is drawn, though. If it gets enforced one day, the account that hurts is yours.

Tools like this save you subscription money. What you put up as collateral is the account itself. Run that math before you install.

There's also the sheer size of the thing. 22 megabytes of code is a lot of surface. Open source is a point in its favor; open source is not the same as audited. Out of 11.8k stars, almost nobody has read the approval chain end to end. My old rule for trusting a tool: look at where it puts your data and where it puts the gates. The rest is marketing.

My take: this got built heavier than it needs to be

I've done full-stack work for 24 years. Back when I led teams, one of the expensive lessons was that the heavier the system, the more of yourself you have to hand over to it.

HumanLayer's full stack (cloud ledger, real-time sync, apps on every surface, tasks, comments on diffs) is built for teams: several people collaborating, an audit trail, multiple agents working in parallel. In that world it earns its weight. If you're one developer who just wants "I can see what the agent is doing, it asks before anything dangerous, and I can approve from my phone," handing every session to a cloud for that one feature doesn't pencil out.

And the vendors left a door open, so the big detour isn't necessary.

Codex ships with a standard interface called the app-server. OpenAI's own editor extension and app talk to Codex through it. Asking the user for approval is a built-in part of that protocol, and it's open source. On the Claude Code side there's ACP, an open protocol (started by the Zed folks) that wraps it, with permission requests as a first-class citizen and ready-made adapters you can download today. Both roads run the official tools through official interfaces, and the tools' own safety limits stay in force, because the tool still stops and asks before dangerous actions on its own.

What's left to build is small. A tiny forwarder. A hook catches the "it wants to ask you" event, pushes the question to your WhatsApp or Telegram, you reply "yes" or "no" in chat, and the forwarder feeds your answer back in. Your session data never leaves your machine (the chat app itself aside). No cloud ledger. Nobody calling an API somewhere else with your subscription login. The amount of code is not in the same universe as the stack above.

Honest downsides: the forwarder road is rough. Your interface is a chat window, with no task board, no diff comments, and no multi-user features. If you run a team and need an audit trail, buy or build the heavy thing with a clear conscience. This road is for solo operators who like things light. It also has a quiet advantage: when a vendor changes its policy one day, what breaks is a small program you wrote, fixable in an afternoon. If you've bet on someone else's whole platform, migration is a different conversation.

I plan to build the little forwarder first and see how it feels. Report back when there's something to report.

Sources: the HumanLayer repo (github.com/humanlayer/humanlayer) and its bundled docs; the open-sourcing announcement at hlyr.dev/blog/open-sourcing-humanlayer. The ban-policy section reflects Anthropic's public 2026 documentation and statements.