Every week there's a new coding agent, and every week someone asks the same question in a slightly different tone: can I actually trust this thing with my codebase?
It's a fair question. A coding agent reads your source, runs commands in your shell, and — increasingly — edits files and opens pull requests on its own. That's a lot of access to hand to a black box. So instead of arguing about which agent is "smartest," I want to talk about a less glamorous property: trust. What does it actually take for a coding agent to earn it?
I'll use SolonCode — an open-source coding agent built in Java on top of Solon AI — as a concrete reference, not because it's the only good answer, but because its design happens to line up with a checklist I think is worth having. Take the checklist with you and hold any agent to it, including this one.
A trust checklist for coding agents
Here are the five questions I ask before I let an agent near a real repo.
| Question | Why it matters |
|---|---|
| Can I read the source? | You can't trust what you can't inspect. |
| Where does my code go? | Every network hop is a place your code can leak. |
| Am I locked into one vendor? | Lock-in quietly removes your ability to walk away. |
| Can I control what it does? | An agent that acts without review is a liability, not a tool. |
| Can I undo a mistake? | Autonomy is only safe when it's reversible. |
Let's go through them.
1. Can I read the source?
The strongest form of trust is the kind you don't have to take on faith. If the agent is open source, you (or your security team) can read exactly how it builds prompts, what it sends over the wire, and where it stores things.
SolonCode is MIT-licensed and fully open source — the CLI, the Web UI, and the desktop client are all in the open. That means the interesting questions ("what exactly gets sent to the model?", "does it phone home?") are answerable by reading code, not by trusting a marketing page.
This is the part that "purity" really comes down to for me: not a vibe, but the fact that there's nothing you can't look at.
2. Where does my code go?
A coding agent has to send something to a model to be useful. The question is what else happens along the way — telemetry, analytics, background uploads.
SolonCode runs locally. You start it from your own machine in whichever form you like:
# terminal (CLI)
soloncode cli
# browser (Web UI)
soloncode web 0
# or the desktop IDE
The agent process lives on your box, works in your workspace, and talks directly to the model endpoint you configured. There's no mandatory middle-tier service that your code has to pass through first. For teams with source that legally cannot leave the building, that distinction is the whole ballgame.
There's a more fundamental angle worth stating on its own: even if it wanted your code, it has no motive to take it and nowhere to use it. A lot of the worry assumes the vendor is quietly harvesting your code to train its own model — but that assumption has a precondition: the vendor needs a first-party model or cloud business for your data to be worth anything to it. SolonCode isn't that kind of player. It has no hosted foundation model of its own, and no official cloud API that your requests are forced to route through — it's a local client that hands the work off to the model you chose. No in-house model to feed means no incentive to scrape user code for training; no mandatory relay means no pipe where your data could be siphoned off. It can't technically, and it has no reason to commercially — and those two things together are more reassuring than any "we promise not to collect" ever could be.
3. Am I locked into one vendor?
A lot of agents are welded to a single model provider. That's convenient right up until pricing changes, a better model ships elsewhere, or your employer mandates a specific vendor.
SolonCode is provider-agnostic. You configure models yourself — through Settings → LLM in the Web UI — and point it at whatever you're allowed to use: a hosted API, an OpenAI-compatible endpoint, or a local model. Because it's built on Solon AI, swapping the underlying model is a configuration change, not a migration.
The practical value: the day a cheaper or smarter model shows up, you switch a setting instead of switching tools.
4. Can I control what it does?
This is the one people underestimate until an agent runs a command they didn't expect. Trust isn't "the agent is always right" — it's "I decide how much rope it gets."
SolonCode makes the autonomy level an explicit choice. Its work modes include:
- Approval execution — the agent proposes actions; you approve before anything runs.
- Automatic editing — for when you've built up trust on a task and want it to move.
- Read-only planning — it can analyze and plan, but cannot touch your files.
- Goal execution — persistent, longer-horizon autonomous work.
The point isn't that one mode is "correct." It's that you pick the risk level per task, instead of the tool picking for you. Reviewing a hairy migration? Read-only planning. Renaming a variable across ten files? Let it run.
5. Can I undo a mistake?
Even a careful agent will occasionally do the wrong thing. What matters is whether that's a shrug or a disaster.
SolonCode keeps persistent session history with rewind and redo, recoverable workspace checkpoints, and safe deletion. If a run goes sideways, you roll back to a known-good checkpoint instead of reconstructing your working tree from memory. Autonomy and reversibility are two sides of the same coin: the more freedom you give an agent, the more you want a clean undo.
Trust is a property you can check
None of these five points are about which agent writes the cleverest code. They're about whether you can verify how it behaves — read its source, see where your code goes, keep your model options open, control its autonomy, and undo its mistakes.
That's a better lens than "which one is smartest," because smart-but-opaque is exactly the combination that gets you into trouble. An agent you can inspect, run locally, point at any model, gate with approvals, and roll back is one you can reason about — and reasoning about your tools is the whole job.
SolonCode is one implementation that scores well on this checklist, and it's open source, so you can confirm every claim above by reading the code:
- Repo: https://github.com/opensolon/soloncode
- Built on Solon AI: https://github.com/opensolon/solon-ai
- Docs: https://solon.noear.org/article/soloncode
Run the checklist against whatever agent you use. The goal isn't loyalty to a tool — it's keeping the power to check, choose, and undo in your own hands.
Top comments (6)
Heads-up to readers: this comment is spam. Please DON'T click the link it contains — it is not affiliated with SolonCode or DEV, and points to a shady "anti-bot" redirect that is a known phishing/scam pattern. Reporting it to the mods. Stay safe!
The checklist is useful, especially separating where the agent runs from where inference happens. A local client can keep code out of a vendor’s relay, but prompts still go to the configured model endpoint—so the endpoint’s retention, region, and logging policies matter too.
Provider choice also changes more than price: latency, output quality, and retry behavior can shift the cost of getting a successful result. Do you see SolonCode giving users a way to compare or constrain those trade-offs when switching models?
Good and fair questions — let me take them one at a time.
On endpoint policy: you're right, and this is the part a client can't fix for you. SolonCode is a local client: it sends prompts to whatever endpoint you configure, so retention, region, and logging are the provider's policy, not the tool's. What SolonCode can do is remove the excuse — it can run fully offline against a local model, or against a self-hosted endpoint you control, and that config lives in plain per-project JSON, so "which endpoint did this repo use?" stays auditable.
On constraining the trade-offs: it's configuration rather than comparison, honestly — there is no built-in benchmark that scores latency vs. quality across models. What is there:
providers+modelsentries, scoped globally or per workspace, so a repo can pin its own endpoint and model set.model:field, so a cheap fast model can plan or summarize while an expensive one implements.summaryModel) — usually the biggest silent cost driver on long sessions.modelRetries,apiRetries,mcpRetries) and reasoning/thinking defaults that clamp to the nearest tier a model actually supports, so switching models degrades predictably instead of failing outright.So: comparison is still on you (and on the provider's published numbers). Constraining, at least, belongs in the config.
That’s a useful distinction: the config can make a choice auditable, but it can’t tell you whether it was the right choice for a real workload. The summaryModel point especially caught my eye—compression costs seem easy to miss when comparing models.
When teams do that comparison, what evidence do they trust most: their own workload tests, production telemetry, or provider benchmarks? And is the gap mainly getting comparable data—or deciding what trade-offs matter? I’m exploring this space with CarbonLayer, so your perspective would be useful.
Fair question, and my honest answer is a ranking with caveats: for switching decisions, your own workload tests beat everything, provider benchmarks beat nothing, and production telemetry is the tiebreaker — but each has a trap worth naming.
Workload tests win because the thing you're actually buying with a model swap (token efficiency on your prompts, latency at your concurrency, retry amplification on your failure modes) is precisely what benchmarks average away. The trap: running them on a toy corpus — a 20-prompt eval correlates with provider-of-the-week far better than with production quality. The
summaryModelpoint fits here: compression costs are workload-specific almost by definition (your summaries have your domain density), so a benchmark's "good at summarization" tells you almost nothing about your bill.Provider benchmarks are still useful, but as screening, not selection — they tell you which three models deserve a workload test, and they age fast. Production telemetry is the one teams underrate: you already have real distributions of prompt lengths, tool-call fan-out, and retry storms; replaying last month's traffic against a candidate model is the cheapest honest eval that exists. The gap is rarely getting comparable data — you can always generate data. The gap is deciding which trade-offs matter before you look, otherwise every model looks best on whichever axis you happened to measure.
Concretely in SolonCode terms: because the client is local and the model config is plain, replaying identical prompts across two endpoint configs is a diff of two config files — which is at least the mechanism for the comparison you're asking about, even though the judgment stays yours.
Really appreciate this breakdown, especially the distinction between using benchmarks to shortlist candidates and using workload-specific tests to make the actual decision.
Your point about defining trade-offs before looking at the results is particularly important. Otherwise, it's too easy to justify whichever model performs best on the metric that happens to look most favorable.
One thing we're exploring at CarbonLayer is the idea of making those decisions explainable after the fact — not just recording which model was selected, but why it was selected, what evidence supported the choice, and which trade-offs were accepted.
The ability to replay identical prompts across SolonCode endpoint configurations seems like a useful foundation for that.
I'm curious: have you considered capturing a structured comparison record alongside those configuration changes? Something that preserves the workload tested, evaluation criteria, observed results, and reason for switching?
That could make model selection easier to revisit when pricing, provider behavior, or workload characteristics change.