DEV Community: Dhruv Malaviya

Nine Keys Can SSH Into Production. Five Owners Don't Work Here Anymore.

Dhruv Malaviya — Tue, 03 Nov 2026 12:20:41 +0000

I cat'd authorized_keys on our oldest box for a boring reason and kept reading: the 2023 contractor, a dead CI pipeline, two ex-engineers, an 'ops-temp' nobody can attribute. The sweep, the smoke alarm, the one-week evidence trick, and keys-that-expire culture.

I opened the file for a boring reason — a connectivity weirdness — and then just kept reading. ~/.ssh/authorized_keys on our oldest production box: nine lines, nine doors.

Matching comments to memory: the 2023 migration contractor (still there), the CI pipeline decommissioned last spring (still there), two engineers who left — one warmly, one not (still there), a key commented ops-temp that nobody including our ops lead can attribute to a human, and one line with no comment at all, which is its own small horror genre.

Four keys belonged to current humans who still needed access. Five were sediment. The door remembered everyone, because nobody had ever asked it to forget.

authorized_keys is an attendance sheet
The oldest access-control file in our profession is a plain-text list on disk: appended to for a decade, reviewed never. Every line was added on someone's worst day — incident, deadline, "just for tonight, add my key." Keys arrive during emergencies; emergencies don't do cleanup; departures miss the file entirely, because it isn't in any system. It's wallpaper. And unlike a password, a key never expires out of embarrassment — it waits for its laptop to be sold, lost, or inherited.

The sweep

#!/bin/bash
# keys-audit.sh — every authorized_keys on every box, with its "who/why" comments
for host in $(cat hosts.txt); do
  ssh "$host" 'for f in /root/.ssh/authorized_keys /home/*/.ssh/authorized_keys; do
    [ -f "$f" ] && {
      echo "== $HOSTNAME $f — $(grep -cvE "^[[:space:]]*$|^[[:space:]]*#" "$f") keys"
      grep -vE "^[[:space:]]*$|^[[:space:]]*#" "$f" | awk "{print \"   \", \$NF}"
    }; done'
done | tee keys-audit.txt
# then the human step: diff the comment column against the current roster.
# every line that doesn't match a living owner is sediment.

Ours diffed to five. Yours will too; that number is your real access list.

The four changes that stuck

Keys got names and reasons. One comment format, enforced at add-time:

echo "$(cat ana.pub) ana@laptop — incident-4821 — 2026-11-03" \
  | sudo tee -a /home/ubuntu/.ssh/authorized_keys
# a key that can't state its reason in six words doesn't get added.
# 'ops-temp' failed this test immediately and was deleted with a small ceremony.

Offboarding gained a server step. Badge, laptop, SSO — and this, on every box:

grep -rn 'ana@' /root/.ssh/authorized_keys /home/*/.ssh/authorized_keys

The step exists because the 2023 contractor taught us it doesn't happen by osmosis.

The file got a smoke alarm. Any change pages someone — not blocks, pages; additions are sometimes legitimate at 2 a.m., silent additions never are:

# auditd: watch writes and attributes to authorized_keys on every box
echo '-w /root/.ssh/authorized_keys -p wa -k ssh_sediment' | sudo tee /etc/audit/rules.d/ssh.rules
sudo augenrules --load
# wire the audit event to your pager; our worst file became our best-monitored one

Deletion ran on evidence, not folklore. We almost kept the unattributed key "in case something depends on it." Instead: a week of fingerprints.

sudo sed -i 's/^#LogLevel INFO/LogLevel VERBOSE/' /etc/ssh/sshd_config && sudo systemctl reload ssh
# after a week — which fingerprints actually authenticated?
sudo journalctl -u ssh --since "-7 days" | grep -oE 'SHA256:[A-Za-z0-9+/=]+' | sort | uniq -c > used.txt
# and per line in the file — which keys never appeared in used.txt?
awk '{print $1" "$2}' /root/.ssh/authorized_keys | while read -r t k; do
  fp=$(echo "$t $k" | ssh-keygen -lf - 2>/dev/null | awk '{print $2}')
  grep -q "$fp" used.txt || echo "UNUSED ALL WEEK: $fp"
done

Nothing used it. Measure before you mourn.

New boxes start at one
The habit stuck because new machines made it easy: on the Cubes we run at Krova Cloud, a box boots with exactly one key — the one handed at creation — and root login is off by default. Additions become a decision you make, not sediment you inherit. Humans increasingly skip static keys entirely: short-lived certs through a bastion for people, per-pipeline keys rotated with the pipeline for robots. Zero sediment, not zero keys.

The honest part

Static keys aren't a sin. Plenty of honest estates run them fine because they run the reviews. The sin is the unreviewed list, not the mechanism.
Ceremony has a cost. Certs and bastions add moving parts and one more thing to fail during the incident that justified the shortcut. We tiered: short-lived access on sensitive boxes, named-and-alarmed keys elsewhere. Perfection is a budget line, not a value.
The alarm will page you once legitimately, at 2 a.m., during a real incident. Thank it. That page is the system proving it can still tell the difference.

The door remembers. You don't have to.
Go cat the file — the actual one, on the actual oldest box, right now. Count the lines; count the owners who are gone. Ours was nine and five. We deleted four, adopted one (the CI key got a name, a rotation, and an apology), and the ninth — the comment-less line — turned out to be me, from a forgotten 2022 incident.

The door remembered me too.

It's Halloween, So Here's My Field Guide to the Monsters Already Living in Your Stack

Dhruv Malaviya — Sat, 31 Oct 2026 05:47:03 +0000

Seven creatures, one per audit: the Zombie process, the Ghost cron, the Trojan paste, the Spider at the door, the Gremlin in CI, the Kraken in the schema, and the Daemon you should adopt. Each with its habitat, its detection one-liner, and its banishment.

Every infrastructure has a bestiary. Nobody writes it down, which is why the monsters keep winning. Mine got catalogued one audit at a time over a year of confessions on this blog — seven distinct creatures, every one still living in somebody's production right now. It's Halloween, so: name them, detect them, banish them. Or adopt them, in one case. Field guides are honest about which animals are pets.

The Zombie 🧟
Died once, kept walking. Deprecated services still serving, restart loops older than the intern who "fixed" them.

# dead-but-walking in the process table
ps -eo pid,ppid,stat,comm | awk '$3 ~ /^Z/'
# and the container flavor: restart loops
docker ps -a --format '{{.Names}}\t{{.Status}}' | grep -iE 'restart|back-off'

Banish: graveyard rules — kill it fully (process, DNS, keys, CI variables) or adopt it with an owner. Half-killed zombies are how sequels happen.

The Ghost 👻
The spirit of a departed colleague, still doing their job. Ours ran at 3:07 AM and reconciled invoices.

for u in $(cut -d: -f1 /etc/passwd); do
  crontab -l -u "$u" 2>/dev/null | grep -vE '^\s*#|^\s*$' | sed "s/^/$u: /"
done   # every line that can't name an owner, on purpose, is a ghost

Banish: the séance — owner, purpose, loud failure, or a funeral. Plus staleness alerts: a ghost that stops working is an incident nobody feels for three weeks.

The Trojan 🐴
Arrives as a gift, opens the door from inside. Pasted snippets, dependency diffs, October PRs titled fix typo.

git diff main...HEAD | grep -nE 'process\.env|os\.environ|getenv'      # env going somewhere
git diff main...HEAD | grep -nE 'base64|eval\(|atob'                   # obfuscation
git diff main...HEAD | grep -nE 'curl .*(-d|--data)|wget |fetch\('     # new outbound call

Banish: review before merge even at twenty lines; provenance comments; pin what you can pin. Most Trojans aren't malicious — just someone else's bug, fossilized with full rights.

The Spider 🕷️
The internet's automated fauna, patiently trying every handle. Lives on your public address and every port you forgot

# before: the porch the spiders sit on
ss -tlnp | grep -E '0\.0\.0\.0|::'          # listeners facing everyone
journalctl -u ssh --since yesterday | grep -c 'Failed password'

# after (no public IP, default-deny, managed ingress): both commands return silence
krova tcp list web-1                        # empty. no porch, no spiders.

Banish: don't fight it at the door; remove the door. My spider population went extinct in a weekend and I did nothing heroic — just changed the default. (The one entry where my hosting is the whole spell: Krova Cubes ship with no public address.)

The Gremlin 👺
Feeds on observation. The CI-only failure, the bug that vanishes under the debugger, timestamps disagreeing by exactly one hour in exactly one region.

# evict its conditions: unseeded randomness and un-pinned clocks in tests
grep -rnE 'Math\.random|random\.random|Date\.now|time\.now' tests/ src/ \
  | grep -viE 'seed|fake|clock'             # every hit is a gremlin bed

Banish: you don't kill gremlins, you bore them out: fixed clocks, seeded randomness, reproducible builds. Boredom is the only exorcism that holds.

The Kraken 🐙
Too big to fight, too central to flee. Tentacles in every schema; migrations scheduled in quarters; a cloud labeled LEGACY.

# count the tentacles before you wake it
grep -rc 'legacy_orders' --include='*.sql' --include='*.py' --include='*.ts' . \
  | sort -t: -k2 -rn | head

Banish: nobody slays the Kraken. Build seams, route one tentacle at a time around it, let it sleep. Anyone selling you a slaying is selling you an outage.

The Daemon 😇
The only monster that works for you. Night-shift process keeping caches warm, certs renewed, queues drained. A daemon is not a demon — it's a guardian spirit we forgot to thank.

systemctl list-units --type=service --state=running --no-pager \
  | awk '{print $1}' | grep -vE '^\.|systemd|ssh|cron'   # your daemons
# adoption papers per daemon: owner, runbook path, silence alert

Banish: absolutely not. Adopt: name it, log it, alert on its silence. A daemon with an owner is infrastructure; a daemon without one is next year's Ghost.

The honest part
None of these are moral failures. Every stack I've audited hosted at least four of the seven simultaneously. The monsters aren't the shame; the unnamed monsters are. An inventory turns fear into a checklist, and a checklist turns 3 AM into Tuesday.

And the seventh entry is the lesson: the difference between a guardian and a ghost is rarely the creature. It's whether somebody wrote its name down.

Name your monsters
Tonight of all nights: run the seven detections. Seven chairs at the table; see who sits down. Then write the field guide — one page per monster: habitat, tells, banishment, or adoption papers if it earns them.

The Meeting Bot Heard the Password. It Wrote It Down. Forever.

Dhruv Malaviya — Fri, 16 Oct 2026 08:10:35 +0000

I searched our meeting transcripts for 'password' and got spoken connection strings, MFA digits, a verbatim angry customer, and a frame-captured dashboard. The sweep, the bot-free policy, the synthetic demo box, and retention as a setting.

I typed one word into our transcript search box: password. Four minutes later I closed the laptop and went for a walk.

Hit one: an API connection string read aloud during pairing ("just paste what I say"). Hit two: an MFA code, digit by digit to IT, transcribed correctly. Hit three: a customer's name and their angriest sentence of the year, verbatim, in a sales summary anyone can open. Hit four wasn't audio: a screen-shared dashboard captured frame by frame, real customer rows in 4K.

The room was never bugged. It was scheduled — recurring, 30 minutes, automatic notes.

Meetings became data. Nobody designed the data.
A transcript is a log no one engineered: nothing scrubs it, nothing scopes it, everything searches it. Retention defaults to forever because forever is what storage does when nobody decides otherwise. The vendor keeps a copy — the vendor is the one transcribing. And unlike every datastore we own, this one ingests secrets, PII, and candid opinions at conversational speed, from everyone, daily, without a schema.

Screen sharing is the same leak with better resolution: the terminal corner in a demo, the .env tab during a "quick look," the autocomplete that finished a customer's email on a projector. We'd never put those in a doc. We put them in a recording, which is a doc with witnesses.

The sweep
Export whatever your tool allows, then:

# spoken-secret shapes
grep -rniE 'password|passphrase|token|api[_-]?key|secret|mfa|verification code|connection string' \
  transcripts/ | grep -viE 'password (policy|manager|reset)'

# real humans in the room-data
while read -r c; do
  grep -rlq "$c" transcripts/ && echo "CUSTOMER IN TRANSCRIPTS: $c"
done < top-accounts.txt

# recordings + screen shares, from the calendar/admin export
jq -r '.[] | select(.recording or .screenShare)
  | "\(.date)\t\(.organizer)\t\(.title)"' meetings.json | sort | tail -30

The customer-name grep is the one that ends careers of assumptions. Ours returned eleven accounts, including two under NDA.

What changed

Secrets are never spoken. Pairing shares a vault link, not a voice. Awkward for a week, then seatbelts.
Sensitive calls are bot-free by default — incidents, credential handling, escalations. Routine calls keep notes; the rule is classification, same as staging: real things don't live in rooms with pretend rules. The enforcement is boring:

# weekly: any flagged meeting next week still carrying a bot/recording flag?
jq -r '.[] | select(.title | test("incident|postmortem|migration|cred"; "i"))
  | select(.bot or .recording) | .title' meetings-next-7d.json
# non-empty output = fix the invite, not the culture

Demos run on a demo box. Screen shares that might show data show synthetic data. Disposable Cube on Krova Cloud: redeployed fresh, seeded fake, deleted after the call:

#!/bin/bash
# demo-box.sh up|down
case "$1" in
  up)
    krova cubes create demo-box --cpu 2 --ram 4 --disk 40 --image ubuntu-24.04 \
      --user-data "$(cat seed-fake-customers.yml)"
    krova domains add demo-box --domain demo.example.com --port 8080 ;;
  down) krova cubes delete demo-box ;;   # nothing true was ever in it
esac

#cloud-config
runcmd:
  - curl -sf https://storage.private/fake-customers.sql.gz | gunzip | sudo -u postgres psql app
  - systemctl restart app

The demo can leak all it wants; there's nothing true in it.

The notes tool got treated like a datastore. Retention set to a number we chose, old exports purged on a schedule (find … -mtime +90 -delete — exports are copies too), search access reviewed, and the vendor's terms actually read for the first time since someone clicked "enable for workspace."

The honest part

Transcripts are genuinely good — searchable decisions, async teammates, onboarding gold. I don't want the tool gone; I want its defaults to have opinions. Burning it down moves the leak back into memory, which retains worse and searches never.
Small teams can't negotiate vendor terms. So minimize what enters the room: the bot can't transcribe a secret nobody says, nor capture a dashboard showing fake rows. Input control beats contract control.
"But the notes!" is a real loss, not resistance. The compromise that stuck: notes everywhere by default, bot-free where stakes say so, and the safe path made easier than the risky one — one-click demo box, vault links in every pairing doc.
This isn't the castle. Walls stay walls: no public IP by default, own kernel per Cube, scoped secrets. Transcript hygiene shrinks what conversations leak; architecture shrinks what machines leak. Both, always.

Forever is a setting
Every "forever" in your stack — transcripts, recordings, screenshots, exports, free-tier syncs — is a default nobody revisited. The scariest datastore you own is probably the one that fills itself while everyone talks.

Go type password into your transcript search. Then token. Then your biggest customer's name. Four minutes. I'll wait.

There's a Cron Job in Production That Nobody Remembers Writing.

Dhruv Malaviya — Thu, 15 Oct 2026 12:06:28 +0000

It runs at 3:07 AM, reconciles our invoices, and its author left in 2022. The crontab séance: inventory every scheduled job, give each an owner + purpose + loud failure or a funeral, crontab-as-code, staleness alerts, and ephemeral job boxes.

Every company has ghosts like this. October is a fine month to introduce yourself.

Scheduled jobs are employees without HR files

A cron job is a colleague: it has a shift, a responsibility, credentials, access. Unlike every human on payroll, it never got an onboarding (what does it need and why), a performance review (does it still succeed, still matter), an offboarding (its author left; its badge stayed active), or a manager (when it fails at 3 AM, who gets paged? nobody — the silence compounds).

Our invoice ghost had a valid API credential embedded in the script, a hardcoded path that survived two migrations by luck, and a failure mode that surfaces as accounting confusion in three weeks, not an alert in three minutes.

The séance
Print every crontab on every box — including the "temporary" ones — and hold one rule over each line: every job gets an owner, a purpose, and a loud failure — or a funeral.

#!/bin/bash
# cron-seance.sh — run per host; collect the output, do the arithmetic, feel the chill
set -euo pipefail
echo "== user crontabs =="
for u in $(cut -d: -f1 /etc/passwd); do
  c=$(crontab -l -u "$u" 2>/dev/null || true)
  [ -n "$c" ] && { echo "--- $u ---"; echo "$c" | grep -vE '^\s*#|^\s*$'; }
done
echo "== system dirs =="
grep -rHvE '^\s*#|^\s*$' /etc/cron.d /etc/cron.daily /etc/cron.hourly /etc/crontab 2>/dev/null
echo "== systemd timers =="
systemctl list-timers --all --no-pager | awk 'NF>6'

Two pages of output, eleven engineers. Then per line: who owns it, what happens when it fails, who knows why it exists. Lines that can't answer are ghosts.

Adopt the load-bearing ones, bury the rest
The invoice job didn't get deleted — it got hired: named owner, repo with tests, credentials out of the script and into scoped runtime injection, and alerts on both outcomes:

# the job's last line — a heartbeat:
touch /var/run/heartbeat/invoice-recon

# checker, on the scheduler host (outside the box it watches):
*/30 * * * *  find /var/run/heartbeat -name invoice-recon -mmin +1500 | grep . \
    && alert "invoice-recon: silent too long"     # "it didn't run" is an incident too

Six jobs got funerals — and funeral meant the line, the script, its credentials, and the downstream thing it fed. Half-buried cron is how ghosts get sequels.

Crontab-as-code, or the ghosts return
"I'll just add it on the server" is the exact sentence that spawns ghosts. Now the box's crontab is generated from a repo file, and CI is the only writer:

# infra/cron/invoice-recon.cron — reviewed like any colleague's PR
SHELL=/bin/bash
MAILTO=""                          # silence is not monitoring
07 3 * * * root /opt/jobs/invoice-recon.sh >> /var/log/jobs/invoice-recon.log 2>&1

# the only path into /etc/cron.d
krova ssh jobs-1 -- 'sudo install -m 644 /srv/infra/cron/*.cron /etc/cron.d/'

Where the jobs live matters too
Some scheduled work moved to ephemeral boxes — spin up, do the thing, die (same pattern as our CI):

CUBE="job-invoice-$(date +%F)"
krova cubes create "$CUBE" --cpu 1 --ram 2 --disk 20 --image ubuntu-24.04
krova ssh "$CUBE" -- 'INVOICE_KEY="'"$INVOICE_KEY"'" /opt/invoice-recon.sh'  # creds at runtime
krova cubes delete "$CUBE"        # leaves a bill line and a log, not a mystery

An ephemeral job can't accumulate drift or host a forgotten second job. Resident crons that remain live behind the usual walls: no public IP, own kernel, default-deny inbound. Ghosts are an ownership problem first, a hosting problem second — but right hosting makes ownership visible.

The honest part

Some ghosts are the business. Deleting the invoice job would have been the incident. The audit finds load-bearing ghosts before they fail; it doesn't ban ghosts.
Silence is the real vulnerability. A job that fails loudly is a bug; one that fails silently is debt with interest. Staleness alerts caught two more ghosts we'd otherwise have met during an accounting argument.
Culture takes a week to feel heavy. Then someone proposed a cron change with a test, an owner, and a rollback note — in a review, like a normal colleague — and the habit had a heartbeat.

Meet your ghosts before they meet you
Tonight, on every box you own: open the crontab, read each line out loud, ask the three questions. Every line that can't answer is a ghost. Some are harmless. One of them, somewhere, is your invoice reconciliation.

I Audited Every Tool That Can Read Our Company. The List Was 43 Long.

Dhruv Malaviya — Tue, 06 Oct 2026 10:56:10 +0000

43 external services held standing access to our Slack, Drive, code, and contacts — signed in modals, forgotten in sprints. The sweep per platform, the grant inventory, export-over-connect, vendor funerals, and breach-news as a revocation trigger.

After the secrets rotation and the docs recon, a vendor breach headline asked the question I couldn't un-ask: who outside this building can read us right now? Not hackers — guests. The answer was forty-three external services with standing access to our words, files, code, or customers. Most I recognized. One was a "free CRM trial" from 2024 that had been syncing our contacts, continuously, for two years.

OAuth is a supply chain we sign ourselves
Every grant is a key handed to another company — to their security posture, their employees, their future acquirers, their worst day. When a vendor gets breached, the attacker doesn't need your password; they need the vendor's, and your grant does the rest.

The audit's greatest hits: a 2021 marketing integration still reading every public Slack channel; a Chrome extension with Drive access installed by someone who left in 2023; a deprecated GitHub App owned by a vendor that got acquired (our code's readers now include a company we never chose); and the two-year CRM sync.

The sweep

# GitHub: every app installed on the org, with its permissions and age
gh api orgs/$ORG/installations --jq '.installations[] |
  "\(.app_slug)\t\(.created_at)\t\(.permissions | to_entries |
   map("\(.key)=\(.value)") | join(","))"'

Then the human pages (no API tells the whole truth):

Slack → Manage apps (workspace admin)
Google Workspace → Security → Access and data controls → Third-party apps & services
Browser: every extension logged into a company account
Payments, DNS, registrar, monitoring dashboards — the boring crown

The inventory (one doc, five columns that matter)

| vendor      | surface | scopes           | data reach | funeral plan          |
|-------------|---------|------------------|------------|-----------------------|
| backup tool | drive   | read             | HIGH       | revoke on migration   |
| standup bot | slack   | channels:history | MED        | downgrade → post-only |
| crm-trial   | contacts| read+sync        | HIGH       | KILL (done, RIP)      |

Owner + "since" column too. You cannot revoke what you haven't listed, and a row with no owner is a grant nobody will ever bury.

Export over connect

# the analytics tool gets a monthly batch, not a standing read:
psql -U app app_db -c "\copy (select * from events) to stdout csv" \
  | gzip > events-$(date +%F).csv.gz
# upload one-way; the connection you never open can't leak

Downgrades are free — most vendors don't notice, which tells you how little they needed the scope.

Funerals and triggers

Vendors attend the same funeral as projects. Tool dies in our stack → its grant dies in the same sitting. A grant is just a project that lives in someone else's building.
Breach news is a revocation trigger, not a newsletter. Vendor in headlines → grant reviewed that day.
The re-audit is scheduled, not remembered:

0 9 1 * *   echo "open the four connected-apps pages. count the keys."

The honest part
Some grants are load-bearing. SSO, backups, the payment processor. Tier vendors by data reach, review in that order, forever. Triage, not purity.
Small teams can't audit 43 vendors' internals. So minimize standing access instead — it requires no vendor questionnaire. You can't inspect their locks; you can stop handing out keys.
The walls still matter. None of this replaces no-public-IP defaults, own kernel per Cube, scoped secrets. Vendor hygiene shrinks the surface of trust; architecture shrinks the surface of reach. Both, always.

Forty-three is a normal number
We weren't careless. We were modern. Every grant was reasonable on the day it was signed; the risk was the accumulation of reasonable decisions, each still active, none still reviewed.

Every OAuth grant is a key you gave to another company's worst day. Count your keys.

It's October. Strangers Are Sending You Code. Here's How I Say Yes.

Dhruv Malaviya — Mon, 05 Oct 2026 12:02:00 +0000

October routes thousands of first-timers to your repo. The review order that keeps CI safe (pipeline first, deps second), grep for the three exfil shapes, fork-PR secret hygiene, per-job disposable runners, and how to reject a PR kindly

First PR of the season arrived October 2nd: account three days old, title fix typo, diff mostly a typo. Mostly.

October is the month strangers send you code, and if you maintain anything with a .git directory, your queue is about to have neighbors. Beautiful — and, if you're not careful, the month your CI meets someone's cousin's script. I've been the maintainer who flinched at a malicious hunk and the first-timer whose tiny PR got reviewed with kindness. Both wrote this post. The goal isn't keeping strangers out; it's keeping the door open without leaving it unlocked.

Review order: pipeline, deps, then everything

# 1. anything that EXECUTES — a typo PR that edits workflows is not a typo PR
git diff main...HEAD --stat -- '.github/' 'Makefile' '*.mk' 'Dockerfile' '*compose*.y*ml'

# 2. dependency deltas — typosquats live here dressed as helpfulness
git diff main...HEAD -- '*requirements*.txt' 'package.json' 'go.mod' 'Cargo.toml'
#    three questions per new package: who maintains it, how old is it, why this one

# 3. then everything else, at the depth its blast radius deserves

The three exfil shapes (grep them)
Most malicious October diffs aren't clever — same three shapes, different variable names:

git diff main...HEAD | grep -nE \
  'process\.env|os\.environ|getenv|/proc/self/environ'   # environment going somewhere
git diff main...HEAD | grep -nE \
  'base64|eval\(|atob|xxd -r'                            # obfuscation in an unobfuscated repo
git diff main...HEAD | grep -nE \
  'curl .*(-d|--data|-F)|wget |requests\.post|fetch\('   # new outbound calls

The real tell is scope creep: the mismatch between the title's promise and the diff's appetite. A fix-typo that adds a network call, a cron, or a download gets the hard look. Paranoia is vague and exhausts you; a checklist is cheap.

The safe room: fork PRs and disposable runners

Fork PRs never see secrets. Keep the platform defaults that make this true (approval required before a fork first-timer's workflow runs; no secret exposure to forks). A stranger's first PR should execute in a room with nothing valuable in it.
Every job gets a throwaway box. My CI runs each job in a fresh microVM — on Krova Cloud a job spins a Cube with its own kernel and no public IP, then dies:

CUBE="ci-job-$PR_NUMBER-$RUN_ID"
krova cubes create "$CUBE" --cpu 2 --ram 4 --disk 40 --image ubuntu-24.04
# run the PR's tests inside; fork PRs mount zero secrets
krova cubes delete "$CUBE"     # teardown is the security feature

A poisoned cache from a malicious build can't greet the next build, because there is no next build on the same machine. When the room is safe, review relaxes from interrogation to conversation.

Be kind on purpose (with receipts)
The three-day-old account is someone's first week in this community. The checklist handles the threat; your words handle the human:

Thanks for this! The typo fix is perfect — merging. 🎉
One note: the workflow change isn't needed for the typo, so I've
split it into its own PR where it can get a proper review.
Welcome aboard — really glad your first one landed here.

Say thanks in every review including rejects. Explain a "no" in two sentences; a silent close teaches nobody. Never talk to a first-timer like a threat actor.

The honest part
You cannot deep-review a flood. Tier it: CI/deps/scripts get the full checklist; docs and typos get a fast, grateful yes. The tiering is the practice.
Automation flags shapes; humans read intent. A perfect auto-merge tool is just a faster door.
The safe room doesn't replace review. Isolation limits what a bad PR can do; only reading limits what a bad PR can become.

Keep the door open
October's queue is a gift pile. Some gifts are typos, some are gems, one might be wired. Open all of them — just open the wired one with the right room behind you.

My Smart Bulb Shares a Network With My Production Keys.

Dhruv Malaviya — Sat, 03 Oct 2026 08:10:56 +0000

Nineteen devices on one flat home LAN: my work laptop with prod access, a 2021-firmware bulb, a camera from a dead brand, and an ESP_4F2A nobody admits to buying. The fix: split the network, promote the router to perimeter, and treat home like a café — tunnels, scoped contexts, no standing creds.

I opened my router's device list like you open a fridge at midnight — without a plan, slightly guilty. Nineteen devices. My work laptop, which can reach production, was on the same flat home network as a $12 smart bulb with 2021 firmware, a camera from a brand whose website no longer exists, a TV phoning home to an unplaceable timezone, and something labeled ESP_4F2A nobody in my household admits to buying.

A home LAN is flat: everything trusts everything. The security of the machine holding my prod keys was, that evening, partially defined by a light bulb I bought on sale.

The living-room test
The café test for laptops has a home version: if the cheapest device on your network gets popped, what's the path from there to the thing you'd cry about? On a flat LAN the answer is usually "a scan, then a try." Not because the bulb is an evil mastermind — because flat networks turn one weak device into a viewing position over everything else.

First, see the whole estate:

# who actually lives on your LAN (router DHCP list is the honest source; this cross-checks)
nmap -sn 192.168.1.0/24 | grep -E 'Nmap scan report|MAC'

The split (free, boring, an afternoon)

work vlan  : laptop, work phone        → reaches SSH endpoints, nothing else inbound
iot vlan   : bulb, camera, TV, ESP_*   → internet only; sandbox where blast radius = each other
guest wlan : visitors                  → internet only, timed
# exceptions are deliberate, visible, few:
#   laptop → printer (print only)
#   phone  → doorbell app

Router settings, zero dollars. The bulb keeps bulbing; it just loses its view of my laptop. Then the router itself got promoted to perimeter: admin password off the sticker, auto-update on, remote management off. It's the only device in the house actually doing security — it deserved the ceremony.

Home is a café now
The laptop rules travel home: no standing production credentials, short-lived sessions, and services reached by tunnel — nothing listens publicly just because I'm on the couch:

#!/bin/bash
# home-is-a-cafe.sh
krova context use home          # scoped: create/wake/ssh ok; delete prod = no

krova ssh db-1  -L 5432:localhost:5432 &   # postgres over the encrypted channel
krova ssh web-1 -L 8080:localhost:8080 &   # app same; internet gets nothing

# local clients now hit localhost; the databases never "listened for me"
psql -h localhost -p 5432 -U app app_db

A popped bulb's "scan, then try" now finds localhost ports bound to encrypted tunnels it can't join, and a context whose worst day is a forgotten dev box — visible in one list, deleted in one word.

The honest part

Segmentation is messy in real life. The printer needs the laptop; the doorbell wants the phone; convenience files weekly complaints. The goal isn't a perfect diagram — it's that the default path from popped bulb to prod keys is gone, and exceptions are deliberate.
You can't audit a household. Kids' tablets, a partner's phone, the in-laws visiting Wi-Fi. The split is the admission that you'll never control every device — so control what they can see instead.
Home hygiene doesn't fix the castle. The walls still live server-side: no public IP by default, own kernel per Cube, default-deny inbound, scoped secrets. Segmentation shrinks the path to those walls; it doesn't replace them.

Go look at your device list
Not tomorrow. Now. Count the devices, find the oldest firmware, and ask the only question that matters: does that device share a network with the keys?

Mine did — neighbors for two years, and the bulb had the worse posture. The fix cost an afternoon and a new SSID name. Best security review I've done all year, and the arithmetic was free.

I Did Recon on My Own Company , Using Only What We Published.

Dhruv Malaviya — Wed, 30 Sep 2026 12:01:37 +0000

Twenty minutes of public-only recon on my own company: the README architecture diagram, job postings as a vendor list, an error-message hostname, .env.example as a secrets schema. The fix: split the manual from the map, scrub errors at the boundary, review jobs like publications.

Quarterly habit: twenty minutes attacking myself with public information only. No credentials, no insider knowledge. This quarter I pointed it at the asset I'd never audited because I never thought of it as one: our documentation.

The notebook I ended up with made me slightly nauseous.

What the public pile gave away

README architecture diagram — service names, queue names, the worker fleet. A map with a legend, updated last quarter.
A support-thread screenshot — an error toast containing an internal hostname and stack path. Errors are involuntary documentation.
Job postings — "must know Kafka, Datadog, Auth0, Terraform" is a vendor list with a budget attached, including which parts we're unhappy enough to replace.
A 2023 conference talk, still indexed, "simplified" diagram 90% accurate.
.env.example — variable names are a schema of your secrets; ours enumerated every third-party integration, alphabetically, with comments.

None of it was a breach. All of it was publishing. Recon is the cheapest phase of an attack, and we were subsidizing it.

Run your own twenty minutes (the sweep)

# 1. involuntary docs: hostnames, internal ranges, vendor names in the repo
grep -rniE '(\.internal|\.local|10\.0\.|192\.168\.|hostname|stack)' docs/ README.md | head -30

# 2. the secrets schema
cat .env.example          # read it like an attacker: that's an inventory

# 3. your jobs page is a publication
grep -iE 'kafka|datadog|auth0|pagerduty|terraform' jobs/*.md

# 4. then do the human part:
#    - search "yourproduct architecture"
#    - site:yourdomain.com diagram
#    - open your last three conference/blog artifacts

The manual vs. the map
Docs have two audiences by default, and only one is friendly; the document can't tell them apart. So we split:

docs-public/    # the MANUAL — behavior, interfaces, examples. Generous.
docs-internal/  # the MAP — topology, internal names, runbooks, vendor wiring.
                # Access-controlled, reviewed like code.

And the habits that cut involuntary leaks:

Errors get scrubbed at the boundary — what happened, never where:

app.use((err, req, res, next) => {
  const id = crypto.randomUUID();
  log.error({ id, err });                    // hostnames + stacks stay in logs
  res.status(err.status || 500).json({
    error: "Something went wrong.",          // no internal names in the toast
    ref: id
  });
});

.env.example documents shape, not inventory:

- # Stripe live key for payments team
- STRIPE_KEY=sk_live_...
- KAFKA_BROKERS=kafka-3.internal:9092
+ PAYMENTS_KEY=        # your payments provider key (scoped, rotated)
+ EVENT_BROKER=        # event pipeline endpoint

Job postings reviewed like publications:

- Own our Kafka + Datadog + Auth0 stack.
+ Own high-throughput event pipelines, observability, and auth integrations.

Candidates understand the second version perfectly. So does the person drafting your attack plan — except they learn less from it.

The honest part

Not security by obscurity. The walls stay walls: no public IP by default, own kernel per Cube, default-deny inbound, scoped secrets. Doc discipline just stops handing out the floor plan of the building the walls protect.
Open source moves the line, not the principle. If your topology is the repo, the discipline shifts to what isn't: your environments, vendors, operational wiring. Publish the software's manual; your instance's manual is yours.
Total erasure is impossible. Mirrored slides live forever. The goal is that today's publishing is deliberate, and the freshest intel an attacker gets is stale.

Run your own twenty minutes
Search your product name plus "architecture." Read your last three job postings like a stranger with a grudge. Open the .env.example. Screenshot one customer-facing error.

Then ask, for each artifact: who needs this to use us — and who else just learned something?

Staging Went Down Last Month. Our Customers Tweeted About It.

Dhruv Malaviya — Sat, 26 Sep 2026 16:27:17 +0000

Our 'staging' environment had real tenants via a forgotten DNS door, a two-year-old prod dump, and debug shortcuts — because environments are labeled by intention, not contents. The fix: classify by data, and make staging ephemeral with nightly rebirth on Krova Cloud.

One guard, courtesy of a commenter: the rebirth has a supply chain. The scrub and seed scripts are built against a schema, and migrations move the schema under them. If they live in a wiki or a separate repo, a migration silently breaks the nightly rebirth and nobody notices until the next demo. So they're versioned right next to the schema migrations — same repo, same PR — and the rebirth ends with a smoke test (healthz plus a sane row count) that alerts on failure. Cleanup became a property of the system; this keeps the system honest about its own breakage.

The alert said staging was unreachable. My first thought was who cares. The second, four seconds later: then why are customers tweeting that they can't log in?

The investigation found staging had been serving real users for months — a forgotten DNS record, a redirect from an old marketing page, customers onboarded during a demo and never moved. Staging hadn't "become production." It had become production for some people, which is the same thing with worse defaults.

What the audit found

A prod dump, copied "just once" two years ago. Real names, real emails, real rows. A copy of production data is production-shaped, whatever the hostname says.
Relaxed rules everyone knew about. Debug endpoints. An admin panel behind a path instead of a password. A test account whose password lives in an offsite whiteboard photo. In prod these are incidents; in staging they were convenience — right up until real people lived there.
No backups, no alerts, no drills. Nobody backs up a costume. Then the costume had tenants.

The rule we now enforce in every planning meeting: production isn't a name. It's wherever your users' data lives.

Classify by data, not hostname
Anything holding real user data gets production care — backups, alerts, auth, patching — even if it's called staging, demo, or bob-box. Real data enters scrubbed or not at all; the "one quick dump" is banned.

And staging itself is now ephemeral by design: it dies and is reborn nightly, scrubbed on boot, deleted every Friday. An environment that dies weekly cannot accumulate secrets, drift, tenants, or folklore.

#!/bin/bash
# staging-reborn.sh — nightly
set -euo pipefail

krova cubes delete staging        # old staging dies completely

krova cubes create staging \
  --image ubuntu-24.04 \
  --ssh-key "$(cat ~/.ssh/deploy.pub)" \
  --vcpu 2 --ram 4 --disk 40 \
  --user-data "$(cat scrub-cloud-init.yml)"

#cloud-config
# scrub-cloud-init.yml — staging is reborn clean every night
runcmd:
  - curl -sf -H "Authorization: Bearer ${DUMP_TOKEN}"
      https://storage.private/scrubbed-dump.sql.gz
      | gunzip | sudo -u postgres psql app
  - systemctl restart app

On Krova Cloud this lifecycle is affordable because it's billed by the minute, and a stopped Cube bills only its disk — Friday's delete is the default state, not an event.

The doors that make environments lie
Orphaned DNS is how staging acquires tenants. The weekly check:

while read -r d; do
  t=$(dig +short "$d" CNAME | head -1)
  # any record still pointing at staging is a door
  [ -n "$t" ] && case "$t" in *staging*) echo "DOOR: $d -> $t";; esac
done < domains.txt

Plus the classification check — ask staging what it's holding, not what it's named:

krova ssh staging -- sudo -u postgres psql app -tAc \
  "SELECT count(*) FROM users WHERE email NOT LIKE '%@example.com';"
# any number above zero means staging is lying to you

krova ssh staging -- ss -tlnp
# debug ports listening? that's a prod-shaped wound on a staging-shaped box

The honest part

Synthetic data has a realism tax. Some bugs only bite real-shaped data — the 400-character name, the emoji in the address line. We use a smaller, weirder synthetic set plus occasional scrubbed real samples, reviewed like code. Perfection isn't on the menu; classification is.
Small teams can't afford full parity. Fine. The rule isn't "staging must equal production." It's "nothing real lives in a place with pretend rules." One sentence, enforceable at any size.
Ephemeral staging doesn't fix prod. Prod still needs real walls: no public IP by default, own kernel per Cube, scoped secrets. This post is about misclassification, not about replacing the castle.

Environments lie. Data doesn't.
Go ask your staging what it's holding — not the hostname, the contents. If the answer includes real names, real emails, or one customer who wanders in through a forgotten door, you don't have a staging environment. You have an unmonitored production wearing a costume.

Ours wore it for months. The customers never noticed. That's not a compliment. That's the scariest part.

I Audited My Dead Side Projects. Some Weren't Dead.

Dhruv Malaviya — Mon, 21 Sep 2026 13:06:00 +0000

Forty minutes of list-making found a VPS still running after two years, a dangling CNAME ripe for subdomain takeover, and a valid API key on a machine I'd mentally buried. The audit scripts, the decommission runbook, and the sleep-instead-of-rot policy.

Last Saturday I listed every project I'd ever deployed — side projects, hackathon demos, "quick tests." Forty minutes to write; reading it back took my breath away. Because dead projects don't die. They wait.

What the graveyard was hiding

Three projects I'd euthanized were still running — a VPS humming for two years with dependencies from the floats era. Forgotten app = dependency museum = vulnerability list.
A cancelled domain still CNAME'd to a service I no longer owned. The record dangled like an unplugged phone line; anyone claiming the orphaned resource receives traffic meant for me. Subdomain takeover, living exactly where nobody looks.
A .env on the old box with a key that was still valid. Two years of "I'll clean that up someday," one working key to my current life.

None of these were hacks. All of them were mine. The scariest attack surface I own isn't production — it's the graveyard.

The audit, automated where possible
Schools teach shipping. Nobody teaches burying. So here's the part of my audit that runs as a script:

#!/bin/bash
# graveyard-audit.sh — dangling DNS + estate inventory
set -euo pipefail

# 1. dangling CNAMEs: target no longer resolves = claimable by strangers
while read -r d; do
  t=$(dig +short "$d" CNAME | head -1)
  [ -n "$t" ] && [ -z "$(dig +short "$t" A)" ] \
    && echo "DANGLING: $d -> $t"
done < domains.txt

# 2. what's actually running, by state
krova list --json | jq -r '.[] | "\(.name)\t\(.state)"'

# 3. secret archaeology in old project dirs — then revoke every hit
grep -rEho '(kro|ghp|aws|sk)_[A-Za-z0-9_-]{8,}' old-projects/ | sort -u

The human part stays human: every CI variable, every service account, every "temporary" panel. The script catches the dangling and the running; you catch the remembered.

The decommission runbook (order matters)
A project isn't dead when the app is down. It's dead when nothing anywhere points at it or opens it:

# memory first, then the box
krova snapshots create oldproj --name burial
krova cubes delete oldproj          # delete dialog preserves a backup by default

# THEN, in the same sitting — this is the part everyone skips:
#  - delete the DNS records (A, CNAME, _acme-challenge, TXT)
#  - revoke the keys it held
#  - remove CI/CD variables and webhooks
#  - cancel service accounts / OAuth apps

Delete the box first and the DNS later, and "later" is where the graveyard grows. One sitting, full kill.

Sleep instead of rot

For the "someday maybe" tier, the respectful middle state: hibernation. On Krova Cloud a powered-off Cube bills only its disk — no running process, no open port, no public IP to begin with:

krova cubes power-off maybe-someday   # pennies/month, zero exposure
krova cubes wake maybe-someday        # revive in seconds when nostalgia strikes

Sleep is what keeps the graveyard from refilling: a project is either alive, asleep, or buried. Rotting is no longer a state I allow.

The honest part

A list isn't a fix. The audit produced a document; the boring weekend of deleting records and revoking keys produced the safety.
You can't audit what you never wrote down. New projects get a row the day they're born — domain, keys, and their planned death: who revokes what.
The graveyard isn't the castle. The living stack still needs real walls: no public IP by default, own kernel per Cube, scoped short-lived secrets. The audit shrinks the estate attackers can explore; it doesn't replace defending what's alive.

Dead projects don't die. They wait.
Every reader has a graveyard. Some of yours are glowing right now — a box you forgot, a record you orphaned, a key that outlived its project. The audit is forty minutes; the fixes are one boring weekend.

I Deleted My Entire Security Stack. My Apps Got Safer.

Dhruv Malaviya — Wed, 16 Sep 2026 19:01:13 +0000

fail2ban, the ufw ruleset, the WAF, the log-watchers — all purged in a month, and my apps got safer, because every one of them was a compensating control for a single inherited default: a public IP. The purge checklist, the before/after, and what I kept.

Uninstalling fail2ban felt like removing a smoke detector. My finger hovered for a genuine minute. Then I pressed enter, and over the next month almost everything went: the ufw ruleset I'd maintained for years, the WAF config, auditd's screaming cron, the nightly "security" scripts whose output I'd stopped reading in 2024.

My apps got safer. That sentence is the opposite of what it sounds like, so let me show the work.

The compensating question

Before the purge I asked what each tool was actually compensating for:

fail2ban → SSH reachable from the entire internet
ufw ruleset → a public address with ports that must be filtered
WAF → the app directly reachable by anything that finds its IP
log-watchers → the noise generated by all of the above

Different tools, one root cause: the machine has a public address — a decision I never made, just inherited, because "server comes with an IP" is how servers work. I wasn't running a security stack. I was running a payment plan on a bad default.

Changing the default
The workloads moved to Cubes on Krova Cloud — Firecracker microVMs, own kernel each, and the part that retired the stack: no public IP. Private NAT'd network, traffic through managed TLS ingress, default-deny inbound, only explicitly opened ports reachable. I open none.

Before/after, same eyes:

# BEFORE — old VPS, the street:
$ ss -tlnp
LISTEN 0.0.0.0:22      # the brute-force magnet
LISTEN 0.0.0.0:80
LISTEN 0.0.0.0:443

# AFTER — Cube, from inside:
$ krova ssh app-1 -- ip -brief addr
eth0  UP  10.0.x.x/24          # private. that's the whole list.

$ krova ssh app-1 -- ss -tlnp
LISTEN 127.0.0.1:8080          # the app, listening to itself

$ krova tcp list app-1
# empty. no port forwards, nothing to filter, nothing to ban.

With no address there's nothing to brute-force, so fail2ban has no job. With default-deny and no surface, the ruleset has no job. One migration weekend retired four tools and ~1,000 lines of compensating config.

The purge checklist (and why it's safe)

apt purge fail2ban          # no public SSH → nothing to ban
ufw disable                 # retired, NOT "off and exposed" —
                            # safe only because the platform default
                            # is private + default-deny inbound
# WAF config → archived; app-level validation stays in the app
crontab -e                  # delete auditd + nightly scanner lines

Read that ufw disable line carefully. It is safe because the exposure left first. On a box with a public IP, the same command is a breach with extra steps. Order matters: default first, meds second.

What I did NOT delete

Secrets hygiene — scoped, short-lived, injected at runtime.
Backups + restore drills — off-host copies, scheduled restores into a fresh Cube.
Patching & dependency cadence — the boring rhythm continues.
Application security — authn/authz, input validation, tenant checks. The purge hit tools defending the door; the app's own locks were never the target.

The honest part

Keep your tools if your symptom is mandatory. Gameservers, UDP media, weird protocols — real public-address workloads exist. This isn't "firewalls are a scam"; it's "ask whether the symptom is optional." For a shocking number of web workloads, it is.
Architecture isn't a sedative. No public IP shrinks the surface; it doesn't review your PRs. The threat model got smaller, not zero.
Ops stay yours. Isolation is the platform's job; patching, backups, drills remain the owner's, same as any self-hosted box.

I didn't remove the locks. I removed the street.
Most of my "security" was pain management for a street I never needed. When the street went away, the pharmacy closed.

Ask your stack the compensating question tonight. You might find a default you inherited, a wound you've been medicating for years — and a weekend where the meds go away.

My Monitoring Missed the Breach. My Credit Card Caught It.

Dhruv Malaviya — Mon, 14 Sep 2026 06:06:19 +0000

A leaked key spun up instances in a region my dashboards didn't watch; the invoice screamed first. Now I treat spend as a security sensor — per-minute resolution, a prepaid fuse, and a nightly audit on Krova Cloud.

The leak was a CI variable I'd forgotten existed, wired to a cloud API key with far too much confidence. Over one weekend, someone's script used it to spin up instances in a region I'd never deployed to.

My dashboards stayed green — they were watching the resources I knew about. Nobody's monitoring watches a region they forgot exists. The first thing that screamed was my credit card statement, Monday morning, looking like a typo.

Spend is the only metric that can't lie by omission
CPU, latency, error rates — they measure what you chose to point them at. Money aggregates everything, including what you never thought to watch. An attacker can stay quiet to avoid your alerts. They cannot consume resources without touching your bill. Every leaked key, runaway loop and forgotten box has one shared property: it costs money per minute it exists.

The invoice isn't accounting. It's the widest-coverage security sensor you own — and the most ignored. The catch is resolution and caps: a monthly invoice is a smoke detector that mails you a letter after the fire. I needed the signal during the fire, and a ceiling on the fire's size.

The setup on Krova
Three properties make spend usable as a control on Krova Cloud:

Per-minute resolution. Metered by the minute, no rounding up. High-resolution billing = high-resolution signal; a loop shows up within the hour, not at month-end.
A literal fuse. Billing is prepaid credit, and when a space's balance hits zero its Cubes are automatically powered off — no data loss. A leaked key physically cannot spend past the credit I chose in advance. Dollar-capped blast radius.
Hourly cost on every Cube. Each Cube reports its own costPerHour, so the audit is arithmetic, not estimation.

The nightly audit — expected list vs. actual list, plus burn rate:

#!/bin/bash
# spend-audit.sh — reality vs. budget, every night
set -euo pipefail
krova context use prod

# what's actually running, and what it burns
krova list --json | jq -r '.[]
  | select(.state=="running")
  | "\(.name)\t\(.resources.vcpu)vcpu\t$\(.costPerHour)/h"'

# total burn rate vs. plan
krova list --json | jq '[.[]
  | select(.state=="running") | .costPerHour] | add // 0'

0 2 * * *   spend-audit.sh | diff - expected.txt && true || alert "reality ≠ budget"
0 * * * *   burn-rate-check.sh   # alert if hourly spend exceeds plan + 20%

A stranger's box can't hide when the whole estate fits in one jq. And for in-guest anomalies, the complement is event-driven: Krova's webhooks include resource.alert.cpu/memory/disk — spend catches the economic anomaly, those catch the hot process. Tripwire plus camera.

Sizing the fuse
Prepaid means choosing your ceiling on purpose:

Sandbox space gets exactly one experiment's worth of credit. If it melts down, the fuse trips before the surprise does.
Prod gets normal load plus headroom — sized like a breaker: above normal, below pain. Misjudge it and the fuse is also an availability incident, so this is a staffing decision, not a slider.

The honest part

The bill says something is wrong, not what. Sensor first, forensics second: when spend twitches, the running-list and logs find the limb.
Cheap attacks hide under the noise. A patient $0.02/hour adversary sits inside your variance. Spend-catching nails the realistic failures — leaks, loops, zombies — not a careful human. For that you still need walls: no public IP by default, own kernel per Cube, default-deny inbound. The bill is a layer, not the castle.
Ops still yours. The audit tells you a box exists; patching, backups and drills remain your job, same as any self-hosted estate.

The cheapest SOC tool you already own
We file the invoice under "finance" instead of "security," then get surprised by it. Move it. Read it daily, alert on its rate of change, cap what it can reach.

My monitoring is good now. But the sensor I'd least want to lose is the one my accountant installed by accident.