DEV Community: Peter Novak

Who's watching private repos turn public? We leaked 75 AWS keys to find out

Peter Novak — Wed, 10 Jun 2026 20:03:04 +0000

Make a private repo public and it stays undiscovered for 6 minutes. Past that, the internet is reading it. We put a live AWS key in 75 throwaway repos, made each public for one of five timed windows from 60 seconds to 12 hours, 15 repos per window, and logged every use. The key was just a tripwire. What we wanted to know is how long it takes anyone watching to notice, and what their intent is once they have.

What follows are the answers, and then the part that matters more for most teams: without enterprise tooling, the repo could go public and nobody would reliably hear of it.

How fast the watchers noticed, and with what intent

Each dot in the chart below is one time something touched a key, placed by how long after the repo went public it landed, on a log scale. The chart here is a static image; the interactive version on the original post lets you hover any dot for the detail.

Two things are visible. First, the earliest action on a newly public repo came at 6 minutes, and most first visits landed around minute 8. Second, keys are used even after the repos were hidden again, so going private is not enough to undo the exposure.

The parties that showed up split three ways. Only one went further than the rest and probed the account behind the key. Most did nothing but check, over and over, that the key still worked. And one was not an outside party at all: AWS, locking the key down on a signal from GitHub.

The one that went further

Outside parties used the keys 139 times. 5 of those came from a single host running the AWS SDK for Python. It called DescribeInstances to list EC2 machines and ListTables to list DynamoDB tables, mapping out the account rather than just confirming the key worked. That was the only hands-on activity in the whole run, and it touched just 2 repos.

The ones that only checked

The harvesters. 126 of the 139 uses came from a single operation on 2 OVH cloud IPs, every call a GetCallerIdentity check that the key was live and nothing more. It touched 32 repos and kept re-checking captured keys for a day; 38 of the 139 hits landed after the repo was already private again.
The TruffleHog-based scanner. 8 uses came from one Hetzner host running TruffleHog, the open-source secret scanner. It called GetCallerIdentity once per repo across 8 repos and moved on.
The email scanners. GitGuardian and a service called leakscanner found the keys and emailed us, GitGuardian on 9 repos, leakscanner on 1. We cannot tie any of the 139 uses to either of them; for all the data shows, they read the repo, emailed, and went no further.

Not an outside party: AWS

The moment a repo went public, GitHub's free, built-in secret scanning kicked in and reported the exposed secret to AWS. AWS then attached a quarantine policy that strips the key of its dangerous permissions. That lockdown, the green dots in the chart, did not land until 4.5 to 9 hours after exposure. Because the catch is GitHub's own, even the 60-second repos had their keys quarantined, the ones no outside party ever reached.

The blind spot below enterprise tools

Whoever showed up, the exposure was the same. A repo goes public, the internet starts reading at minute 6, and the source sits exposed beside the key. And the flip isn't always an accident: turning a private repo public is a recognized way to exfiltrate code, watched for by detection rules and catalogued by MITRE. You can rotate a key; you can't un-share code.

GitHub logs the flip and tells you nothing, the same as branch protection coming off, a check dropped, a reviewer removed, or scanning switched off. Tools that catch it do exist. The security-posture platforms big companies run will flag it, but they assume a security team to run them and a budget to match, so a smaller team that only wants to know when a repo setting changes has nothing built for it.

Closing the blind spot

That's what we're building Codatus for: a read-only install that watches the settings this post is about, a repo going public, branch protection coming off, a check or reviewer dropped, scanning switched off, and tells you the moment one changes.

We're planning to launch at $99/month, with early users locked in at that rate. Reserve early access and we'll tell you when it's ready.

Only 2 of 128 YC-backed dev tools companies block unchecked merges

Peter Novak — Thu, 28 May 2026 17:12:47 +0000

We scored 6,195 public GitHub repos at 128 YC-backed dev tools companies on four rules. The median scored 21 out of 100; no company cleared 80. Apollo took the top spot at 71.

But the most interesting thing isn't the scores. It's the pattern in the data: of the 44 companies that enable branch protection on most of their repos, only 2 block unchecked merges.

Scores

The rules

Each rule is summarized below; full definitions and the limits of public-scan mode live in a previous post. Scanner source: github.com/CodatusHQ/scanner.

Has branch protection. The default branch requires a pull request before changes can land. Median pass rate across the 128: 33%.
Has required checks. At least one check (status check, workflow, code scan, or deployment) must succeed before a merge. Median: 2%.
Has CODEOWNERS. A CODEOWNERS file exists at .github/, the repo root, or docs/. Median: 2%.
Has CI workflow. A recognized CI configuration is committed to source. Median: 45%.

CI passes most often, branch protection next, and required checks and CODEOWNERS almost never. 209 of 6,195 repos pass all four (3.4%); 1,398 pass two or three (22.6%); the remaining 4,588 (74.1%) pass zero or one.

The ranking

#	Company	YC batch	Score	BP	Chk	CO	CI
1	Apollo	Summer 2011	71	74%	58%	67%	86%
2	Formance	Summer 2021	69	96%	61%	38%	83%
3	Supabase	Summer 2020	61	100%	22%	38%	85%
4	Mezmo	Winter 2015	58	93%	26%	33%	80%
5	ParadeDB	Summer 2023	58	100%	0%	64%	71%
6	Seam	Summer 2020	56	52%	34%	65%	73%
7	Doppler	Winter 2019	55	100%	22%	16%	83%
8	RevenueCat	Summer 2018	54	100%	23%	36%	58%
9	Tailor	Summer 2022	54	73%	6%	60%	80%
10	QuestDB	Summer 2020	52	100%	7%	12%	90%
11	MagicBell	Winter 2021	51	91%	33%	0%	83%
12	Rainforest	Summer 2012	50	93%	44%	0%	65%
13	authzed	Winter 2021	49	81%	23%	23%	72%
14	Tempo	Summer 2023	46	100%	8%	11%	68%
15	Replit	Winter 2018	44	78%	29%	24%	48%
16	Rootly	Summer 2021	44	90%	28%	0%	61%
17	Docker	Summer 2010	43	78%	15%	21%	60%
18	Reflex	Winter 2023	43	100%	0%	5%	68%
19	Massdriver	Winter 2022	42	75%	0%	23%	72%
20	Infisical	Winter 2023	42	100%	1%	3%	65%
21	Infracost	Winter 2021	42	97%	2%	8%	62%
22	Teleport	Summer 2015	42	96%	9%	9%	56%
23	Aviator	Summer 2021	42	76%	29%	17%	47%
24	MindsDB	Winter 2020	41	84%	6%	12%	62%
25	Svix	Winter 2021	41	40%	20%	46%	60%
26	Imgix	Summer 2011	39	50%	28%	30%	48%
27	Embrace	Summer 2019	38	37%	8%	41%	66%
28	BotCity	Winter 2022	37	100%	0%	0%	50%
29	Castle	Winter 2016	36	90%	3%	0%	53%
30	Tiptap	Summer 2023	36	90%	0%	5%	50%
31	SuperTokens	Summer 2020	35	58%	22%	4%	58%
32	Mindee	Winter 2021	35	71%	4%	4%	61%
33	ReadMe	Winter 2015	34	48%	6%	25%	60%
34	DeepSource	Winter 2020	34	100%	0%	0%	37%
35	Fintoc	Winter 2021	34	76%	7%	0%	53%
36	Mux	Winter 2016	33	37%	2%	39%	54%
37	PropelAuth	Winter 2022	32	76%	0%	2%	52%
38	GrowthBook	Winter 2022	32	39%	12%	3%	75%
39	Airbyte	Winter 2020	32	48%	18%	14%	48%
40	Heroic Labs	Summer 2015	32	100%	0%	0%	30%
41	Amplitude	Winter 2012	31	50%	3%	7%	65%
42	Cortex	Winter 2020	31	46%	10%	20%	50%
43	Escape	Winter 2023	31	28%	4%	40%	52%
44	Trigger.dev	Winter 2023	30	100%	1%	0%	22%
45	Avo	Winter 2019	30	84%	3%	0%	34%
46	Hubble Network	Winter 2022	30	47%	0%	11%	64%
47	PostHog	Winter 2020	29	33%	26%	12%	46%
48	Depot	Winter 2023	29	28%	2%	0%	87%
49	PagerDuty	Summer 2010	29	83%	3%	3%	27%
50	Lamin	Summer 2022	29	23%	0%	0%	94%
51	Signadot	Winter 2020	29	66%	0%	6%	46%
52	OneSignal	Summer 2011	28	45%	2%	11%	55%
53	Exa	Summer 2021	28	100%	0%	0%	14%
54	WarpBuild	Summer 2021	28	41%	0%	8%	66%
55	Porter	Summer 2020	26	54%	2%	0%	50%
56	Convoy	Winter 2022	25	56%	3%	0%	43%
57	Alpaca	Winter 2019	24	45%	2%	10%	40%
58	Zeplin	Summer 2015	24	46%	3%	0%	50%
59	Dailybot	Summer 2021	24	33%	25%	0%	41%
60	BuildBuddy	Winter 2020	23	35%	17%	0%	41%
61	Raycast	Winter 2020	23	40%	0%	0%	53%
62	FifthTry	Winter 2021	22	2%	0%	0%	86%
63	Skyhook	Winter 2023	22	2%	0%	2%	86%
64	Beam	Winter 2022	22	36%	0%	0%	52%
65	Supernova	Winter 2019	21	82%	0%	0%	2%
66	Mintlify	Winter 2022	21	70%	3%	3%	11%
67	Alokai	Winter 2021	21	37%	0%	18%	31%
68	Elementary	Winter 2022	21	26%	6%	6%	46%
69	Dagger	Winter 2019	20	62%	0%	3%	15%
70	Bitmovin	Summer 2015	19	32%	12%	2%	31%
71	Wasmer	Summer 2019	19	17%	12%	3%	47%
72	CodeCrafters	Summer 2022	19	8%	7%	0%	62%
73	hoop.dev	Winter 2021	18	14%	0%	4%	57%
74	Vellum	Winter 2023	18	8%	8%	8%	50%
75	Continue	Summer 2023	17	7%	4%	1%	59%
76	Quicknode	Winter 2021	17	42%	0%	0%	27%
77	Superwall	Summer 2021	17	12%	4%	8%	44%
78	Courier	Summer 2019	16	27%	3%	1%	35%
79	Glide	Winter 2019	16	16%	6%	6%	39%
80	Nango	Winter 2023	16	20%	13%	0%	33%
81	Flowglad	Winter 2020	16	16%	8%	0%	41%
82	Algolia	Winter 2014	15	22%	8%	3%	30%
83	Font Awesome	Summer 2015	15	12%	0%	0%	48%
84	Rulebricks	Winter 2021	15	0%	0%	0%	62%
85	Speedscale	Summer 2020	15	12%	6%	0%	43%
86	SigNoz	Winter 2021	14	22%	6%	8%	20%
87	Inconvo	Summer 2023	14	25%	0%	0%	33%
88	Roboflow	Summer 2020	13	23%	3%	6%	23%
89	Shuttle	Summer 2020	13	14%	2%	4%	34%
90	Retool	Winter 2017	13	33%	0%	2%	17%
91	Windmill	Summer 2022	13	8%	0%	4%	43%
92	Ultralight	Winter 2019	13	0%	0%	0%	53%
93	Hyperbeam	Winter 2022	13	54%	0%	0%	0%
94	hotglue	Summer 2021	12	0%	0%	0%	51%
95	Ditto	Winter 2020	12	30%	10%	0%	10%
96	Evidently AI	Summer 2021	12	10%	0%	0%	40%
97	Jovian	Summer 2021	12	30%	0%	0%	20%
98	Bitrise	Winter 2017	11	30%	13%	0%	3%
99	DrDroid	Winter 2023	11	30%	0%	0%	16%
100	AssemblyAI	Summer 2017	11	20%	4%	0%	20%
101	Artillery	Summer 2021	11	10%	0%	5%	31%
102	Evidence	Summer 2021	10	14%	3%	0%	25%
103	Okteto	Winter 2019	9	14%	0%	5%	19%
104	Curvenote	Winter 2021	9	6%	0%	0%	30%
105	HackerRank	Summer 2011	9	15%	5%	1%	15%
106	Pipekit	Summer 2021	9	27%	0%	0%	11%
107	Lightdash	Summer 2020	8	3%	0%	3%	29%
108	Parea	Summer 2023	8	7%	0%	0%	28%
109	Datasaur	Winter 2020	8	25%	0%	0%	8%
110	Expo	Summer 2016	7	8%	1%	3%	19%
111	Karate Labs	Winter 2022	7	7%	0%	0%	23%
112	Webiny	Winter 2021	6	7%	5%	0%	14%
113	Nullstone	Winter 2022	6	4%	0%	0%	22%
114	Inkeep	Winter 2023	5	2%	0%	2%	16%
115	Boundary	Winter 2023	5	2%	2%	0%	17%
116	Draftbit	Winter 2018	5	3%	3%	0%	15%
117	Firecrawl	Summer 2022	4	4%	0%	1%	12%
118	Medplum	Summer 2022	4	11%	2%	2%	4%
119	LiteLLM	Winter 2023	4	4%	4%	0%	11%
120	Mito	Summer 2020	4	0%	0%	0%	17%
121	Cosmic	Winter 2019	3	8%	0%	0%	4%
122	NanoNets	Winter 2017	3	0%	0%	0%	12%
123	Velt	Winter 2022	2	0%	0%	2%	6%
124	Dockup	Winter 2019	2	0%	0%	0%	8%
125	Cosine	Winter 2023	1	1%	0%	0%	6%
126	Release	Winter 2020	1	0%	0%	0%	6%
127	Termii	Winter 2020	0	0%	0%	0%	0%
128	Jet Admin	Winter 2020	0	0%	0%	0%	0%

Column key: BP = branch protection, Chk = required checks, CO = CODEOWNERS, CI = CI workflow. Each percentage is the share of an org's scanned repos that pass that rule. Score is the weighted aggregate (0-100).
Sortable version with per-company scorecards: codatus.com/blog/only-2-of-128-yc-backed-dev-tools-companies-block-unchecked-merges/.

How the 128 were chosen

The starting universe of 549 companies is the union of YC's developer-tools (532) and devops (50) tags, pulled from yc-oss.github.io and deduplicated on slug. We narrowed from there:

Operating companies. Companies whose YC status reads "Inactive" or "Acquired" were removed. Companies marked "Public-on-stock-market" were kept; they're still operating dev tools businesses, just at different scale. 142 dropped. 407 remaining.
Mature batches. Batches Winter 2024 and later were removed. Companies that recently entered YC haven't been around long enough to have settled engineering practices. 166 dropped. 241 remaining.
Verified GitHub org. We matched each company to a GitHub organization via homepage links and GitHub search, requiring either a domain match or an exact name match to avoid mis-attributions. 47 dropped. 194 remaining.
Non-trivial public footprint. We required at least 10 active (non-fork, non-archived) public repos per org. 66 dropped. 128 remaining.

The cohort includes two publicly-traded YC alumni: Amplitude (Winter 2012, rank 41, score 31) and PagerDuty (Summer 2010, rank 49, score 29). GitLab (Winter 2015) passed the earlier filters but drops at the public-footprint step; their GitHub footprint is two forked repos because they host on gitlab.com.

Pattern

Something jumped out while we were scoring the cohort: branch protection passes for a real chunk of the dataset, but required checks barely register. To see how this plays out, we plotted each company on both: branch protection pass rate on one axis, required checks pass rate on the other.

Three of the four quadrants have companies in them. The top-left is empty: required checks attach to a protected branch, so the configuration can't exist.

The top-right is the rare exception. Of the 44 companies with branch protection on most of their repos, only 2 also require a check: Apollo (BP 74%, Chk 58%) and Formance (BP 96%, Chk 61%).

That leaves 42 in the bottom-right. They enable branch protection on most of their repos without requiring any check. Every change opens a PR; nothing has to pass for the PR to merge. Supabase is the extreme case (BP 100%, Chk 22%).

The bottom-left holds the remaining 84 companies. Branch protection isn't enabled on most of their repos, so there's no workflow to gate.

The pattern is clear across the cohort: most companies have either no gate or a workflow that doesn't enforce anything.

See where you land

The 128 companies in the leaderboard are public-scan results. Install Codatus on your own GitHub org for a full scan, private repos included.