DEV Community

Cover image for Only 2 of 128 YC-backed dev tools companies block unchecked merges
Peter Novak
Peter Novak

Posted on • Originally published at codatus.com

Only 2 of 128 YC-backed dev tools companies block unchecked merges

We scored 6,195 public GitHub repos at 128 YC-backed dev tools companies on four rules. The median scored 21 out of 100; no company cleared 80. Apollo took the top spot at 71.

But the most interesting thing isn't the scores. It's the pattern in the data: of the 44 companies that enable branch protection on most of their repos, only 2 block unchecked merges.

Scores

The rules

Each rule is summarized below; full definitions and the limits of public-scan mode live in a previous post. Scanner source: github.com/CodatusHQ/scanner.

  • Has branch protection. The default branch requires a pull request before changes can land. Median pass rate across the 128: 33%.
  • Has required checks. At least one check (status check, workflow, code scan, or deployment) must succeed before a merge. Median: 2%.
  • Has CODEOWNERS. A CODEOWNERS file exists at .github/, the repo root, or docs/. Median: 2%.
  • Has CI workflow. A recognized CI configuration is committed to source. Median: 45%.

Median pass rates across the four rules: branch protection 33%, required checks 2%, CODEOWNERS 2%, CI 45%

CI passes most often, branch protection next, and required checks and CODEOWNERS almost never. 209 of 6,195 repos pass all four (3.4%); 1,398 pass two or three (22.6%); the remaining 4,588 (74.1%) pass zero or one.

The ranking

# Company YC batch Score BP Chk CO CI
1 Apollo Summer 2011 71 74% 58% 67% 86%
2 Formance Summer 2021 69 96% 61% 38% 83%
3 Supabase Summer 2020 61 100% 22% 38% 85%
4 Mezmo Winter 2015 58 93% 26% 33% 80%
5 ParadeDB Summer 2023 58 100% 0% 64% 71%
6 Seam Summer 2020 56 52% 34% 65% 73%
7 Doppler Winter 2019 55 100% 22% 16% 83%
8 RevenueCat Summer 2018 54 100% 23% 36% 58%
9 Tailor Summer 2022 54 73% 6% 60% 80%
10 QuestDB Summer 2020 52 100% 7% 12% 90%
11 MagicBell Winter 2021 51 91% 33% 0% 83%
12 Rainforest Summer 2012 50 93% 44% 0% 65%
13 authzed Winter 2021 49 81% 23% 23% 72%
14 Tempo Summer 2023 46 100% 8% 11% 68%
15 Replit Winter 2018 44 78% 29% 24% 48%
16 Rootly Summer 2021 44 90% 28% 0% 61%
17 Docker Summer 2010 43 78% 15% 21% 60%
18 Reflex Winter 2023 43 100% 0% 5% 68%
19 Massdriver Winter 2022 42 75% 0% 23% 72%
20 Infisical Winter 2023 42 100% 1% 3% 65%
21 Infracost Winter 2021 42 97% 2% 8% 62%
22 Teleport Summer 2015 42 96% 9% 9% 56%
23 Aviator Summer 2021 42 76% 29% 17% 47%
24 MindsDB Winter 2020 41 84% 6% 12% 62%
25 Svix Winter 2021 41 40% 20% 46% 60%
26 Imgix Summer 2011 39 50% 28% 30% 48%
27 Embrace Summer 2019 38 37% 8% 41% 66%
28 BotCity Winter 2022 37 100% 0% 0% 50%
29 Castle Winter 2016 36 90% 3% 0% 53%
30 Tiptap Summer 2023 36 90% 0% 5% 50%
31 SuperTokens Summer 2020 35 58% 22% 4% 58%
32 Mindee Winter 2021 35 71% 4% 4% 61%
33 ReadMe Winter 2015 34 48% 6% 25% 60%
34 DeepSource Winter 2020 34 100% 0% 0% 37%
35 Fintoc Winter 2021 34 76% 7% 0% 53%
36 Mux Winter 2016 33 37% 2% 39% 54%
37 PropelAuth Winter 2022 32 76% 0% 2% 52%
38 GrowthBook Winter 2022 32 39% 12% 3% 75%
39 Airbyte Winter 2020 32 48% 18% 14% 48%
40 Heroic Labs Summer 2015 32 100% 0% 0% 30%
41 Amplitude Winter 2012 31 50% 3% 7% 65%
42 Cortex Winter 2020 31 46% 10% 20% 50%
43 Escape Winter 2023 31 28% 4% 40% 52%
44 Trigger.dev Winter 2023 30 100% 1% 0% 22%
45 Avo Winter 2019 30 84% 3% 0% 34%
46 Hubble Network Winter 2022 30 47% 0% 11% 64%
47 PostHog Winter 2020 29 33% 26% 12% 46%
48 Depot Winter 2023 29 28% 2% 0% 87%
49 PagerDuty Summer 2010 29 83% 3% 3% 27%
50 Lamin Summer 2022 29 23% 0% 0% 94%
51 Signadot Winter 2020 29 66% 0% 6% 46%
52 OneSignal Summer 2011 28 45% 2% 11% 55%
53 Exa Summer 2021 28 100% 0% 0% 14%
54 WarpBuild Summer 2021 28 41% 0% 8% 66%
55 Porter Summer 2020 26 54% 2% 0% 50%
56 Convoy Winter 2022 25 56% 3% 0% 43%
57 Alpaca Winter 2019 24 45% 2% 10% 40%
58 Zeplin Summer 2015 24 46% 3% 0% 50%
59 Dailybot Summer 2021 24 33% 25% 0% 41%
60 BuildBuddy Winter 2020 23 35% 17% 0% 41%
61 Raycast Winter 2020 23 40% 0% 0% 53%
62 FifthTry Winter 2021 22 2% 0% 0% 86%
63 Skyhook Winter 2023 22 2% 0% 2% 86%
64 Beam Winter 2022 22 36% 0% 0% 52%
65 Supernova Winter 2019 21 82% 0% 0% 2%
66 Mintlify Winter 2022 21 70% 3% 3% 11%
67 Alokai Winter 2021 21 37% 0% 18% 31%
68 Elementary Winter 2022 21 26% 6% 6% 46%
69 Dagger Winter 2019 20 62% 0% 3% 15%
70 Bitmovin Summer 2015 19 32% 12% 2% 31%
71 Wasmer Summer 2019 19 17% 12% 3% 47%
72 CodeCrafters Summer 2022 19 8% 7% 0% 62%
73 hoop.dev Winter 2021 18 14% 0% 4% 57%
74 Vellum Winter 2023 18 8% 8% 8% 50%
75 Continue Summer 2023 17 7% 4% 1% 59%
76 Quicknode Winter 2021 17 42% 0% 0% 27%
77 Superwall Summer 2021 17 12% 4% 8% 44%
78 Courier Summer 2019 16 27% 3% 1% 35%
79 Glide Winter 2019 16 16% 6% 6% 39%
80 Nango Winter 2023 16 20% 13% 0% 33%
81 Flowglad Winter 2020 16 16% 8% 0% 41%
82 Algolia Winter 2014 15 22% 8% 3% 30%
83 Font Awesome Summer 2015 15 12% 0% 0% 48%
84 Rulebricks Winter 2021 15 0% 0% 0% 62%
85 Speedscale Summer 2020 15 12% 6% 0% 43%
86 SigNoz Winter 2021 14 22% 6% 8% 20%
87 Inconvo Summer 2023 14 25% 0% 0% 33%
88 Roboflow Summer 2020 13 23% 3% 6% 23%
89 Shuttle Summer 2020 13 14% 2% 4% 34%
90 Retool Winter 2017 13 33% 0% 2% 17%
91 Windmill Summer 2022 13 8% 0% 4% 43%
92 Ultralight Winter 2019 13 0% 0% 0% 53%
93 Hyperbeam Winter 2022 13 54% 0% 0% 0%
94 hotglue Summer 2021 12 0% 0% 0% 51%
95 Ditto Winter 2020 12 30% 10% 0% 10%
96 Evidently AI Summer 2021 12 10% 0% 0% 40%
97 Jovian Summer 2021 12 30% 0% 0% 20%
98 Bitrise Winter 2017 11 30% 13% 0% 3%
99 DrDroid Winter 2023 11 30% 0% 0% 16%
100 AssemblyAI Summer 2017 11 20% 4% 0% 20%
101 Artillery Summer 2021 11 10% 0% 5% 31%
102 Evidence Summer 2021 10 14% 3% 0% 25%
103 Okteto Winter 2019 9 14% 0% 5% 19%
104 Curvenote Winter 2021 9 6% 0% 0% 30%
105 HackerRank Summer 2011 9 15% 5% 1% 15%
106 Pipekit Summer 2021 9 27% 0% 0% 11%
107 Lightdash Summer 2020 8 3% 0% 3% 29%
108 Parea Summer 2023 8 7% 0% 0% 28%
109 Datasaur Winter 2020 8 25% 0% 0% 8%
110 Expo Summer 2016 7 8% 1% 3% 19%
111 Karate Labs Winter 2022 7 7% 0% 0% 23%
112 Webiny Winter 2021 6 7% 5% 0% 14%
113 Nullstone Winter 2022 6 4% 0% 0% 22%
114 Inkeep Winter 2023 5 2% 0% 2% 16%
115 Boundary Winter 2023 5 2% 2% 0% 17%
116 Draftbit Winter 2018 5 3% 3% 0% 15%
117 Firecrawl Summer 2022 4 4% 0% 1% 12%
118 Medplum Summer 2022 4 11% 2% 2% 4%
119 LiteLLM Winter 2023 4 4% 4% 0% 11%
120 Mito Summer 2020 4 0% 0% 0% 17%
121 Cosmic Winter 2019 3 8% 0% 0% 4%
122 NanoNets Winter 2017 3 0% 0% 0% 12%
123 Velt Winter 2022 2 0% 0% 2% 6%
124 Dockup Winter 2019 2 0% 0% 0% 8%
125 Cosine Winter 2023 1 1% 0% 0% 6%
126 Release Winter 2020 1 0% 0% 0% 6%
127 Termii Winter 2020 0 0% 0% 0% 0%
128 Jet Admin Winter 2020 0 0% 0% 0% 0%

Column key: BP = branch protection, Chk = required checks, CO = CODEOWNERS, CI = CI workflow. Each percentage is the share of an org's scanned repos that pass that rule. Score is the weighted aggregate (0-100).
Sortable version with per-company scorecards: codatus.com/blog/only-2-of-128-yc-backed-dev-tools-companies-block-unchecked-merges/.

How the 128 were chosen

The starting universe of 549 companies is the union of YC's developer-tools (532) and devops (50) tags, pulled from yc-oss.github.io and deduplicated on slug. We narrowed from there:

  • Operating companies. Companies whose YC status reads "Inactive" or "Acquired" were removed. Companies marked "Public-on-stock-market" were kept; they're still operating dev tools businesses, just at different scale. 142 dropped. 407 remaining.
  • Mature batches. Batches Winter 2024 and later were removed. Companies that recently entered YC haven't been around long enough to have settled engineering practices. 166 dropped. 241 remaining.
  • Verified GitHub org. We matched each company to a GitHub organization via homepage links and GitHub search, requiring either a domain match or an exact name match to avoid mis-attributions. 47 dropped. 194 remaining.
  • Non-trivial public footprint. We required at least 10 active (non-fork, non-archived) public repos per org. 66 dropped. 128 remaining.

The cohort includes two publicly-traded YC alumni: Amplitude (Winter 2012, rank 41, score 31) and PagerDuty (Summer 2010, rank 49, score 29). GitLab (Winter 2015) passed the earlier filters but drops at the public-footprint step; their GitHub footprint is two forked repos because they host on gitlab.com.

Pattern

Something jumped out while we were scoring the cohort: branch protection passes for a real chunk of the dataset, but required checks barely register. To see how this plays out, we plotted each company on both: branch protection pass rate on one axis, required checks pass rate on the other.

Scatter plot of branch protection pass rate vs required checks pass rate, by company; most of the 128 cluster in the bottom-left, 42 sit in the bottom-right with high branch protection but low required checks, only 2 sit in the top-right

Three of the four quadrants have companies in them. The top-left is empty: required checks attach to a protected branch, so the configuration can't exist.

The top-right is the rare exception. Of the 44 companies with branch protection on most of their repos, only 2 also require a check: Apollo (BP 74%, Chk 58%) and Formance (BP 96%, Chk 61%).

That leaves 42 in the bottom-right. They enable branch protection on most of their repos without requiring any check. Every change opens a PR; nothing has to pass for the PR to merge. Supabase is the extreme case (BP 100%, Chk 22%).

The bottom-left holds the remaining 84 companies. Branch protection isn't enabled on most of their repos, so there's no workflow to gate.

The pattern is clear across the cohort: most companies have either no gate or a workflow that doesn't enforce anything.

See where you land

The 128 companies in the leaderboard are public-scan results. Install Codatus on your own GitHub org for a full scan, private repos included.

Top comments (0)