On Saturday, November 14, at 12:08 UTC I opened ten issues on orm-lens, a static analyzer for Django models and migrations. The first pull request arrived 29 minutes later. By 20:00 there were nine PRs from eight people, and seven of them were merged the same day.
The week before, a batch of ten went the same way: six people, all ten closed in two days. Five days after the second batch the count stood at 22 external pull requests from 12 people, and when I opened the next batch this morning, the first PR came in 25 minutes.
I didn't do anything clever with labels or bots. What changed is where the issues came from.
Where the issues come from
orm-lens reads Django source without importing Django, so it can be wrong in ways its own test suite never shows. The only way I know to find those is to run it on code it has never seen.
For this batch I picked four large open-source Django projects: pretix, Weblate, Label Studio and Taiga. I ran every command on each one and read the output looking for answers that were wrong, not just ugly:
- migrations named
0162b_...pywere silently skipped, so the migration graph had holes -
@receiver(signal=post_save, sender=Order)with the signal passed by keyword lost the signal name -
ArrayFieldimported by name (from django.contrib.postgres.fields import ArrayField) dropped the field from the model -
select_related("date_from")was suggested for a plain datetime attribute
Each of those is a small, self-contained bug with an obvious expected output. That's the whole trick.
What goes in the issue
Every issue had the same five sections:
- What happens. One paragraph, with the real project and file where it showed up.
-
Repro. A minimal
models.py(or migration) of 5–15 lines and the exact command. - Actual vs expected output. Copied from the terminal, not described.
- Where. The file and roughly which function to start in.
- How to test. Which test file to add to, and what the assertion should be.
Writing these takes longer than writing the fix would. That's fine. The point isn't to save my time on this bug; it's that someone new can go from "never seen this repo" to a green PR in an evening, and come back for the next one.
What I'd skip
- Vague tickets. "Improve the docs" or "add more tests" sat open for months in my repo. Nobody knows when they're done.
-
Tickets that need design decisions. Two of the ten were bigger: a field declared under
if/elsein the class body, and field classes defined in the project itself. I labelled thosehelp wanted, notgood first issue, and the one that got a PR needed a longer review. -
Waiting to review. CI on fork PRs needs a manual approval on GitHub. A PR that sits in
action_requiredfor a day loses its author. I approve the run after reading the diff, and review the same day.
The review is part of the issue
Not every PR was right the first time. One fixed the reported case but broke the opposite one: a field declared under a condition and then declared again unconditionally kept the conditional type. Another fixed the bug but more than doubled the run time on a large project. Both reviews came in the same format as the issues: a minimal repro of the problem and the output I expected.
If the issue is precise, the review can be precise too, and a newcomer can act on it without a long back-and-forth.
Try it on your own project
If you maintain a linter, a parser, a code generator, anything that reads other people's code: pick three big open-source projects in your niche, run your tool on them, and file every wrong answer with a repro. You get a better tool and a queue of issues people actually want to pick up.
And if you work with Django, the open ones are here. To see what the tool does on your code:
uvx orm-lens migration-risk # migrations that break on a populated table
uvx orm-lens nplusone # N+1 loops, with the select_related to add
Top comments (0)