Your good first issue queue might be lying to contributors — so I built StarterQueue

vyxordev — Sat, 24 Oct 2026 08:17:43 +0000

A good first issue label looks like an invitation.

But sometimes you open the issue and find out it was assigned weeks ago. Or it has been inactive for months. Or it is blocked by another issue. Or there is already an open pull request that will close it.

None of those things necessarily remove the good first issue label.

So the label slowly stops meaning what contributors think it means.

I kept wondering: could this be checked automatically without trying to guess whether an issue is “easy”?

That question turned into StarterQueue.

The problem wasn't finding issues. It was trusting them.

For someone trying to make their first contribution, a good first issue label carries a lot of weight.

You assume someone has already looked at the issue and decided:

it is still available;
nobody else is actively working on it;
it is not blocked;
the conversation is still open;
and the issue is actually ready for a contributor to pick up.

But GitHub labels don't update themselves when the state around an issue changes.

An issue can still say good first issue even after somebody gets assigned to it.

It can sit untouched for months.

It can depend on another unfinished issue.

It can belong to an old milestone.

And, probably the most frustrating case, there can already be an open pull request that is going to close it.

The label isn't technically wrong because of any one of these things. Different projects have different workflows.

But they are all signals a contributor should probably know about before spending time understanding the issue, setting up the project, and preparing a change.

That distinction became important when I started designing StarterQueue.

I didn't want to build a tool that says:

“AI thinks this issue is too difficult for a beginner.”

That would be subjective, difficult to verify, and very easy to get wrong.

I wanted something much more boring — and much more defensible:

look at GitHub's actual state, detect objective signals, and explain exactly what was found.

Turning that into rules

StarterQueue currently has nine rules, each with a stable code from SQ001 to SQ009.

The important part is that each rule maps to something GitHub can actually verify.

For example:

SQ001 checks whether the issue is already assigned.
SQ003 checks whether it has been inactive for too long.
SQ004 checks whether GitHub already knows about an open pull request that will close the issue.
SQ006 checks whether GitHub reports open issue dependencies blocking it.
SQ007 checks whether the conversation is locked.
SQ008 and SQ009 look at milestone state.

Not every signal deserves the same severity.

An assigned issue, for example, might still be open to collaboration in one repository and completely unavailable in another.

So StarterQueue separates detection from judgment.

The default balanced policy treats some signals as warnings and stronger contradictions as failures. A stricter policy is available for repositories that want assignment and dependencies to count as failures too.

And if a maintainer disagrees with just one rule, they can override that rule's severity without inventing a whole new policy.

That was one of the design decisions I cared about most:

the tool should tell you what GitHub says, not pretend it understands your repository better than you do.

Testing it on a real repository

A rules engine is easy to make look good with fake examples.

So I wanted to see what happened when StarterQueue met a real, large starter-issue queue.

I ran it against rust-lang/rust-clippy.

117 issues later

The audit found:

2 PASS
76 WARN
39 FAIL

That doesn't mean 115 issues were “bad”.

It means 115 issues had at least one signal worth showing to someone before they started working on them.

Some warnings were relatively mild — things like assignment or long inactivity.

Some failures were much stronger.

For example, I opened one of the failing issues and StarterQueue showed three separate findings:

SQ001 — the issue was already assigned;
SQ003 — it had seen no activity for 358 days;
SQ004 — there was already an open pull request linked by GitHub to close the issue.

That last one is exactly the kind of thing I wanted StarterQueue to catch.

A contributor could otherwise discover the issue through the good first issue label, spend time reading it, clone the repository, understand the codebase — and only later realize somebody already has an open PR solving it.

StarterQueue surfaces that information before the contributor invests that time.

And because the finding comes from GitHub's structured closing-PR relationship, it isn't guessing based on a random mention in a comment.

One important caveat: this audit is a snapshot of repository state, not a judgment of the Rust Clippy project or its maintainers. Large open-source repositories change constantly, and different projects intentionally manage starter issues in different ways.

The point of the test was simply to see whether the rules produced useful information on a real queue.

They did.

I didn't want another CLI that only works in screenshots

At first, StarterQueue was mostly a traditional CLI.

You ran an audit, got a list of results, and that was it.

That worked well for automation, but it wasn't great for actually exploring a repository with dozens — or hundreds — of starter issues.

So for v0.5.0, I built an interactive terminal interface.

Running:

starterqueue

opens a keyboard-driven TUI where you can:

enter any GitHub repository;
watch the audit run;
browse PASS, WARN, and FAIL results;
filter by severity;
search across issues;
open an issue's detailed findings;
and jump directly to the issue in your browser.

The CLI and TUI use the same audit engine.

That matters because I didn't want the interactive interface to become a separate “pretty demo” with different behavior. The same rules, policies, and GitHub data power both the human-facing interface and automation.

For CI, StarterQueue still works as a normal command:

starterqueue audit owner/repo

And for repositories that want the audit to run automatically, it also ships as a GitHub Action.

Making it useful without making it invasive

One constraint has stayed in place from the beginning:

StarterQueue is read-only.

It doesn't remove labels, close issues, assign contributors, post comments, or try to “clean up” a repository automatically.

It only audits what already exists and reports what it finds.

That makes it much easier to try on a real project without handing a new tool permission to start changing things.

For maintainers, StarterQueue can also run automatically as a GitHub Action:

permissions:
  contents: read
  issues: read
  pull-requests: read

steps:
  - name: Audit starter queue
    uses: VyxorDev/starterqueue@v0.5.0

By default, the Action can fail a workflow when StarterQueue finds a FAIL, which means it can act as a lightweight health check for a repository's starter queue.

Or maintainers can make it less strict, change the stale threshold, use a different starter label, or override individual rule severities.

There is one authenticated check worth mentioning.

SQ004 — detecting an open pull request that GitHub has registered as closing an issue — requires a GitHub token when using the CLI.

I chose to make that explicit rather than silently replacing structured GitHub data with unreliable text matching.

Without a token, StarterQueue still runs the other rules and tells you that SQ004 was skipped.

I'd rather show an incomplete audit honestly than present a false-clean result.

Where I want to take this next

I don't want StarterQueue to grow by adding rules just because they sound clever.

The useful next steps are going to come from running it against more real repositories.

I'm especially interested in cases where:

StarterQueue reports something that isn't actually useful;
a rule produces a false positive;
a clearly unavailable starter issue passes the audit;
a repository uses good first issue in a way I haven't accounted for;
or GitHub exposes another objective signal that would make a good deterministic rule.

If you maintain an open-source project with a starter-issue queue, I'd genuinely like to know what StarterQueue finds.

And if you're looking to contribute, there is plenty of room for work around audit rules, GitHub API edge cases, tests, documentation, and the terminal interface.

You can find the project here:

GitHub: https://github.com/VyxorDev/starterqueue

The current release is v0.5.0.

If you want to try the CLI on Windows:

Invoke-WebRequest `
  -Uri "https://github.com/VyxorDev/starterqueue/releases/download/v0.5.0/starterqueue-0.5.0.tgz" `
  -OutFile "starterqueue-0.5.0.tgz"

npm install --global .\starterqueue-0.5.0.tgz

$env:GH_TOKEN = gh auth token
starterqueue

Or, if you're a maintainer, the GitHub Action is probably the easiest way to try it without changing your existing workflow.

If you test StarterQueue on your repository, I'd especially appreciate hearing about the wrong results, not just the correct ones.

Those are the cases that will make the project better.

DEV Community: vyxordev