"Write issues like specs" is the one most maintainers skip. A good first issue that names the files to touch and says what done looks like is worth more than any label. One addition that pairs well with it: a "run it locally in 5 minutes" section, or a devcontainer, so first-timers don't quietly give up on setup before they ever open a PR. Happy last day of Hacktoberfest!
The fresh database check is the part I’d copy. I’d also put one expected screen or sample response at the end of the setup guide, so I can tell that I reached a working app rather than just a running process. For a first contribution, that small confirmation would make it much easier for me to start debugging the actual issue.
Thanks, Nathan! I’d love to see that kind of expected-output example in more setup docs. It gives a first-time contributor a clear point to say, okay, my environment works.
Your number two is the whole post, and you said so yourself…
I build cairn, a work harness for coding agents (github.com/eventually-consistent-code-org/cairn2), and the thing that made it work was arriving at the same issue shape you did from the other direction. Every issue cairn files has a problem, a fix, a "done when" checklist, and an out-of-scope list, because an agent given a vague issue does exactly what your newcomer does: guesses the scope, guesses the wording, and guesses what done means, then ships the guess with confidence. The not-in-this-issue section turned out to matter more for the agent than for the human. Your #336 is the best-written one I've seen on a project this new, the rules and the deliberate "no down-votes, anywhere" are the part a contributor can't infer.
The other half of your post that cairn has an opinion on is number four. The PRs that were easy to review came with what changed, why, and how to try it. cairn refuses to close an issue without that: the close carries the command that was run and what it printed, not a sentence saying tests pass, and the PR body is written from that record. Review is fast because the reviewer reads a receipt, not a claim…
So here's the offer rather than the theory. I'd like to take #336, the feedback up-votes, on a fork, run it through cairn in its pair mode where I claim and decide and it keeps the verification honest, and send you a PR that reads the way your post asks for. I read your AGENTS.md, so it'll arrive as mine with no agent names anywhere in it, and the receipt lines will be in the PR body in plain words. The claim comment goes on the issue first, per your contributing guide, unless someone beats me to it today.
To your question: what got my first outside contributors was a bug report with a reproduction I could run in under a minute. Your three-step setup is the same move, you just did it before anyone asked.
Early contribution energy is valuable, but a project keeps it when the first contribution has a clear contract: a small issue, a reproducible local setup, a visible definition of done, and prompt review. That turns enthusiasm into a repeatable maintainer workflow instead of a lucky launch week.
Yes—that is the pattern I try to use: make the first task small enough to finish, state the local setup and acceptance check, then keep the feedback loop visible. It removes ambiguity before a contributor has invested a weekend in the wrong direction.
Labeling by skill instead of by type feels obvious only after someone writes it down. The worst Hacktoberfest moment is opening a 'good first issue' and realising you need to understand the whole auth flow first. Writing issues like specs does more than any label, because 'here's the file, here's what done looks like' is already half the PR.
Labelling issues by skill is such a smart move. With those labels, contributors get the answer to their first question, "is this one for me?", before they even open the issue. The part about writing issues like specs matches what we see in docs work too: the clearer the starting point, the braver people feel. Which label or issue section brought the most first-timers in?
Exactly, Kate!
My goal is to make the process and decision as easy as possible. What I try and avoid is someone starting work on something but they are unsure what done looks like.
A technologist working in automation. Here I write up what I actually learn building tools, engineering AI agents, and keeping systems running, along with the workflows and habits that stick.
Separating the drafting agent from the verification check against retrieved sources keeps hallucinations from slipping into the auto-reply path. The part that usually trips this pipeline in production is the feedback loop for resolutions. When a resolution gets saved back into knowledge, stale answers or outdated workarounds from old issues can poison later retrievals. Pinning a freshness timestamp or tying each saved chunk to its git commit sha helps purge obsolete answers when documentation updates.
Hey @reidmarlow, thanks for this. Stale answers coming back through retrieval is something we are actively working on.
Here's what we have today. Each team's knowledge base is seperated by org, and a connected GitHub repo is one source feeding it. When a team re-syncs that repo, answerLoops rebuilds that repo's content from the current files and swaps it in for the old copy. So if a doc was deleted or rewritten, the old answer from it is gone after the next sync. We don't track the commit SHA, though. It reads the latest version and stores when it last synced, so your SHA idea is a real improvement.
Saved resolutions are also on the list. A person has to review and save them, and each one keeps a link back to its ticket. An owner or admin can also mark an article verified, and if anyone edits the text afterward, it shows as changed since it was verified. But nothing flags one that has simply aged, and a repo re-sync never touches them, so an old workaround can sit there until someone notices.
What we do have is tracing. Every answer records which articles it used, and every article shows which answers used it, so a bad answer leads back to the article behind it.
Your comment turned into an issue. It covers recording the commit SHA with each sync, showing how old each saved resolution is, and flagging ones saved before the latest docs sync: github.com/answerLoops/answerLoops...
If you have a better rule for tying a resolution to the docs it came from, I'd like to hear it. That's the open question on the issue.
Code enthusiast shaping tomorrow's tech landscape, one algorithm at a time. Committed to pushing boundaries and sharing knowledge with the DEV community.
Glad to hear your project is getting contributors. That's a great validator when someone lands on your GitHub, and it shows that it's clearly maintained and has people contributing to it.
Nice job!
Thanks Fliin, I am very open-source-centric, and that is my thought exactly, plus I love building in public with other developers who have great ideas, and add feature requests.
DevOps advocate on a mission to streamline development workflows. Empowering fellow developers to conquer challenges and scale new heights in the DEV world.
the second agent checking the draft against sources is the bit id want numbers on. how often does it reject a draft the first agent was fully confident about
Hey Andrew, No. Auto-reply only happens when two things are true. Auto-reply has to be turned on for that channel / integration, and the checker(2nd AI Mastra agent) has to score the draft at or above the confidence bar. The default bar is 80%, and a team can set its own per channel.
If the check itself fails or comes back empty, the draft is scored as zero confidence and goes to a person to review. It never goes out unchecked.
We're a place where coders share, stay up-to-date and grow their careers.
"Write issues like specs" is the one most maintainers skip. A good first issue that names the files to touch and says what done looks like is worth more than any label. One addition that pairs well with it: a "run it locally in 5 minutes" section, or a devcontainer, so first-timers don't quietly give up on setup before they ever open a PR. Happy last day of Hacktoberfest!
Totally agree Aliza, thanks for the feedback.
Adding running it in five minutes is a great idea, thank you!
The fresh database check is the part I’d copy. I’d also put one expected screen or sample response at the end of the setup guide, so I can tell that I reached a working app rather than just a running process. For a first contribution, that small confirmation would make it much easier for me to start debugging the actual issue.
Totally agree Emily, nice tip, thank you!
Thanks, Nathan! Having the expected result next to the setup steps would make that first contribution easier to verify.
100% agree!
Thanks, Nathan! I’d love to see that kind of expected-output example in more setup docs. It gives a first-time contributor a clear point to say, okay, my environment works.
Your number two is the whole post, and you said so yourself…
I build cairn, a work harness for coding agents (github.com/eventually-consistent-code-org/cairn2), and the thing that made it work was arriving at the same issue shape you did from the other direction. Every issue cairn files has a problem, a fix, a "done when" checklist, and an out-of-scope list, because an agent given a vague issue does exactly what your newcomer does: guesses the scope, guesses the wording, and guesses what done means, then ships the guess with confidence. The not-in-this-issue section turned out to matter more for the agent than for the human. Your #336 is the best-written one I've seen on a project this new, the rules and the deliberate "no down-votes, anywhere" are the part a contributor can't infer.
The other half of your post that cairn has an opinion on is number four. The PRs that were easy to review came with what changed, why, and how to try it. cairn refuses to close an issue without that: the close carries the command that was run and what it printed, not a sentence saying tests pass, and the PR body is written from that record. Review is fast because the reviewer reads a receipt, not a claim…
So here's the offer rather than the theory. I'd like to take #336, the feedback up-votes, on a fork, run it through cairn in its pair mode where I claim and decide and it keeps the verification honest, and send you a PR that reads the way your post asks for. I read your AGENTS.md, so it'll arrive as mine with no agent names anywhere in it, and the receipt lines will be in the PR body in plain words. The claim comment goes on the issue first, per your contributing guide, unless someone beats me to it today.
To your question: what got my first outside contributors was a bug report with a reproduction I could run in under a minute. Your three-step setup is the same move, you just did it before anyone asked.
Hey @eventuallyconsistentcode, I appreciate the breakdown here, I'm reviewing your PR as we speak :)
Thanks you by the way!
Happy to contribute, it's a cool project!
Thank you, I appreciate that!
Thank you!
Some really awesome tips for open-source maintainers. I'm going to use them for Resume Matcher too!
Thanks Saurabh, I love Resume Matcher!
Dang, I just checked out your GitHub. It's amazing that it has 29k stars, nice job!
Early contribution energy is valuable, but a project keeps it when the first contribution has a clear contract: a small issue, a reproducible local setup, a visible definition of done, and prompt review. That turns enthusiasm into a repeatable maintainer workflow instead of a lucky launch week.
That's totally right Alex. Is that how you do it?
Yes—that is the pattern I try to use: make the first task small enough to finish, state the local setup and acceptance check, then keep the feedback loop visible. It removes ambiguity before a contributor has invested a weekend in the wrong direction.
Alex, nice, I also try to follow this process. It's really important for early contributors to get quick wins with small tasks.
Labeling by skill instead of by type feels obvious only after someone writes it down. The worst Hacktoberfest moment is opening a 'good first issue' and realising you need to understand the whole auth flow first. Writing issues like specs does more than any label, because 'here's the file, here's what done looks like' is already half the PR.
Agreed Nilamadhab!
Labelling issues by skill is such a smart move. With those labels, contributors get the answer to their first question, "is this one for me?", before they even open the issue. The part about writing issues like specs matches what we see in docs work too: the clearer the starting point, the braver people feel. Which label or issue section brought the most first-timers in?
Exactly, Kate!
My goal is to make the process and decision as easy as possible. What I try and avoid is someone starting work on something but they are unsure what done looks like.
Separating the drafting agent from the verification check against retrieved sources keeps hallucinations from slipping into the auto-reply path. The part that usually trips this pipeline in production is the feedback loop for resolutions. When a resolution gets saved back into knowledge, stale answers or outdated workarounds from old issues can poison later retrievals. Pinning a freshness timestamp or tying each saved chunk to its git commit sha helps purge obsolete answers when documentation updates.
Hey @reidmarlow, thanks for this. Stale answers coming back through retrieval is something we are actively working on.
Here's what we have today. Each team's knowledge base is seperated by org, and a connected GitHub repo is one source feeding it. When a team re-syncs that repo, answerLoops rebuilds that repo's content from the current files and swaps it in for the old copy. So if a doc was deleted or rewritten, the old answer from it is gone after the next sync. We don't track the commit SHA, though. It reads the latest version and stores when it last synced, so your SHA idea is a real improvement.
Saved resolutions are also on the list. A person has to review and save them, and each one keeps a link back to its ticket. An owner or admin can also mark an article verified, and if anyone edits the text afterward, it shows as changed since it was verified. But nothing flags one that has simply aged, and a repo re-sync never touches them, so an old workaround can sit there until someone notices.
What we do have is tracing. Every answer records which articles it used, and every article shows which answers used it, so a bad answer leads back to the article behind it.
Your comment turned into an issue. It covers recording the commit SHA with each sync, showing how old each saved resolution is, and flagging ones saved before the latest docs sync: github.com/answerLoops/answerLoops...
If you have a better rule for tying a resolution to the docs it came from, I'd like to hear it. That's the open question on the issue.
Labeling issues by skill, not just type, is a small change with a clear payoff. A newcomer can tell at a glance whether the work fits them.
That is so true @robertpaschal!
Appreciate that, Nathan. Skill labels make the first step a lot less guesswork for a newcomer.
Glad to hear your project is getting contributors. That's a great validator when someone lands on your GitHub, and it shows that it's clearly maintained and has people contributing to it.
Nice job!
Thanks Fliin, I am very open-source-centric, and that is my thought exactly, plus I love building in public with other developers who have great ideas, and add feature requests.
Nice walkthrough of how you started boosting your project on GitHub. Thanks for sharing!
Thanks David, yeah, it's something I'm really passionate about. I appreciate the feedback!
the second agent checking the draft against sources is the bit id want numbers on. how often does it reject a draft the first agent was fully confident about
Cool man, love this!
Thanks Ferguson!
Nice writeup!
I'll check out answerLoops, looks like a cool project I could use.
Thank you Steve.
I would appreciate any feedback along the way. Please reach out if you have any questions.
Nice article.
What happens if the second AI agent fails or returns nothing? Could an unchecked answer go out automatically?
Hey Andrew, No. Auto-reply only happens when two things are true. Auto-reply has to be turned on for that channel / integration, and the checker(2nd AI Mastra agent) has to score the draft at or above the confidence bar. The default bar is 80%, and a team can set its own per channel.
If the check itself fails or comes back empty, the draft is scored as zero confidence and goes to a person to review. It never goes out unchecked.