DEV Community

Cover image for My open source project is brand new and already has contributors. Here's what I did.
Nathan Tarbert
Nathan Tarbert

Posted on Edited on

My open source project is brand new and already has contributors. Here's what I did.

Actionable tips on writing issues like clear specs

In the spirit of Hacktoberfest, with it being the last day of October, I wanted to share my story about open source and building in public.

I'm a DevRel. In my day job, I help people who've never contributed to open source get their first code shipped. So when I started building answerLoops, an open source, self-hostable AI support tool, one thing bugged me. Was my own project actually easy to jump into?

What is answerLoops?

answerLoops turns your knowledge source into answers for your community. It's self-hostable, so you can run it on your own infrastructure with the AI provider you choose, or use the hosted app.

Here's the workflow:

  1. A question comes in from a community channel like Discord, Slack, Discourse, GitHub, or website chat.
  2. answerLoops searches your knowledge: your docs, files, GitHub repos, and Notion pages.
  3. It drafts a reply with the sources attached.
  4. A second AI agent checks the draft against those sources.
  5. Your team approves the reply, or you turn on auto-reply for the questions that pass the check.
  6. Useful resolutions are saved as knowledge, and knowledge gaps show what your docs are missing.

That workflow is why the labels below make sense. frontend work touches the dashboard where a team reviews and approves replies. backend work touches the pipeline and the channel integrations behind it.

Explore answerLoops on ⭐️ GitHub

The project is new, so the hard part right now is getting off the ground and getting people to find it. I posted about it on Reddit and X. It didn't bring in many stars. So I changed where I put my effort. Instead of trying to reach everyone, I focused on developers who want to build something and on making it easy for them to start.

Over the last week, developers started showing up. They sent pull requests, and I merged them. Some built something from an open issue. Others found a problem on their own and fixed it.

I can't prove any of this is cause and effect. I have a handful of data points, and some of them didn't pick up an issue at all. But I did set a few things up beforehand, and I think they're worth writing down for anyone starting out.

1. I labeled issues by skill, not just by type

At first my issues only had the usual labels: enhancement, bug, documentation. That tells a contributor what kind of change it is, but not whether it's for them.

I added two labels: frontend and backend. Right now I have 58 open issues, 41 tagged frontend and 33 tagged backend. Someone can click one label and see only the work that matches their skills, without reading all 58.

I also use good first issue and help wanted. good first issue is the one most people already know. The skill labels are the new part.

Most of the PRs I merged closed an issue with one of these labels, so they look like they're doing something. I'm still careful about how much I claim.

One side effect: only 8 good first issue tickets are open now. Labels only work while there's something behind them, so I'm writing more.

2. I wrote issues like specs

I'd have undersold this one if I'd only written about labels.

The issue my first contributor picked up was tagged frontend, good first issue, and help wanted. It was also written so that someone who had never seen the codebase could start. It had:

  • A Problem section: the data was already being collected and stored, but nothing in the UI showed it.
  • A Scope section: exactly what to show, where to start looking in the code, and what the wording should be.
  • A Not in this issue section: what to leave alone, so nobody drifts into a bigger project.
  • Acceptance criteria as a checklist.

It also said it was part of a larger tracking issue, so a contributor could see where it fit.

A vague issue like "show usage on the article page" leaves a newcomer guessing about scope, wording, and what "done" means. A scoped one lets them start. If I had to pick one thing to copy, it's this.

Every issue also ends with a link to the contributing guide, and it's now built into the Issues template.

3. I made the project easy to run

You can't contribute to something you can't start. I rewrote the README to be a short overview, and added a CONTRIBUTING guide plus issue forms for bugs, features, integrations, and docs, so new reports come in with the details I need.

The first version of the setup steps wasn't good enough. It ran the Docker path, then said "or run it natively," and I couldn't tell you which one to pick. If I was confused, a newcomer was too. I rewrote it as three steps: clone the app, set the values in one .env, and run it. Then I ran the native steps myself against a fresh database to make sure they work.

Contributors have already improved the docs from there, including a diagram of the install flow in the self-hosting quickstart.

4. Review was fast, and the PRs made it easy

The PRs came with a clear description of what changed, why, and how to try it. That made them easy to review, and I could merge quickly.

Fast feedback matters. A first-time contributor who waits two weeks for a reply rarely comes back.

5. I said thank you in public

The contributors are credited in the release notes and in the docs, and I thanked them publicly. Open source is mostly volunteers, so it's the least I can do. It also shows the next person what happens when they contribute.

I also posted on LinkedIn and X.

Thanks Aaqibhafeez Khan for the kind words

What I got wrong, or am not sure of

  • I don't know if the labels worked 100%. Nothing else about the project changed much, but I also posted about it in a few places, which brings people in on its own.
  • Not everyone took a labeled issue. Some contributors found a problem nobody had filed, and the labels can't explain that.
  • Return visits are the real test. What matters is whether people come back for a second PR.
  • The supply runs out. Contributors finish good first issue tickets quickly. Writing good ones is now a regular chore for me.

If you want to try this

  • Label issues by the skill needed, not just the type of change.
  • Write one issue as a spec: problem, scope, what's out of scope, and acceptance criteria.
  • Make sure someone can run the project, and test those steps yourself.
  • Review fast.
  • Thank people in public.

If you're new to open source and want help with a first PR, leave a comment or message me. I'm happy to walk you through it. The open issues are here: https://github.com/answerLoops/answerLoops/issues

And if you maintain a project: what got your first outside contributors to show up?

P.S. If you're reading this, starring ⭐️ the repo also really helps people find it :)

Thanks for reading!

Top comments (38)

Collapse
 
stackmasteraliza profile image
Aliza Ali •

"Write issues like specs" is the one most maintainers skip. A good first issue that names the files to touch and says what done looks like is worth more than any label. One addition that pairs well with it: a "run it locally in 5 minutes" section, or a devcontainer, so first-timers don't quietly give up on setup before they ever open a PR. Happy last day of Hacktoberfest!

Collapse
 
nathan_tarbert profile image
Nathan Tarbert •

Totally agree Aliza, thanks for the feedback.

Adding running it in five minutes is a great idea, thank you!

Collapse
 
emilywoodsnyc profile image
Emily Woods •

The fresh database check is the part I’d copy. I’d also put one expected screen or sample response at the end of the setup guide, so I can tell that I reached a working app rather than just a running process. For a first contribution, that small confirmation would make it much easier for me to start debugging the actual issue.

Collapse
 
nathan_tarbert profile image
Nathan Tarbert •

Totally agree Emily, nice tip, thank you!

Collapse
 
emilywoodsnyc profile image
Emily Woods •

Thanks, Nathan! Having the expected result next to the setup steps would make that first contribution easier to verify.

Thread Thread
 
nathan_tarbert profile image
Nathan Tarbert •

100% agree!

Thread Thread
 
emilywoodsnyc profile image
Emily Woods •

Thanks, Nathan! I’d love to see that kind of expected-output example in more setup docs. It gives a first-time contributor a clear point to say, okay, my environment works.

Collapse
 
eventuallyconsistentcode profile image
John Reed •

Your number two is the whole post, and you said so yourself…

I build cairn, a work harness for coding agents (github.com/eventually-consistent-code-org/cairn2), and the thing that made it work was arriving at the same issue shape you did from the other direction. Every issue cairn files has a problem, a fix, a "done when" checklist, and an out-of-scope list, because an agent given a vague issue does exactly what your newcomer does: guesses the scope, guesses the wording, and guesses what done means, then ships the guess with confidence. The not-in-this-issue section turned out to matter more for the agent than for the human. Your #336 is the best-written one I've seen on a project this new, the rules and the deliberate "no down-votes, anywhere" are the part a contributor can't infer.

The other half of your post that cairn has an opinion on is number four. The PRs that were easy to review came with what changed, why, and how to try it. cairn refuses to close an issue without that: the close carries the command that was run and what it printed, not a sentence saying tests pass, and the PR body is written from that record. Review is fast because the reviewer reads a receipt, not a claim…

So here's the offer rather than the theory. I'd like to take #336, the feedback up-votes, on a fork, run it through cairn in its pair mode where I claim and decide and it keeps the verification honest, and send you a PR that reads the way your post asks for. I read your AGENTS.md, so it'll arrive as mine with no agent names anywhere in it, and the receipt lines will be in the PR body in plain words. The claim comment goes on the issue first, per your contributing guide, unless someone beats me to it today.

To your question: what got my first outside contributors was a bug report with a reproduction I could run in under a minute. Your three-step setup is the same move, you just did it before anyone asked.

Collapse
 
nathan_tarbert profile image
Nathan Tarbert •

Hey @eventuallyconsistentcode, I appreciate the breakdown here, I'm reviewing your PR as we speak :)
Thanks you by the way!

Collapse
 
eventuallyconsistentcode profile image
John Reed •

Happy to contribute, it's a cool project!

Thread Thread
 
nathan_tarbert profile image
Nathan Tarbert •

Thank you, I appreciate that!

Thread Thread
 
nathan_tarbert profile image
Nathan Tarbert •

Thank you!

Collapse
 
srbhr profile image
𝚂𝚊𝚞𝚛𝚊𝚋𝚑 𝚁𝚊𝚒 •

Some really awesome tips for open-source maintainers. I'm going to use them for Resume Matcher too!

Collapse
 
nathan_tarbert profile image
Nathan Tarbert •

Thanks Saurabh, I love Resume Matcher!

Dang, I just checked out your GitHub. It's amazing that it has 29k stars, nice job!

Collapse
 
alexshev profile image
Alex Shev •

Early contribution energy is valuable, but a project keeps it when the first contribution has a clear contract: a small issue, a reproducible local setup, a visible definition of done, and prompt review. That turns enthusiasm into a repeatable maintainer workflow instead of a lucky launch week.

Collapse
 
nathan_tarbert profile image
Nathan Tarbert •

That's totally right Alex. Is that how you do it?

Collapse
 
alexshev profile image
Alex Shev •

Yes—that is the pattern I try to use: make the first task small enough to finish, state the local setup and acceptance check, then keep the feedback loop visible. It removes ambiguity before a contributor has invested a weekend in the wrong direction.

Thread Thread
 
nathan_tarbert profile image
Nathan Tarbert •

Alex, nice, I also try to follow this process. It's really important for early contributors to get quick wins with small tasks.

Collapse
 
madhabcoder profile image
Nilamadhab Senapati •

Labeling by skill instead of by type feels obvious only after someone writes it down. The worst Hacktoberfest moment is opening a 'good first issue' and realising you need to understand the whole auth flow first. Writing issues like specs does more than any label, because 'here's the file, here's what done looks like' is already half the PR.

Collapse
 
nathan_tarbert profile image
Nathan Tarbert •

Agreed Nilamadhab!

Collapse
 
kate_astrid profile image
kate astrid •

Labelling issues by skill is such a smart move. With those labels, contributors get the answer to their first question, "is this one for me?", before they even open the issue. The part about writing issues like specs matches what we see in docs work too: the clearer the starting point, the braver people feel. Which label or issue section brought the most first-timers in?

Collapse
 
nathan_tarbert profile image
Nathan Tarbert •

Exactly, Kate!
My goal is to make the process and decision as easy as possible. What I try and avoid is someone starting work on something but they are unsure what done looks like.

Collapse
 
reidmarlow profile image
Reid Marlow •

Separating the drafting agent from the verification check against retrieved sources keeps hallucinations from slipping into the auto-reply path. The part that usually trips this pipeline in production is the feedback loop for resolutions. When a resolution gets saved back into knowledge, stale answers or outdated workarounds from old issues can poison later retrievals. Pinning a freshness timestamp or tying each saved chunk to its git commit sha helps purge obsolete answers when documentation updates.

Collapse
 
nathan_tarbert profile image
Nathan Tarbert • • Edited

Hey @reidmarlow, thanks for this. Stale answers coming back through retrieval is something we are actively working on.

Here's what we have today. Each team's knowledge base is seperated by org, and a connected GitHub repo is one source feeding it. When a team re-syncs that repo, answerLoops rebuilds that repo's content from the current files and swaps it in for the old copy. So if a doc was deleted or rewritten, the old answer from it is gone after the next sync. We don't track the commit SHA, though. It reads the latest version and stores when it last synced, so your SHA idea is a real improvement.

Saved resolutions are also on the list. A person has to review and save them, and each one keeps a link back to its ticket. An owner or admin can also mark an article verified, and if anyone edits the text afterward, it shows as changed since it was verified. But nothing flags one that has simply aged, and a repo re-sync never touches them, so an old workaround can sit there until someone notices.

What we do have is tracing. Every answer records which articles it used, and every article shows which answers used it, so a bad answer leads back to the article behind it.

Your comment turned into an issue. It covers recording the commit SHA with each sync, showing how old each saved resolution is, and flagging ones saved before the latest docs sync: github.com/answerLoops/answerLoops...

If you have a better rule for tying a resolution to the docs it came from, I'd like to hear it. That's the open question on the issue.

Collapse
 
robertpaschal profile image
Odinaka Robert Nnamani •

Labeling issues by skill, not just type, is a small change with a clear payoff. A newcomer can tell at a glance whether the work fits them.

Collapse
 
nathan_tarbert profile image
Nathan Tarbert •

That is so true @robertpaschal!

Collapse
 
robertpaschal profile image
Odinaka Robert Nnamani •

Appreciate that, Nathan. Skill labels make the first step a lot less guesswork for a newcomer.

Collapse
 
fliin_0788 profile image
Fliin •

Glad to hear your project is getting contributors. That's a great validator when someone lands on your GitHub, and it shows that it's clearly maintained and has people contributing to it.
Nice job!

Collapse
 
nathan_tarbert profile image
Nathan Tarbert • • Edited

Thanks Fliin, I am very open-source-centric, and that is my thought exactly, plus I love building in public with other developers who have great ideas, and add feature requests.