DEV Community

Floor Drees
Floor Drees

Posted on • Updated on

GitHub integration for Orbit for anyone with multiple communities

Orbit is "mission control for your community", a single, shared view of members and activity.

For my work for k6 I've been adding integrations to our Orbit "workspace" so that I can make correlations between data points, and to act accordingly.

For instance: this person asking a question on Twitter, posted the same question in our Discourse, let's link to our explanation in our reply. Discourse after all allows for more characters than Twitter, and anyone passing by might learn about our community Discourse and check it out.

I've used the GitHub Actions workflow for this, which is just adding a bunch of YAML files and "secrets" to a personal repo.

YAML files in a repository

Next I wanted to hook up our public repositories on GitHub, to scan those for activity, and I ran into an issue. k6 was recently acquired by Grafana Labs, and in August we moved all k6 repositories under the Grafana Labs organization on GitHub. Grafana uses Orbit to listen to all the repos in that organization, and it seems Orbit can't send data from one organization to two different workspaces.

Nicolas Goutay, Sr. Software Engineer, Ben Greenberg, (their former) Developer Advocate, and Josh Dzielak, Co-Founder & CTO, responded unanimously to my request: it's just not possible. Yet.

Now I think Orbit should give this feature request some priority, because I think that many larger (open source) players have multiple product communities they support - Grafana has Prometheus, Grafana itself, Tempo, etc, Red Hat has OpenShift, OpenStack, Ansible, HashiCorp has Consul, Terraform, Vagrant. Currently information on all these different project feed into one workspace and I can imagine the signal-to-noise ratio is... not great.

Colleague, and Tempo maintainer, Daniel GonzΓ‘lez Lopes to the rescue! Part of the SRE team at k6, Daniel helped me (ab)use the GitHub API to search GitHub for public Grafana repos with "k6" in the name, and get all the recent events for each one of them.

From those events we create new members (or update them if they already exist), and create new activities for each one of them. While functional in the first iteration, Daniel spent some time to update the activity types to be more legible / without all the breadcrumb info.

Before:
Activity graph before

After:
Activity graph after

We're hoping tags in Orbit Activities will help to filter by repo in the future. But in Orbit's reference material tags are marked as "experimental". In all caps.

Anyway, the idea is that we for this as well use GitHub Actions. We'll let the implementation simmer for a few weeks, make tweaks here and there, and the gradually add the other integrations. With every integration we add we expect to see duplication, and I'd rather deal with those on a case by case basis, than to go through 300+ possible matches at once.

"Fun" anecdote: Daniel thought it'd be neat to use "starring a repo" as an activity as well, which exposed some rate limits on Orbit's end. We plan to run the script hourly, to avoid big data (not to be confused with Big Data) imports moving forward.

Orbit's reports / graphs are pretty rudimentary, so I would love to see someone create an Orbit data source plugin for Grafana. Imagine being able to view community data with your other project's health metrics... Looking at Marcus Olsson πŸ‘€

Our little integration lives on GitHub for you to fork, copy, or take inspiration from.

Discussion (1)

Collapse
dzello profile image
Josh Dzielak πŸ”†

Thanks Floor for sharing the integration and the write up! Really cool to see what you're doing with the API, this is a brilliant use for it and I know other folks want the same thing, either to split org data into multiple workspaces or monitor repos that aren't necessarily part of their org (e.g. a community-hosted plugin).

Thank you also for providing the opportunity for me to speak to a few of the discoveries aka snags that you hit :)

  • GitHub orgs funneling into multiple workspaces - super important for the kind of larger companies you mention and it's on the roadmap for next year
  • Activity tags - this is getting upgraded to a non-experimental solution very soon (1-2 months), which will allow a set of key value properties to be applied to any activity. For example repository: "postman-to-k6". The Orbit UI will detect these properties and allow you to use them anywhere you can filter, including members, activities, and reports pages.
  • Rate limits - we have a pretty generous limit of ~120 calls per minute but maybe you are getting more GitHub stars than I think :) we're happy to look at what happened there or if the limit needs to be raised, feel free to DM me
  • Reports - lots coming on this front in 2022, but nothing that rivals the power of Grafana πŸ˜† we make the data easy to export so that folks can analyze it easily outside of Orbit and it would be really cool to have a Grafana plugin

Again, thanks for sharing the journey. Happy to answer any questions if I missed anything.