Monitoring GitHub Pull Requests with Prometheus

The problem

Have you ever wanted to track your open source contributions? Or perhaps monitor contributions made by multiple users for some sort of event? Well I had the exact the same problem 😊.
With Hacktoberfest just around the corner, I wanted a way to automatically track open source contributions to be able to incentivize participation via prizes or similar. It's fairly difficult to track who opens pull requests to what projects on GitHub at scale within a large organization, but boy do I have the solution for you.

The solution

I built a prometheus exporter that takes in a config file with a list of users and will use the GitHub search API to find pull requests created by said users. This exporter exposes some useful data such as user, created_at, status and link.

What is a prometheus exporter?

Prometheus is an open source time series database that uses a "pull" model where it reaches out to configured clients (basically plugins) called exporters. Then ingests the data from the exporters via configured interval which is typically 15 seconds.

I want to see some users' pull requests, what now?

If you're familiar with prometheus, you can view the github-pr-exporter docs here.

Maybe you're not familiar with prometheus, follow along for how to get started!

How does it work?

The exporter is written in Go and utilizes a GitHub client library to execute searches using the GitHub search API. The exporter is basically a CLI that takes in configuration options via arguments then runs a web server in a loop that looks for new pull request data periodically.

Searches

The exporter uses the search API and runs this search for every user: "type:pr author:<user> created:>=<calculated timeframe>"

The search is executed once per user instead of a bulk search due to the 256 character limit of the search API, but because the search API has a rate limit of 10 searches per minute for unauthenticated clients, there is a hard 6 second wait between collecting pull request data for each user to avoid said rate limit. This isn't a huge deal since this would process around 1000 users per 90 minutes which completely fine for this kind of data.

Configuration Options

• --config YAML config file with usernames
• --days-ago How many days prior to the current to get pull requests from. Defaults to 90
• --ignore-user-repos Ignore pull requests that the user made to their own repositories (no cheating!). Defaults to false.
• --interval How long (in seconds) to sleep between checking for new data. Defaults to 21600 (6 hours)
• --port Which port to run the web server on. Defaults to 8080

Deploying github-pr-exporter, prometheus, and grafana

There are 3 components needed to get up and running to make use of the data.

• github-pr-exporter
• Prometheus
• Grafana

You will also need docker, docker-compose, and git installed. This will run the 3 components above in the form of containers and connect them together via a docker network.

• Start by cloning the repo and cd'ing into the directory

git clone https://github.com/circa10a/github-pr-exporter.git
cd github-pr-exporter

• Start the containers

docker-compose up -d

• Profit!

You should now be able to access grafana at http://localhost:3000 and the preconfigured dashboard at http://localhost:3000/d/h_PRluMnk/pull-requests?orgId=1

The default admin login is username: admin password: admin

Monitoring your own user or others

To change the default configuration, simply update examples/config.yaml to include your GitHub username or others then recreate the containers by running:

docker-compose down
docker-compose up -d

Then check out the new details on the dashboard! Please note that it can take around 2 minutes for the dashboard to reflect the new data so just try refreshing or turn on auto-refresh.