DEV Community

Jayson DeLancey
Jayson DeLancey

Posted on • Updated on

Query GitHub Repo Topics Using GraphQL

Introduced in 2020, the GitHub user profile README allow individuals to give a long-form introduction. This multi-part tutorial explains how I setup my own profile to create dynamic content to aid discovery of my projects:

You can visit github.com/j12y to see the final result of what I came up with for my own profile page.

My GitHub Profile

The GitHub Repo Gallery

The intended behavior for my repo gallery is to create something similar to pinned repositories but with a bit more visual pizzazz to identify what the projects are about.

In addition to source code, the repo can have metadata associated with it:

✔️ Name of the repository
✔️ Short description of the project
✔️ Programming language used for the project
✔️ List of tags / topics
✔️ Image that can be used for social cards

About

The About has editable fields to set the description and topics.

About Repos Description and Topics

Settings

The Settings includes a place to upload an image for social media preview cards.

Repos Settings Social Preview

If you don't set a preview card image, GitHub will generate one automatically that includes some basic profile statistics and your user profile image.

Getting Started with the GitHub REST API

The way I structured this project is to build a library of any functions related to querying GitHub in src/gh.ts. I used a .env file to store my personal access (classic) token for authentication during local development.

├── package.json
├── .env
├── src
│   ├── app.ts
│   ├── gh.ts
│   └── template
│       ├── README.liquid
│       ├── contact.liquid
│       └── gallery.liquid
└── tsconfig.json
Enter fullscreen mode Exit fullscreen mode

I started by using REST endpoints with the Octokit library and TypeScript bindings.

// src/gh.ts
import { Octokit } from 'octokit';
import { RestEndpointMethodTypes } from '@octokit/plugin-rest-endpoint-methods'
const octokit = new Octokit({ auth: process.env.TOKEN});

export class GitHub {
    // GET /users/{user}
    // https://docs.github.com/en/rest/users/users#get-a-user
    async getUserDetails(user: string): Promise<RestEndpointMethodTypes['users']['getByUsername']['response']['data']> {
        const { data } = await octokit.rest.users.getByUsername({
            username: user
        });

        return data;
    };
}
Enter fullscreen mode Exit fullscreen mode

From src/app.ts I initialize the GithHub class, fetch the results, and can inspect the data being returned as a way to get comfortable with the various endpoints.

// src/app.ts
import dotenv from 'dotenv';
import { GitHub } from "./gh";

export async function main() {
  dotenv.config();
  const gh = new GitHub()

  const details = await gh.getUserDetails();
  console.log(details);
}
main();
Enter fullscreen mode Exit fullscreen mode

I typically get started on projects with simple tests like this to make sure all the various pieces to an integration can be configured and work together before getting too far.

Use the GitHub GraphQL Endpoint

To get the data needed for the gallery layout, it would be necessary to make multiple calls to REST endpoints. In addition there is some data not yet available from the REST endpoint at all.

Switching to query using the GitHub GraphQL interface becomes helpful. This single endpoint can process a number of queries and give precise control over the data needed.

💡 The GitHub GraphQL Explorer was fundamentally useful for me to get the right queries defined

This query needs authorization with the personal access token to fetch profile details about followers similar to some of the details returned from the REST endpoints.

// src/gh.ts

const { graphql } = require("@octokit/graphql")

export class GitHub 
    // https://docs.github.com/en/graphql
    graphqlWithAuth = graphql.defaults({
        headers: {
            authorization: `token ${process.env.TOKEN}`
        }
    })

    async getProfileOverview(name: string): Promise<any> {
        const query = `
            query getProfileOverview($name: String!) { 
                user(login: $name) { 
                    followers(first: 100) {
                        totalCount
                        edges {
                            node {
                                login
                                name
                                twitterUsername
                                email
                            }
                        }
                    }
                }
            }
        `;
        const params = {'name': name};

        return await this.graphqlWithAuth(query, params);
    }
}
Enter fullscreen mode Exit fullscreen mode

There are other resources such as Learn GraphQL if you haven't written many queries yet which explains the basics around syntax, schemas, and types.

Getting used to GitHub's GraphQL schema primarily involves walking a series of edges to find linked nodes for objects of interest and their data attributes. In this case, I started by querying a user profile, finding the list of linked followers, and then inspecting their corresponding node's login, name, and email address.

   ┌────────────┐
   │    user    │
   └─────┬──────┘
         │
         └──followers
               │
               ├─── totalCount
               │
               └─── edges
                     │
                     └── node

Enter fullscreen mode Exit fullscreen mode

Faceted Search by Topic Frequency

I often want to find repositories by a topic. The user interface makes it easy to filter among many repositories by programming language such as python but unless you know which topics are relevant can become hit or miss. Was it nlp or nltk I used to categorize related repositories. Did I use dolby or dolbyio to identify repos I have for work projects.

Filter repo by topic

A faceted search that narrows down the number of matching repositories can be helpful for finding relevant projects like this. Given topics on GitHub are open-ended and not constrained to fixed values, it can be easy to accidentally categorize repos with variations like lambda and aws-lambda such that searches only identify partial results.

To address this, a GraphQL query gathering topics by frequency of usage within an organization or individual account can help with identifying the most useful topics.

The steps for this would be:

  1. Query repository topics
  2. Process results to group topics by frequency
  3. Use a template to render the gallery

1 - Query Repository Topics

I used the following GraphQL query to fetch my repositories and their corresponding topics.

const query = `
    query getReposOverview($name: String!) {
        user(login: $name) {
            repositories(first: 100 ownerAffiliations: OWNER) {
                edges {
                    node {
                        name
                        url
                        description
                        openGraphImageUrl
                        repositoryTopics(first: 100) {
                            edges {
                                node {
                                    topic {
                                        name
                                    }
                                }
                            }
                        }
                        primaryLanguage {
                            name
                        }
                    }
                }
            }
        }
    }
`;
Enter fullscreen mode Exit fullscreen mode

This query starts by filtering by user owned repositories (not counting forks) along with the metadata such as the social image.

2 - Process Results and Group Topics by Frequency

Iterating over the results of the query the convention used was to look for anything with the topic github-gallery as something to be featured in the gallery. We also get a count of usage for each of the other topics and programming languages.

var topics: {[id: string]: number } = {};
var languages: {[id: string]: number } = {};
var gallery: {[id: string]: any } = {};

const repos = await gh.getReposOverview(user);
for (let repo of repos.user.repositories.edges) {
  // Count occurrences of each topic
  repo.node.repositoryTopics.edges.forEach((topic: any) => {
    if (topic.node.topic.name == 'github-gallery') {
      gallery[repo.node.name] = repo;
    } else {
      topics[topic.node.topic.name] = topic.node.topic.name in topics ? topics[topic.node.topic.name] + 1 : 1;
    }
  });

  // Count and include count of language used
  if (repo.node.primaryLanguage) {
    languages[repo.node.primaryLanguage.name] = repo.node.primaryLanguage.name in languages ? languages[repo.node.primaryLanguage.name] + 1 : 1;
  }
}
Enter fullscreen mode Exit fullscreen mode

3 - Use a template to render the gallery

The topics are ordered by how often they are used. From the previous post on setting up a dynamic profile, I'm passing scope to the liquid engine for any data to be made available in a template.

  // Share topics sorted by frequency of use for filtering repositories
  // from the organization
  scope['topics'] = Object.entries(topics).sort(function (first, second) {
    return second[1] - first[1];
  });
  scope['languages'] = Object.entries(languages).sort(function (first, second) {
    return second[1] - first[1];
  });

  // Gather topics across repos
  scope['gallery'] = Object.values(gallery);

Enter fullscreen mode Exit fullscreen mode

The repository page on GitHub uses query parameters to sort and filter, so items like topic:nltk can be passed directly in the URL to load a filtered view of repositories. The shields create a nice looking button for navigating to the topic, and use of icons for programming languages helps find relevant code samples.

<p>Explore some of my projects: <br/>
{% for language in languages %}<a href="https://github.com/j12y?tab=repositories&q=language%3A{{language[0]}}&type=&language=&sort="><img src="https://img.shields.io/badge/{{ language[0] }}-{{ language[1] }}-lightgrey?logo={{ language[0] }}&label={{ language[0] }}&labelColor=000000" alt="{{ language[0] }}"/></a> {% endfor %}
{% for topic in topics %}<a href="https://github.com/j12y?tab=repositories&q=topic%3A{{topic[0]}}&type=&language=&sort="><img src="https://img.shields.io/static/v1?label={{topic[0]}}&message={{ topic[1] }}&labelColor=blue"/></a> {% endfor %}
</p>
Enter fullscreen mode Exit fullscreen mode

List of topics and languages

The presentation includes a 3-column row in a table for displaying the metadata about each featured gallery project. This could display all repositories, but limiting to one or two rows seems sensible for managing screen space.

{% for tile in gallery limit:3 %}
<td width="25%" valign="top" style="padding-top: 20px; padding-bottom: 20px; padding-left: 30px; padding-right: 30px;">
<a href="{{ tile.node.url }}"><img src="{{ tile.node.openGraphImageUrl }}"/></a>
<p><b><a href="{{ tile.node.url }}">{{ tile.node.name }}</b></a></p>
<p>{{ tile.node.description }}<br/>
{% for topic in tile.node.repositoryTopics.edges %} <a href="https://github.com/j12y?tab=repositories&q=topic%3A{{topic.node.topic.name }}&type=&language=&sort="><img src="https://img.shields.io/badge/{{ topic.node.topic.name | replace: "-", "--" }}-blue?style=pill"/></a> {% endfor %}
</p>
</td>
{% endfor %}
Enter fullscreen mode Exit fullscreen mode

With all of that put together, we now have a gallery that displays a picture along with the name, description, and tags. The picture can highlight a user interface, architectural diagram, or some other branded visual to help identify the purpose of the project visually.

Gallery row

We can also use this to maintain our list of topics and make finding relevant topics for an audience easier to discover.

Learn more

I hope this overview helps with getting yourself sorted. The next article will dive into some of the other ways of aggregating content.

Did this help you get your own profile started? Let me know and follow to get notified about updates.

Top comments (0)