Why does my GitHub OAuth2 Token not have the scopes I requested?

Why doesn't my access token work?

If you're building a GitHub integration into your app that uses GitHub's OAuth authorization method (as opposed to their App authorization, which is similar but has important differences), you may have noticed that sometimes the scopes that your access token has been granted are different from what you requested. This can be a particularly difficult issue to debug since it usually rears its head when you're trying to use an API protected by one of the scopes you requested, but the API request fails for just one of your users. However, when you try to reproduce this issue, it seems to work just fine. What's going on here?

GitHub makes use of a little known part of the OAuth2 spec that allows the authorization server (in this case GitHub) to: "fully or partially ignore the scope requested by the client, based on the authorization server policy or the resource owner's instructions". The way GitHub does this is by allowing your user to pick and choose which of your requested scopes to actually grant to your application through a process we call "line item grants".

As you can see, this is a completely valid use of the OAuth2 spec, but since most providers don't use this, you may not have been ready for it. But not to worry, while the spec includes the ability for the authorization server (GitHub) to ignore your scope, it does provide you some recourse: "if the issued access token scope is different from the one requested by the client, the authorization server MUST include the scope response parameter to inform the client of the actual scope granted."

Verifying the Granted Scope

The scope parameter in the Access Token response (which you were probably ignoring until now) will contain the scope that your user actually granted you, and which you therefore actually have access to with your token. See GitHub's docs about this process here.

Unfortunately, here's where GitHub goes off the rails. While GitHub allows you to specify the scope you want in your request by space-delimiting each scope string (e.g. read:org read:user), which matches the OAuth2 Spec, it returns scope by comma-delimiting the scope strings (e.g. read:org,read:user) in violation of the specification. Which means you may have to serialize and parse your scope using different patterns.

Scope Normalization

In addition, GitHub transforms the scopes through a process called "scope normalization". Basically, they take your requested scopes and narrow them down to the smallest set of logical scopes that will be granted. So, for example, if you request the repo scope, but you also request public_repo and delete:repo, GitHub will return just repo, since the other requested scopes are already included in the repo scope.

The actual normalization process (which scopes are removed when requesting two or more) is only partially documented, but based on our reading of the documentation and subsequent testing, here's the normalization tree:

repo
┣ delete:repo
┣ repo:status
┣ repo:invite
┣ repo_deployment
┣ public_repo
┃ ┗ admin:repo_hook
┃   ┗ write:repo_hook
┃     ┗ read:repo_hook
┗ security_events
admin:org
┗ write:org
  ┗ read:org
admin:public_key
┗ write:public_key
  ┗ read:public_key
user
┣ read:user
┣ user:email
┗ user:follow
write:discussion
┗ read:discussion
write:packages
┗ read:packages
admin:gpg_key
┗ write:gpg_key
  ┗ read:gpgp_key
workflow
gist
notifications
admin:org_hook

Conclusion

If you're building a GitHub integration, be sure to check the normalized, comma-delimited scopes that your user actually granted you via GitHub's API to make sure that your token has the scopes you expect it to.

Of course, if you want to avoid building (or heck, even learning) all that, you can use Xkit's GitHub OAuth Connector and be up and running with always-valid access tokens in a half hour. We have built-in checks to make sure that your tokens have the scopes you expect them to so you don't have to worry about it.