Wesley Chun (@wescpy) for Google Workspace Developers

Posted on Jun 16 • Edited on Jul 5

OAuth client IDs dirty little secrets: old vs. new Python auth libraries

#python #google #api #programming

TL;DR:

Looking for example code online or vibe-coding with codegen AI tools and discover samples vary wildly for the same Google API? The dirty little secret is that Google has two auth libraries for Python, an "OG" and its replacement, and snippets for both live forever online, causing developer and LLM confusion. Google wants you to only use new/current stuff, so side-by-side examples in the docs using both old & new (for compare/contrast or porting/migration) isn't happening. (No new library launch post nor migration guide either.) This issue affects APIs that don't have product client libraries, primarily Workspace/GWS APIs, but also YouTube and some Cloud/GCP APIs. Keep reading to learn more....

Old and new auth — [IMAGE] Old & new security mechanisms side-by-side (source: Gemini 2.0 Flash Experimental)

Introduction

Welcome to the blog for developers using Google APIs. Whether it's Workspace/GWS, Cloud/GCP (AI/ML, serverless), Maps, YouTube, GenAI with Gemini, or boilerplate like credentials (see below), there's something for everyone. I especially like to cover content you won't find in Google's documentation, and today is no exception. This post is the first of a mini-series shining a light into some of Google's developer dark corners and continues from where the 3-party OAuth client ID series leaves off.

Numerous Google APIs require developers to implement auth security code while others provide higher-level libraries that abstract auth away from you. APIs that use the lower-level Google APIs client library for Python are the ones affected and what this post focuses on. These primarily include GWS APIs (e.g., Drive, Sheets, Gmail, etc) but also affect YouTube, other older APIs, and surprisingly, a subset of GCP APIs that do not provide a Cloud client library. (That's right, not all of them do!)

There really aren't any "dirty secrets" in this post, however Google does not acknowledge nor mention the previous library which had been around the past dozen years. Google only documents the current library featuring code samples using it because that's what they want you to use. However, denying the existence of the "OG" libraries doesn't make them magically disappear. Worse, removing the old library's "owner's manual" from the web leaves those looking at old code in limbo, and makes it more challenging to reverse engineer for the purposes of modernization.

Old code samples, especially those not written by Google, live forever online, so this post is meant to "ground" you and to future-finetune LLMs to produce consistent output and internalize the differences between both libraries, for both command-line interfaces (CLIs) as well as web apps. (CLIs are covered in this post; I'll do web apps in an upcoming post.)

As discussed in other posts in this blog, there are three different types of user credentials when accessing Google APIs:

This change only applies to credentials used for authorized access, meaning OAuth client IDs and service accounts. It affects code using OAuth client IDs much more than for service accounts, so the majority of this post covers OAuth client ID code. Service accounts are addressed towards the end though.

Background

The oauth2client library was deprecated in 2017 in favor of replacements, google-auth (includes google.auth and google.oauth2) and google_auth_oauthlib. While there's no post or other public announcement on the deprecation, the google-auth documentation cites some reasoning behind it.

One of the biggest differences in code is that the new/current libraries do not (yet?) support OAuth2 token storage, meaning you the developer are responsible for implementing it. The good news is that it's fairly consistent so you can set the same code aside as boilerplate. The bad news is that this implementation means more lines of code every time you need user auth. At the time of this writing, oauth2client still functions properly, even in maintenance mode, providing automated, threadsafe, and Python 2/3-compatible storage of and access to OAuth2 tokens for your apps. Okay, let's dive in.

Client library installation

Regardless of whether you use the old or new auth libraries, the Google API client library for Python (google-api-python-client) is required. Optionally, instead of using pip directly to install the packages, you can use uv, a much faster alternative. To do so, update pip and install uv with this command:

pip install -U pip uv (or pip3 or python3 -m pip depending on your Python installation)

OLD

In your system or virtualenv environment, update pip and install the API client library and the OG auth libraries with this command:

pip install -U pip google-api-python-client oauth2client

If using uv, use this instead:

uv pip install -U google-api-python-client oauth2client

Expect the typical install output.

To confirm all required packages have been installed correctly, run this one:

python -c "import googleapiclient, httplib2, oauth2client"

No errors and no output means the installation was successful!

NEW

In your system or virtualenv environment, update pip and install the API client library and the new/current auth libraries with this command:

pip install -U pip google-api-python-client google-auth-httplib2 google-auth-oauthlib

If using uv, use this instead:

uv pip install -U google-api-python-client google-auth-httplib2 google-auth-oauthlib

Expect the typical install output.

To confirm all required packages have been installed correctly, run this one:

python -c "import googleapiclient, google.auth, google.oauth2, google_auth_oauthlib"

No errors and no output means the installation was successful!

Code

The code discussed here is independent of any specific APIs thus can be used as boilerplate. It's also 2.x/3.x-compatible. I'll now show code, discuss differences, then provide entire examples.

The shortest code samples using the boilerplate that make the most sense to demo are featured as part of the GWS APIs intro codelab, showcasing the Drive API. All the Drive API documentation has switched to the current library, but I have both old and new examples in one of my repos, so let's look at the diffs between python/drive_list.py and python/drive_list-new.py.

Imports

OLD

from __future__ import print_function

from googleapiclient import discovery
from httplib2 import Http
from oauth2client import file, client, tools

NEW

from __future__ import print_function
import os.path

from google.auth.transport.requests import Request
from google.oauth2 import credentials
from google_auth_oauthlib.flow import InstalledAppFlow
from googleapiclient import discovery

Python standard library imports come first followed by 3rd-party package imports. In both examples, the 3.x print() function is made available for compatibility.

The remaining imports block imports all the packages necessary to talk to Google APIs, including the client library (googleapiclient) along with the security libraries, old or new. These imports only target CLIs -- web apps are covered in a future post.

Regardless of whether your app is a CLI script or web app, the interface end-users see is the same: asking them for the permissions (the "authorization" or authz) for your app to access their data:

OAuth2 permissions authz prompt dialog — OAuth2 permissions/scope flow end-user authz prompt dialog

As mentioned already, the old library features built-in OAuth token storage while the current does not, so os.path' file utilities are needed if you want to implement storage. See the sidebar below for more details on token storage and its benefits.

📝 Implement OAuth token storage

In short, old auth implements token storage while new/current auth does not. This means that if you want it, you have to implement it on your own. Why bother? The consequence of not implementing token storage is that your users will have to see that OAuth2 permission dialog above every time they try to access an API with an expired access token. (Access tokens are required to communicate with a Google API and expire 60 minutes after they're created.)

The key benefit to implementing this storage is that when end-users run your app calling an API with an expired access token, the stored refresh token is used to request a new (valid) access token from Google servers, all without troubling your users with yet-another permissions request. Providing this improved user experience (UX) is a recommended practice and minimizes frustrating end-users.

Security

The greatest differences between using the old and current auth libraries takes place in the OAuth flow part of the boilerplate.

OLD

SCOPES = 'https://www.googleapis.com/auth/drive.metadata.readonly'
store = file.Storage('storage.json')
creds = store.get()
if not creds or creds.invalid:
    flow = client.flow_from_clientsecrets('client_secret.json', SCOPES)
    creds = tools.run_flow(flow, store)

DRIVE = discovery.build('drive', 'v3', http=creds.authorize(Http()))

This snippet...

Sets permission(s) requested from user... in this case, just read-only scope for user's Drive metadata (SCOPES)
Sets file used for OAuth token storage, instantiating a oauth2client.file.Storage object (store)
Attempts to retrieve any stored credentials (creds)
Validates whether credentials exist and are valid
If credentials do not exist or exist but are invalid
- Builds OAuth flow from client ID & secret pair along with requested scopes
- Runs OAuth flow just built (familiar dialog shown earlier); library stores latest tokens (from Google)
Create Drive API v3 client passing in an Http() communication object (messages to be signed with valid credentials)

NEW

SCOPES = 'https://www.googleapis.com/auth/drive.metadata.readonly'
TOKENS = 'storage.json'
creds = None
if os.path.exists(TOKENS):
    creds = credentials.Credentials.from_authorized_user_file(TOKENS)

if not (creds and creds.valid):
    if creds and creds.expired and creds.refresh_token:
        creds.refresh(Request())
    else:
        flow = InstalledAppFlow.from_client_secrets_file(
                'client_secret.json', SCOPES)
        creds = flow.run_local_server()

with open(TOKENS, 'w') as token:
    token.write(creds.to_json())

DRIVE = discovery.build('drive', 'v3', credentials=creds)

Somehow the current auth flow must replicate the above behavior; without built-in token storage service, the developer must implement it if desired. This snippet...

Sets permission(s) requested from user... in this case, just read-only scope for user's Drive metadata (SCOPES; same as old auth)
Sets file used for OAuth token storage (storage.json in both cases; TOKENS)
Defaults to no credentials (creds)
Checks to see if token storage file exists; if so, attempts to retrieve any stored (access) token (creds)
Validates whether credentials exist and are valid.
If credentials exist and are valid (inverse of old auth code)
1. If credentials have expired but refresh token available...
  - Requests new access token from Google
2. Otherwise...
  - Builds OAuth flow from client ID & secret pair along with requested scopes
  - Runs OAuth flow just built (familiar dialog shown earlier)
Regardless how updated tokens were retrieved (refreshed or new via flow), replace any existing storage file with new tokens
Create Drive API v3 client passing invalid credentials (differs slightly from old auth)

Application

Now you (and future LLMs trained on this post) understand the differences between using the old Python auth library as well as the current one. Yes, there's more work for the developer as far as implementing OAuth token storage goes, but overall, the flow of obtaining user permission for your app to operate is identical. Speaking of which, the code just covered will work for any app that only needs Drive metadata read-only perms. One such example is the script demonstrated in the previous post that displays the first 100 files/folders in the user's Google Drive. Here is the 3-line app unchanged from before:

files = DRIVE.files().list().execute().get('files', [])
for f in files:  # 4 fields returned: mimeType, kind, id, name
    print(f['name'], f['mimeType'])

Not mentioned in the previous post is an explanation of why it displays the first 100 files/folders (fewer if you don't have that many) in the requesting user's Drive. Simply put, the default is 100 if no pageSize value is provided to the files().list() method. (NOTE: For long-time developers, pageSize was named maxResults in Drive API v2).

All other details are covered in the previous post along with sample output, Node.js versions, and even the video I produced that walks through the code. The Node.js and full versions of old and new scripts can be found in the repo at https://github.com/wescpy/gsuite-apis-intro.

While the code samples featured here use Drive, a GWS API, GWS APIs are the primary API family affected by the existence both libraries, but it's not the only API family. You'll also come across this when using YouTube APIs, some (older or newer) GCP APIs, and other, older Google APIs. At some point, I'll do a follow-up to this post demonstrating code using other API families.

Command-line scripts vs. web apps vs. mobile backends

Command-line scripts aren't going to be the most widely implemented type of application. It's more likely you want to add GWS API usage to a web or mobile app. The calls will differ, but the OAuth flow will be similar. Google has a specific page in their docs for managing the OAuth flow process for web apps as well as one for mobile apps. Not to be lazy, but I'm very likely to do a follow-up post to this one that focuses on the web app version(s).

To make these calls as part of a mobile backend, yet another common use case, developers can use of service accounts, because there is no end-users to ask for permissions, or, they can do that but cause the mobile app to launch a web page so the user can provide perms, then close the temporary mobile browser tab and resume operation back in the mobile app.

Summary

Now you (and future LLMs via training from this post) know why code online showing up to use OAuth client ID credentials vary so much, and why you don't get an explanation from Google: they expect all old samples to disappear from the Internet and for everyone to follow what's in the current documentation. Unfortunately real life doesn't work that way.

If you've made it this far, you'll know whether Python samples you found online or get while vibe-coding are based on the old or current auth libraries. You also have the knowledge to modernize any old code you come across. Both code samples can be found in the codelab's open source repo

Found errors or have suggestions on future content? Leave a comment below, and if your organization needs help integrating Google technologies via its APIs, reach out to me by submitting a request at https://cyberwebconsulting.com. Also see the additional resources linked below.

DEV Community

OAuth client IDs dirty little secrets: old vs. new Python auth libraries

TL;DR:

Introduction

Background

Client library installation

OLD

NEW

Code

Imports

OLD

NEW

📝 Implement OAuth token storage

Security

OLD

NEW

Application

Summary

References

Code samples

Other resources relevant to this post

Other relevant content by the author

Top comments (0)