DEV Community

Žan Mažgon
Žan Mažgon

Posted on

Python + Google Sheets on autopilot: why your script dies after 7 days (and 4 more errors)

Your Python script writes to a Google Sheet perfectly on your laptop. Then you put it on a schedule and it breaks a few days later. Here are the five errors behind most of those failures, what they actually mean, and how to fix them.

I'll use gspread in the examples, but the causes are about Google's APIs, so they apply to any client library.

1. 403: The caller does not have permission

A service account is a separate Google identity. It can only touch files that were shared with it.

Fix: open the Sheet, press Share, and add the client_email value from your service-account JSON key (it looks like name@project.iam.gserviceaccount.com) as Editor. Viewer can read but not write.

A close relative is 403: Google Sheets API has not been used in project ... or it is disabled. You enabled the API in a different Cloud project than the one that created your key. Enable the Sheets API (and the Drive API if you open sheets by title) in the project that owns the key, then wait a minute or two.

2. invalid_grant: Token has been expired or revoked after exactly 7 days

If your script uses OAuth user credentials and the OAuth consent screen is in Testing status, Google issues refresh tokens that expire after 7 days. Spreadsheet scopes are not exempt from that rule (only basic name/e-mail/profile scopes are).

Fix: for unattended jobs, use a service account. There is no user token to expire. If you must use OAuth, publish the consent screen (read what that changes first) and make sure failures are loud (see "make failures visible" below).

3. 429: Quota exceeded ... Write requests per minute per user

The Sheets API limit counts requests, not cells. At the time of writing, Google's usage-limits page lists 300 read and 300 write requests per minute per project, and 60 per minute per user per project. Check the page, because limits change.

Writing a 5,000 row x 12 column table:

Strategy Requests Minimum time at 60/min
one request per cell (update_cell in a loop) 60,000 about 16 hours
one request per row (append_row in a loop) 5,000 about 84 minutes
one batch 1 under a minute

Fix, in this order: (1) batch your writes, (2) then add backoff for the occasional 429/5xx.

import random, time, gspread

gc = gspread.service_account(filename="service_account.json")
ws = gc.open_by_key("YOUR_SHEET_ID").worksheet("Data")

rows = [["date", "usd"], ["2026-10-27", 1.1150]]   # your whole table, as a list of lists
ws.update(values=rows, range_name="A1", value_input_option="USER_ENTERED")  # one request

def call_with_backoff(fn, *args, tries=6, **kwargs):
    for n in range(tries):
        try:
            return fn(*args, **kwargs)
        except gspread.exceptions.APIError as e:
            if e.code not in (429, 500, 502, 503) or n == tries - 1:
                raise                    # not retryable, or out of attempts
            time.sleep(min(2 ** n + random.random(), 64))   # exponential backoff + jitter
Enter fullscreen mode Exit fullscreen mode

Retry only what is worth retrying. A 403 "permission denied" will never fix itself; retrying it just hides the problem.

4. Works locally, fails in CI: Could not deserialize key data

The private_key inside the JSON contains \n line breaks. If you paste the key into a CI secret and something converts or strips them, the key can no longer be parsed (you may also see Invalid JWT Signature if the key was deleted).

Fix: store the entire JSON file, unmodified, as one secret, and load it in the job. In GitHub Actions:

# .github/workflows/daily.yml
on:
  schedule: [{ cron: "15 6 * * *" }]   # UTC
  workflow_dispatch:                    # manual "run now" button
jobs:
  run:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
      - uses: actions/setup-python@v5
        with: { python-version: "3.12" }
      - run: pip install gspread
      - run: python job.py
        env:
          SA_JSON: ${{ secrets.SA_JSON }}   # whole JSON file content
          SHEET_ID: ${{ secrets.SHEET_ID }}
Enter fullscreen mode Exit fullscreen mode
import json, os, gspread
gc = gspread.service_account_from_dict(json.loads(os.environ["SA_JSON"]))
Enter fullscreen mode Exit fullscreen mode

Two scheduling gotchas: scheduled workflows run from the default branch only, and on public repositories GitHub disables scheduled workflows after 60 days without repository activity. Always keep the manual workflow_dispatch trigger so you can test the job on demand.

5. Duplicate rows after a retry

If a job fails halfway and you re-run it, appended rows are duplicated. Appending is right for logs; reports need an upsert: use a key (for example the date), update the row if the key exists, append if it does not.

def upsert_by_date(ws, date_str, values):
    dates = ws.col_values(1)               # one read
    if date_str in dates:
        r = dates.index(date_str) + 1
        ws.update(values=[[date_str, *values]], range_name=f"A{r}")
    else:
        ws.append_row([date_str, *values], value_input_option="USER_ENTERED")
Enter fullscreen mode Exit fullscreen mode

(For many rows, read the key column once and write all updates in a single batch_update.)

Make failures visible

  • Write a "last success (UTC)" timestamp into a cell, and colour it red with conditional formatting when it's older than two days.
  • Log counts, not data: "fetched 4, inserted 1, updated 3".
  • Make the CI job exit non-zero on failure, so GitHub e-mails you.
  • Keep a "does my connection work?" script that checks credentials, sharing, clock skew and a scratch-tab write before you debug anything else.

A free cheat sheet, and the long version

I put the five errors above on a one-page cheat sheet: https://whop.com/whop-0bab/fix-the-5-most-common-python-google-sheets-errors-free-1-page-cheat-sheet/.

Disclosure: I'm the author of a paid starter kit that covers all of this in depth (a 47-page guide, a tested Python toolkit, a connection-checking tool, GitHub Actions workflows and three Sheet templates): https://whop.com/whop-0bab/python-google-sheets-running-on-autopilot-the-starter-kit/. Everything in this article works without it.

Top comments (0)