Dependabot and Renovate are great at one job: changing the version number. For patch and minor releases, that's usually the whole job.
For a major version, it's maybe 5% of it.
What a major version actually changes
Here's the OpenAI Python SDK going from 0.x to 1.x. The version bump is one line:
- openai==0.28.1
+ openai==1.51.0
The code that breaks is everywhere else:
- import openai
- openai.api_key = os.environ["OPENAI_API_KEY"]
- resp = openai.ChatCompletion.create(model=MODEL, messages=msgs)
- text = resp["choices"][0]["message"]["content"]
+ from openai import OpenAI
+ client = OpenAI()
+ resp = client.chat.completions.create(model=MODEL, messages=msgs)
+ text = resp.choices[0].message.content
Module-level calls became a client. Dict access became attributes. Every call site has to move, and the bump PR touches none of them. So CI goes red, and the PR waits until somebody has an afternoon.
Two jobs, one automated
- Bump the version. Automated for years.
- Migrate the call sites. Still done by hand: read the migration guide, grep, rewrite, run the tests, repeat.
The second job is the expensive one, and it's the reason old majors pile up.
Doing the second job safely
If you automate the migration, the order of operations matters more than the model:
- Read the vendor's migration guide for the exact before and after.
- Find call sites with structural search, not with an LLM. A tool like ast-grep matches code by syntax tree, so the set of files touched is deterministic and reviewable.
- Rewrite only those call sites.
- Run the project's own typecheck, build and tests, before and after.
- Open a pull request only if they pass. Never push to the default branch.
That's the pipeline Versyka runs. It works alongside Dependabot rather than replacing it: Dependabot keeps bumping, and Versyka adds an impact note to those PRs saying what the change touches in your code, then handles the majors that need code changes.
A quick way to see your own backlog
Paste a package.json or requirements.txt into the upgrade check to see which SDKs are major versions behind, with links to the release notes. No sign-up.
Top comments (5)
Structural search is a strong way to bound the migration surface. It is also worth checking configuration and runtime behavior changes that may not appear at call sites, then comparing representative integration results before and after the upgrade.
Agreed, and that's the gap call-site search can't close on its own. The quiet breakages are usually changed defaults: a timeout, a retry policy or a serialization setting that no call site mentions. Running the same integration tests before and after the upgrade is the best way to catch those, which is why we compare results on the unchanged code first and only count new failures.
With the OpenAI 0.28 to 1.51 jump, which call site hurt most, the client setup or the response handling?
Response handling, by a wide margin.
The client setup is a few lines in one place: openai.api_key and openai.ChatCompletion.create become a client = OpenAI() and client.chat.completions.create. Annoying, but you find it in five minutes.
The responses are the part that spreads. 0.28 returned dicts, 1.x returns typed objects, so every response["choices"][0]["message"]["content"] becomes response.choices[0].message.content. Those reads are scattered through the code, often far from the call itself, and they don't fail until that line actually runs. Streaming chunks and the error classes (openai.error.RateLimitError becomes openai.RateLimitError) have the same problem.
So the setup breaks loudly on import, and the response access breaks quietly in production. That's why we search for where the response is used, not just where the call is made.
Streaming chunks feel like the harder half of that. A bad attribute read fails on one line, but a bad chunk can poison everything downstream before anything complains.