dbt 1.8 breaks on update

On updating dbt-bigquery to latest 1.8.0: No module named 'dbt.adapters.factory'

Issue - dbt 1.8.0 won't run

I spun up a Codespace on pypi_vulnerabilities to resolve a failing build.
My auto-update ran as usual, updating all my Python dependencies to latest stable release and I got dbt-core 1.8.0, now out of beta as of 2024-05-09.

The first thing my auto-update does that depends on dbt - install/update dbt packages - failed.

Successfully installed daff-1.3.46 dbt-adapters-1.1.1 dbt-bigquery-1.8.0 dbt-common-1.0.4 dbt-core-1.8.0 dbt-semantic-interfaces-0.5.1
install or upgrade dbt dependencies
Traceback (most recent call last):
  File "/workspaces/pypi_vulnerabilities/venv/bin/dbt", line 5, in <module>
    from dbt.cli.main import cli
  File "/workspaces/pypi_vulnerabilities/venv/lib/python3.11/site-packages/dbt/cli/__init__.py", line 1, in <module>
    from .main import cli as dbt_cli  # noqa
    ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/workspaces/pypi_vulnerabilities/venv/lib/python3.11/site-packages/dbt/cli/main.py", line 14, in <module>
    from dbt.cli import requires, params as p
  File "/workspaces/pypi_vulnerabilities/venv/lib/python3.11/site-packages/dbt/cli/requires.py", line 10, in <module>
    from dbt.adapters.factory import adapter_management, register_adapter, get_adapter
ModuleNotFoundError: No module named 'dbt.adapters.factory'

Clean Install to Fix

Clearing my venv and reinstalling from scratch gets things back up and running. Here's the specific procedure I used:

$ deactivate # deactivate the venv, if active
$ rm -r venv # recursively delete the venv folder
$ python -m venv venv
$ . venv/bin/activate
$ pip install -U -r requirements.txt
$ dbt deps

I actually ran my init_and_update.sh script rather than those highlighted lines, but those are the relevant things it does.

This also explains why my build wasn't breaking (well, not for this reason anyway) - it always starts from scratch.

Changes in Adapter Dependencies

That's how you can fix the No module named 'dbt.adapters.factory' error if it comes up.
As I read the v1.8 upgrade docs I also learned that dbt Labs intend to change the way Python adapter dependencies work.

Beginning in v1.8, dbt-core and adapters are decoupled. Going forward, your installations should explicitly include both dbt-core and the desired adapter.

Scanning through the related technical docs, I think the auto-update approach I use will still work as intended without explicitly specifying the core dependency.

Up to now adapters have been required release a new minor version to declare compatibility with dbt-core’s minor version. Post dbt-core version 1.8, adapters will not need to do this. Instead maintainers will need to declare their compatibility with dbt-adapters’ versions.

So, if you're setting your adapter dependency with an open upper bound (eg. dbt-bigquery>=1.8.0 as mine will be shortly) then you should also get the latest compatible dbt-core version each time you run pip install -U -r requirements.txt on your development environment or your build system. I'll post how that shakes out over time.