If you're a data engineer or data scientist, you might be using or learning to use dbt (data build tool), the powerful data modeling and transformation tool from Fishtown Analytics. I've helped colleagues understand and install dbt a number of times, and I know there are a few things that data engineers can get hung up on during installation.
Let's take a peek at two big ones so you can avoid getting tripped up by the same errors.
If you didn't already have Python installed in your working environment, you've probably been told to install it. So you did! And likely you installed the latest version as anyone would do, which is Python 3.9.
But currently when you run pip install dbt in your working environment in the hopes that you can install dbt, the tool's setup.py installs dbt 0.19.0, which is not ideal. Why? Because this latest version of dbt is incompatible with Python 3.9 as of this post on the official dbt community from November 2020.
Have no fear, all you need to do to fix this is to install Python 3.8.6 instead.
OK, you installed the right version of Python and you ran the install command again. Now you're stuck here:
Collecting PyICU>=2.4.2 Downloading PyICU-2.6.tar.gz (233 kB) |████████████████████████████████| 233 kB 2.2 MB/s ERROR: Command errored out with exit status 1: command: 'c:\users\pc\appdata\local\programs\python\python38\python.exe' -c 'import sys, setuptools, tokenize; sys.argv = '"'"'C:\\Users\\pc\\AppData\\Local\\Temp\\pip-install-m92rew9z\\pyicu\\setup.py'"'"'; __file__='"'"'C:\\Users\\pc\\AppData\\Local\\Temp\\pip-install-m92rew9z\\pyicu\\setup.py'"'"';f=getattr(tokenize, '"'"'open'"'"', open)(__file__);code=f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, __file__, '"'"'exec'"'"'))' egg_info --egg-base 'C:\Users\pc\AppData\Local\Temp\pip-pip-egg-info-uns2aquj' cwd: C:\Users\pc\AppData\Local\Temp\pip-install-m92rew9z\pyicu\ Complete output (53 lines): Traceback (most recent call last): File "C:\Users\pc\AppData\Local\Temp\pip-install-m92rew9z\pyicu\setup.py", line 63, in <module> ICU_VERSION = os.environ['ICU_VERSION'] File "c:\users\pc\appdata\local\programs\python\python38\lib\os.py", line 675, in __getitem__ raise KeyError(key) from None KeyError: 'ICU_VERSION'
If this looks familiar to you, don't panic. It's another quick fix. dbt 0.19.0 is incompatible with a dependency that's inside PyICU, called agate. Fishtown Analytics seems to be on the way to solving this, but who knows when it will be in the pip package. For now, try this.
Basically you just need to install agate 1.6.1 because 1.6.2 is incompatible and will cause the PyICU library to fail during dbt install. Run the following command:
pip install agate==1.6.1
This should succeed. When it does, run
pip install dbt
It should install properly. You have dbt ready to rock!
Did this post solve your problem? If not, leave a comment and I'll see if I can help!
I hope these quick facts about dbt installation are helpful for you. If you'd like to see a dbt project in action, please feel free to clone my dbtTestExamples repository on Github and learn how to connect a dbt model and tests to a Google Big Query instance.