Troubleshoot dependencies in Pipfile/pipenv

The Docker build failed in one of my projects on building pip wheel. As a Python newbie, I was not sure what it refers to, all I got is a dependency build stack trace that ends with AttributeError: cython_sources. I assume this is one of the packages hidden in the pip dependency tree. Eventually I managed to locate it but it took me for a while, thus I think it might be helpful to note down the troubleshooting process that I went through - at least for future me who has gold fish memory.

First I scratched the pip surface:

• pip list to print the installed dependencies, got a looong list but nothing exactly matched the errored source. (Also tried pip freeze but didn't get any luck.)
• searched in Pipfile.lock file for cython but nothing popped up.

Then I went to Uncle Google for cython and got this issue from last year:
https://github.com/yaml/pyyaml/issues/724
Good news - it's closed!

From this issue, I learnt that cython is a dependency for pyyaml. I probably can look for this thing instead.

I figured that we can install and print out the pip dependency tree from the pipfile project, some Q&As on Stack Overflow, and the helpful manual of pipenv cli.

1. pipenv shell to spawn a shell within the virtualenv;
2. pipenv verify to ensure the Pipfile.lock is up-to-date;
3. pipenv sync -d to install all packages and dev packages specified in the Pipfile.lock;
4. pipenv graph to display currently installed dependency tree.

MY Pipfile:

[[source]]
url = "https://pypi.python.org/simple"
verify_ssl = true
name = "pypi"

[packages]
flask = "*"
requests = "*"

[requires]
python_version = "3.10"

[dev-packages]
pre-commit = "*"
black = "*"

The dependency tree I got:

$ pipenv graph
black==22.10.0
├── click [required: >=8.0.0, installed: 8.1.3]
├── mypy-extensions [required: >=0.4.3, installed: 0.4.3]
├── pathspec [required: >=0.9.0, installed: 0.10.1]
├── platformdirs [required: >=2, installed: 2.5.2]
└── tomli [required: >=1.1.0, installed: 2.0.1]
Flask==2.2.2
├── click [required: >=8.0, installed: 8.1.3]
├── itsdangerous [required: >=2.0, installed: 2.1.2]
├── Jinja2 [required: >=3.0, installed: 3.1.2]
│   └── MarkupSafe [required: >=2.0, installed: 2.1.1]
└── Werkzeug [required: >=2.2.2, installed: 2.2.2]
    └── MarkupSafe [required: >=2.1.1, installed: 2.1.1]
pre-commit==2.20.0
├── cfgv [required: >=2.0.0, installed: 3.3.1]
├── identify [required: >=1.0.0, installed: 2.5.7]
├── nodeenv [required: >=0.11.1, installed: 1.7.0]
│   └── setuptools [required: Any, installed: 65.5.0]
├── PyYAML [required: >=5.1, installed: 6.0]
├── toml [required: Any, installed: 0.10.2]
└── virtualenv [required: >=20.0.8, installed: 20.16.5]
    ├── distlib [required: >=0.3.5,<1, installed: 0.3.6]
    ├── filelock [required: >=3.4.1,<4, installed: 3.8.0]
    └── platformdirs [required: >=2.4,<3, installed: 2.5.2]
requests==2.28.1
├── certifi [required: >=2017.4.17, installed: 2022.9.24]
├── charset-normalizer [required: >=2,<3, installed: 2.1.1]
├── idna [required: >=2.5,<4, installed: 3.4]
└── urllib3 [required: >=1.21.1,<1.27, installed: 1.26.12]

Finally I found PyYAML is one of the dependencies of the pre-commit dev packages.

What solved my issue is to re-sync Pipfile.lock file from Pipfile, even tho I didn't update anything in it. Because I use wildcard * to pull the latest version for dev packages, when I do pipenv install, there are many dependency changes in the lock file. PyYAML has bumped from 6.0 to 6.0.1 - and this fixed the docker build issue. Yay!