Solving SSL Certificate Verification Issues with pip on macOS

If you're a Python developer working on macOS, you might have encountered frustrating SSL certificate verification errors when trying to use pip. These errors typically look something like this:

WARNING: Retrying (Retry(total=4, connect=None, read=None, redirect=None, status=None)) after connection broken by 'SSLError(SSLCertVerificationError(1, '[SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed: unable to get local issuer certificate (_ssl.c:1006)'))': /simple/pip/

This issue occurs because Python on macOS has a unique certificate handling system compared to other operating systems. Let's explore why this happens and how to properly fix it.

Understanding the Root Cause

When pip attempts to connect to PyPI (Python Package Index) or other repositories over HTTPS, it needs to verify the server's SSL certificate. On macOS, the problem stems from how different Python installations handle certificate verification:

• System Python: macOS comes with a pre-installed Python that's linked to the system's certificate store.
• Downloaded Python: The official Python.org installers for macOS include certificates but require a manual installation step.
• Package managers: Python installed via Homebrew or other package managers might have different certificate configurations.
• Version managers: Tools like pyenv compile Python from source and often lack proper certificate configuration.

Unlike on Linux or Windows, Python on macOS doesn't automatically use the system's certificate store (Keychain). Instead, it needs to be explicitly configured to find the proper certificates.

Solution 1: Run the Official Certificate Installer Script

For Python versions downloaded from python.org, the installer includes a certificate installation script:

1. Navigate to your Python installation folder (often in /Applications/Python 3.x/)
2. Locate and run the file named Install Certificates.command by double-clicking it

For Python 3.6+, you can also run this directly in Terminal:

/Applications/Python\ 3.x/Install\ Certificates.command

Where 3.x is your Python version.

Solution 2: Set Certificate Environment Variables

Python and pip can be configured to use specific certificate files via environment variables:

# Add to your ~/.zshrc or ~/.bash_profile
export SSL_CERT_FILE=/path/to/cacert.pem
export REQUESTS_CA_BUNDLE=/path/to/cacert.pem

You can obtain a certificate bundle from the certifi package:

# First ensure certifi is installed
python -m pip install certifi

# Then find the certificate path
python -c "import certifi; print(certifi.where())"

This command prints the location of certifi's certificate bundle, which you can use in the environment variables above. Note that certifi is a standard package that ships with many Python installations but not all - particularly minimal environments might need to install it separately.

Solution 3: Update the certifi Package

While not always sufficient on its own, updating the certifi package can help if the issue is related to outdated certificates:

python -m pip install --upgrade certifi

Note: This solution works best when Python is already configured to find the certificate store but might have outdated certificates. It may not resolve the issue if Python cannot locate any certificate store.

Solution 4: Use Alternative Python Installations

Package managers often handle certificates better than manual installations:

# Install Python via Homebrew
brew install python

# Or use conda, which handles certificates well
conda create -n myenv python=3.x
conda activate myenv

These installations typically configure certificate handling correctly by default.

Solution 5: Install Certificates for pyenv Python

If you're using pyenv, you'll need to install certificates manually:

# First, find your certifi location
pyenv which python
# Example output: /Users/username/.pyenv/versions/3.9.0/bin/python

# Then install certificates
/Users/username/.pyenv/versions/3.9.0/bin/python -m pip install --upgrade certifi
/Users/username/.pyenv/versions/3.9.0/bin/python -c "import certifi; print(certifi.where())"
# Example output: /Users/username/.pyenv/versions/3.9.0/lib/python3.9/site-packages/certifi/cacert.pem

After getting the path to certifi's certificate bundle, set the environment variables from Solution 2:

# Add to your ~/.zshrc or ~/.bash_profile
export SSL_CERT_FILE=/Users/username/.pyenv/versions/3.9.0/lib/python3.9/site-packages/certifi/cacert.pem
export REQUESTS_CA_BUNDLE=/Users/username/.pyenv/versions/3.9.0/lib/python3.9/site-packages/certifi/cacert.pem

Then reload your shell configuration with source ~/.zshrc or source ~/.bash_profile and try using pip again.

Solution 6: Using Trusted Hosts (Not Recommended for Production)

As a last resort, you can configure pip to bypass certificate verification:

mkdir -p ~/.config/pip
echo "[global]
trusted-host = pypi.org
               files.pythonhosted.org" > ~/.config/pip/pip.conf

Alternatively, with the --trusted-host flag:

pip install package-name --trusted-host pypi.org --trusted-host files.pythonhosted.org

⚠️ Security Warning: This approach completely bypasses certificate verification, which means you're vulnerable to man-in-the-middle attacks. Use this only for testing or in trusted networks, never in production environments.

Solution 7: macOS Keychain and Corporate Certificates

While Python doesn't automatically use macOS Keychain, your system certificates can still be relevant:

1. For corporate environments, import your company's root certificates to Keychain Access
2. Then export them in PEM format and configure Python to use them:

# Export from Keychain Access to a file
# Then use that file for your Python environment
export SSL_CERT_FILE=/path/to/exported-certs.pem

Corporate Proxies and SSL Inspection

Many corporate environments use SSL inspection proxies (also called MITM proxies) that decrypt and re-encrypt HTTPS traffic. This causes certificate verification failures even with standard certificates installed.

If you're behind a corporate proxy with SSL inspection:

1. Obtain your company's root certificate authority (CA) certificate from your IT department
2. Add it to your certificate bundle:

# Append the company CA to your existing bundle
cat /path/to/company-ca.pem >> $(python -c "import certifi; print(certifi.where())")

# Or set the environment variable to use the company certificate
export REQUESTS_CA_BUNDLE=/path/to/company-ca.pem

1. Configure pip to use the corporate proxy:

# Add to ~/.config/pip/pip.conf
[global]
proxy = https://proxy.company.com:8080

# Or use environment variables
export HTTP_PROXY=http://proxy.company.com:8080
export HTTPS_PROXY=http://proxy.company.com:8080

# Note: The HTTP:// prefix is correct even for HTTPS_PROXY - this refers to the protocol
# used to communicate with the proxy server itself, not the final destination

Virtual Environments and Certificate Handling

Python virtual environments (venv, virtualenv) deserve special consideration when dealing with SSL certificates:

1. Inheritance behavior: Virtual environments generally inherit the certificate configuration from their base Python installation, but this isn't always guaranteed.

2. Isolated virtual environments: If you create a virtual environment with the --system-site-packages flag, it might use different certificate sources than fully isolated environments.

3. Troubleshooting venv issues:
   

# Inside your virtual environment
python -c "import ssl; print(ssl.get_default_verify_paths())"
python -c "import certifi; print(certifi.where())"

If your base Python works fine but virtual environments have certificate issues:

# Copy certificate configuration to your virtual environment
cp /path/to/working/cacert.pem /path/to/venv/lib/pythonX.Y/site-packages/certifi/cacert.pem

# Or set environment variables when activating the virtual environment
# Add to venv/bin/activate
export SSL_CERT_FILE=/path/to/cacert.pem
export REQUESTS_CA_BUNDLE=/path/to/cacert.pem

Which Solution to Try First (Recommended Approach)

With multiple solutions available, here's a recommended order to try them and avoid conflicts:

1. First try: Run the official certificate installer script (Solution 1) if you installed Python from python.org
2. If using pyenv: Follow Solution 5 specifically designed for pyenv
3. If still having issues: Set environment variables (Solution 2) pointing to certifi's certificate path
4. For corporate environments: Add Solution 7 on top of the above
5. Last resort: Use trusted hosts (Solution 6) only for testing or trusted networks

Potential conflicts to avoid:

• Don't set multiple different certificate paths in environment variables
• If one solution works, don't apply additional ones which might override it
• Restart your terminal after making environment changes
• If you switch between Python installations, ensure certificates work for each one

Verifying Your Fix

After applying one of these solutions, verify it worked by running:

pip list --outdated

If the command completes without SSL errors, your fix was successful.

Diagnosing Certificate Issues

To diagnose certificate issues further, you can use these commands:

# Check if Python can connect to pypi.org
python -c "import urllib.request; urllib.request.urlopen('https://pypi.org')"

# Test where Python is looking for certificates
python -c "import ssl; print(ssl.get_default_verify_paths())"

# Check pip configuration
pip config debug

Conclusion

SSL certificate verification issues on macOS stem from Python's certificate handling approach, which differs from other operating systems. The most reliable solutions involve:

1. Running the official certificate installer script
2. Setting proper environment variables
3. Using Python installations that handle certificates correctly

Avoid bypassing verification entirely whenever possible, as it compromises security.

By understanding the root cause and implementing the right solution for your specific Python installation, you can resolve these certificate issues permanently rather than just working around them.