Python package conflicts

When developing a Python package, users might encounter dependency conflicts if different versions of the same dependency are required. For example, if your package requires requests==2.26.0, but the user's system needs requests==2.25.1, both cannot coexist since Python doesn’t allow multiple versions of the same package to be installed simultaneously.

Approaches to Avoid Dependency Conflicts:

A. Vendor Approach:

• Vendoring Dependencies: This involves including necessary dependencies directly in your package. It's useful for controlling versions but may increase package size.
• Pure-Python Packages: Vendoring works well for pure Python packages without their own dependencies.
• Packages with Dependencies: Vendoring becomes problematic if the vendored package has its own dependencies, leading to potential conflicts.

Issues:

• Dependency Clashes: Vendoring a package with dependencies may lead to conflicts in the user's environment.
• Version Control: Keeping vendored dependencies updated is crucial for security.
• Size: Vendoring can increase package size.

Example:

• Scenario 1: If requests had no dependencies, bundling it with your package ensures the correct version is used.
• Scenario 2: Since requests relies on libraries like urllib3, including it may cause conflicts if other packages require different versions of urllib3.

Note: If you do vendoring, you need to comply with the vendoring policy. Check it here.

B. Virtual Environment Approach:

• Dependency conflicts are often out of your control, especially in third-party apps, even if virtual environments are used.

Issues:

• Out of Our Control: How users set up virtual environments is beyond our influence.
• Third-Party Apps: They might still face conflict issues, even in virtual environments.

C. Fork Approach:

• You can fork the conflicting package, rename it (e.g., mypackage-requests==2.26.0), and use the forked version in your package.

Issues:

• Maintenance: Forking requires keeping the fork updated with the original package.
• Child Dependencies: If the forked package has dependencies, you may need to fork and manage those as well.

Conclusion:

Each approach has its benefits and challenges, and the choice depends on your specific use case and how much control you want over the dependencies. As a rule of thumb, it’s better to resolve conflicts by maintaining the package properly, ensuring compatibility with the broader Python ecosystem.

Resources:

• How do you manage conflicting packages in your requirements.txt?
• Vendoring Policy
• python-vendorize
• How do you feel about vendored packages?