DEV Community

SHIGRAF SALIK
SHIGRAF SALIK

Posted on

How to Install `pairtools` Properly in Python 3.12

If you're trying to install pairtools—a toolkit for parsing, sorting, and analyzing Hi-C paired-end sequencing data—but run into linking issues with pysam, you're not alone.

A common and frustrating issue is this:

ImportError: libchtslib.cpython-312-x86_64-linux-gnu.so: cannot open shared object file: No such file or directory
Enter fullscreen mode Exit fullscreen mode

Even after using --no-build-isolation, you might still see errors like:

/tmp/pip-build-env-*/overlay/lib/python3.12/site-packages/pysam/libchtslib.cpython-312-x86_64-linux-gnu.so => not found
Enter fullscreen mode Exit fullscreen mode

This happens because the compiled shared object is wrongly linked to temporary build paths. In this guide, I'll show you how to fix this for good.

✅ Goal

To install pairtools in an isolated environment (like GitHub Codespaces), ensuring that:

  • pysam is installed from source.
  • Shared libraries are not linked to /tmp or other temporary build paths.
  • All 17 tests pass using pytest.

📦 Step-by-Step Installation

🔄 1. Start Fresh (Use a Clean Codespace or Environment)

Delete and recreate the Codespace or virtual environment to prevent cached artifacts from interfering.

🔧 2. Install System Dependencies 

sudo apt update && sudo apt install -y build-essential libhts-dev python3-dev
Enter fullscreen mode Exit fullscreen mode

This ensures your system has the compilers and headers required for building C extensions like those in pysam.

❌ 3. Uninstall Existing Builds

Remove any previously installed versions of pysam or pairtools:

pip uninstall pysam pairtools -y
Enter fullscreen mode Exit fullscreen mode

🧪 4. Build pysam from Source

This is crucial. We want to avoid using binary wheels:

pip install "pysam==0.23.0" --no-binary pysam --no-cache-dir --verbose
Enter fullscreen mode Exit fullscreen mode

✅ Check for .so files:

ls /home/codespace/.python/current/lib/python3.12/site-packages/pysam/*.so
Enter fullscreen mode Exit fullscreen mode

You should see something like:

libchtslib.cpython-312-x86_64-linux-gnu.so
csamtools.cpython-312-x86_64-linux-gnu.so
Enter fullscreen mode Exit fullscreen mode

🔍 5. Verify Proper Linking of pysam 

ldd /home/codespace/.python/current/lib/python3.12/site-packages/pysam/libchtslib.cpython-312-x86_64-linux-gnu.so
Enter fullscreen mode Exit fullscreen mode

Ensure the output does not reference any /tmp paths.

Expected:

linux-vdso.so.1 => ...
libc.so.6 => /lib/x86_64-linux-gnu/libc.so.6 (0x00007f...)
Enter fullscreen mode Exit fullscreen mode

📂 6. Clone and Install pairtools

Navigate to your cloned pairtools directory:

cd /workspaces/pairtools
Enter fullscreen mode Exit fullscreen mode

Install in editable mode with --no-build-isolation:

pip install -e .[all] --no-build-isolation --no-cache-dir --verbose
Enter fullscreen mode Exit fullscreen mode

The [all] includes all optional dependencies, and --no-cache-dir avoids reuse of broken builds.

🧬 7. Confirm pairtools Linking 

ldd /workspaces/pairtools/pairtools/lib/parse_pysam.cpython-312-x86_64-linux-gnu.so
Enter fullscreen mode Exit fullscreen mode

You should see it link to:

/home/codespace/.python/current/lib/python3.12/site-packages/pysam/libchtslib.cpython-312-x86_64-linux-gnu.so
Enter fullscreen mode Exit fullscreen mode

There should be no reference to /tmp/pip-build-env-*.

✅ 8. Run Tests

Now validate the setup:

/home/codespace/.python/current/bin/python -m pytest -v
Enter fullscreen mode Exit fullscreen mode

You should see:

collected 17 items
...
17 passed in ...
Enter fullscreen mode Exit fullscreen mode

Note: At the time of writing this article, the Pairtools had 17 tests in total. Kindly recheck it to see if there has been any addition to the tests using:

pytest --collect-only
Enter fullscreen mode Exit fullscreen mode

🧹 If Issues Persist: Manual Build

If you still see broken paths in the ldd output, do this:

cd /workspaces/pairtools
rm -rf build dist *.egg-info pairtools/lib/*.so
python setup.py build_ext --inplace --verbose
pip install -e .[all] --no-build-isolation --no-cache-dir --verbose
Enter fullscreen mode Exit fullscreen mode

Also check and clear any conflicting environment variables:

unset LD_LIBRARY_PATH
unset PYTHONPATH
Enter fullscreen mode Exit fullscreen mode

🧠 Why This Works

  • --no-build-isolation forces pip to use your installed pysam, not a temp one.
  • Building from source ensures .so files are in the correct, persistent location.
  • Cleaning build artifacts ensures you're not pulling in a mislinked .so.

💡 Pro Tip: Verify Shared Object Path

Always check your .so dependencies using:

ldd <your-module>.so
Enter fullscreen mode Exit fullscreen mode

If it points to /tmp, rebuild from scratch.

📌 Final Words

Installing pairtools correctly in Python 3.12, especially in ephemeral environments like GitHub Codespaces, requires care. By following this guide, you can ensure proper linking and a smooth installation with all tests passing.

Top comments (0)