So, you've got a super-secret Python package that you want to share with the world—but without exposing your source code. The good news? You can publish your package on PyPI while keeping your code hidden. This guide will show you how to use Cython and Poetry to protect your code like a ninja in the shadows.
Why Hide Your Source Code?
There are many reasons why developers might want to conceal their Python source code:
- Prevent reverse engineering of proprietary algorithms.
- Protect intellectual property from competitors.
- Ensure security by keeping sensitive implementation details private.
- Reduce code tampering risks in production environments.
While Python is an interpreted language and usually exposes source code, we can use Cython to convert .py
files into compiled binary files, making the original code inaccessible.
Step 1: Setting Up Cython for Source Code Protection
Cython is a tool that compiles Python code into C and then generates a binary (.so
or .pyd
) file. This means that instead of distributing .py
files, you share compiled binaries.
To get started, install Cython in your development environment:
poetry add --dev Cython
Then, configure Poetry to use a custom build script instead of generating a default setup.py
file.
Modify pyproject.toml
for Custom Builds
Add the following to your pyproject.toml
file:
[tool.poetry.build]
script = "build.py"
generate-setup-file = false
Now, let's create build.py
, our custom script that will compile Python files into binary form.
Step 2: Writing the build.py
Script
This script automates the conversion of Python files into C extensions:
import multiprocessing
from pathlib import Path
from typing import List
from setuptools import Extension, Distribution
from Cython.Build import cythonize
from Cython.Distutils.build_ext import new_build_ext as cython_build_ext
SOURCE_DIR = Path("SRC") # Replace with your source directory
BUILD_DIR = Path("BUILD")
def get_extension_modules() -> List[Extension]:
return [
Extension(
name=str(py_file.with_suffix("")).replace("/", "."),
sources=[str(py_file)]
)
for py_file in SOURCE_DIR.rglob("*.py")
]
def cythonize_helper(extension_modules: List[Extension]) -> List[Extension]:
return cythonize(
module_list=extension_modules,
build_dir=BUILD_DIR,
nthreads=multiprocessing.cpu_count() * 2,
compiler_directives={"language_level": "3"}
)
extension_modules = cythonize_helper(get_extension_modules())
distribution = Distribution({"ext_modules": extension_modules, "cmdclass": {"build_ext": cython_build_ext}})
build_ext_cmd = distribution.get_command_obj("build_ext")
build_ext_cmd.ensure_finalized()
build_ext_cmd.inplace = 1
build_ext_cmd.run()
This script:
-
Finds all
.py
files in the source directory. - Converts them into compiled binaries using Cython.
- Prepares them for distribution via Poetry.
Step 3: Exclude Source Code from Distribution
By default, Poetry includes all files in your source directory. To prevent .py
files from being uploaded to PyPI, modify pyproject.toml
:
[tool.poetry]
exclude = ["SRC/**/*.py"] # Replace SRC with your actual source directory
include = ["SRC/**/*.so"] # Include compiled files
This ensures that only compiled binaries are included in the final package.
Step 4: Building & Publishing the Secure Package
Now that everything is set up, build your package using Poetry:
poetry build --format wheel
Verify that the package contains only .so
files (or .pyd
for Windows) by extracting the .whl
file:
unzip dist/*.whl -d check_package
ls check_package
If everything looks good, publish your package to PyPI:
poetry publish --build
Note: Do not publish an
sdist
(source distribution) package, as it includes uncompiled source code. Stick to wheel packages (.whl
).
Step 5: Supporting Multiple Platforms & Python Versions
Wheel packages contain pre-compiled code, so you must build different versions for different Python versions and OS platforms. Popular projects like TensorFlow use this approach.
To automate this, use GitHub Actions or another CI/CD pipeline with a build matrix:
- Build Linux, Windows, and macOS versions.
- Compile for Python 3.7, 3.8, 3.9, 3.10, and 3.11.
A simple GitHub Actions workflow example:
jobs:
build:
runs-on: ubuntu-latest
strategy:
matrix:
python-version: ["3.7", "3.8", "3.9", "3.10", "3.11"]
steps:
- uses: actions/checkout@v2
- uses: actions/setup-python@v2
with:
python-version: ${{ matrix.python-version }}
- run: poetry install
- run: poetry build --format wheel
This generates multiple wheels for different Python versions, ready to be uploaded to PyPI.
Final Thoughts & Resources
By following this guide, you've successfully:
✅ Protected your Python source code using Cython.
✅ Configured Poetry to exclude uncompiled .py
files.
✅ Created a custom build process using build.py
.
✅ Published your package securely on PyPI.
Need a working example? Check out the sample repository: GitHub - prasad89/pypi-publish-secure. If you have questions or need help, feel free to open an issue there!
If this guide helped you, share it with fellow Python developers! 🚀
Top comments (0)