Releasing my Python Project

I have been working on til-page-builder for this whole semester, and as the term approaches its end, I thought it was a good time to release the project.

I have published the package to Python Package Index, commonly called PyPi, and in this post, I'll be sharing the steps I had to follow in the process.

Note: I will be using TestPyPi in this tutotal, which is a separate server for testing python packages, since the original Pypi has temporarily suspended new users registration.

Table of Contents

 1. Registering as a New User 🧑🏻
 2. Creating an API Key 🔑
       2.1. Enabling MFA 🔒
       2.2. Getting the Token 💳
 3. Adjusting Project Structure
 4. Configuring a Build Backend
 5. Generating Distribution Packages
 6. Publishing the Package
 7. Installing the Package 💻
       7.1. User Testing
 8. Conclusion 🎇
 9. Attributions

Registering as a New User 🧑🏻

The very first step is to create a new PyPi account that we will use to publish our packages.

Once we have an account, we can login using those credentials

Creating an API Key 🔑

After creating and logging into your account, the next step is to create an API key that will be used to upload packages to PyPi under our account.

Enabling MFA 🔒

However, to be eligible to create an API key, you first need to go to Account Settings, and enable 2 factor authentication - which is again a 2-step process. You first generate and store recovery codes for your account, after which you can enable MFA.
Be advised that only one authentication application can be linked and in my case, I used Microsoft Authenticator - a pretty popular and standard option.

Getting the Token 💳

Getting an API token is pretty simple once you are eligible. All you have to do is go to the API Token section under Account Settings and click on Add Token.

which will take you to the following screen

Here, you need to name your token and set a scope for its validity, which can be a specific project or all projects like I have selected.

Now click on Create Token and you'll see your token in the following format.

Adjusting Project Structure

The most crucial step in the entire process is to reorganize the project into a structure expected by PyPi.

Let's discuss the role of each file in the figure.

1. LICENSE: This file contains information about the rights and permissions granted to users regarding the use, modification, distribution, and sharing of the software. I already had an MIT License in my project.
2. pyproject.toml: It is a configuration file typically used for specifying build requirements and backend build systems for Python projects. I was already using this file for Black code formatter configuration.
3. README.md: Used as a documentation file for your project, typically includes project overview, installation instructions and optionally, contribution instructions.
4. example_package_YOUR_USERNAME_HERE: One big change I had to face was restructuring my project, essentially packaging all files in this directory. The name of this directory should be what you want to name your package and shoud not conflict with any of the existing packages.
Of course, since its a Python Package, it needs to have an __init__.py.
5. tests/: This is where you put all your unit and integration tests, I think its optional as not all projects will have tests.
The rest of the project remains as is.

⚠️Important Note: Make sure you are using absolute imports throughout the project, as relative imports might not behave as you expect, when your package is installed on other machines.

I spent quite a while to figure it out.

and had to go through 3 patch fixes to finally get it to work.

Configuring a Build Backend

From official documentation,

Tools like pip and build do not actually convert your sources into a distribution package (like a wheel); that job is performed by a build backend. The build backend determines how your project will specify its configuration, including metadata (information about the project, for example, the name and tags that are displayed on PyPI) and input files.

I used Hatchling for my purpose, but there are other options available as well.

To configure Hatchling as your build backend, add the following lines to pyproject.toml file.

[build-system]
requires = ["hatchling"]
build-backend = "hatchling.build"

And the following metadata needs to be set

[project]
name = "til_page_builder"
version = "1.0.3"
authors = [
  { name="Amnish Singh Arora", email="[e-mail]" },
]
description = "A command-line tool for authoring \"Today I Learned\" posts in Markdown, which can be converted to HTML for publishing on the web."
readme = "README.md"
requires-python = ">=3.8"
classifiers = [
    "Programming Language :: Python :: 3",
    "License :: OSI Approved :: MIT License",
    "Operating System :: OS Independent",
]

I started out with version 1.0.0, but got to 1.0.3 while getting everything to work correctly.

Generating Distribution Packages

The next step is to generate distribution packages by building the project. These are the files that are uploaded to PyPi and installed as packages on systems.

First you need to make sure you have the latest package builder.

py -m pip install --upgrade build

Now make sure you're at the root of your project (at the same level as pyproject.toml), and run

py -m build

Once it is successfull, you'll see a tarball and a whl file generated in the dist directory.

These are the files that need to be uploaded/published to PyPi.

Publishing the Package

Congratulation for making to the last step of the process.
We'll use twine to upload the distribution packages.

py -m pip install --upgrade twine

Once installed, we can run

py -m twine upload --repository testpypi dist/*

And that's how I was able to publish my first python package to get it in the hands of users.

📌 Notice that we are using a --repository flag to upload the package to TestPyPi instead of the real thing due to the issue I described in the beginning. But we can omit this flag to upload to the real server if we have the credentials.

Installing the Package 💻

Now that we have the package published, its time to install and test if it works when installed.

py -m pip install --index-url https://test.pypi.org/simple/ --no-deps til-page-builder==<version_number_optional>

Let's try to use it now 🤞

py -m til_page_builder.til_builder_main -v

py -m til_page_builder.til_builder_main -h

That works. Let's try the main functionality now.

py -m til_page_builder.til_builder_main examples --output voila

And that works!

User Testing

I also updated the README file of my project and asked Katie to help me with testing it locally.

And thanks to her, I was able to find a critical defect. In my distribution package, I wasn't specifying the dependencies that had to be installed which is why she couldn't run my tool on her system.
She asked me to refer to her blog and I was able to fix this problem by adding dependencies to pyproject.toml and creating a setup.py file.

pyproject.toml

...
...
[build-system]
requires = ["hatchling"]
build-backend = "hatchling.build"

[project]
name = "til_page_builder"
version = "1.0.4"
...
...
dependencies = [
    "yattag==1.15.1",
    "tomli==2.0.1"
]
...
...

setup.py

from setuptools import setup, find_packages
from src.til_page_builder.version import __version__ as version

with open("requirements.txt") as f:
    requirements = f.read().splitlines()

setup(
    name="til_page_builder",
    version=version,
    packages=find_packages(),
    install_requires=requirements,
    entry_points={
        "console_scripts": [
            "til_page_builder=til_page_builder.til_builder_main:main",  # Adjust 'module_name' and 'main' accordingly
        ],
    },
    python_requires=">=3.8",
    classifiers=[
        "Programming Language :: Python :: 3",
        "License :: OSI Approved :: MIT License",
        "Operating System :: OS Independent",
    ],
    author="Amnish Singh Arora",
    author_email="[e-mail]",
    description="A command-line tool for authoring 'Today I Learned' posts in Markdown, which can be converted to HTML for publishing on the web.",
    long_description=open("README.md").read(),
    long_description_content_type="text/markdown",
    url="https://github.com/pypa/sampleproject",
    project_urls={
        "Homepage": "https://github.com/pypa/sampleproject",
        "Issues": "https://github.com/pypa/sampleproject/issues",
    },
)

Conclusion 🎇

In this post, I shared my experience uploading my first ever Python Package to PyPi, as I wrap up my open source class for this semester. I am really glad that I was able to carry my project to its 1.0 release, and will continue making improvements so it actually becomes a useful tool.

Here's one to the 1.0 release 🍾

Attributions

Cover Photo by Hitesh Choudhary on Unsplash

Comments

[Juan F Gonzalez]

👏👏👏