DEV Community: sirfuzzalot

How to Protect Your Python Code Health 🐍🩺

sirfuzzalot — Sun, 06 Jun 2021 22:25:53 +0000

Code can be long and complex. Let's configure a few good tools to help keep our Python codebase healthy. Flake8, Black, and isort.

Linting and Formatting

Flake8 is a linter. It's designed to find mistakes in your code that can later become bugs when your code is run.

A formatter arranges our code so that it's more readable on the screen, but does not change what our code does. Black and isort are formatters.

Installation

Let's install our linting and formatting tools. For a healthy coding environment use a virtual environment like venv or virtualenv (venv comes with Python).

Create a virtual environment and activate it.

# bash
python3 -m venv venv
source venv/bin/activate

# pwsh
py -3 -m venv venv
venv\Scripts\activate

Install the packages.

pip install flake8 black isort

Configuration

You'll want to configure these tools a little to make them play nicely together.

Flake8 📋

There's a lot you can configure with Flake8, but you don't have to. Add the bare minimum first to get Black and Flake8 to cooperate (also we'll exclude a few directories from linting). In the root of your project create a file called .flake8.

# .flake8

[flake8]
max-line-length = 88
extend-ignore = E203
exclude =
    .git,
    __pycache__,
    build,
    dist,
    venv

Black 🖊️

Black is famous as a highly opinionated "uncompromising code formatter". While there are a few settings you can configure, nearly all of the formatting choices are made for you. That's actually half of its appeal. The other half is getting back all the time you previously used to tweak where this comma was or that spacing. Embrace the way Black will do it all for you and you'll thank me later.

In the root of your project create a file called pyproject.toml.

# pyproject.toml

[tool.black]
exclude = 'venv'

isort 🗂️

Short for "import sort" or just "I Sort", this utility will arrange your module's imports in a consistent opinionated way, much like Black does for the rest of your code.

Let's configure isort to play nicely with Black. Add the following to our pyproject.toml.

# pyproject.toml

[tool.isort]
profile = "black"
multi_line_output = 3
skip = ["venv"]

Checking Your Code's Health 🩺

Let's run our tools individually first.

python -m flake8

python -m black .

python -m isort .

Now let's make this a little easier for ourselves by setting up a pre-commit hook. In your project create a file called .git/hooks/pre-commit.

#!/bin/sh
echo "----------------------"
echo "--- Pre-Commit Hook --"
echo "----------------------"
echo "📋 Flake8 📋"
python -m flake8 && echo "No Errors"
echo ""

echo "🖊️  Black 🖊️"
python -m black .
echo ""

echo "🗂️  isort 🗂️"
python -m isort . && echo "Imports Sorted"
echo ""
echo "----------------------"

When you run git commit this script will run and check your code health before a commit is made. It will show you what issues Flake8 has found and it will format your code. You'll then get to go back and fix the issues. Finally, you'll run git commit again.

Here's what it looks like with errors.

----------------------
--- Pre-Commit Hook --
----------------------
📋 Flake8 📋
.\src\packagecake\__main__.py:2:1: F401 'sys' imported but unused
.\src\packagecake\__main__.py:6:1: E302 expected 2 blank lines, found 1

🖊️  Black 🖊️
reformatted src\packagecake\__main__.py
All done! ✨ 🍰 ✨
1 file reformatted, 4 files left unchanged.

🗂️  isort 🗂️
Skipped 1 files
Imports Sorted

-----------------

Here's what it will look like when you're code is healthy.

----------------------
--- Pre-Commit Hook --
----------------------
📋 Flake8 📋
No Errors

🖊️  Black 🖊️
All done! ✨ 🍰 ✨
5 files left unchanged.

🗂️  isort 🗂️
Skipped 1 files
Imports Sorted

-----------------

Automation 🤖

With code health, the more often you can check it the better. You've seen how to setup a Git hook. Here's a few other ways to run Flake8, Black, and isort.

IDE

Your code editor likely integrates with these tools using only some light configuration. You'll benefit from spellcheck-like behavior from Flake8, and format on save from Black and isort.

Some info on setting up VSCode for linting and formatting.

Pre-Commit

The Pre-Commit Framework is Git hooks with super powers. Its easy setup and configuration give you access to an extensive list of shared Git hooks.

CI/CD Pipelines

While a Git hook is great for catching errors before they go to GitHub/GitLab you still want the safety of your CI/CD Pipeline running these tools again. That's because sometimes people forget to configure their Git hooks 🙃.

For our formatters, there is a special CI/CD mode that will throw an error when the formatting is incorrect.

python -m black . --check
python -m isort . --check

Source Code Example 💻

You're all set with the tools you'll need to keep your code healthy. Remember the goal is to make our code bug free, and easy to maintain. Flake8, Black, and isort can help you get there.

Check out packagecake for an example of what we've covered here.

sirfuzzalot / packagecake

Turn your Python package into a delicious cake 🎂

Find out what type of cake your Python package is by running the following:

>>> python -m pip install packagecake
>>> python -m packagecake bake [your package name]
🍰

Cover Photo Credit: Alexander Sinn

Take Control of Your VSCode Workspaces 🛠️

sirfuzzalot — Sat, 08 May 2021 23:02:51 +0000

I work on a lot of different projects. You probably do too.

Back in 2018, I created an extension for VSCode to help solve the problem of managing many different projects, using many different workspaces. It's called Workspace Explorer.

Workspace Explorer provides a file-browser-like experience for your workspaces. Some of the things you can do:

Workspaces
- Create Workspaces
- Delete Workspaces
- Rename Workspaces
- Open a Workspace in Another Window
- Open a Workspace in the Same Window
Sub-Folders
- Organize Workspaces in Sub-folders
- Delete Sub-folders
- Rename Sub-Folders
Custom Icons
- Set Custom Icons for Workspaces and Sub-folders

Workspace Explorer 2.0.0

Today I'm releasing version 2.0.0 of Workspace Explorer. This release brings support for jpg icons, environment variables in config paths, and patches several bugs. I've incremented the major version as support for SVG icons has been removed, due to an issue in VSCode.

Icon Format Support

~~svg~~
png
jpg (new)

Environment Variable Support

${env:NAME_OF_ENV_VAR}/remainder/of/your/path

/root/of/my/path/${env:NAME_OF_ENV_VAR}/remainder/of/your/path

${env:USERPROFILE}\my\windows\path

Workspace Explorer is developed open source on GitHub. Special thanks to all the contributors and issue reporters over the years.

sirfuzzalot / workspace-explorer

Organize, iconize and open your collection of VSCode workspaces

How to Bake A Python Package Cake🐍+📦=🎂

sirfuzzalot — Mon, 19 Apr 2021 00:12:34 +0000

Let's bake a package cake 🎂! You will need...

Ingredients

Item	Location	Description
library or application	`./src/[name of library or application]/[your code]`	Your code that does something useful or fun
pyproject.toml	`./pyproject.toml`	Build configuration information
setup.cfg	`./setup.cfg`	Package metadata for PyPI (name of your project, etc.)
README.md or README.rst	`./README.md` or `./README.rst`	Brief docs for your package to display on PyPI
.gitignore	`./.gitignore`	Things you don't want in Git, including some build artifacts.
LICENSE or LICENSE.txt	`./LICENSE` or `./LICENSE.txt`	Project's license telling others how they can use your work
setup.py	`./setup.py`	Technically optional, you'll want this so you can better develop locally

Package Cake Recipe 📋

Add one LICENSE or LICENSE.txt file. Picking a license can be tough, but there are tools to help like Choose A License.

MIT License

Copyright (c) 2021 Cake Packaging Foundation.

Permission is hereby granted, free of charge, to any person obtaining a copy
of this software and associated documentation files (the "Software"), to deal
in the Software without restriction, including without limitation the rights
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
copies of the Software, and to permit persons to whom the Software is
furnished to do so, subject to the following conditions:

The above copyright notice and this permission notice shall be included in all
copies or substantial portions of the Software.

THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
SOFTWARE.

Add one pyproject.toml file. Unless you're building native code or using an alternative build system (i.e. poetry), you can copy and paste this.

[build-system]
requires = [
    "setuptools>=42",
    "wheel"
]
build-backend = "setuptools.build_meta"

Add one setup.cfg. This is where you will specify the information that appears on PyPI and what is essential for your package to work.

# This is essentially a static version of setuptools.setup() and is the
# preferred method of indicating package metadata.
# https://packaging.python.org/guides/distributing-packages-using-setuptools/#setup-args

[metadata]
# A unique name for your package. Search through PyPI for duplicates.
name = packagecake

# Here you use SemVer for versioning - https://semver.org/
# You can also get fancy and pull this value from a file
# like so attr: packagecake.__version__
version = 1.0.0

# Your name or the organization's name. You can actually leave out the
# email and PyPI will still accept your package.
author = Example Author
author_email = author@example.com

# This is your sales pitch, your one-liner, your logline. Make it count
description = Turn your package into a delicious cake in less than 100ms

# Set this to the file you want displayed on PyPI.
# content-type can also be text/x-rst, or text/plain
long_description = file: README.md
long_description_content_type = text/markdown

# This will show in PyPI as your package's Homepage link
url = https://github.com/pypa/sampleproject

# These links will show up in PyPI under the Homepage link.
# Include at least Tracker.
project_urls =
    Tracker = https://github.com/pypa/sampleproject/issues
    Documentation = https://example.com/docs
    Source = https://github.com/pypa/sampleproject/
    Funding = https://example.com/funding
    Custom URL = https://example.com/

# Classifiers - https://pypi.org/classifiers/
# Make sure to include your license
classifiers =
    Development Status :: 3 - Alpha
    Intended Audience :: Developers
    License :: OSI Approved :: GNU General Public License v3 (GPLv3)
    Programming Language :: Python
    Programming Language :: Python :: 3
    Programming Language :: Python :: 3 :: Only
    Programming Language :: Python :: 3.8
    Programming Language :: Python :: 3.9
    Operating System :: OS Independent
    Topic :: Utilities

[options]
# Path to our libary/application
package_dir =
    = src
packages = find:

# Version of Python needed to use the package
python_requires = >=3.8

# Our package depends on these other external packages
install_requires =
    requests == 2.25.1

[options.packages.find]
where = src

Add one README.md or README.rst. Briefly document your tool, giving users a taste of what they can do with it. Then point to your full length docs with a link. You'll want to save full length docs for another site as PyPI does have some limitations particularly with links to section headers. Alternatively, you can make a file just for PyPI and leave your README nice and detailed for your GitHub repo.

# Package Cake 🎂

**Package Cake** is a simple utility that takes your package and turns it
into a cake 🍰.

Checkout the [documentation](https://example.com/docs).

-> Add installation example here <-

-> Usage example here <-

Add one .gitignore. You'll want to update your .gitignore file to exclude some of the build artifacts.

# Packaging
dist
build
*.egg-info/

Add one setup.py. While setup.cfg handles static values, setup.py is for dynamic metadata. Favor setup.cfg whenever possible. You will however, want to use setup.py to help you test your package locally. Read on to see how you can test.

import sys

from setuptools import setup

if sys.version_info < (3, 8, 0):
    sys.exit("Python 3.8.0+ Required")

setup()

Add one library or application. This is why you're publishing and you'll want to put this in a specific spot.

./src/[python package name]
    /__init__.py <- control your package's namespace
    /__main__.py <- optionally allow your package to be invoked from the command line
    /[your modules]

Bake Until Built 🔥

Next you're going to build the package into a wheel and a source archive.

A wheel is a built distribution. If you have any binaries there will be pre-compiled copies in the wheel. This makes it a much faster process for users to get up and running with your package.

The source archive contains the raw source code of the package and let's pip or the user do any compilation locally instead. Wheels are generally preferred in most use cases.

Create a virtual environment

Unix/MacOS

cd [your project]
python3 -m venv venv
source venv/bin/activate

Windows

cd [your project]
py -m venv venv
.\venv\Scripts\activate

Install and run build. This will generate a ./build and ./dist directory in your project's root, along with creating a .whl and .tar.gz distribution package. Finally, you will also see a .egg-info directory in your ./src/[my project] directory.

pip install build
python -m build

./build
./dist
    /packagecake-1.0.0-py3-none-any.whl
    /packagecake-1.0.0.tar.gz
./src/[python package name]
    /packagecake.egg-info

Publish Your Package Cake 🌐

Now that our package cake is built you'll want to upload it to PyPI. I highly recommend testing your package configuration by first publishing to Test PyPI.

In your virtual environment install twine.

pip install twine

Ensure the version number is properly incremented. You might keep the version number in setup.cfg, setup.py, or as __version__ in one of your other files. Make sure they all get updated.

   # setup.cfg
   [metadata]
   name = packagecake
   version = 1.0.1

Create a PyPI account.
Create an API token from PyPI. Optionally setup a credentials file to store your API token.

# [user home directory]/.pypirc
[testpypi]
username = __token__
password = <my API token>

[pypi]
username = __token__
password = <my API token>

Publish the package

twine upload dist/*

# For Test PyPI
twine upload --repository testpypi dist/*

Taste the Package Cake 🍽️

You'll want to download and test the package now. Let's also look at how you can test before publishing.

Download and Verify the publish.

pip install packagecake

# For Test PyPI. No deps is safer, though you can only verify package contents
pip install --index-url https://test.pypi.org/simple/ --no-deps packagecake

python
>>> import packagecake
🥮

# or if running from command-line

python -m packagecake
🍰

And now what you can do before you publish. This let's you import and run your package as though it was downloaded with pip. It's called an editable package. You can then perform the steps you did above to verify the package.

python -m pip install -e .

My First Python Package 🐣

I hope this helps you publish your package. A few quick resources I want to point out that helped me and of course a link to a package I built/baked 😊.

sirfuzzalot / pywhoami

A Simple HTTP Request Analysis Server

Pywhoami is inspired by the whoami Go server by Traefik Labs. Send a request to one of the endpoints to get back details from your HTTP request. With pywhoami you can help answer questions like, what headers were added to my original request by a proxy server.

Using the PyPI Package

Installation

bash

python3 -m pip install pywhoami

powershell

py -m pip install pywhoami

Running the Server

bash

>>> python3 -m pywhoami
[2021-04-17 15:00:25 -0700] [4400] [INFO] Running on http://127.0.0.1:8080 (CTRL + C to quit)

powershell

>>> py -m pywhoami
[2021-04-17 15:00:25 -0700] [4400] [INFO] Running on http://127.0.0.1:8080 (CTRL + C to quit)

Send it a test HTTP request.

…

View on GitHub

Cover Photo Credit: Domonique Davenport