Aspects of Writing Solid Python Applications

Heya fellows,

as developers we're always lacking time during the day, so we have to make smart decisions and use our time efficiently when turning ideas to code. In general it's a good strategy to build a minimal viable product at first to avoid wasting time in writing unnecessary features.

But nevertheless even a minimal viable product should match some standards to attract users and other developers joining the project. In this series I will shed some light on several aspects of writing solid Python applications. We will create a little HTTP client and thereby we will:

• Turn a script into a cli app with sys/argparse
• Compare shell script installation vs. setup.py file
• Write tests with pytest, tox and github actions
• Use sphinx to create documentation
• Publish our app at PyPI and Anaconda

1. Writing a Simple Script

First we will create a simple script which can perform GET requests. I keep all my scripts within a MyScripts folder. Then I create a project folder, and a virtual environment using pipenv.

$ cd ~/MyScripts
$ mkdir tinyHTTPie
$ cd tinyHTTPie

$ pipenv --python 3.7
$ pipenv shell

Then we have to create our project file and install dependencies:

$ touch tihttp.py
$ pip install requests

Let's write our script. We want to print the header of the response. I used the OrderedDict object imported from the collections library to sort the keys of the dictionary.

import requests
import collections

resp = requests.get('https://the-coding-lab.com/')

header = dict(collections.OrderedDict(resp.headers))
body = resp.text

for section in sorted(header.items()):
    print(f"{section[0]}: {section[1]}")

When we execute the script...

$ python tihttp.py

...the metadata of the request are returned successfully.

Accept-Ranges: bytes
Access-Control-Allow-Origin: *
Age: 124
Cache-Control: max-age=600
Connection: keep-alive
Content-Encoding: gzip
Content-Length: 5764
Content-Type: text/html; charset=utf-8
Date: Tue, 16 Feb 2021 14:39:59 GMT
...

But when you realize that a script of yours is pretty useful, you might think "How could I turn this script into an easy-to-use command?". At first, you have to add a shebang line to stick the scripts virtual environment to it permanently. The which python command or pipenv --py will return the path to the Python interpreter of the virtual environment. It will look like:

/home/niklas/.local/share/virtualenvs/tinyHTTPie-iqhOkNUA/bin/python ~/tinyHTTPie/tihttp.py

Now this path has to be added to the head of the script as a shebang line. We can do this from the command line.

$ pybin=$(pipenv --py)
$ echo -e "#\!${pybin}\n\n$(cat tihttp.py)" > tihttp.py

The head of the file should look like this:

#!/home/niklas/.local/share/virtualenvs/tiny-HTTPie-iqhOkNUA/bin/python

import requests
...

To make the script executable we have to give permissions. Then we can just type the path to the script and it will be interpreted by the specified Python interpreter.

$ chmod +x tiny_HTTPie_clone.py
$ ./tihttp.py

Giving the path an alias will make it easier for us to remember/use our tool. To allow us to use the alias in every new shell session, we add it to the .bashrc file (or wherever you're storing your aliases).

$ echo "\nalias tihttp='~/MyScripts/tiny_HTTPie_clone.py'" >> ~/.bashrc
$ source ~/.bashrc

Now you can call your script in each new shell session no matter where you are on the filesystem.

$ tihttp

But there is still a problem: every time our tool sends a GET request to a website we have to change the URL by opening the file, changing the URL, saving and executing it. Would it not be nicer to add the URL as argument to our tihttp command? Python's sys module from the standard library comes here to our rescue!