Introduction to Python Programming - Organizing Your Code With Modules

#programming #python #beginners #tutorial

Introduction
An Example Module
Importing From Modules
Using Modules as Scripts
Some Built-in Modules

Introduction

When writing a Python program, things can quickly get out of hand when using a single file.

While functions can help you group code within a Python file or script, sometimes they are not enough. Organizing related variables and functions in separate files enables code cleanliness, maintenance, and consistency. Each file is a module.

The file name is the module name with the suffix .py appended.

You can import variables and functions from one module into another or the main module (entry point of a Python program). Also, you can run the module as a script from a Python REPL. This way you have flexibility while keeping your code clean and organized.

Note: Within a module, a global variable __name__ (a string) exists and represents the module name.

An Example Module

Programmers typically write their tutorials in markdown. The code sections are often within three backticks. Let's say we want functions that extract and execute each markdown file code block.

You could extract all codes from that markdown file you downloaded from Github.

Or you could ensure that your code runs as expected in the markdown file you created. Like the one I used for this article.

Here's an example module.

Note, I've replaced line.startswith() arguments due to Devto's template errors. See this article's image above for the correct arguments.

# parse.py

import sys

def extract_code_blocks(markdown_file):
    """
      Extracts the code blocks from a markdown file
      Args:
        markdown (file) - The file from which we extract the code blocks
      Returns:
        code_blocks (list) - A list containing the code blocks
    """
    start_of_code_block = False
    code_blocks = []
    codes = ""
    with open(markdown_file, encoding='utf8') as file:
        for line in file:

            # the start of a code block. Move to the next line
            if line.startswith("see_post_image") or line.startswith("see_post_image"):
                start_of_code_block = True
                continue

            # the end of a code block
            if line.startswith("see_post_image"):
                code_blocks.append(codes)
                codes = ""
                start_of_code_block = False
                continue

            # We are now in a code line. Add to the code string
            if start_of_code_block:
                codes += line

            # There's nothing else to do
            continue

    return code_blocks

def write_to_file(filename):
    """
    Write the extracted codes to a file
    Args:
      filename: The filename.md to read from. filename.py will be written to
    """
    code_blocks = extract_code_blocks(filename)
    output, _ = filename.split('.')

    with open(f'{output}.py', 'w') as f:
        f.writelines(code_blocks)


def execute_codes(filename):
    """
    Execute the extracted codes of a markdown file
    Args:
      filename: The filename.md to read from
    """
    for code in extract_code_blocks(filename):
        try:
            # NOTE THAT EXECUTING UNTRUSTED CODES (FROM OTHERS)
            # USING EXEC OR ANY OTHER MEANS IS VERY DANGEROUS. 
            exec(code)
        except Exception as e:
            print(e)


# Run as a script
if __name__ == "__main__":
    args = sys.argv

    if len(args) == 3:

        _, command, filename = args

        if command not in {'execute', 'extract'}:
            print("Usage: python parse.py [execute|extract] file.md")
        elif command == 'execute':
            execute_codes(filename)
        elif command == 'extract':
            write_to_file(filename)
    else:
        print("Usage: python parse.py [execute|extract] file.md")

Importing (from) a Module

Using the import keyword, you can use the functions extract_code_blocks(), write_to_file(), and execute_codes() in another file or module in the following ways:

import the entire module import parse and then use parse.extract_code_blocks() to assess extract_code_blocks() or the other functions.
import all functions (and variables) in the module using asterisks from parse import *. Be careful about doing this to avoid surprises.
import one function from parse import extract_code_blocks.
import more than one function from parse import extract_code_blocks, write_to_file, execute_codes.

# test.py

#1 Import the module
import parse

# use the module functions (or variables if any)
print(parse.write_to_file)
print(parse.execute_codes)
print(parse.extract_code_blocks)

#2 Import everything from the module at once using *
from parse import *

# use them
print(write_to_file)
print(execute_codes)
print(extract_code_blocks)


#3 Import a single function
from parse import write_to_file

# use
print(write_to_file)


#3 Import many functions at one
from parse import write_to_file, extract_code_blocks, execute_codes 

# use them
print(write_to_file)
print(execute_codes)
print(extract_code_blocks)

Be careful when sharing codes between modules so as not to introduce circular dependency. A situation where modules A and B both depend on each other.

You can also import the modules from a Python REPL. To do so quickly, you need to open a Python terminal in the same directory as the module.

Using a Module as a Script

As stated earlier, a global variable __name__ (a string) is the module name. For example

# Another module or REPL
import parse

print(parse.__name__)

When running the module as a script, the value of __name__ is __main__. That means you can detect if your module is a script. For example, the section below enables you to run the parse.py as a Python script.


# Run as a script
if __name__ == "__main__":
    args = sys.argv

    if len(args) == 3:

        _, command, filename = args

        if command not in {'execute', 'extract'}:
            print("Usage: python parse.py [execute|extract] file.md")
        elif command == 'execute':
            execute_codes(filename)
        elif command == 'extract':
            write_to_file(filename)
    else:
        print("Usage: python parse.py [execute|extract] file.md")

The sys.argv contains the arguments passed to a Python script.

To run a module as a script, use python module name one_or_more_arguments_separated_by_spaces in the terminal. For example, run python parse.py execute markdownfile.md to execute all Python code blocks in markdownfile.md. Also, running python parse.py extract markdownfile.md will write the Python code blocks to a file named markdownfile.py.

Some Built-in Modules

The Python standard library consists of packages, modules, and submodules to assist you with common tasks. Here are a few worth mentioning.

Module	Description
`asyncio`	Contains functions for performing asynchronous operations
`csv`	Contains functions for reading and writing CSV files
`datetime`	Contains functions for working with dates and times
`functools`	Contains functions for functional programming
`inspect`	Inspect your python functions or objects
`itertools`	Contains functions for working with iterators
`json`	Contains functions for working with JSON objects
`math`	Contains mathematical functions
`os`	Contains functions for working with the operating system
`pdb`	Useful for debugging your Python codes
`pprint`	Useful for pretty printing
`re`	Contains functions for working with regular expressions
`statistics`	Contains some basic statistics functions
`sys`	Contains functions for working with the operating system
`tkinter`	Useful for creating graphical user interfaces
`timeit`	Time the execution of your functions
`unittest`	Contains functions for testing your Python codes
`urllib`	Contains functions for working with urls

You can find more of these modules here.

Example: Reading and Writing CSV Files

The code below will read a CSV file.

import csv

with open('your_csvfile.csv', 'r') as csv_file:
    reader = csv.reader(csv_file, delimiter=',')
    # If it has headings
    headings = next(reader)

    # The rest of the rows
    for row in reader:
        # do something with each row
        # row[0] contains the first column's cell value
        # row[1] contains the second column's cell value
        # row[2] contains the third column's cell value
        # ...
        # row[n] contains the (nth - 1) column's cell value
        print(row)

The function below will write to a CSV file. rows is a list of dictionaries while header is a list containing the dictionary keys.

import csv

def write_csv(filename: str, header: list[str], rows: list[dict[str, any]]):
    with open(filename, 'w', newline='') as csvfile:
        writer = csv.DictWriter(csvfile,
            fieldnames=header,
            quoting=csv.QUOTE_ALL)
        try:
            writer.writeheader()
            writer.writerows(rows)
        except BaseException as e:
            print(e)


# example
header = ['name', 'age']
data = [dict(name='M', age=11), dict(name='J', age=12)]
write_csv('test.csv', header, data)

Summary

You have seen how to organize your code in separate files called modules. You also saw how to import the content of a module into another module or REPL using the import keyword. The __name__ variable can help detect whether your Python module is running as a script. The Python standard library comes with many packages and modules.