DEV Community: Ian Johnson

I Installed Omakub Five Times So You Can Do It Just Once

Ian Johnson — Mon, 12 May 2025 13:58:51 +0000

TL;DR

Omakub is a tool to turn a fresh installation of Ubuntu 24.04 into a friendly system for developers new to Linux. Install it with:

wget -qO- https://omakub.org/install | bash

and reap the benefits!

Optional directions are included for setting up VNC to connect to a remote desktop in the cloud.

Linux and Me

I have used Linux for over twenty years. I've run many distributions, including Ubuntu, Debian, Fedora, CentOS, and Arch. There are many reasons why I enjoy Linux such as package management, openness to modification, and philosophy. But this is not about me; it's about you!

Linux and You

Linux is a great system. Ubuntu is a fantastic distribution to start with because of its ease to install and setup. It's also intuitive and easy to use, especially when coming from Windows or MacOS. It is a free (as in speech) and open-source system that keeps you out of the grips of Microsoft or Apple. In particular, it's great for developers and especially for web development -- since your web app is likely also running on Linux.

If you are a Linux aficionado with a highly-prized Arch setup, this tutorial is not for you. This article is intended for developers new to Linux who would like to try it out using a bootstrapped setup to get up and running quickly and decide if Linux is for them.

Creating a New Server in Hetzner

I don't have a spare computer to install Ubuntu and Omakub on, so I chose to do this in the cloud with Hetzner. To allow for the desktop to be usable, I also setup a VNC server using TigerVNC. You can follow this setup with any cloud provider. Just make sure you are using an image that is Ubuntu 24.04 or higher using x86_64 architecture. Also, make sure you have port 22 open for SSH and port 5901 open for VNC.

However, you will likely want to install this on your local computer. If you are installing this on your local machine, install Ubuntu 24.04 and then skip to the Installing Omakub section.

Since this was an experiment, I chose a server with a shared CPU, using x86_64. I chose the Ubuntu 24.04 image and the following server specs:

4 VCPU
8 GB RAM
160 GB SSD

Additionally, I created an SSH key and added firewall rules to open ports for IPv4 and IPv6 for TCP for ports 22 and 5901.

If you are planning to create this in the cloud, you may want to consider using a dedicated VCPU with enough RAM and disk space to support your needs. You may also choose to forward the port for VNC, rather than opening the port in your cloud provider because it is more secure.

Note: If you would like to forward the port, you can do so like this:

ssh -L 5901:localhost:5901 <username>@<machine-ip>

Again, you will likely want to install this on your local machine. If that's what you are doing, skip to the Installing Omakub section.

Initial System Setup

Connect to the remote server.

ssh -i ~/.ssh/<private-key-file> root@<machine-ip>

Update packages on the server.

apt update && apt upgrade -y

Install Gnome and friends on the server.

apt install ubuntu-gnome-desktop gnome-session gnome-terminal gnome-tweaks gnome-shell-extensions zsh git curl dbus-x11 -y

Why do we have to install all these extra things? The reason for this is that Gnome uses Wayland by default. Unfortunately, TigerVNC only supports X11 so we have to install these packages and add some extra configuration to ensure that Gnome uses X11 instead.

Create your new user and add sudo privileges.

adduser <username>
usermod -aG sudo <username>

Installing and Configuring TigerVNC

Install TigerVNC on the server.

apt install tigervnc-standalone-server tigervnc-xorg-extension tigervnc-viewer -y

Switch to your new user.

su <username>

Set the VNC password.

vncpasswd

Open the VNC port.

sudo ufw allow 5901/tcp

Create the VNC server startup file.

cd ~
vim ~/.vnc/xstartup

Insert the following text into the ~/.vnc/xstartup file. If you are new to Vim, press i, then paste in the following text:

#!/bin/sh

unset SESSION_MANAGER
unset DBUS_SESSION_BUS_ADDRESS

export XDG_SESSION_TYPE=x11
export GDK_BACKEND=x11

[ -r $HOME/.Xresources ] && xrdb $HOME/.Xresources

export XDG_RUNTIME_DIR="/run/user/$(id -u)"
dbus-launch --exit-with-session gnome-session

Then press <esc> and type :wq<enter>.

Create a script to start the VNC server.

vim ~/start-vnc.sh

Insert the following text into the ~/start-vnc.sh file. Again, press i, then paste in the following text:

#!/bin/bash
sudo mkdir -p /run/user/$UID
sudo chown $UID:$UID /run/user/$UID
vncserver -localhost no :1

Then press <esc> and type :wq<enter>.

Make both scripts executable.

chmod +x ~/.vnc/xstartup
chmod +x ~/start-vnc.sh

To start the VNC server:

cd ~
./start-vnc.sh

Connecting to the VNC Server

There are several options for connecting to the VNC server. The one I chose was Jump Desktop since I'm on a Mac. Jump Desktop does cost money. Alternative VNC clients that are free (as in beer) you can use include:

There are also additional clients for specific operating systems, but these two are available for Windows, MacOS, and Linux.

Connect to your VNC server using this as a target:

vnc://<machine-ip>:5901

If you forwarded the port, use this instead:

vnc://localhost:5901

You will be prompted for the VNC password. Enter it and you should be logged into your user.

Installing Omakub

Installing Omakub is easy! Just run this single command:

wget -qO- https://omakub.org/install | bash

If you are very new to Linux, you'll want to run this command in the Terminal application.

The command will prompt you to make some choices for your preferred tools and setup.

And you're done! To see your updated system, you'll have to log out and back in again.

Using the System

The best way to understand to how get started with using Omakub is to watch the video from DHH on the Omakub website. If you prefer a text version, check out The Omakub Manual.

Conclusion

Omakub is an easy way to bootstrap a fresh installation of Ubuntu 24.04. If you want to try out Linux but don't want to waste time setting up tools and configuration to use it quickly and evaluate if it's right for you, I highly suggest trying Omakub.

Giving Our Tests Context

Ian Johnson — Wed, 17 Jan 2024 11:30:03 +0000

At the end of the last post, we added tests that cause errors and failures, so let's recap what that looks like.

@pytest.fixture
def fake_file():
    mock = mock_open(read_data='Buy Milk')
    return patch('{}.open'.format(__name__), mock, create=True)


@pytest.fixture
def task_list(fake_file):
    with fake_file:
        with open('todo.txt', 'a') as todo:
            with open('done.txt', 'a') as done:
                return TaskList(todo, done)


def test_adding_a_task_displays_output(task_list, capsys):
    task_list.add("Buy Bread")
    captured = capsys.readouterr()
    assert "Buy Bread" in captured.out


def test_adding_a_task_increases_list_size(task_list):
    before = len(task_list.all())
    task_list.add("Buy Bread")
    after = len(task_list.all())
    assert after == before + 1


def test_adding_a_task_appends_that_task(task_list):
    task_list.add("Buy Bread")
    task = task_list.all()
    assert "Buy Bread" in task.description


def test_listing_tasks_displays_output(task_list, capsys):
    task_list.ls()
    captured = capsys.readouterr()
    assert "Buy Milk" in captured.out

poetry run pytest
# FAILED tests/test_task_list.py::test_adding_a_task_increases_list_size - assert 0 == (1 + 1)
# FAILED tests/test_task_list.py::test_adding_a_task_appends_that_task - AttributeError: 'list' object has no attribute 'description'

Here we have two problems. First, our task list size is not increasing. Second, we are referencing an attribute on the Task that does not exist. We'll add this functionality next time. First, we'll add more context to our tests.

Adding Context

Notice the length of our test names. Notice how they are attempting to describe the test case. Now, imagine all of the different things that a TaskList could do. How does that scale for our tests? Currently, we would have to add a test for each piece of functionality. Then, hopefully, we would DRY up duplication. But we would end up with many disparate tests that use a variety of fixtures to describe the system under test. It would be better if we could group our tests. Let's add another dependency to get this done.

poetry add pytest-describe

With pytest-describe, we can write contextualized tests similar to RSpec. We can add a describe function surrounding our first test to see how this works.

def describe_task_list():
    def test_adding_a_task_displays_output(task_list, capsys):
        task_list.add("Buy Bread")
        captured = capsys.readouterr()
        assert "Buy Bread" in captured.out

Running tests:

poetry run pytest -v
# tests/test_task.py::test_marking_task_done SKIPPED (unconditional skip)
# tests/test_task_list.py::describe_task_list::test_adding_a_task_displays_output PASSED
# tests/test_task_list.py::test_adding_a_task_increases_list_size FAILED
# tests/test_task_list.py::test_adding_a_task_appends_that_task FAILED
# tests/test_task_list.py::test_listing_tasks_displays_output PASSED

Notice the output of our new test?

tests/test_task_list.py::describe_task_list::test_adding_a_task_displays_output PASSED

We have added context to our test. Now we know that our test is describing a task list. Great! Let's indent the rest of the tests.

poetry run pytest -v
# tests/test_task.py::test_marking_task_done SKIPPED (unconditional skip)
# tests/test_task_list.py::describe_task_list::test_adding_a_task_displays_output PASSED
# tests/test_task_list.py::describe_task_list::test_adding_a_task_increases_list_size FAILED
# tests/test_task_list.py::describe_task_list::test_adding_a_task_appends_that_task FAILED
# tests/test_task_list.py::describe_task_list::test_listing_tasks_displays_output PASSED

Awesome! Even though this gives us more context, it still has limited value. Let's revisit our first test to give it more value.

tests/test_task_list.py::describe_task_list::test_adding_a_task_displays_output PASSED

First, having the prepended test makes the sentence a little harder to read. Plus, we already know it's a test. Let's rename the test to start with it, in RSpec style.

def describe_task_list():
    def it_displays_output_when_adding_a_task(task_list, capsys):
        task_list.add("Buy Bread")
        captured = capsys.readouterr()
        assert "Buy Bread" in captured.out

poetry run pytest -v
# tests/test_task_list.py::describe_task_list::it_displays_output_when_adding_a_task PASSED

We are just focusing on this test for now. Then we'll apply the change to the rest of our tests. This sentence reads a little better, but it's still not quite English-like. Let's swap the predicate to not be a passive voice:

def describe_task_list():
    def when_adding_a_task_it_displays_output(task_list, capsys):
        task_list.add("Buy Bread")
        captured = capsys.readouterr()
        assert "Buy Bread" in captured.out

poetry run pytest -v
# tests/test_task_list.py::describe_task_list::when_adding_a_task_it_displays_output PASSED

Adding Custom Prefixes

This is better, but it's not quite there. Notice the tests starts with when. What does this tell us? That we have additional context. Let's add one more layer of context.

def describe_task_list():
    def when_adding_a_task():
        def it_displays_output(task_list, capsys):
            task_list.add("Buy Bread")
            captured = capsys.readouterr()
            assert "Buy Bread" in captured.out

poetry run pytest -v

Whoa! Our test is gone! Why? Because pytest-describe is only looking for functions with a describe prefix. One option is to add a describe before our when for this new function name. But, that's going the opposite direction that we are headed (because it won't be English-like). To solve this, we'll add a configuration option for pytest-describe to our pyproject.toml:

[tool.pytest.ini_options]
describe_prefixes = ["describe_", "when_"]

Here we are just adding the prefix when for pytest-describe to look for. I chose when because it sounds natural and using Given, When, Then statements is really good model for contextualizing behavior. Currently, we only need a when, so we're just adding that for now.

Run tests again:

poetry run pytest -v
# tests/test_task_list.py::describe_task_list::when_adding_a_task::it_displays_output PASSED

And the test is back! And it looks great! Now we know that this test is describing a task list. And that this test is asserting that when a task is added to the task list, it displays output. Let's wrap the remaining tests:

def describe_task_list():
    def when_adding_a_task():
        def it_displays_output(task_list, capsys):
            task_list.add("Buy Bread")
            captured = capsys.readouterr()
            assert "Buy Bread" in captured.out

        def it_increases_list_size(task_list):
            before = len(task_list.all())
            task_list.add("Buy Bread")
            after = len(task_list.all())
            assert after == before + 1

        def it_appends_that_task(task_list):
            task_list.add("Buy Bread")
            task = task_list.all()
            assert "Buy Bread" in task.description

Notice our function scoping here. All of our add tests can just be indented under the new when_adding_a_task. One area that could be improved here is adding a describe block to identify that we are testing the add method. It would look like describe_add. This has limited value. It doesn't make our test more English-like, but it does give us additional context. As a developer, we would know that the system under test is Task.add. If we do this, we don't necessarily need the configuration for the when prefix anymore. But, we'll keep it because it will allow us to further scope. Let's rename this function and continue.

def describe_task_list():
    def describe_add():
        def it_displays_output(task_list, capsys):
            task_list.add("Buy Bread")
            captured = capsys.readouterr()
            assert "Buy Bread" in captured.out

        def it_increases_list_size(task_list):
            before = len(task_list.all())
            task_list.add("Buy Bread")
            after = len(task_list.all())
            assert after == before + 1

        def it_appends_that_task(task_list):
            task_list.add("Buy Bread")
            task = task_list.all()
            assert "Buy Bread" in task.description

Now we can scope our last test: which is testing Task.ls:

def describe_task_list():
    def describe_add():
        def it_displays_output(task_list, capsys):
            task_list.add("Buy Bread")
            captured = capsys.readouterr()
            assert "Buy Bread" in captured.out

        def it_increases_list_size(task_list):
            before = len(task_list.all())
            task_list.add("Buy Bread")
            after = len(task_list.all())
            assert after == before + 1

        def it_appends_that_task(task_list):
            task_list.add("Buy Bread")
            task = task_list.all()
            assert "Buy Bread" in task.description


    def describe_list():
        def it_displays_output(task_list, capsys):
            task_list.ls()
            captured = capsys.readouterr()
            assert "Buy Milk" in captured.out

It would be nice to DRY up that task_list argument, but if we do that it will break the call chain of pytest-describe. Here's what that error would look like for reference:
ERROR tests/test_task_list.py::describe_task_list::describe_add - TypeError: describe_task_list.<locals>.describe_add() missing 1 required positional argument: 'task_list'

So why are we keeping when? Because it will be useful as we move forward. Let's point out a use-case for it. Take a look at describe_list. Is there a case that is not covered here? What if the task list has nothing in it? Let's spec that out:

def describe_list():
    def when_there_are_tasks():
        def it_displays_no_tasks(task_list, capsys):
            task_list.ls()
            captured = capsys.readouterr()
            assert "No tasks" in captured.out


    def when_there_are_no_tasks():
        def it_displays_output_in_a_list(task_list, capsys):
            task_list.ls()
            captured = capsys.readouterr()
            assert "Buy Milk" in captured.out

poetry run pytest -v
# tests/test_task.py::test_marking_task_done SKIPPED (unconditional skip)
# tests/test_task_list.py::describe_task_list::describe_add::it_displays_output PASSED
# tests/test_task_list.py::describe_task_list::describe_add::it_increases_list_size FAILED
# tests/test_task_list.py::describe_task_list::describe_add::it_appends_that_task FAILED
# tests/test_task_list.py::describe_task_list::describe_list::when_there_are_tasks::it_displays_no_tasks FAILED
# tests/test_task_list.py::describe_task_list::describe_list::when_there_are_no_tasks::it_displays_output_in_a_list PASSED

Amazing! Now we have test cases that read like English. What's great about this is that even a non-developer can look at these test descriptions and understand what they are doing. That is, they are executable product specifications (or documentation).

Next up, we'll make these tests pass by updating our code.

Key Takeaways

Adding context to tests helps with organization and understanding
pytest-describe gives us an RSpec-like API that allows us to better contextualize our tests
Adding custom prefixes allows us more control over scope and naming
Tests are executable documentation

Refactoring Our Tests

Ian Johnson — Wed, 17 Jan 2024 11:30:03 +0000

Now that we have green tests, we can keep promoting methods from Task to TaskList. But before we do, let's do a little investigation and consider our tests.

Task Deletion

Following is the current implementation of deleting a task. Again, deleting a task is something that this list should do.

def deL(no):
    try:
        no = int(no)
        Task.nec()
        with open("todo.txt", "r+") as f:
            lines = f.readlines()
            f.seek(0)
            for i in lines:
                if i.strip('\n') != d[no]:
                    f.write(i)
            f.truncate()
        print(f"Deleted todo #{no}")

    except Exception as e:
        print(f"Error: todo #{no} does not exist. Nothing deleted.")

Notice the call to Task.nec. This is the second time we've run into this. What is it?

def nec():
    try:
        f = open('todo.txt', 'r')
        c = 1
        for line in f:
            line = line.strip('\n')
            d.update({c: line})
            c = c+1
    except:
        sys.stdout.buffer.write("There are no pending todos!".encode('utf8'))

Honestly, I'm not sure what nec is referencing. This is a naming issue. So what is this method doing? Well, it:

Opens the todo file
Updates the (global) dictionary with each line from the file, with the new line stripped off
Writes a message to stdout if anything goes wrong

So what is it actually doing? It seems like it's just state management. We don't need this anymore if we aren't going to use a global. Also, this is very close to using exceptions for flow control. In general, this is not a good idea. However, sometimes it does makes sense. For example, if the file does not exist. Okay, let's hop back to deleting tasks.

First, let's give this a better name. deL is just a shortened version of delete. Make sure not to use del, as that is a Python keyword (and likely the reason this method is named slightly different).

Also, take note that deL takes a no (number) argument. todo.txt refers to tasks by number, so we need to pass that in. What is the number? It's the line number! Recall the assertion from our last implmented test:

assert "[1] Buy Milk" in captured.out

Remember how I said this test wasn't that good? One of the issues here is that it is expecting the wrong string. It's assuming that Buy Milk is the first task in the file. We can even see this evidenced in the mock, as we had to do the following:

mock = mock_open(read_data='[1] Buy Milk')

In our real todo.txt file, there is no number. Instead the content is just Buy Milk. So, we really should update our test data here. This will also force us to implement the task numbering so we can refer to it in other places.

Considering Our Tests

Let's ask ourselves the bigger question: are we testing the right thing? Perhaps not. We are invoking our system under test and asserting that it works based on output to stdout. It is not testing anything about the logical consequences of the test. In a case like list, checking for stdout seems fine -- and testing the correct message is sent to stdout is a valid test.

What about add? Here there is something we can test. We can test that if we add a task to the task list, then that item is now in the task list. We could do this by either:

Asserting the last task in the task list is the task that was added
Asserting that the length of the task list increases by one

Refactoring Our Tests

Yes, because tests are code, they too can be refactored. And DRY matters with tests too! Let's clean some of this up so we can easily add our remaining methods and complete separating our domain objects.

Here is the current state of our tests for TaskList. There is quite a bit of duplication that is obvious, so let's start there.

@pytest.fixture
def todo_file():
    mock = mock_open(read_data='[1] Buy Milk')
    return patch('{}.open'.format(__name__), mock, create=True)

def test_adding_task_to_list(todo_file, capsys):
    with todo_file:
        with open('todo.txt', 'a') as todo:
            with open('done.txt', 'a') as done:
                task_list = TaskList(todo, done)
                task_list.add("Buy Bread")
    captured = capsys.readouterr()
    assert "Buy Bread" in captured.out


def test_listing_tasks(todo_file, capsys):
    with todo_file:
        with open('todo.txt', 'a') as todo:
            with open('done.txt', 'a') as done:
                task_list = TaskList(todo, done)
                task_list.ls()
    captured = capsys.readouterr()
    assert "[1] Buy Milk" in captured.out

First, let's take note that the todo_file fixture is also being used for done.txt, so let's rename that.

@pytest.fixture
def fake_file():
    mock = mock_open(read_data='[1] Buy Milk')
    return patch('{}.open'.format(__name__), mock, create=True)


def test_adding_task_to_list(fake_file, capsys):
    with fake_file:
        with open('todo.txt', 'a') as todo:
            with open('done.txt', 'a') as done:
                task_list = TaskList(todo, done)
                task_list.add("Buy Bread")
    captured = capsys.readouterr()
    assert "Buy Bread" in captured.out


def test_listing_tasks(fake_file, capsys):
    with fake_file:
        with open('todo.txt', 'a') as todo:
            with open('done.txt', 'a') as done:
                task_list = TaskList(todo, done)
                task_list.ls()
    captured = capsys.readouterr()
    assert "[1] Buy Milk" in captured.out

Running tests:

poetry run pytest
# tests/test_task_list.py ..

Okay, now let's dry up the TaskList into its own fixture.

@pytest.fixture
def fake_file():
    mock = mock_open(read_data='[1] Buy Milk')
    return patch('{}.open'.format(__name__), mock, create=True)

@pytest.fixture
def task_list(fake_file):
    with fake_file:
        with open('todo.txt', 'a') as todo:
            with open('done.txt', 'a') as done:
                return TaskList(todo, done)

def test_adding_task_to_list(task_list, capsys):
    task_list.add("Buy Bread")
    captured = capsys.readouterr()
    assert "Buy Bread" in captured.out


def test_listing_tasks(task_list, capsys):
    task_list.ls()
    captured = capsys.readouterr()
    assert "[1] Buy Milk" in captured.out

Running tests again:

poetry run pytest
# tests/test_task_list.py ..

Let's also remove the task numbering. We'll address that soon.

@pytest.fixture
def fake_file():
    mock = mock_open(read_data='Buy Milk')
    return patch('{}.open'.format(__name__), mock, create=True)


@pytest.fixture
def task_list(fake_file):
    with fake_file:
        with open('todo.txt', 'a') as todo:
            with open('done.txt', 'a') as done:
                return TaskList(todo, done)


def test_adding_task_to_list(task_list, capsys):
    task_list.add("Buy Bread")
    captured = capsys.readouterr()
    assert "Buy Bread" in captured.out


def test_listing_tasks(task_list, capsys):
    task_list.ls()
    captured = capsys.readouterr()
    assert "Buy Milk" in captured.out

Running tests again:

poetry run pytest
# tests/test_task.py s
# tests/test_task_list.py ..

Great! Also note that I've pointed out above that we are skipping the Task test for now. I'm intentionally doing this to focus on the TaskList. To do this, I just added a pytest mark to that test. Here's what that looks like:

@pytest.mark.skip
def test_marking_task_done(task, capsys):
    task.done(1)
    captured = capsys.readouterr()
    assert "Buy Bread" in captured.out

Since we have marked the test to be skipped, pytest will happily skip it and report it in our test as s. So now let's get a little more information from out tests. Run them with a -v flag.

poetry run pytest -v
# tests/test_task.py::test_marking_task_done SKIPPED (unconditional skip)
# tests/test_task_list.py::test_adding_task_to_list PASSED
# tests/test_task_list.py::test_listing_tasks PASSED

Note that we get an output of our test method names. This is great, but it would be even better if they read like sentences.

Naming matters! It's one of the hard problems in programming; don't take it lightly -- even with tests.

Let's rename those tests to be more descriptive.

@pytest.fixture
def fake_file():
    mock = mock_open(read_data='Buy Milk')
    return patch('{}.open'.format(__name__), mock, create=True)


@pytest.fixture
def task_list(fake_file):
    with fake_file:
        with open('todo.txt', 'a') as todo:
            with open('done.txt', 'a') as done:
                return TaskList(todo, done)


def test_adding_a_task_displays_output(task_list, capsys):
    task_list.add("Buy Bread")
    captured = capsys.readouterr()
    assert "Buy Bread" in captured.out


def test_listing_tasks_displays_output(task_list, capsys):
    task_list.ls()
    captured = capsys.readouterr()
    assert "Buy Milk" in captured.out

Run tests with -v again:

poetry run pytest -v
# tests/test_task.py::test_marking_task_done SKIPPED (unconditional skip)]
# tests/test_task_list.py::test_adding_a_task_displays_output PASSED
# tests/test_task_list.py::test_listing_tasks_displays_output PASSED

Now our tests are much more descriptive. What's the benefit? Now I can just read my tests and I know what they are testing. These descriptions are better, but they are still not quite english-like. We are going to extend that soon.

This also allows us to easily add our other cases.

@pytest.fixture
def fake_file():
    mock = mock_open(read_data='Buy Milk')
    return patch('{}.open'.format(__name__), mock, create=True)


@pytest.fixture
def task_list(fake_file):
    with fake_file:
        with open('todo.txt', 'a') as todo:
            with open('done.txt', 'a') as done:
                return TaskList(todo, done)


def test_adding_a_task_displays_output(task_list, capsys):
    task_list.add("Buy Bread")
    captured = capsys.readouterr()
    assert "Buy Bread" in captured.out


def test_adding_a_task_increases_list_size(task_list):
    before = len(task_list.all())
    task_list.add("Buy Bread")
    after = len(task_list.all())
    assert after == before + 1


def test_adding_a_task_appends_that_task(task_list):
    task_list.add("Buy Bread")
    task = task_list.all()
    assert "Buy Bread" in task.description


def test_listing_tasks_displays_output(task_list, capsys):
    task_list.ls()
    captured = capsys.readouterr()
    assert "Buy Milk" in captured.out

Adding these tests does cause errors and failures, but this is the direction that we are heading. Before we do, notice that our tests are now named well, organized well, and are more focused on the system under test.

Key Takeaways

It's helpful to consider what should be tested when ensuring test coverage
- Ask yourself: Am I testing the right thing?
Naming matters
Tests are code too -- don't be afraid to change them

Separating Domain Objects

Ian Johnson — Tue, 16 Jan 2024 11:30:03 +0000

Before we continue, let's do a quick recap of the current state of our code.

`todo.py`

import sys
from todo.task import Task


if __name__ == '__main__':
    args = sys.argv
    if(args[1] == 'add'):
        if(len(args[2:]) == 0):
            sys.stdout.buffer.write(
                "Error: Missing todo string. Nothing added!".encode('utf8'))
        else:
            with open('todo.txt', 'a') as f:
                    task = Task(f)
                    task.add(''.join(args[2:]))

    elif(args[1] == 'done' and len(args[2:]) == 0):
        if(len(args[2:]) == 0):
            sys.stdout.buffer.write(
                "Error: Missing NUMBER for marking todo as done.".encode('utf8'))
        else:
            Task.done(args[2:])

    elif(args[1] == 'del' and len(args[2:]) == 0):
        if(len(args[2:]) == 0):
            sys.stdout.buffer.write(
                "Error: Missing NUMBER for deleting todo.".encode('utf8'))
        else:
            Task.deL(args[2])

    elif(args[1] == 'ls'):
        Task.ls()

`todo/task.py`

import sys
import datetime

d = {}
don = {}

class Task:
    def __init__(self, todo_file, done_file):
        self.todo_file = todo_file
        self.done_file = done_file

    def add(self, s):
        self.todo_file.write(s)
        self.todo_file.write("\n")
        s = '"'+s+'"'
        print(f"Added todo: {s}")

    def help():
        sa = """Usage :-
    $ ./todo add "todo item" # Add a new todo
    $ ./todo ls          # Show remaining todos
    $ ./todo del NUMBER  # Delete a todo
    $ ./todo done NUMBER     # Complete a todo
    $ ./todo help            # Show usage
    $ ./todo report      # Statistics"""
        sys.stdout.buffer.write(sa.encode('utf8'))

    def ls(self):
        try:

            Task.nec()
            l = len(d)
            k = l

            for i in d:
                sys.stdout.buffer.write(f"[{l}] {d[l]}".encode('utf8'))
                sys.stdout.buffer.write("\n".encode('utf8'))
                l = l-1

        except Exception as e:
            raise e

    def deL(no):
        try:
            no = int(no)
            Task.nec()
            with open("todo.txt", "r+") as f:
                lines = f.readlines()
                f.seek(0)
                for i in lines:
                    if i.strip('\n') != d[no]:
                        f.write(i)
                f.truncate()
            print(f"Deleted todo #{no}")

        except Exception as e:
            print(f"Error: todo #{no} does not exist. Nothing deleted.")

    def done(self, no):
        try:

            nec()
            no = int(no)
            st = 'x '+str(datetime.datetime.today()).split()[0]+' '+d[no]
            self.done_file.write(st)
            self.done_file.write("\n")
            print(f"Marked todo #{no} as done.")

            with open("todo.txt", "r+") as f:
                lines = f.readlines()
                f.seek(0)
                for i in lines:
                    if i.strip('\n') != d[no]:
                        f.write(i)
                f.truncate()
        except:
            print(f"Error: todo #{no} does not exist.")

    def report():
        nec()
        try:

            nf = open('done.txt', 'r')
            c = 1
            for line in nf:
                line = line.strip('\n')
                don.update({c: line})
                c = c+1
            print(
                f'{str(datetime.datetime.today()).split()[0]} Pending : {len(d)} Completed : {len(don)}')
        except:
            print(
                f'{str(datetime.datetime.today()).split()[0]} Pending : {len(d)} Completed : {len(don)}')

    def nec():
        try:
            f = open('todo.txt', 'r')
            c = 1
            for line in f:
                line = line.strip('\n')
                d.update({c: line})
                c = c+1
        except:
            sys.stdout.buffer.write("There are no pending todos!".encode('utf8'))

`todo/task_list.py`

class TaskList:
    def __init__(self, todo_file, done_file):
        pass

    def add(self, task_description):
        print('Buy Bread')

    def ls(self):
        print('[1] Buy Milk')

`tests/test_task.py`

import pytest
from unittest.mock import mock_open, patch
from todo.task import Task

# Skipping this test until we finish re-organizing
@pytest.mark.skip
def test_marking_task_done(task, capsys):
    task.done(1)
    captured = capsys.readouterr()
    assert "Buy Bread" in captured.out

`tests/test_task_list.py`

import pytest
from unittest.mock import mock_open, patch
from todo.task_list import TaskList

@pytest.fixture
def todo_file():
    mock = mock_open()
    return patch('{}.open'.format(__name__), mock, create=True)

def test_adding_task_to_list(todo_file, capsys):
    with todo_file:
        with open('todo.txt', 'a') as todo:
            with open('done.txt', 'a') as done:
                task_list = TaskList(todo, done)
                task_list.add("Buy Bread")
    captured = capsys.readouterr()
    assert "Buy Bread" in captured.out

def test_listing_tasks(todo_file, capsys):
    with todo_file:
        with open('todo.txt', 'a') as todo:
            with open('done.txt', 'a') as done:
                task_list = TaskList(todo, done)
                task_list.ls()
    captured = capsys.readouterr()
    assert "[1] Buy Milk" in captured.out

And if we run tests:

poetry run pytest
# tests/test_task.py s
# tests/test_task_list.py ..

Separating Domain Objects

So let's continue to separate our domain objects. What do I mean by this? When I refer to your domain, I mean the domain of the product that you are building. We are building a todo app, so that's our domain here. Our domain consists of:

A task list
Tasks

And any other concepts that may be helpful to us, such as:

Projects
Tags
Priority

What is not in our domain is also vitally important. Our domain does not include:

Files
Databases
Network

Or any other third-party dependencies.

Separating domain objects is simply splitting them up. In our case, Task does way too much. It is also doing the work of a TaskList. So, in accordance with the single responsibility principle, we should separate this.

Currently, we are separating the Task from the TaskList. Let's start with add. Right now, our test for adding to a task list is passing. Although, it is not a good test. We also have some stubbed code in our implementation just to make the test pass. Let's take the next step and promote the add behavior from the Task to the TaskList.

First, just move the implementation:

class TaskList:
    def __init__(self, todo_file, done_file):
        pass

    def add(self, task_description):
        self.todo_file.write(s)
        self.todo_file.write("\n")
        s = '"' + s + '"'
        print(f"Added todo: {s}")

And if we run tests:

poetry run pytest
# E       AttributeError: 'TaskList' object has no attribute 'todo_file'

Here, we get an error. We are referring to a todo_file that we never assigned. Update the constructor:

class TaskList:
    def __init__(self, todo_file, done_file):
        self.todo_file = todo_file

    def add(self, task_description):
        self.todo_file.write(s)
        self.todo_file.write("\n")
        s = '"' + s + '"'
        print(f"Added todo: {s}")

Running tests again:

poetry run pytest
# E       UnboundLocalError: cannot access local variable 's' where it is not associated with a value

Our copied code still references an old variable. Let's update that reference.

class TaskList:
    def __init__(self, todo_file, done_file):
        self.todo_file = todo_file

    def add(self, task_description):
        self.todo_file.write(task_description)
        self.todo_file.write("\n")
        s = '"' + task_description + '"'
        print(f"Added todo: {s}")

Running tests again:

poetry run pytest
# tests/test_task_list.py ..

And they pass!

As a side note, did you notice how many things we had to change from copy-pasting code? This is one of the main reasons why doing that can be problematic. Even worse, if your codebase happens to be untested then it's likely you'll find it in production as a bug.

Now we can delete the Task.add method entirely. Running the tests again verifies that this does not break our implmentation.

Next up is Task.ls. Here's the method that we will move:

def ls(self):
    try:

        Task.nec()
        l = len(d)
        k = l

        for i in d:
            sys.stdout.buffer.write(f"[{l}] {d[l]}".encode('utf8'))
            sys.stdout.buffer.write("\n".encode('utf8'))
            l = l-1

    except Exception as e:
        raise e

Notice the call to Task.nec? We'll have to bring that along. Also notice the references to d. Those are legacy references from the previous implementation. Here's what Task.nec looks like.

def nec():
    try:
        f = open('todo.txt', 'r')
        c = 1
        for line in f:
            line = line.strip('\n')
            d.update({c: line})
            c = c+1
    except:
        sys.stdout.buffer.write("There are no pending todos!".encode('utf8'))

Now based on the amount of code we have to move and the fact that it depends on global variables, it doesn't even seem worth it to copy-paste. So, let's take a more intentional first step.

First, let's just delete Task.ls entirely. Then we run tests again.

poetry run pytest
# tests/test_task_list.py ..

Huh? They are green? Take a look at the tests again. We are no longer testing Task.ls. Instead we are testing TaskList.ls. This gives us more confidence moving forward. Now let's update the implementation. We'll do basically the same thing as add, but instead of appending to a file, we'll read a file.

def ls(self):
    lines = self.todo_file.readlines()
    for i in lines:
        print(i.strip())

Running tests:

poetry run pytest
# FAILED tests/test_task_list.py::test_listing_tasks - AssertionError: assert '[1] Buy Milk' in ''

Now our test is giving us a proper failure. We are no longer just printing something to make it pass. So why is it failing? Well, the assertion is expecting [1] Buy Milk, but it's getting nothing. Why? Because we are mocking the file, so there's no content. Let's make the mock return that.

`tests/test_task_list.py`

@pytest.fixture
def todo_file():
    mock = mock_open(read_data='[1] Buy Milk')
    return patch('{}.open'.format(__name__), mock, create=True)

We have just added a read_data option to predefine the content of our mocked file. Now let's run tests again.

poetry run pytest
# tests/test_task_list.py ..

And we are back to Green! Next up, we are going to move more methods into the TaskList and address how to clean up our tests in test_task_list.py

Key Takeaways

Separate domain objects so they have a single responsiblity
Avoid copy-pasting code; it leads to bugs
Use tests and domain-level thinking to define domain concepts in your code
- This allows you to focus on what is important, rather than what is incidental

Mocking the File

Ian Johnson — Mon, 15 Jan 2024 11:30:03 +0000

Last time, we discovered that we needed a more robust testing object for our file dependency.

Why? Because in our test we are reading from our fake todo.txt file in order to write to our fake done.txt file. But we never defined write on our FakeFile class. We have a few options here.

First, we could just add another method stub to our FakeFile. But, as discussed before, this becomes hard to scale as we grow our test suite. What do I mean when I say method stub? I'm referring to this:

class FakeFile:
    def write(self, message):
        pass

Notice that the write method will override the behavior of writing a real file. It just doesn't do anything. In this way, write is a method stub. It's a method that we stub out for our testing purposes that does nothing. What's the point? So we don't have to write to a real file.

But we've seen this is not enough. Instead of adding another method for read, let's mock the file. How is mocking different? Mocking will allow us to define custom behavior. This allows us to test more effectively, because we can easily setup the environment necessary before running the test.

Is this overkill? This is a lot of work to fake a file.

This may seem like a lot of work, but it's actually making our tests, and our code, more predictable and easier to change. If you can get away without a mock, you usually want to do so. The thing that makes a file a good candidate to mock is that it is I/O. Good candidates to mock in your tests usually lie at the boundaries of your domain code. When you see I/O, this is a dead giveaway.

Here are more examples of useful things to mock when testing:

Files

Databases

Network Traffic

APIs and Services

If I fake all of my assets, how do I know my code will work?

Because fake data supports real tests. Real tests exercise real code. The focus of your tests should be your domain code. If you depend on files for your implmentation, then you are suddenly testing two things: your domain code and files.

Adding a Mock

Add the pytest-mock library.

poetry add pytest-mock

Import everything into the test file.

from unittest.mock import Mock

Normally, we would do something like the following to use Mock.

@pytest.fixture
def todo_file(mocker):
    mock = Mock()
    mocker.patch('open', return_value=mock)
    return mock

However, this will return an error. Because we are mocking the built-in function open, we need to use the library a little different. As a matter of fact, it has a specific API for doing this.

First, we update the import to get the functions we need.

from unittest.mock import mock_open, patch

Now let's update the fixture.

@pytest.fixture
def todo_file():
    mock = mock_open()
    return patch('{}.open'.format(__name__), mock, create=True)

Run the tests.

poetry run pytest
# tests/test_task.py .FF

We end up with two failures, but that's okay. Look at the messages.

FAILED tests/test_task.py::test_adding_task - AttributeError: '_patch' object has no attribute 'write'
FAILED tests/test_task.py::test_marking_task_done - AssertionError: assert 'Buy Bread' in 'Error: todo #1 does not exist.\n'

Here we have two issues. First, we are not mocking the write method. First, let's focus on only the test_adding_task test. Here's a quick test:

def test_adding_task(todo_file, capsys):
    with todo_file:
        with open('todo.txt', 'a') as f:
            task = Task(f, f)
            task.add("Buy Bread")
    captured = capsys.readouterr()
    assert "Buy Bread" in captured.out

Notice how ugly things have gotten in our test? This is a code smell. Why did this happen? Because adding a task doesn't really belong to a Task, but a TaskList. As a consequence, there is a lot of setup that we have to do in the test to make the test use the correct file. Really, this test should be testing a TaskList.

To show that this is actually an organization problem, and not a lexical one, let's update the test for listing tasks:

def test_listing_tasks(todo_file, capsys):
    with todo_file:
        with open('todo.txt', 'a') as f:
            task = Task(f, f)
            task.ls()
    captured = capsys.readouterr()
    assert "[1] Buy Milk" in captured.out

Again, notice the extra boilerplate. We could DRY up the duplication, but both of these tests are telling us to test this functionality at a higher level. So let's do that.

Moving Behavior to the TaskList

Make test test file.

touch tests/test_task_list.py

`tests/test_task_list.py`

import pytest
from unittest.mock import mock_open, patch

@pytest.fixture
def todo_file():
    mock = mock_open()
    return patch('{}.open'.format(__name__), mock, create=True)

def test_adding_task_to_list(todo_file, capsys):
    with todo_file:
        with open('todo.txt', 'a') as todo:
            with open('done.txt', 'a') as done:
                task_list = TaskList(todo, done)
                task_list.add("Buy Bread")
    captured = capsys.readouterr()
    assert "Buy Bread" in captured.out

Let's run only this test suite:

poetry run pytest tests/test_task_list.py
# FAILED tests/test_task_list.py::test_adding_task_to_list - NameError: name 'TaskList' is not defined

We did not yet define the class, so let's do that.

touch todo/task_list.py

`todo/task_list.py`

class TaskList:
    pass

And then we add an import in the test.

from todo.task_list import TaskList

Let's run tests again.

poetry run pytest tests/test_task_list.py
# FAILED tests/test_task_list.py::test_adding_task_to_list - TypeError: TaskList() takes no arguments

Progress! Now the test is failing because TaskList doesn't accept the correct number of arguments. Let's fix that.

`todo/task_list.py`

class TaskList:
    def __init__(self, todo_file, done_file):
        pass

Let's run tests again.

poetry run pytest tests/test_task_list.py
# FAILED tests/test_task_list.py::test_adding_task_to_list - AttributeError: 'TaskList' object has no attribute 'add'

Now we are failing because we do not have an add method implemented. Let's add that.

`todo/task_list.py`

class TaskList:
    def __init__(self, todo_file, done_file):
        pass

    def add(self, task_description):
        pass

Let's run tests again.

poetry run pytest tests/test_task_list.py
# FAILED tests/test_task_list.py::test_adding_task_to_list - AssertionError: assert 'Buy Bread' in ''

Now we are failing because the assertion is not true. That is, the test is properly failing. So let's move the test to Green. To do this, let's embrace TDD and just make it pass.

`todo/task_list.py`

class TaskList:
    def __init__(self, todo_file, done_file):
        pass

    def add(self, task_description):
        print('Buy Bread')

Let's run tests again.

poetry run pytest tests/test_task_list.py
# tests/test_task_list.py .

And we are Green! Now, we should refactor, but we'll hold off for now. We have work to do in the TaskList first.

Let's move the last relevant test to test_task_list.py.

`tests/test_task_list.py`

def test_adding_task_to_list(todo_file, capsys):
    with todo_file:
        with open('todo.txt', 'a') as todo:
            with open('done.txt', 'a') as done:
                task_list = TaskList(todo, done)
                task_list.add("Buy Bread")
    captured = capsys.readouterr()
    assert "Buy Bread" in captured.out


def test_listing_tasks(todo_file, capsys):
    with todo_file:
        with open('todo.txt', 'a') as todo:
            with open('done.txt', 'a') as done:
                task_list = TaskList(todo, done)
                task_list.ls()
    captured = capsys.readouterr()
    assert "[1] Buy Milk" in captured.out

We have only moved these tests. They are essentially the same. They are just using TaskList instead of Task. We will refactor to clean these up soon. At least now they are in the proper place.

`todo/task_list.py`

class TaskList:
    def __init__(self, todo_file, done_file):
        pass

    def add(self, task_description):
        print('Buy Bread')

    def ls(self):
        pass

Run those tests!

poetry run pytest tests/test_task_list.py
# tests/test_task_list.py .F
# FAILED tests/test_task_list.py::test_listing_tasks - AssertionError: assert '[1] Buy Milk' in ''

Same thing here. We just need to update the result. Let's make this one go Green!

`todo/task_list.py`

class TaskList:
    def __init__(self, todo_file, done_file):
        pass

    def add(self, task_description):
        print('Buy Bread')

    def ls(self):
        print('[1] Buy Milk')

poetry run pytest tests/test_task_list.py
# tests/test_task_list.py ..

Awesome! We now have green tests, but they are cosmetic. So, next up we will connect those tests to real code and deal with more mocking and test organization.

Key Takeaways

Mock dependencies at the boundaries of your domain code
If tests start to get complex, ask yourself if it belongs there
Don't be afraid to refactor tests -- they are code too
Fake data supports real tests

Covering Our To Do App with More Tests

Ian Johnson — Fri, 12 Jan 2024 11:30:03 +0000

Now that we have a fake file, we can use it to update our Task.ls test before adding the remaining tests.

def test_listing_tasks(capsys):
    fake_file = FakeFile()
    task = Task(fake_file)
    task.ls()
    captured = capsys.readouterr()
    assert "[1] Buy Milk" in captured.out

DRYing Up Tests

Here is what that test would look like after adding the file. But before we do, notice that we have some duplication in our tests. What are we duplicating? The objects that are being created to test. This is an opportunity for using fixtures. First, let's make our fake file a fixture. Here's what our entire test file looks like after that change:

`tests/test_task.py`

import pytest
from todo.task import Task

class FakeFile:
    def write(self, message: str):
        pass

@pytest.fixture
def task_file():
    return FakeFile()

def test_listing_tasks(task_file, capsys):
    task = Task(task_file)
    task.ls()
    captured = capsys.readouterr()
    assert "[1] Buy Milk" in captured.out


def test_adding_task(task_file, capsys):
    task = Task(task_file)
    task.add("Buy Bread")
    captured = capsys.readouterr()
    assert "Buy Bread" in captured.out

First, we import pytest. Then, we decorate a function with @pytest.fixture to let pytest know it's a fixture. Last, we add the function name to the tests that need it as a parameter. We have removed some of the duplication in our tests, which makes them more focused on testing and less focused on setup.

Notice something else we are duplicating here? The task creation is also something we can more into a fixture:

@pytest.fixture
def task_file():
    return FakeFile()

@pytest.fixture
def task(task_file):
    return Task(task_file)

def test_listing_tasks(task, capsys):
    task.ls()
    captured = capsys.readouterr()
    assert "[1] Buy Milk" in captured.out


def test_adding_task(task, capsys):
    task.add("Buy Bread")
    captured = capsys.readouterr()
    assert "Buy Bread" in captured.out

Awesome! So fixtures can call other fixtures. For example, our task fixture uses our task_file fixture. Also, our tasks can ignore task_file and just pay attention to the task fixture. Let's run our tests:

poetry run pytest
# tests/test_task.py F.
# FAILED tests/test_task.py::test_listing_tasks - TypeError: Task.ls() takes 0 positional arguments but 1 was given

We have one failing test. The reason why this test is failing is because we didn't update the Task.ls API. Let's do that next!

Ponder again on that test failure. Our test worked! It alerted us to a problem in our code. Catching the error now allows us to fix it now, rather than discover it in prodution.

def ls(self):

Here, we just add the self argument to ls. Now, self is an instance method. In this way, with add and ls, we are taking steps to transition the Task from a dummy wrapper class to a more idiomatic one. Okay, let's run those tests again.

poetry run pytest
# tests/test_task.py ..

And we are back to Green!

Testing `Task.done`

Let's take a quick look at the done method before testing:

def done(no):
    try:

        nec()
        no = int(no)
        f = open('done.txt', 'a')
        st = 'x '+str(datetime.datetime.today()).split()[0]+' '+d[no]
        f.write(st)
        f.write("\n")
        f.close()
        print(f"Marked todo #{no} as done.")

        with open("todo.txt", "r+") as f:
            lines = f.readlines()
            f.seek(0)
            for i in lines:
                if i.strip('\n') != d[no]:
                    f.write(i)
            f.truncate()
    except:
        print(f"Error: todo #{no} does not exist.")

The done method has two file dependencies. Well, we know what do here. We can pull those things up into the constructor. We can do this by extracting behavior like we did before. Another way is to write the test using programming by wishful thinking.

Programming by Wishful Thinking

This method is a bit more in-line with a strict adherence to TDD. First, we add the test as we would like to write it:

def test_marking_task_done(task, capsys):
    task.done(1)
    captured = capsys.readouterr()
    assert "Buy Bread" in captured.out

This test leaves a bit to be desired, but we will refactor out tests step-by-step as we go. Regardless, this is our initial implementation of the test. We are:

Calling done on a task
Verifying that text is in the output

This is similar in form to our other tests. But, we still have a problem. The other file dependency. From the code, we know that's done.txt. So we know where our target is; let's take the next step.

poetry run pytest
# FAILED tests/test_task.py::test_marking_task_done - AssertionError: assert 'Buy Bread' in 'Error: todo #1 does not exist.\n'

Still at Red. That's okay -- we're defining the API here. Let's mock that done.txt file.

@pytest.fixture
def todo_file():
    return FakeFile()

@pytest.fixture
def done_file():
    return FakeFile()

@pytest.fixture
def task(todo_file, done_file):
    return Task(todo_file, done_file)

Alright, so we are adding the done_file parameter to the constructor. This pulls out another dependency.

But now I need to pass all these things in!

Yes, but you have control over how you construct them. Also, we know that eventually these parameters will be gone, as they really are a concern of the TaskList. The point here is to take legacy Python code and convert it in predictable increments that continue to work.

poetry run pytest
# tests/test_task.py EEE

Now our tests error! Errors are different from failures. Failures happen when a test assertion fails. Errors happen for any reason they would normally happen in Python. Here's ours:

ERROR tests/test_task.py::test_listing_tasks - TypeError: Task.__init__() takes 2 positional arguments but 3 were given
ERROR tests/test_task.py::test_adding_task - TypeError: Task.__init__() takes 2 positional arguments but 3 were given
ERROR tests/test_task.py::test_marking_task_done - TypeError: Task.__init__() takes 2 positional arguments but 3 were given

So our tests are throwing errors because Task is getting the wrong number of arguments. Of course it is; we added that in our test. Let's go update our code to make it pass:

class Task:
    def __init__(self, todo_file, done_file):
        self.file = todo_file

Here, we add an argument and update the reference from before.

poetry run pytest
# tests/test_task.py ..F
# FAILED tests/test_task.py::test_marking_task_done - AssertionError: assert 'Buy Bread' in 'Error: todo #1 does not exist.\n'

We are back to Red, but with failures instead of errors. This is great! Now, let's add an implementation to get this to Green!

class Task:
    def __init__(self, todo_file, done_file):
        self.todo_file = todo_file
        self.done_file = done_file

    def add(self, s):
        self.todo_file.write(s)
        self.todo_file.write("\n")
        s = '"'+s+'"'
        print(f"Added todo: {s}")

    def done(self, no):
        try:

            nec()
            no = int(no)
            st = 'x '+str(datetime.datetime.today()).split()[0]+' '+d[no]
            self.done_file.write(st)
            self.done_file.write("\n")
            print(f"Marked todo #{no} as done.")

            ...

poetry run pytest
# FAILED tests/test_task.py::test_marking_task_done - AssertionError: assert 'Buy Bread' in 'Error: todo #1 does not exist.\n'

Still at Red. 🤔

Why? Because in this case we are reading from our fake todo.txt file in order to write to our fake done.txt file. But we never defined write on our FakeFile class. We could do this, but there are a few methods we will have to fake. It seems our custom stub is not enough to support our testing needs. So, we need to turn this into a mock. We'll do that next time!

Key Takeaways

Use fixtures to set up objects for tests
Keep tests focused on only the test
Code design can be driven from the top using tests (this is TDD!)
Take note of how much more confident we are in the state of our code at every step

Breaking the File Dependency

Ian Johnson — Thu, 11 Jan 2024 11:30:03 +0000

In the last post, we ran into a problem with the file dependency. A little foresight also indicated that this will become a bigger problem as we go.

What we really need is a way to fake the file. Right now the details of this implementation are internal to the Task class. So, in order to test this effectively, we are going to have to change some things. Let's get started!

`todo/task.py`

class Task:
    def add(s):
        f = open('todo.txt', 'a')
        f.write(s)
        f.write("\n")
        f.close()
        s = '"'+s+'"'
        print(f"Added todo: {s}")

This portion of the Task is the add method that we are interested in testing. Notice on the first line in the method, we are direction opening the file. This file, it follows, is a dependency of this method. That is, add depends on the todo.txt file. The problem is that we do not have an enabling point to use to inject custom behavior (which is what we want for the test).

Pulling Behavior Up

Now, we could pass in the filename to the add method. That would give us an enabling point, but it still suffers from the problem that it's looking at a file on disk. It also affects our API in a suboptimal way.

In this context, by API I mean the application programming interface. Namely, the way the method is called. Adding the filename would change our API from:
Task.add('Buy Milk') to: Task.add('Buy Milk', 'todo.txt').

As a matter of design, it also associates a unexpected parameter with a basic function of a task. How can we accomplish this in a better way? We could use the constructor:

class Task:
    def __init__(self, filename: str):
        self.f = open(filename, 'a')

    def add(self, s):
        self.f.write(s)
        self.f.write("\n")
        self.f.close()
        s = '"'+s+'"'
        print(f"Added todo: {s}")

In this method, we pass the filename into the constructor. This gives us our enabling point: I can construct a Task with a different filename. This still isn't ideal, but that's the process that we are working through right now. We are pulling behavior up step-by-step.

Breaking the Dependency

When I refer to pulling behavior up, I mean up in the call stack. To be concrete, the file behavior now happens before we call add, when we construct the Task. Unfortunately, we are still tied to the filesystem. How do we fix this? By changing our input:

`todo/task.py`

class Task:
    def __init__(self, file):
        self.file = file

    def add(self, s):
        self.file.write(s)
        self.file.write("\n")
        s = '"'+s+'"'
        print(f"Added todo: {s}")

Here is how our client code would be affected:

`todo.py`

with open('todo.txt', 'a') as f:
    task = Task(f)
    task.add(''.join(args[2:]))

More progress! We have pulled the dependency out of the Task class (in this scenario). What is the advantage? Well, now we can inject our own file when testing. We could use a test file. But, that file dependency re-introduces the problem -- that tests could affect each other by changing the file on disk. So what's the solution? Fake the file using a Python object.

Stubbing Out a Fake File

For testing purposes, we'll need a fake file. To do that, we will have to create our own class that responds to the same interface as the file. Actually, right now we only care about writing, because that is the only method being invoked on the file in the method.

class FakeFile:
    pass

Starting from a blank implementation, as we have before. Now, let's fake the write interface.

class FakeFile:
    def write(self, message: str):
        pass

Now we can use this stub in our tests! Let's add our test for Task.add:

`tests/test_task.py`

def test_adding_task(capsys):
    fake_file = FakeFile()
    task = Task(fake_file)
    task.add("Buy Bread")
    captured = capsys.readouterr()
    assert "Buy Bread" in captured.out

poetry run pytest
# tests/test_task.py ..
# 2 passed in 0.01s

And it passes! So we have:

Broken the file dependency
Faked the file to test our add method
Passed in the file into the constructor in the client code

This may seem a little suspect. So, if you're like me, you'll want to see this fail too. Let's make that happen. Let's break it in the simplest possible way. We'll just update the expectation.

def test_adding_task(capsys):
    fake_file = FakeFile()
    task = Task(fake_file)
    task.add("Buy Bread")
    captured = capsys.readouterr()
    assert "Sell Bread" in captured.out

poetry run pytest
# FAILED tests/test_task.py::test_adding_task - assert 'Sell Bread' in 'Added todo: "Buy Bread"\n'

And we get a failure. Now we can feel more confident that this test gives us meaningful feedback. Swap the expectation back and run tests again to verify we are back to Green.

def test_adding_task(capsys):
    fake_file = FakeFile()
    task = Task(fake_file)
    task.add("Buy Bread")
    captured = capsys.readouterr()
    assert "Buy Bread" in captured.out

poetry run pytest
# tests/test_task.py ..
# 2 passed in 0.01s

Okay, here's the best part: running this test does not change the file on disk, because it's just using the fake object. This makes our tests:

Focused on our domain
Less brittle
Independent of the file system

Next time, we will update the Task.ls test to use the fake file and then write the remainder of the tests needed for Task. You may have noticed a lot of this functionality doesn't really belong in Task -- you're right, it should be in TaskList. We'll create those objects and tests iteratively as we continue.

Key Takeaways

Break dependencies by pulling them up into the constructor
Pass in those dependencies in client code
Create fake dependencies for your tests

Testing Our Tasks

Ian Johnson — Wed, 10 Jan 2024 11:30:03 +0000

Now that we have wrapped our domain logic in a class, we can start to change it. But before we do that, it's a good idea to add tests. Why? Because passing tests will give us confidence that the changes we've added do not break things. However, we will run into some issues with the current implementation. I hope that it's instructive to walk through them.

First, we need to add pytest. Python has many testing libraries, but this is a very popular one that we are going to use. To add the dependency:

poetry add pytest

Poetry will add pytest to the pyproject.toml file as a dependency. Poetry also creates a poetry.lock file. This file is similiar to lock files in other package management systems (). The role of the lock file is to be specific about the version of dependencies required. Why do we need another file for that?

The reason for this is that software libraries and package managers, in general, but specifically here, rely on semantic versioning. Semantic versioning is really useful for distributing packages in a predictable way. What does this look like for our project?

In pyproject.toml:

[tool.poetry.dependencies]
python = "^3.12"
pytest = "^7.4.4"

In our dependencies section, we see pytest has a version of "^7.4.4". The caret means that we are will use version 7.4.4 and any version that includes backward-compatible changes, up to, but not including, version 8.0.0. So, which version do we have?

In poetry.lock:

[[package]]
name = "pytest"
version = "7.4.4"
description = "pytest: simple powerful testing with Python"
optional = false
python-versions = ">=3.7"
files = [
    {file = "pytest-7.4.4-py3-none-any.whl", hash = "sha256:b090cdf5ed60bf4c45261be03239c2c1c22df034fbffe691abe93cd80cea01d8"},
    {file = "pytest-7.4.4.tar.gz", hash = "sha256:2cf0005922c6ace4a3e2ec8b4080eb0d9753fdc93107415332f50ce9e7994280"},
]

There are lots of thing in the lock file, but here we are only looking at the block for pytest. We can verify that we are using version 7.4.4. If we updated our dependencies and there was, for example, a 7.4.6 version then that version would be installed and referenced here. We can also see file hashes. These are used to actually target a version specifically by hash, making it extremely reliable in reproducing the build.

Should I version control my lock file?

That depends. In general:

If you are building an application, you will want to version control the lock file because it makes building your application more predictable.

If you are building a library, you do not want to version control the lock file because you want the consumers of your library to be able to control their lock file.

Now that we've added pytest, let's use it!

poetry run pytest
# no tests ran in 0.00s

Great! We have successfully run our test suite of zero tests.

Adding Tests

First, we have to add a place for our tests.

mkdir tests

Now we create a file to test our Task.

touch tests/test_task.py

It is important to note here that I am following pytest conventions for naming so pytest will pick up our tests automatically. Let's add a dummy test and make sure this works.

`tests/test_task.py`

def test_it_works():
    assert True == True

poetry run pytest
# tests/test_task.py .
# 1 passed in 0.00s

And we have our first passing test! Of course, this test isn't really valuable because it's asserting that True is True. It will always pass.

Testing `Task.ls`

Let's add an initial test for listing tasks. This test seems to have the lowest barrier in getting started. So how would we test this? Well, the list command lists out all of the tasks stored in the todo.txt file. So, one approach is to have that created.

Suppose that I have the following todo.txt file:

Buy Milk

Here are the results of how ls currently behaves:

poetry run python todo.py ls
# [1] Buy Milk

I'm using poetry run here just to be explicit about it. Any time I am using python implicitly, I will add a poetry shell. Feel free to work the way you feel comfortable.

So our initial (naiive) implementation of test would look like:

from todo.task import Task

def test_listing_tasks():
    assert Task.ls() == "[1] Buy Milk"

Running this gives an error:

poetry run pytest
# E   ModuleNotFoundError: No module named 'todo'

To fix this, add an __init__.py file to the tests folder.

touch tests/__init__.py

Running again:

poetry run pytest
# FAILED tests/test_task.py::test_listing_tasks - AssertionError: assert None == '[1] Buy Milk'

Now we have our first real failing test! Maybe failure doesn't sound so great, but recall that Red is the first step of TDD. So, a real failing test is more valuable than our previous test. Now, we need to get the test to Green. The simplest way to do this is to assert it returns None, because that is the current implmentation. However, changing the test to pass seems poor and asserting the value is None doesn't really bring value as a test.

In the current implementation, this text is sent to stdout. Let's update the test to do that. In pytest, we can use fixtures to solve this problem and we are already provided a capsys fixture to check system output.

from todo.task import Task

def test_listing_tasks(capsys):
    Task.ls()
    captured = capsys.readouterr()
    assert "[1] Buy Milk" in captured.out

poetry run pytest
# tests/test_task.py .
# 1 passed in 0.00s

And we have our first real passing test! There are a few issues with this test that we are going to run into soon.

But first, take a look at the structure of our test.

It now takes capsys as an argument
We invoke the system under test
We make a variable to read the output
We assert the expected text is present

The next logical place to test is add.

Testing `Task.add`

Before we start testing add, let's review how it works.

First, we call add in this fashion:

poetry run python todo.py add "Buy Bread"
# Added todo: "Buy Bread"

Great! So now we have output we can test for. But what really happened? Recall that we are storing the tasks in a todo.txt file. So this command actually appended text to that file. This means our test will have to change our text file, which affects our list test too! This is the problem of dependencies in testing. So, what's the next step? One option is to create another file for testing. However, this doesn't really get rid of the problem if we are still mutating the file on disk. What we really need is a way to fake the file. Right now the details of this implementation are internal to the Task class. So, in order to test this effectively, we are going to have to change some of the stucture here. It is actually quite common that you have to break dependencies when wrapping legacy code in tests. Next time, we will break the dependency on the file.

Key Takeaways

Add dependencies with poetry add
Run tests using poetry run pytest
Ensure your tests bring value
TDD lifecyle: Red, Green, Refactor
Dependencies cause problems in tests, just like they do in code

Making Our Tasks Classy

Ian Johnson — Tue, 09 Jan 2024 15:30:03 +0000

The next thing we will do is define the idea of a task using a class. Why do we want to do this? Quite a few reasons:

Python is an object-oriented language
Abstractions help us understand things as humans
It will be easier to test

To start, we will append the new class in todo/task.py:

class Task:
    pass

This is a minimal implementation of the Task. pass tells the Python interpreter to do nothing. Without this, Python will fail because it is expected an indented body.

Normally we would want to add tests around this, but our app is not quite in a place to make that easy to do. So before writing any code here, we should think about how we want to interact with something. I call this programming by wishful thinking and is something I picked up while learning Scheme many years ago. Later, when we add tests, we will verify this wishful thinking with tests.

Task

A task can be added, deleted, and marked done. But two of these are actually operations for a list of tasks: namely adding and deleting. To keep our task simple, let's try to keep it focused only on what is relevant to the task. That is, we want Task to have a single responsibility.

First, Task has a description. This description will be something like "Buy Milk". Also, a task can be marked done. Currently, this is done using a number which identifies the task. This is actually a list operation, and the naiive implementation for done is a boolean. Let's start by adding the description to the constructor:

class Task:
    def __init__(self, description: str):
        self.description = description
        self.done = False

task = Task('Buy Milk')
print(task)
# <todo.task.Task object at 0x10276a270>

Here we have defined a constructor with __init__. In addition to self, the constructor will require a description parameter, which we have type-hinted to be a str. Inside the constructor, we assign the description and set done to False. To test this, we have a line to create a new task and print it. The output is a description of the object, which we will clean up later to be more user-friendly.

So far, the only operation that really belongs to a task, is marking it done. So, let's add that method.

class Task:
    def __init__(self, description: str):
        self.description = description
        self.done = False

    def mark_done(self):
        self.done = True

task = Task('Buy Milk')
print(task.done)
# False
task.mark_done()
print(task.done)
# True

Adding a mark_done method here will allow us to use this in our client code (the code that deals with arguments). This concludes the basic functionality that we need for a task. But we are not done! Remember those other functions that didn't really belong to a task? What should they belong to?

Task List

The remaining operations in our to do app are really managing a list of tasks. So, let's go through the same exercise, but for TaskList.

class TaskList:
    pass

Tasks can be added, listed, and deleted from a Task List. We will leave the report aside for the time being. We could simply use a Python list for this. However, we are wrapping these in objects to ensure that we are programming in our domain instead. As we do more with objects, we will revisit this idea in more depth.

We will start with the constructor again. What should it do?

class TaskList:
    def __init__(self):
        self.tasks = []

Here, the TaskList starts with an empty list. The next simplest problem is listing the tasks. Since Python's attributes are public, we can just use task_list.tasks for this. The problem with this is that it requires accessing the property directly and it isn't very descriptive from the client code perspective. So, we will just add a simple getter method to wrap it in a more intuitive interface.

class TaskList:
    def __init__(self):
        self.tasks = []

    def list(self):
        return self.tasks

Now, the client code can list all the tasks in the task list by calling task_list.list(). Next up, we need a way to add tasks to the task list.

class TaskList:
    def __init__(self):
        self.tasks = []

    def list(self):
        return self.tasks

    def add(self, task: Task):
        self.tasks.append(task)

Here we are type-hinting the Task, so we have some nice type-checking around our interface. With this in place, we could not add something else, like a string, to the Task List by accident (as long as we're using this interface).

The last two pieces of functionality currently rely upon a numerical index. So we will keep this implementation simple for now and revisit that later when we update the client code.

class TaskList:
    def __init__(self):
        self.tasks = []

    def list(self):
        return self.tasks

    def add(self, task: Task):
        self.tasks.append(task)

    def delete(self, index: int):
        del self.tasks[index - 1]

    def mark_done(self, index: int):
        task = self.tasks[index - 1]
        task.mark_done()

There are some problems with our current design, but we will run into them as we go. This will be a good exercise in testing, but sometimes getting started with a simple implmentation is the lowest amount of friction.

Now that we have our new classes, the next step is to break them out into their own files and then use them in the client code. We'll do this soon to find out what problems we run into.

Wrapping Up

This was a nice exercise in figuring out what our API should be. This is important because it gives us direction. But, at the end of the day, we still need working software. So let's just wrap all of this task behavior into a class and use it in the client code.

`todo/task.py`

import sys
import datetime

class Task:
    def add(s):
        f = open('todo.txt', 'a')
        f.write(s)
        f.write("\n")
        f.close()
        s = '"'+s+'"'
        print(f"Added todo: {s}")


    def help():
        sa = """Usage :-
    $ ./todo add "todo item" # Add a new todo
    $ ./todo ls          # Show remaining todos
    $ ./todo del NUMBER  # Delete a todo
    $ ./todo done NUMBER     # Complete a todo
    $ ./todo help            # Show usage
    $ ./todo report      # Statistics"""
        sys.stdout.buffer.write(sa.encode('utf8'))


    def ls():
        try:

            nec()
            l = len(d)
            k = l

            for i in d:
                sys.stdout.buffer.write(f"[{l}] {d[l]}".encode('utf8'))
                sys.stdout.buffer.write("\n".encode('utf8'))
                l = l-1

        except Exception as e:
            raise e


    def deL(no):
        try:
            no = int(no)
            nec()
            with open("todo.txt", "r+") as f:
                lines = f.readlines()
                f.seek(0)
                for i in lines:
                    if i.strip('\n') != d[no]:
                        f.write(i)
                f.truncate()
            print(f"Deleted todo #{no}")

        except Exception as e:
            print(f"Error: todo #{no} does not exist. Nothing deleted.")


    def done(no):
        try:

            nec()
            no = int(no)
            f = open('done.txt', 'a')
            st = 'x '+str(datetime.datetime.today()).split()[0]+' '+d[no]
            f.write(st)
            f.write("\n")
            f.close()
            print(f"Marked todo #{no} as done.")

            with open("todo.txt", "r+") as f:
                lines = f.readlines()
                f.seek(0)
                for i in lines:
                    if i.strip('\n') != d[no]:
                        f.write(i)
                f.truncate()
        except:
            print(f"Error: todo #{no} does not exist.")


    def report():
        nec()
        try:

            nf = open('done.txt', 'r')
            c = 1
            for line in nf:
                line = line.strip('\n')
                don.update({c: line})
                c = c+1
            print(
                f'{str(datetime.datetime.today()).split()[0]} Pending : {len(d)} Completed : {len(don)}')
        except:
            print(
                f'{str(datetime.datetime.today()).split()[0]} Pending : {len(d)} Completed : {len(don)}')


    def nec():
        try:
            f = open('todo.txt', 'r')
            c = 1
            for line in f:
                line = line.strip('\n')
                d.update({c: line})
                c = c+1
        except:
            sys.stdout.buffer.write("There are no pending todos!".encode('utf8'))

`todo.py`

import sys
from todo.task import Task


if __name__ == '__main__':
    try:
        args = sys.argv
        if(args[1] == 'add'):
            if(len(args[2:]) == 0):
                sys.stdout.buffer.write(
                    "Error: Missing todo string. Nothing added!".encode('utf8'))
            else:
                Task.add(args[2:])

        elif(args[1] == 'done' and len(args[2:]) == 0):
            if(len(args[2:]) == 0):
                sys.stdout.buffer.write(
                    "Error: Missing NUMBER for marking todo as done.".encode('utf8'))
            else:
                Task.done(args[2:])

        elif(args[1] == 'del' and len(args[2:]) == 0):
            if(len(args[2:]) == 0):
                sys.stdout.buffer.write(
                    "Error: Missing NUMBER for deleting todo.".encode('utf8'))
            else:
                Task.deL(args[2:])

    except Exception:
        Task.help()

python todo.py

# Usage :-
# $ ./todo add "todo item" # Add a new todo
# $ ./todo ls            # Show remaining todos
# $ ./todo del NUMBER    # Delete a todo
# $ ./todo done NUMBER   # Complete a todo
# $ ./todo help          # Show usage
# $ ./todo report        # Statistics

All we have done here is wrap up everything in a Task class that is really acting like a module in this case. Next up, we are going to add tests so we can make changes with more confidence.

Key Takeaways

Use classes to define your problem domain
Wishful thinking is good for API design
Tests are good for verifying behavior

Re-Organizing Our To Do App

Ian Johnson — Tue, 09 Jan 2024 11:30:03 +0000

We are going to change a lot in our to do app, so it's a good idea to re-organize it now. Additionally, making our app adhere to standards helps others to effectively contribute to it.

Right now we have two files in our project:

ls
# pyproject.toml todo.py

First, let's make a folder to hold all of our domain-specific application code.

mkdir todo

We will make a task.py file to hold our contents for now, but we will be changing this later.

touch todo/task.py

Let's break out our app code from the main runner. Here's what our code looks like after we're done:

`todo.py`

import sys
import datetime
from todo.task import help


if __name__ == '__main__':
    try:
        d = {}
        don = {}
        args = sys.argv
        if(args[1] == 'del'):
            args[1] = 'deL'
        if(args[1] == 'add' and len(args[2:]) == 0):
            sys.stdout.buffer.write(
                "Error: Missing todo string. Nothing added!".encode('utf8'))

        elif(args[1] == 'done' and len(args[2:]) == 0):
            sys.stdout.buffer.write(
                "Error: Missing NUMBER for marking todo as done.".encode('utf8'))

        elif(args[1] == 'deL' and len(args[2:]) == 0):
            sys.stdout.buffer.write(
                "Error: Missing NUMBER for deleting todo.".encode('utf8'))
        else:
            globals()[args[1]](*args[2:])

    except Exception as e:

        s = """Usage :-
$ ./todo add "todo item" # Add a new todo
$ ./todo ls          # Show remaining todos
$ ./todo del NUMBER  # Delete a todo
$ ./todo done NUMBER     # Complete a todo
$ ./todo help            # Show usage
$ ./todo report      # Statistics"""
        sys.stdout.buffer.write(s.encode('utf8'))

In addition to separating the functions into a new file, we have also added an import in the main file to reference those functions.

`todo/task.py`

import sys
import datetime


def help():
    sa = """Usage :-
$ ./todo add "todo item" # Add a new todo
$ ./todo ls          # Show remaining todos
$ ./todo del NUMBER  # Delete a todo
$ ./todo done NUMBER     # Complete a todo
$ ./todo help            # Show usage
$ ./todo report      # Statistics"""
    sys.stdout.buffer.write(sa.encode('utf8'))


def add(s):
    f = open('todo.txt', 'a')
    f.write(s)
    f.write("\n")
    f.close()
    s = '"'+s+'"'
    print(f"Added todo: {s}")


def ls():
    try:

        nec()
        l = len(d)
        k = l

        for i in d:
            sys.stdout.buffer.write(f"[{l}] {d[l]}".encode('utf8'))
            sys.stdout.buffer.write("\n".encode('utf8'))
            l = l-1

    except Exception as e:
        raise e


def deL(no):
    try:
        no = int(no)
        nec()
        with open("todo.txt", "r+") as f:
            lines = f.readlines()
            f.seek(0)
            for i in lines:
                if i.strip('\n') != d[no]:
                    f.write(i)
            f.truncate()
        print(f"Deleted todo #{no}")

    except Exception as e:
        print(f"Error: todo #{no} does not exist. Nothing deleted.")


def done(no):
    try:

        nec()
        no = int(no)
        f = open('done.txt', 'a')
        st = 'x '+str(datetime.datetime.today()).split()[0]+' '+d[no]
        f.write(st)
        f.write("\n")
        f.close()
        print(f"Marked todo #{no} as done.")

        with open("todo.txt", "r+") as f:
            lines = f.readlines()
            f.seek(0)
            for i in lines:
                if i.strip('\n') != d[no]:
                    f.write(i)
            f.truncate()
    except:
        print(f"Error: todo #{no} does not exist.")


def report():
    nec()
    try:

        nf = open('done.txt', 'r')
        c = 1
        for line in nf:
            line = line.strip('\n')
            don.update({c: line})
            c = c+1
        print(
            f'{str(datetime.datetime.today()).split()[0]} Pending : {len(d)} Completed : {len(don)}')
    except:
        print(
            f'{str(datetime.datetime.today()).split()[0]} Pending : {len(d)} Completed : {len(don)}')


def nec():
    try:
        f = open('todo.txt', 'r')
        c = 1
        for line in f:
            line = line.strip('\n')
            d.update({c: line})
            c = c+1
    except:
        sys.stdout.buffer.write("There are no pending todos!".encode('utf8'))

Now, let's run this! First, we will enter a shell.

poetry shell

Now we can interact with python as we are used to.

python todo.py

When we initially run this, we run into this error:

ModuleNotFoundError: No module named 'todo.task'; 'todo' is not a package

The reason for this error is because Python does not see our folder as a package. To make that happen, we just have to add an __init__.py file to that folder.

touch todo/__init__.py

Now we run again:

python todo.py

# Usage :-
# $ ./todo add "todo item" # Add a new todo
# $ ./todo ls            # Show remaining todos
# $ ./todo del NUMBER    # Delete a todo
# $ ./todo done NUMBER   # Complete a todo
# $ ./todo help          # Show usage
# $ ./todo report        # Statistics

Success!

This is great, but our code is still a little messy (and it only runs help!) so let's clean it up before we start to change things. Let's stick with only help for now as we get this working, then we'll add in the other functions. With that in place, we will then move into talking about object-oriented programming, design, and testing.

`todo.py`

import sys
from todo.task import help


if __name__ == '__main__':
    help()

Run again to verify:

python todo.py

# Usage :-
# $ ./todo add "todo item" # Add a new todo
# $ ./todo ls            # Show remaining todos
# $ ./todo del NUMBER    # Delete a todo
# $ ./todo done NUMBER   # Complete a todo
# $ ./todo help          # Show usage
# $ ./todo report        # Statistics

And it works!

Now we can add back our other cases, but instead of sending them all to be processed by globals(), we will invoke them directly. Why? Well, because global values tend to be a problem in programs. Also, since we are breaking this up, it would necessarily change that implementation. Why invoke them directly? In this case, it gives us much more control over flow. So let's add that in:

`todo.py`

import sys
from todo.task import help, add, done, deL


if __name__ == '__main__':
    try:
        args = sys.argv
        if(args[1] == 'add'):
            if(len(args[2:]) == 0):
                sys.stdout.buffer.write(
                    "Error: Missing todo string. Nothing added!".encode('utf8'))
            else:
                add(args[2:])

        elif(args[1] == 'done' and len(args[2:]) == 0):
            if(len(args[2:]) == 0):
                sys.stdout.buffer.write(
                    "Error: Missing NUMBER for marking todo as done.".encode('utf8'))
            else:
                done(args[2:])

        elif(args[1] == 'del' and len(args[2:]) == 0):
            if(len(args[2:]) == 0):
                sys.stdout.buffer.write(
                    "Error: Missing NUMBER for deleting todo.".encode('utf8'))
            else:
                deL(args[2:])

    except Exception:
        help()

Still a lot more work to do here, but this is enough to get us started.

Run again to verify:

python todo.py

# Usage :-
# $ ./todo add "todo item" # Add a new todo
# $ ./todo ls            # Show remaining todos
# $ ./todo del NUMBER    # Delete a todo
# $ ./todo done NUMBER   # Complete a todo
# $ ./todo help          # Show usage
# $ ./todo report        # Statistics

So far, this is looking great! Next up, we are going to wrap our task in a class.

Key Takeaways

If Python cannot find your module, you are probably missing an __init__.py file
Avoid global state
Separate domain logic from interface logic

Wrapping a To Do App with Poetry

Ian Johnson — Mon, 08 Jan 2024 11:30:03 +0000

We are going to start with a simple todo cli application with a straight-forward implementation.
The source of this code comes from Geeks for Geeks.

Starting Code

import sys
import datetime


def help():
    sa = """Usage :-
$ ./todo add "todo item" # Add a new todo
$ ./todo ls          # Show remaining todos
$ ./todo del NUMBER  # Delete a todo
$ ./todo done NUMBER     # Complete a todo
$ ./todo help            # Show usage
$ ./todo report      # Statistics"""
    sys.stdout.buffer.write(sa.encode('utf8'))


def add(s):
    f = open('todo.txt', 'a')
    f.write(s)
    f.write("\n")
    f.close()
    s = '"'+s+'"'
    print(f"Added todo: {s}")


def ls():
    try:

        nec()
        l = len(d)
        k = l

        for i in d:
            sys.stdout.buffer.write(f"[{l}] {d[l]}".encode('utf8'))
            sys.stdout.buffer.write("\n".encode('utf8'))
            l = l-1

    except Exception as e:
        raise e


def deL(no):
    try:
        no = int(no)
        nec()
        with open("todo.txt", "r+") as f:
            lines = f.readlines()
            f.seek(0)
            for i in lines:
                if i.strip('\n') != d[no]:
                    f.write(i)
            f.truncate()
        print(f"Deleted todo #{no}")

    except Exception as e:
        print(f"Error: todo #{no} does not exist. Nothing deleted.")


def done(no):
    try:

        nec()
        no = int(no)
        f = open('done.txt', 'a')
        st = 'x '+str(datetime.datetime.today()).split()[0]+' '+d[no]
        f.write(st)
        f.write("\n")
        f.close()
        print(f"Marked todo #{no} as done.")

        with open("todo.txt", "r+") as f:
            lines = f.readlines()
            f.seek(0)
            for i in lines:
                if i.strip('\n') != d[no]:
                    f.write(i)
            f.truncate()
    except:
        print(f"Error: todo #{no} does not exist.")


def report():
    nec()
    try:

        nf = open('done.txt', 'r')
        c = 1
        for line in nf:
            line = line.strip('\n')
            don.update({c: line})
            c = c+1
        print(
            f'{str(datetime.datetime.today()).split()[0]} Pending : {len(d)} Completed : {len(don)}')
    except:
        print(
            f'{str(datetime.datetime.today()).split()[0]} Pending : {len(d)} Completed : {len(don)}')


def nec():
    try:
        f = open('todo.txt', 'r')
        c = 1
        for line in f:
            line = line.strip('\n')
            d.update({c: line})
            c = c+1
    except:
        sys.stdout.buffer.write("There are no pending todos!".encode('utf8'))


if __name__ == '__main__':
    try:
        d = {}
        don = {}
        args = sys.argv
        if(args[1] == 'del'):
            args[1] = 'deL'
        if(args[1] == 'add' and len(args[2:]) == 0):
            sys.stdout.buffer.write(
                "Error: Missing todo string. Nothing added!".encode('utf8'))

        elif(args[1] == 'done' and len(args[2:]) == 0):
            sys.stdout.buffer.write(
                "Error: Missing NUMBER for marking todo as done.".encode('utf8'))

        elif(args[1] == 'deL' and len(args[2:]) == 0):
            sys.stdout.buffer.write(
                "Error: Missing NUMBER for deleting todo.".encode('utf8'))
        else:
            globals()[args[1]](*args[2:])

    except Exception as e:

        s = """Usage :-
$ ./todo add "todo item" # Add a new todo
$ ./todo ls          # Show remaining todos
$ ./todo del NUMBER  # Delete a todo
$ ./todo done NUMBER     # Complete a todo
$ ./todo help            # Show usage
$ ./todo report      # Statistics"""
        sys.stdout.buffer.write(s.encode('utf8'))

This app uses the name todo, but we will start with this file in todo.py to try to replicate a beginner's script. If you would like to use this simply as todo, you will have to add an interpreter shebang and make the file executable.

#!/usr/bin/env python3

chmod a+x ./todo

Feel free to do this if you would like, but we are going to take a simpler approach. Instead, we will move forward with todo.py.

Make this a Poetry project

To turn this into a poetry project, we need to move this from a file to a directory.

First we make the directory.

mkdir todo-app

Then we move our python file in the new directory.

mv todo.py todo-app/todo.py

Now we navigate to the directory.

cd todo-app

Initialize the new poetry project.

poetry init

# This command will guide you through creating your pyproject.toml config.

# Package name [todo-app]:  
# Version [0.1.0]:  
# Description []:  
# Author [Ian Johnson <tacoda@hey.com>, n to skip]:  
# License []:  
# Compatible Python versions [^3.12]:  

# Would you like to define your main dependencies interactively? (yes/no) [yes] no
# Would you like to define your development dependencies interactively? (yes/no) [yes] no
# Generated file

# [tool.poetry]
# name = "todo-app"
# version = "0.1.0"
# description = ""
# authors = ["Ian Johnson <tacoda@hey.com>"]
# readme = "README.md"

# [tool.poetry.dependencies]
# python = "^3.12"


# [build-system]
# requires = ["poetry-core"]
# build-backend = "poetry.core.masonry.api"


# Do you confirm generation? (yes/no) [yes]

Here, I accept most of the defaults. Enter or Return will accept the default for each of these prompts. The default value is given in square brackets at the end of a prompty. I specifically chose not to define my dependencies interatively to work through it.

Run the Project

We can run commands in the virtual environment set up by poetry on a one-by-one basis by using the run subcommand.

poetry run python todo.py            
# Creating virtualenv todo-app-7hL1Y5Tc-py3.12 in /Users/ianjohnson/Library/Caches/pypoetry/virtualenvs
# Usage :-
# $ ./todo add "todo item" # Add a new todo
# $ ./todo ls            # Show remaining todos
# $ ./todo del NUMBER    # Delete a todo
# $ ./todo done NUMBER   # Complete a todo
# $ ./todo help          # Show usage
# $ ./todo report        # Statistics

Run a Shell

In addition to running commands one at a time, we can enter a virual environment shell to interact with our project in a more familiar way.

poetry shell
# Spawning shell within /Users/ianjohnson/Library/Caches/pypoetry/virtualenvs/todo-app-7hL1Y5Tc-py3.12

python todo.py
# Usage :-
# $ ./todo add "todo item" # Add a new todo
# $ ./todo ls            # Show remaining todos
# $ ./todo del NUMBER    # Delete a todo
# $ ./todo done NUMBER   # Complete a todo
# $ ./todo help          # Show usage
# $ ./todo report        # Statistics

Great! We have now wrapped our script inside Poetry. In addition to our todo.py script, we now have a pyproject.toml file that was generated by Poetry.

Key Takeaways

Wrap Python scripts with Poetry using init
Poetry uses a pyproject.toml file
Run commands using run
Enter a virtual environment shell using shell

Modern Python with Poetry

Ian Johnson — Sat, 06 Jan 2024 11:30:03 +0000

The intended audience here is advanced beginners. Maybe you've taken a Python course, or completed a tutorial. Maybe you have a basic, but solid understanding of the language. Maybe you have a Python script in place that does work for you. Great!

Now what? How do we move that understanding into real, working software?

In this series, we will be working with Python 3. Poetry requires at least Python 3.8.

Pipx

Pipx is a package manager that distributes Python code.

I am on a Mac, so I can install Pipx via Brew:

brew install pipx

Poetry

Poetry is a tool for dependency management and packaging. Traditionally a tool called virtualenv has been the standard in Python, but it seems like that is transitioning from requirements.txt to pyproject.toml.

pipx install poetry

Poetry provides some really great documentation to help get you started.
In this series, we will be using Poetry because it has a modern interface that adheres to the pyproject.toml standard. It will also enable us to learn how to take our Python projects from a beginner's script to a complete package.

DEV Community: Ian Johnson

I Installed Omakub Five Times So You Can Do It Just Once

TL;DR

Linux and Me

Linux and You

Creating a New Server in Hetzner

Initial System Setup

Installing and Configuring TigerVNC

Connecting to the VNC Server

Installing Omakub

Using the System

Conclusion

Giving Our Tests Context

Adding Context

Adding Custom Prefixes

Key Takeaways

Refactoring Our Tests

Task Deletion

Considering Our Tests

Refactoring Our Tests

Key Takeaways

Separating Domain Objects

todo.py

todo/task.py

todo/task_list.py

tests/test_task.py

tests/test_task_list.py

Separating Domain Objects

tests/test_task_list.py

Key Takeaways

Mocking the File

Adding a Mock

Moving Behavior to the TaskList

tests/test_task_list.py

todo/task_list.py

todo/task_list.py

todo/task_list.py

todo/task_list.py

tests/test_task_list.py

todo/task_list.py

todo/task_list.py

Key Takeaways

Covering Our To Do App with More Tests

DRYing Up Tests

tests/test_task.py

Testing Task.done

Programming by Wishful Thinking

Key Takeaways

Breaking the File Dependency

todo/task.py

Pulling Behavior Up

Breaking the Dependency

todo/task.py

todo.py

Stubbing Out a Fake File

tests/test_task.py

Key Takeaways

Testing Our Tasks

Adding Tests

tests/test_task.py

Testing Task.ls

Testing Task.add

Key Takeaways

Making Our Tasks Classy

Task

Task List

Wrapping Up

todo/task.py

todo.py

Key Takeaways

Re-Organizing Our To Do App

todo.py

todo/task.py

todo.py

todo.py

Key Takeaways

Wrapping a To Do App with Poetry

Starting Code

Make this a Poetry project

Run the Project

`todo.py`

`todo/task.py`

`todo/task_list.py`

`tests/test_task.py`

`tests/test_task_list.py`

`tests/test_task_list.py`

`tests/test_task_list.py`

`todo/task_list.py`

`todo/task_list.py`

`todo/task_list.py`

`todo/task_list.py`

`tests/test_task_list.py`

`todo/task_list.py`

`todo/task_list.py`

`tests/test_task.py`

Testing `Task.done`

`todo/task.py`

`todo/task.py`

`todo.py`

`tests/test_task.py`

`tests/test_task.py`

Testing `Task.ls`

Testing `Task.add`

`todo/task.py`

`todo.py`

`todo.py`

`todo/task.py`

`todo.py`

`todo.py`