DEV Community: dmikhr

A Step-by-Step Tutorial for Writing a Bash Installer Script for Your App

dmikhr — Mon, 26 May 2025 09:33:47 +0000

Intro

Consider the situation: you've spent a considerable amount of time developing an app, either as a personal project or as part of your job, and now it's time to share it with your target audience.

Note: here I focus mostly on console apps running on 🐧 Linux and 🍎 MacOS.

CLI apps can be executed either from a directory where the binary is located and it is fine at the development stage or if you share your app with your team, but as an end user, you want the app usage to be as straightforward as possible. Recently, I developed an app for extracting images from PDF documents pdfjuicer which was intended for both manual usage and being integrated into data processing pipelines.

Calling it like this

/Users/qwe/code/cli_apps/pdfjuicer/bin/pdfjuicer

is not a convenient way to run the app. I want users to be able to call the app just like any others, simply typing its name in the terminal

pdfjuicer

This way, it is convenient both for users and for calling the app from other software without bothering about the app location.

Problem analysis

Making the app run simply by calling its name in a terminal is achieved by adding your app path to the environment PATH. You can also add your app to software repositories like apt repositories, homebrew, etc. However, especially at the first stages it might be an overwhelming thing to do, and you might want to strike a balance between installation convenience and ease of solution maintainability. Here, I'll explore the option of developing a custom installation script.

However, here we need to look at the situation on a larger scale and consider the following questions:

How tech-savvy is our audience?
Is providing manual instructions enough?
What are the installation steps?
If choosing automated installation, should the whole process of installation be automated?
Can we test the installation process before actually installing an app?
What tool or language should be used for automation?

All these questions naturally arise one from another. First, we need to understand how complex the process is. We need to answer the question of whether our target audience has enough expertise to follow manual instructions or automation is needed. If an automated route is taken, maybe not all steps should be automated, giving user options or protecting user from unintended changes in system settings. And finally, if something goes wrong, can we find potential issues in advance by performing a test installation?

Working out the plan 🔧

So my next step was to get answers to these questions.

How tech-savvy is our audience?

You might think that users of console apps are mostly tech savvy, however, it is more like a spectrum. Consider CLI apps like ffmpeg or imagemagic which can be used by a wide range of users doing tasks like converting videos to certain formats or resizing photos from family photoarchive. I my case pdfjuicer was made for users who need to extract pages from PDF files as images with custom image size, format and thumbnails generation. The major target audience are educators and content creators. Users in these categories do not always want or have time for digging into the intricacies of application setup. Even if we consider users like devops engineers and software engineers, they also will appreciate quick and easy ways to install an app rather than delving into documentation and figuring out how to do it themselfes.

Is providing manual instructions enough? 📄

This question naturally arises from the previous one and my decision was not to go with the manual route. Even many tech-savvy users will prefer to save time by using a ready automated solution instead of installing the app manually or developing their own automation script. However, by taking the route which involves automated installation script, there is a problem of the installation process being hidden, which might concern some users: What will happen after installation? Where the app will be installed?

Solution is to print messages during installation showing which steps are performed now with additional information like file and folder names which are involved in the installation process, script status (start, finish, error); also adding test mode or dry-run mode will introduce additional layer of comfort and safety for a user avoiding concerns like What will actually happen after I'll run the script? and answers our question Can we test installation process before actually installing an app?

Also user can check the source code of an installation script to check the process in more detail. And finally, making script output formatted with ANSI Escape Codes will add more clarity to output messages, for example, making diagnostic information green and errors red.

Installation steps 💻

First, let's lay down steps which must be performed to install the CLI app with brief considerations for each step:

Find binary of an app - can be done by entering a path to the binary as a parameter to the installation script or app can be put into the same directory with the installation script.

Transfer binary to bin directory - at this step 2 new questions arise.

Transfer binary to system bin directory or in user-specific ~/bin?
Move binary or copy to bin directory?

Let's address both.

Since this app is not installed from official repositories and can be considered as custom software, user-specific ~/bin seems like a preferable choice. It will also avoid permissions issues that might arise when installing in the system bin and potential conflicts of overwriting binaries there. Overall, in this case, custom ~/bin simplifies app management and avoids risks of interfering with operating system.

Regarding whether it is better to move or copy binary - better to choose copying. In this way, if installation fails, it can be started again, otherwise, if the binary was moved, user will need to take the binary from ~/bin and put it into an original folder or download it again to retry the installation process.

Also, going with user-specific ~/bin directory creates extra optional step of creating such a directory if I did not exist.

Set permissions to binary - binary should be executable and execution permission will be limited to its owner since the app is installed in user user-specific binary directory, which can be achieved by using chmod u+x app_binary which makes user (owner) u right to execute x the binary.

Adding binary to PATH - this step makes app executable by simply calling it by name in terminal. To achieve that path to app must be added to shell configuration file (.bashrc, .bash_profile, .profile or .zshrc) using export command export PATH="$PATH:$target_bin_dir". In order to apply changes user will need to reload terminal which can be done by executing source ~/.bashrc (or targeting another appropriate file for particular user's shell config).

Should the whole process of installation be automated?

Here it is crucial to focus on steps which have the most uncertainty in the produced outcome and risk of interfering with the operation system functioning and let the user manage these steps. In this installation process, this is the last step of adding the app to PATH. It is better to allow user to edit shell config file manually, choosing where to add new path - precede other paths or make it the last one. It will also help to avoid risks of potential overwriting of existing settings or duplicating settings or writing into the wrong shell configuration file.

Which tool or language should be used for automation?

To make installation compatible among Linux, Mac OS and BSD systems, it makes sense to implement installation using bash script. However, my app is intended to be used not only by users but also by other software with scenarios of the app being installed on VPS or in Docker containers, including lightweight containers based on operating systems like Alpine Linux, which provides only sh but not bash.

Additionally, such an approach will improve overall compatibility across different operating systems and their distributives. However, it means that while implementing the installation script, it is important to be mindful about using syntax that is supported in sh and avoid bash specific syntax, which might mean potential tradeoffs like certain parts of code being more verbose.

What about other scripting languages?

One can ask that there are other options for scripting languages which will get this job done, like Python, Ruby or Lua and in terms of syntax, they will make the script easier to develop and easier to maintain if in the future it will require upgrades and extensions. However, we will pay the price of portability complexity. Choosing sh (Bourne shell) allows us to be as much agnostic about user's operating system environment as possible. If we choose bash, we add a layer of uncertainty: what if the host operating system has only sh installed?

With other scripting languages like Python, Ruby or Lua uncertainty rises even higher. Now we need to take care of interpreter version (Python 2 and 3), guess wether chosen scripting language is installed on the user's machine at all, manage situations like installing scripting language if it is not presented in operating system which will significantly increase installation complexity like extra installation steps, longer installation time, need to download additional packages, more risks of things go wrong due to extra steps in installation.

So in this case my choice is sh - the most available scripting language on UNIX-like systems, including minimal distributives used in Docker containers.

Implementation 🚀

Add ANSI escape codes for formatting messages

reset="\033[0m"
bold="\033[1m"
color_green="\033[32m"
color_red="\033[31m"

In this script we will use green and red colors, bold font and also reset code that will return output format to default.

Red color will be used to notify user that installation script is running in testing mode when no actual operations are performed. Green will be used to print message about installation success. Bold text will be used in large chunks of text to highlights key moments.

Set work directory

workdir=$(pwd)

Multiple flags

Since script has 2 flags --dev and --dry-run and they can be used either both or just one of them and in any order to check for current flag the following construction will be used which will check whether target flag is 1st or the 2nd argument:

if [ "$1" = "--flag" ] || [ "$2" = "--flag" ]; then
    ...
fi

Dev flag

In addition to regular installation, script supports another mode which is activated by --dev flag and intended to be used when you compile app from source.
App was written in Go and project contains bin directory where compiled binary is stored. In regular mode, installation script expects binary to be in the same directory with script, or in case of calling install.sh from source code (script is located in the root of the project) -- dev flag modifies path to binary adding current workdir/bin as folder.

if [ "$1" = "--dev" ] || [ "$2" = "--dev" ]; then
  binary_file="bin/pdfjuicer"
else
  binary_file="pdfjuicer"
fi

Dry-run

Check if this is a test/dry run:

dry_run=false
if [ "$1" = "--dry-run" ] || [ "$2" = "--dry-run" ]; then
    printf "${bold}${color_red}This is dry run, no actual actions will be performed. Use this for testing installation script.${reset}\n"
    dry_run=true
fi

If dry run is enabled show message about it in red color and with bold color using defined previously ANSI escape codes.

Setting paths to target path where it will be copied (~/bin) and original binary location.

target_bin_dir="$HOME/bin/"

source="$workdir/$binary_file"

Create ~/bin directory if it doesn't exist

echo "Creating directory $target_bin_dir for binary (if not exist)"
if ! $dry_run; then
  mkdir -p "$target_bin_dir"
fi

Copy binary to ~/bin

echo "Copying $source to $target_bin_dir"
if ! $dry_run; then
  cp "$source" "$target_bin_dir"
fi

Add execute permission using chmod, in the message to user script will show user name who will be able to execute binary using whoami command. If everything is running correctly it will show name of a user running the script.

echo "Adding execute permission for user $(whoami)"
if ! $dry_run; then
  chmod u+x "$target_bin_dir$binary_file"
fi

And finally installation script will show instructions for the final step which user will do manually: adding binary to PATH.

Calling installation script

The install script has 2 modes of running: real installation and test (dry-run) when the user sees messages for each step of an installation process but no actual operations are performed.

# testing script
bash ./install.sh --dry-run

# actual installation
bash ./install.sh

And in case of installing compiled binary from source code:

# testing script
bash ./install.sh --dev --dry-run

# actual installation from 
bash ./install.sh --dev

Such practice enables user to check if script is working properly: binary is found in correct folder, it will be copied to correct path, script doesn't fail during the execution.

Conclusions

Shipping app with installation script simplifies app installation compared to providing manual instructions. At the same time it's important to identify steps which are better to be performed manually for installation safety reasons. In this case user will manually add app to PATH to avoid potential mistakes and conflicts in shell configuration. Having an installation script provides a balance between manual setup and adding app to official repositories which can be time consuming especially at early stages of app development or when app is developed for internal usage in a company.

Complete source code of installation script can be found in pdfjuicer project here: install.sh

Extend Python VENV: Organize Dependencies Your Way

dmikhr — Thu, 11 May 2023 12:49:42 +0000

Introduction

Virtual environments are great way to organise development process by isolating project specific packages. In such way Python has a built-in tool venv for creating virtual environments. In this tutorial we are going to explore how to extend its functionality by implementing a feature that stores information about production and development related packages in separate requirements file.

This tutorial will guide you how to implement such feature, explaining logic behind the solution and purpose of each function in the script. Prerequisites for this tutorial are basic knowledge of Python and experience with virtual environments. Examples from this tutorial requires Python 3.5+.

Tools

There are different ways to split dependencies in Python. Poetry provides versatile options to configure Python app including staging: production, development and testing. It also provides a lot of other useful options for Python developers.

But, despite powerfull functionality there might be reasons to use simpler alternatives like venv. If you are working on a complex project, planning to publish your project on python repository like PyPi or it's historically based on Poetry then staying with Poetry makes sense.

Other alternatives worth mentioning are virtualenv and pipenv. In data science and scientific computing also conda has substantial popularity. While these tools provide options to customize your virtual environment it's not always needed.

Simpler alternatives

If you are working on a pet project, simple service or just don't want to complicate code with extra third party tools then using venv can be an ideal solution. It's built-in Python library and has limited set of commands which makes it easy to learn. In the simplest scenario there no need to learn venv at all. Just navige to the project directory and create virtual environment:

python -m venv .venv

What it will do is to call python module venv and create virtual environment in .venv directory under the current directory.

Other common names for virtual environment folders are venv, env, .env. Now this folder contains executables for python and it will store installed packages. Let's now install some packages. But before we can do that virtual environment must be activated

source .venv/bin/activate

When you no longer need virtual environment active deactivate it by simply typing deactivate.

In our hypothetical scenario let's assume that we are building a web app on Flask, but before that let's check that we are really in the virtual environment by calling which python. It will produce a path to currently used python executable. If path leads to the current directory and ends like .venv/bin/python then environment is activated and we are good to go.

Installing Flask

pip install Flask

Packages especially complex ones like Flask doesn't exist in isolation, it depends on other python packages that will be installed alongside. To ensure that this is the case call pip freeze. It shows a list of installed packages in our virtual environment with their corresponding versions.

A good practice is to maintain a text file requirements.txt with relevant dependencies so other developers can setup environment for work by installing all required packages by calling pip install -r requirements.txt.

Creating requirements.txt

pip freeze > requirements.txt

Apart from packages that are required for running an app usually there is a need in tools that are used in development and testing. For this example we choose a package for testing pytest, for maintaining code style in accordance with PEP8 let's install flake8 (for manual check) and autopep8 that will format our code properly.

pip install pytest flake8 autopep8

Type pip freeze to see that now there are more packages. And finally save to file: pip freeze > requirements.txt

Keeping dependencies separate

Here we came to the point where venv simplicity gives us a bit of inconvenience. Currently installed packages can be divided into 2 groups - one group of packages related to Flask are necessary for running the app. However it's redundant to install test packages into production environment.

The solution is to maintain two separate files - one for app packages (requirements.txt) another for development packages (requirements-dev.txt). In this case testing packages are also stored in -dev file like for example in Flasgger. It's possible to split dependencies manually removing them from original requirements.txt and saving them into requirements-dev.txt.

However if new packages will be installed this process should be repeated. At this point we have two options: either moving to more advances third-party virtual environment management tools or we can automate this process.

Automation

Basically this process consists of two steps: save current packages list into file: pip freeze > requirements.txt and then some script sould sort packages keeping app related packages in requirements.txt while moving development packages to its development counterpart.

This sequence can be executed manually from terminal however on UNIX based operation systems including MacOS there is more convenient solution: make utility. Basic usage is make command-name. This utility when called search for file named Makefile and execute sequence of commands under command name section.

Let's create our Makefile and command

freeze:
    pip freeze > requirements.txt
    python -m split_dependencies.py

Calling make freeze will save dependencies into requirements.txt and call python script to split dependencies between separate files. To run it correctly first let's create the basis for a future script. Create file split_dependencies.py and put the following code into it:

if __name__ == "__main__":
    pass

Now run make freeze. If everything has been done right terminal will produce the list of executed commands:

pip freeze > requirements.txt
python -m split_dependencies

At this point the following file structure is presented in working directory:

.
├── .venv
├── Makefile
├── requirements.txt
└── split_dependencies.py

Script architecture

It's time to start developing the script. But before coding let's think about things that our script should be doing. It's going to consists of multiple functions and main executable function run() that will be responsible for executing other functions.

It's a good practice to follow single responsibility principle while coding function. So, every function will have only one task. For example list of packages should be loaded from file (load_requirements) but for cleaning data from newline character another function can be used. Then to obtain cleared list of packages these function will be called in a sequence one passing data to another forming data pipeline.

A list of functions with their interfaces that will be used in the script separated by their responsibility:

Load

load_requirements(fname="requirements.txt")
clean_list(items: list) - remove newline character from each string loaded from requirements.txt

Logic

is_dev_requirement(item: str) - check if current package is related to development category
is_prod_requirement(item: str) - check if current package is related to production (or app running) category
extract(criteria: Callable, data: list) - filter list of packages by criteria (is_dev_requirement, is_prod_requirement)

Save

prepare_data(data: list) - prepare data for saving (joining list of filtered packages to string)
save_requirements(fname: str, data: str) - save requirements to file (requirements.txt and requirements-dev.txt)

Since packages related to development and testing usually much less in quantity compared to app related packages it makes sense to store information about them for filtering purposes. Here is an example of a list with packages that might be used for development:

DEV_REQUIREMENTS = ["autopep8", "black", 
                    "flake8", "pytest-asyncio", 
                    "pytest", "Faker"]

Also during the coding we are going to use python typing to make code more readable in terms what datatypes are used and your IDE can use this information for providing type hinting.

Further type checkers like mypy can be used to enable static typing (Python by design is dynamic typing language). Since Python 3.5 typing is a built-in feature. However some additional functionality for typing can be achieved by using typing package (shipped with Python distribution).

Coding

Using developed script architecture let's code the script. It's going to be managed by run() function where script logic will be implemented.

Step 1: loading list of packages from requirements.txt and cleaning it from newline characters.

Step 2: forming two lists of packages - for production and development

Step 3: preparing data and saving it into two text files.

Here is a final version of a script. Feel free to experiment with it:

from typing import Callable

DEV_REQUIREMENTS = ["autopep8", "black", 
                    "flake8", "pytest-asyncio", 
                    "pytest", "Faker"]


def run():
    dependencies = clean_list(load_requirements())
    dev_dependencies = extract(is_dev_requirement, 
                               dependencies)
    prod_dependencies = extract(is_prod_requirement, 
                                dependencies)
    save_requirements("requirements-dev.txt", 
                      prepare_data(dev_dependencies))
    save_requirements("requirements.txt", 
                      prepare_data(prod_dependencies))


def extract(criteria: Callable, data: list) -> list:
    return list(filter(lambda item: criteria(item), data))


def is_dev_requirement(item: str) -> bool:
    package_name, _ = item.split("==")
    return package_name in DEV_REQUIREMENTS


def is_prod_requirement(item: str) -> bool:
    return not is_dev_requirement(item)


def load_requirements(fname="requirements.txt") -> list:
    with open(fname, "r") as f:
        dependencies = f.readlines()
    return dependencies


def save_requirements(fname: str, data: str) -> None:
    with open(fname, "w") as f:
        f.write(data)


def prepare_data(data: list) -> str:
    return "\n".join(data) + "\n"


def clean_list(items: list) -> list:
    return list(map(lambda item: item.strip(), items))


if __name__ == "__main__":
    run()

Commentary

load_requirements - loads packages from file to list using readlines method
clean_list removes newline character from each string through map which applies item.strip() to each element of items list
extract - filters initial list using built-in function filter
is_dev_requirement - takes a string with information about package (example: Flask==2.2.3), extract package name and its version and checks whether package in the list of development packages
is_prod_requirement - check whether package is production related, calls is_dev_requirement and returns negated result
prepare_data joins list of packages by newline character and adds one at the end since according to convention in UNIX based opearation systems
save_requirements - writes data to file, called twice since packages are saved into two separate files. Writing is performed with w flag since this is text data
run - implements script logic and ensures data exchange between functions

Now if someone wants to work on the project in order to set it up dependencies should be installed from both requirements files:

pip install -r requirements.txt
pip install -r requirements-dev.txt

Conclusions

In this tutorial we looked at the way of extending venv functionality with a simple Python script that can be shipped with a project. Regarding the options on extensibility of this script there are plenty of things that can be done. For example splitting packages into three parts instead of two by introducing testing stage or applying sorting to make package list more convenient to navigate. The latter can be especially handy in large projects with lots of dependencies.

You can find the files and code used in this tutorial here: https://github.com/dmikhr/split-dependencies-demo

Building CLI time tracker with Python

dmikhr — Wed, 19 Oct 2022 14:49:58 +0000

So, why another time tracker?

There are plenty of time tracking apps and services, however it doesn't mean there is no room for a new app in this niche. Like there are established clothing brands with mass production capabilities doesn't eliminate custom made clothing brands. Making time tracker app as a personal project was both a personal challenge and opportunity to create an app that is tailored to specific time management techniques like a tailor can make a suit that's made by your specific requirements.

However custom made doesn't mean "only for yourself". Like they say "If you had an idea it's likely someone came up with the same idea before". Same situation here — if you developed particular way for doing things might mean that you simply discovered an approach that has been adopted by some people for a long time. It might be a niche approach but nevertheless your are not likely to be alone. This way of thinking was encouraging for me since it's more motivating when you build something that can be useful not only for yourself but for other people as well.

My first approach to time tracking was quite straightforward: jot down on a piece of paper or a sticker when task started, then put finish time and a comment what task was about. Initially time tracking was purely manual.

Though I enjoy taking notes manually and handwriting in general (especially with a good fountain pen) it's hard to do much with these data after it was written. Next step was tracking time in digital form but still manually using sticky notes app:

Looks neater and now it's possible to do something with these data. So idea to write a script that going to show amount of time spent on a specific task by supplying time slots as parameter was originated.

To automate time spent on each activity a ruby script was written that took a string with time ranges when task was active:

ruby st.rb "10:30 - 11:25 11:45 - 12:30"

Now it's possible to do something with data about time spent on different tasks, but it was still a semi-manual approach. Track time manually, then pass it in a specific format to the script didn't feel convenient and was more like a compromise. So the idea to build an app to track time was born.

Prototyping phase

Having all these in place the question was what features should be included in the app. The first obvious step was to get rid of manually typing time ranges, so the idea was that user evoke the app with a specific flag and task name that indicates whether task is active or finished, while app tracks time itself.

There were lots of ideas what features such application might have. But first thing first: before building full featured app let's make a prototype, a proof of concept. Then it's going to be clear whether it's gonna stick — am I going to use it consistently? If app proves to be handy further development efforts are justified.

Overall the following minimal functionality must be in an app:

add new task
remove task
start tracking time for a given task
finish time tracking

Also feature that shows time stats (how much time has been spent on each task during the day) was added.
The idea was to build a console app. CLI apps have something in common with writing on paper - it's basic, nothing extra, but this is an appeal for both. Also doing software development projects made me get used to command line interfaces, so CLI app was a no brainer option.

Further app architecture was built in the same manner - identify app component and separate them:

console interface
app logic (time tracking, statistics)
database management (creating database, add, remove data)

Console interface

Console interface was built with argparse. However it worth noting that alternatives were considered and one that stands out is click. This module takes a different approach to building command line interfaces - instead of building methods for processing incoming arguments it offers a way to build CLI interface by decorating functions that will be evoked by a given flag (code example). While click seems like a promising library, for initial app version I decided to stay with tried and tested argparse. However in the future when CLI interface will comprise more features decorator approach that is used by click seems to be more scalable.

The following commands were included in the initial app version:

options:
-h, - help show this help message and exit
-s START, - start START
Start task by providing its name
-f, - finish Finish current task. Task also can be finished by
starting a different task
-a ADD, - add ADD Add new task
-r REMOVE, - remove REMOVE Remove the task
-l, - list List of all tasks
-st, - stats Stats about tasks

Database

With database design — let's keep it simple, one table for storing tasks and another for time tracking. Each working session spent on a specific task will be stored as a record in work_blocks table. Each task may have any number of work blocks, including none if task hasn't been tracked yet. ER-diagram of the app database is presented on the following picture:

As a database engine SQLite has been chosen — lightweight, data is stored in a single file, overall good option for a first version of an app. For database management an SQLAlchemy ORM was chosen to separate data manipulation from specific database implementation. Since it's a utility with local database and large amounts of data are not expected, so moving to distributed databases was not in a plan, auto incremented id was chosen for primary keys instead of uuid.

Storing timestamps
One issue with SQLite — it doesn't have datetime data type and this is a time tracking app! The solution was to store time as a Unix timestamp using Integer data type. This approach is recommended by SQLite documentation, since it can support storing numbers in up to 8 bytes which corresponds to bigint in database engines such as PostgreSQL. Storing time in Unix timestamp might bring some inconveniences down the road with features like obtaining daily stats, however it doesn't seem like a real roadblock for further development.

Database location and schema
To store database user files directory was used. The issue here is that each OS has it's own path for this directory. But there is no problem, Python is known for having libraries almost for everything and identifying user directory is no exception: for this job appdirs has been chosen. When app starts for the first time it will check whether database exists and if no database file was found it will create an empty database with the following schema:

class Task(Base):
   __tablename__ = 'tasks'

   id = Column(Integer, primary_key=True)
   name = Column(String)
   description = Column(String)
   children = relationship("WorkBlock",
                           cascade="all,delete",
                           backref="tasks")

class WorkBlock(Base):
   __tablename__ = 'work_blocks'

   id = Column(Integer, primary_key=True)
   task_id = Column(Integer, ForeignKey("tasks.id"))
   start_time = Column(Integer)
   finish_time = Column(Integer)

App logic

App logic consists of methods that evoked through CLI interface and interact with database as well as a collection of private methods that serve auxiliary purpose.

Start task - app creates a record in work_blocks table related to a given task that is stored in tasks table with a unix timestamp of a current time in start_time field while finish_time field remains empty.

Finish task — no arguments are required for this options. If there is an active task (finish_time==NULL) current time in Unix timestamp format will be recorded in the finish_time time, thus session for a current task will be ended.

Add new task — in order to track tasks there should be a way to store them. This options requires task name as input and record data in tasks table.

Remove task — if task is no more needed it can be removed. In the current version removing the task from tasks table triggers 'delete on cascade' and all corresponding work_block will be removed as well. In the future versions of an app an additional option might be added that's going to archive tasks instead of complete deletion. The motivation behind this is that even if task is no more needed, storing data about it can be useful for further data analysis. For example for understanding performance bottlenecks that can occur while working on similar tasks.

List all tasks — show all tasks that has been added to an app. If one of these tasks is active there will be (in progress) status near it. For example:

Project_work
Exercising (in progress)
Flask_API_project
Some_task

Statistics — shows only tasks that were active today. For each task time spent on a it is shown in two formats: hr:min and in decimal format. For example: 1:20 (1.33). The reason behind presenting time in decimal format is that it can be useful if you track time in spreadsheets and it's easier to use time in such format for further analysis like counting total time spent on a task during the last week, month, etc. I've been using this approach for identifying issues with prioritisation - track time on each task/project and at the end of the week see how much time was spend on each. If there is an outlier (task that consumed way more time that others) and there is no clear reason why it has happened (close deadline, high importance of the task) that it's the reason to change priorities and dedicate more time to other responsibilities.

Testing

Testing was implemented using PyTest framework. Test data was generated with Faker. In order to isolate test data for each test fixtures were used. Fixtures are convenient way to generate test data and ensure its consistency among different tests. If you're not familiar with them you may check this example to get an idea how it works.

Since application features mostly use database it was important to use a test database. In the simplest case requests to database can be stubbed, but then most test won't be useful in this particular case.

First idea was to create temporary SQLite database file each time tests run and delete it after they're finished, however this approach didn't feel right. Thankfully it's possible to create SQLite database in memory. This approach doesn't require any read/write operations with file system, no need to think about deleting database file after testing. In case of in-memory database memory will be cleared automatically after connection is closed.

While choosing in-memory database for testing it's important to consider how much test data is planned to put into database. In this case it was about dozens of records at most, so exceeding memory limit was not an issue.
To establish connection and create in-memory sqlite database:

from sqlalchemy import create_engine
engine = create_engine(f"sqlite+pysqlite:///:memory:")

Testing approach was to cover with tests all public methods. Mostly I tried to follow TDD approach by writing tests first and then implementing a feature, followed by several iterations of test and implementation improvements and code refactoring.

Giving a proper name to project

When you're working on a personal project it might seem that project name is not a big deal. However giving a project a proper name achieves at least two important goals. First, when personal project has a name it feels subconsciously that I began to take it more seriously, not just an exercise for improving programming skills but more like I'm building a tool or a product that might be useful not only for personal purpose but for other people too. Secondly, if you decide to share your project with community via repository such as pypi or npm you will need a unique name for a project. In case of this project I gave it a name tiempo-tracker (tiempo means time in Spanish).

Conclusions

So far I found myself using this app consistently during my work process and new ideas were born on how to improve an app and what features might be useful to add. However it's important not to be overwhelmed with implementing all ideas at once but instead keeping track of them and planning on implementing them when they're truly needed.

Pet-projects is something that can be useful and educational. Using a self-developed app tailored to your own requirements is an interesting experience, however personal projects take time and the more features app has the more maintenance it requires: more tests, bug-fixes. It's important not to be overly consumed in this activity since a lot of other things requires our attention both professional responsibilities and personal. However it's something that easier to be said than done especially when you're passionate about your project.

A short demo is available on Asciinema

You can check the source code of tiempo-tracker on Github.