Python one-liner for reading files

qoqosz — Mon, 03 Oct 2022 11:08:46 +0000

I was today years old when I realized that instead of a lengthy open context manager:

with open('myfile.txt') as f:
    print(f.read())

I can use read_text():

from pathlib import Path

print(Path('myfile.txt').read_text())

Of course, if you factor in the extra import it is no longer a one-liner. Anyway, it was a nice little discovery for me.

Config class aware of your Python application changes

qoqosz — Tue, 27 Sep 2022 20:08:16 +0000

Minimalistic approach

When my application requires a centralized place for a config, I usually start with a dedicated file holding a dict() object:

# file: config.py
config = dict()

The usage across the application is really simple - import and use it:

# file: app.py
from config import config

config['db'] = 'uat01'
config.update(host='localhost', username='user', 
              password='secret pass')

Such config has a nice property - it is a singleton without any implementation tricks.

However, I don't find it particularly handy to use dict syntax of accessing elements with []. Quick fix would be to either:

1. Create aliases for __*attr__ methods:

# file: config.py
class Config(dict):
    __getattr__ = dict.get
    __setattr__ = dict.__setitem__
    __delattr__ = dict.__delitem__

config = Config()

2. Use SimpleNamespace from types however, it lacks update() equivalent method.

When things are changing

Let's stick to the first choice and have a look at the following scenario:

# file: app.py
from config import config
from types import Optional

def fibonacci(
    count: int,
    first: Optional[int] = None,
    second: Optional[int] = None
) -> int:
    a, b = first or config.first, second or config.second
    for _ in range(count):
        a, b = b, a + b
    return a

# file: main.py
from app import fibonacci
from config import config

if __name__ == '__main__':
    config.update(first=1, second=1)
    print(f'{fibonacci(6)=}')

    config.update(first=3, second=5)
    print(f'{fibonacci(6)=}')

The result won't change between two calls to our fibonacci function.

fibonacci(6)=8
fibonacci(6)=8

Make it aware of the changes

# file: config.py
import importlib
import inspect

class Config:
    _data = dict()
    _referrers = set()

    def __getattr__(self, name):
        caller = inspect.currentframe().f_back
        module_name = caller.f_globals['__name__']

        if '__main__' not in module_name:
            self._referrers.add(module_name)

        try:
            return self._data[name]
        except KeyError:
            raise AttributeError('no attr %s', name)

    def __setattr__(self, name, value):
        self._data[name] = value

    def _reload_modules(self):
        for module_name in self._referrers.copy():
            module = importlib.import_module(module_name)
            importlib.reload(module)

    def __repr__(self):
        return 'Config(%r)' % self._data

    def update(self, reload=True, **kwargs):
        for k, v in kwargs.items()
            self.__setattr__(k, v)

        if reload:
            self._reload_modules()

config = Config()

# set defaults
config.update(first=0, second=1)

Now, running main.py yields the following result:

fibonacci(6)=8
fibonacci(6)=55

What's going on?

Ok, the newly defined Config class is a bit lengthy and requires some explanation.

Let's go through the code:

import importlib
import inspect

These two imports will come in handy in a minute, bare with me.

class Config:
    _data = dict()
    _referrers = set()

The Config class encapsulates two elements: 1. _data, the actual key-value container and 2. _referrers, the set of other modules that import config and access its entries.

    def __getattr__(self, name):
        caller = inspect.currentframe().f_back
        module_name = caller.f_globals['__name__']

        if '__main__' not in module_name:
            self._referrers.add(module_name)

Custom __getattr__ enables dot access syntax. Additionally, it is the moment in which I record modules that refer to config. When config is updated, it will allow me to let those modules know that something has changed and it is time to reload.

The rest is pretty straightforward. In:

    def _reload_modules(self):
        for module_name in self._referrers.copy():
            module = importlib.import_module(module_name)
            importlib.reload(module)

I loop over related modules and reload them.

Finally, at the end of the file, I have to set default values for entries that are used in other parts of the code. The other option would be to load a config file and parse it.

Also, by design I decided that only call to update refresh referred modules. Setting them one by one with __setattr__ does not have this behavior.

What is your approach to config?

Python debug function

qoqosz — Fri, 23 Sep 2022 06:53:18 +0000

In Rust you find the dbg! macro that prints to what a given expression evaluates to. This on its own wouldn't be as helpful. Thus, additionally the macro returns the value of the evaluated expression.

It is possible to mimic this functionality in Python. You may find the original implementation on Raphael's blog. The one I use, is roughly:

import inspect
import sys
import typing

_ExpType = typing.TypeVar('_ExpType')


def dbg(exp: _ExpType) -> _ExpType:
    for frame in inspect.stack():
        line = frame.code_context[0]

        if 'dbg' in line:
            start = line.find('(') + 1
            end = end if (end := line.rfind(')')) >= 0 else len(line) 

            print(f'[{frame.filename}:{frame.lineno}] {line[start:end]} = {exp!r}', 
                  file=sys.stderr)
            break

    return exp

Whether in it's a *.py file or in a notebook, it works great. Exemplary output:

>>> a, b = 1, 2
... dbg(a + b)
[(...):2] a + b = 3
3