Towards Effortless Python Configuration Files Version 2

Introduction

In the previous article, we managed to create a simulated property dataclass to manage configuration values for an application. However, it was basically just a template that we had to re-implement for each usage. In version 2, of my work I succeeded in implementing a reusable class.

• It requires a pair of decorators on the property definitions.
• Plus, the implementation requires a Sections data type that contains a Section entry for each configuration section.
• Each section requires a list of ConfigurationNameValue entries

Class Representation

The following class diagram shows the basic reusable class and the data structures needed for a developer to use this functionality.

Subclass Configuration Properties

The developer starts the process by subclassing Configuration Properties as follows.

BASE_FILE_NAME: str = 'config.ini'
MODULE_NAME:    str = 'version2properties'

class ConfigurationPropertiesVersion2(ConfigurationProperties, metaclass=SingletonV3):

    def __init__(self):
        self.logger: Logger = getLogger(LOGGER_NAME)

        super().__init__(baseFileName=BASE_FILE_NAME, moduleName=MODULE_NAME, sections=CONFIGURATION_SECTIONS)

        self._configParser.optionxform = self._toStr    # type: ignore

        self._loadConfiguration()

The call to super creates a fully qualified path to the configuration file. The code follows a subset of XDG Base Directory Specification. The code first tries XDG_CONFIG_HOME, then HOME, and finally uses the current directory if both fail. In line 13, the developer calls the protected method to get the configuration parser ready. Additionally, that call will

• Ensures the configuration file exists, if not it creates an empty one
• Creates missing sections
• Creates missing keys in the sections

The previous capabilities then result in lots of reusable code and bootstraps configuration files. Additionally, we make the class a Singleton to mitigate creation costs as we instantiate it across our application code base.

Define Sections

Notice the parameter sections in the super call. This definition of sections looks like this:

from codeallybasic.ConfigurationProperties import Sections

CONFIGURATION_SECTIONS: Sections = Sections(
    {
        SectionName('General'):  SECTION_GENERAL,
        SectionName('Database'): SECTION_DATABASE,
    }
)

The above is a dictionary where the key is a section name and the value is a section.

Define Section

A Section is just a list of ConfigurationNameValue entries. A ConfigurationNameValue is a data class with 2 values, a PropertyName and its default value. Here are our sections.

from codeallybasic.ConfigurationProperties import Section
from codeallybasic.ConfigurationProperties import ConfigurationNameValue
from codeallybasic.ConfigurationProperties import PropertyName

SECTION_GENERAL: Section = Section(
    [
        ConfigurationNameValue(name=PropertyName('debug'),                           defaultValue='False'),
        ConfigurationNameValue(name=PropertyName('logLevel'),                     defaultValue='Info'),
        ConfigurationNameValue(name=PropertyName('phoneyEnumByValue'),  defaultValue=DEFAULT_PHONEY_ENUM_BY_VALUE.value),
        ConfigurationNameValue(name=PropertyName('impostorEnumByName'), defaultValue=DEFAULT_IMPOSTOR_ENUM_BY_NAME.name),
    ]
)

SECTION_DATABASE: Section = Section(
    [
        ConfigurationNameValue(name=PropertyName('dbName'), defaultValue='example_db'),
        ConfigurationNameValue(name=PropertyName('dbHost'), defaultValue='localhost'),
        ConfigurationNameValue(name=PropertyName('dbPort'), defaultValue='5432'),
    ]
)

Notice that I have specified two new enumeration properties. One where we wish to persists is value and the other its name.

Here are there enumeration definitions.

class PhoneyEnumByValue(Enum):
    TheWanderer = 'The Wanderer'
    Mentiroso   = 'Mentiroso'
    FakeBrenda  = 'Faker Extraordinaire'
    NotSet          = 'Not Set'

    @classmethod
    def deSerialize(cls, value: str) -> 'PhoneyEnumByValue':

    @classmethod
    def deSerialize(cls, value: str) -> 'PhoneyEnumByValue':

        match value:
            case PhoneyEnumByValue.TheWanderer.value:
                phoneyEnum: PhoneyEnumByValue = PhoneyEnumByValue.TheWanderer
            case PhoneyEnumByValue.Mentiroso.value:
                phoneyEnum = PhoneyEnumByValue.Mentiroso
            case PhoneyEnumByValue.FakeBrenda.value:
                phoneyEnum = PhoneyEnumByValue.FakeBrenda
            case _:
                raise Exception('Unknown PhoneyEnumByValue')

        return phoneyEnum


class ImpostorEnumByName(Enum):
    Low       = 0.1
    Medium = 0.5
    High     = 1.0
    NotSet = -1.0

We'll see how these affect the property definitions in a developer class

Property Definitions

String properties are defined as follows.

    @property
    @configurationGetter(sectionName='General')
    def debug(self) -> str:
        return ''       # never executed

    @debug.setter
    @configurationSetter(sectionName='General')
    def debug(self, newValue: str):
        pass

What we have gotten rid of is the boiler plate to access the configParser to get and set the values. What we added was the configurationGetter and configurationSetter decorators. I am not going to cover the implementation details of the decorators and leave this as an exercise for the reader. These decorators take care of interacting with the configuration parser to get and set values. On setting values the configurationSetter decorator does the write-through.

Developers define integer properties as follows.

    @property
    @configurationGetter(sectionName='Database', deserializeFunction=int)
    def dbPort(self) -> int:
        return -1

    @dbPort.setter
    @configurationSetter(sectionName='Database',)
    def dbPort(self, newValue: int):
        pass

Notice that the configurationGetter decorator has an optional parameter. It is a function that takes the string property value and converts it to an appropriate typed value before returning to the caller properties. This can be applied to float properties.

Enumeration properties where we want to persist the enumeration name are defined as follows:

    @property
    @configurationGetter(sectionName='General')
    def impostorEnumByName(self) -> ImpostorEnumByName:
        return ImpostorEnumByName.NotSet      # Never executed

    @impostorEnumByName.setter
    @configurationSetter(sectionName='General', enumUseName=True)
    def impostorEnumByName(self, newValue: ImpostorEnumByName):
        pass

Besides using the appropriate decorators note that to persist the enumeration name use the enumUseName parameter and set it to True.

The following is an enumeration property where the developer want to persist its value. Notice that the setter decorator indicates that it is an enumeration. Additionally, notice that the developer must provide a deserialize method that can convert the value to the specific enumeration value.

    @property
    @configurationGetter(sectionName='General', deserializeFunction=PhoneyEnumByValue.deSerialize)
    def phoneyEnumByValue(self) -> PhoneyEnumByValue:
        return PhoneyEnumByValue.NotSet      # Never executed

    @phoneyEnumByValue.setter
    @configurationSetter(sectionName='General', isEnum=True)
    def phoneyEnumByValue(self, newValue: PhoneyEnumByValue):
        pass

Accessing and Modifying Properties

Accessing and modifying properties is exactly the same as in version 1.

    basicConfig(level=INFO)

    config: ConfigurationPropertiesVersion2 = ConfigurationPropertiesVersion2()

    logger: Logger = getLogger(LOGGER_NAME)

    logger.info(f'{config.debug=}')
    logger.info(f'{config.logLevel=}')
    logger.info(f'{config.phoneyEnumByValue=}')
    logger.info(f'{config.impostorEnumByName=}')
    logger.info('Database Properties Follow')
    logger.info(f'{config.dbName=}')
    logger.info(f'{config.dbHost=}')
    logger.info(f'{config.dbPort=}')
    logger.info('Mutate Enumeration Properties')
    config.phoneyEnumByValue = PhoneyEnumByValue.TheWanderer
    logger.info(f'{config.phoneyEnumByValue=}')
    config.impostorEnumByName = ImpostorEnumByName.Low
    logger.info(f'{config.impostorEnumByName=}')

The above snippet produces the following output.

INFO:Tutorial:config.debug='False'
INFO:Tutorial:config.logLevel='Info'
INFO:Tutorial:config.phoneyEnumByValue=<PhoneyEnumByValue.FakeBrenda: 'Faker Extraordinaire'>
INFO:Tutorial:config.impostorEnumByName='High'
INFO:Tutorial:Database Properties Follow
INFO:Tutorial:config.dbName='example_db'
INFO:Tutorial:config.dbHost='localhost'
INFO:Tutorial:config.dbPort=5432
INFO:Tutorial:Mutate Enumeration Properties
INFO:Tutorial:config.phoneyEnumByValue=<PhoneyEnumByValue.TheWanderer: 'The Wanderer'>
INFO:Tutorial:config.impostorEnumByName='Low'

Conclusion

The source code for this article is here. See the support class SingletonV3. See the implementation of ConfigurationProperties

The result of the implementation left me satisfied as a consumer of the code. I was able to get and set typed properties. It did not remove as much code as I had initially hoped for. However, it did provide me with reusable code. However, it did prompt me to write a Live Template in PyCharm to get me over generating individual properties.

Advantages

• Easy type safe access to application properties
• Reusable parent class for different implementations
• Data structure driven code to add new sections and configuration keys

Disadvantages

• Still lots of boiler plate code
• I have feeling that using decorators was a misuse of them

My next post I implemented what I call dynamic properties. It entirely removed all the boiler plate code and still maintained the aforementioned advantages.