Read cloud specific configuration from configuration files

Rahul Kashyap — Thu, 10 Oct 2024 05:54:09 +0000

Configuration specific to cloud provider can be separated in a config file by setting cloud provider name as a section.

Define a configuration file

Create a configuration file - config.ini in the src package and define cloud provider specific configuration similar to below.

[aws]
bucket_name: test-aws-bucket

[gcp]
bucket_name: test-gcp-bucket

Read the configuration in the code

Read cloud provider from environment variable.

cloud_provider = os.environ.get('CLOUD_PROVIDER')

Declare a config parser in python and read the configuration file

config = configparser.ConfigParser()
config_path = os.path.join(os.path.dirname(__file__), 'config.ini')
config.read(config_path)
bucket_name = config.get(cloud_provider,'bucket_name')

In this way, we can separate cloud provider specific configuration in config files.

Please feel free to comment with any suggestions/feedback.

Developing cloud agnostic application

Rahul Kashyap — Thu, 10 Oct 2024 05:44:05 +0000

Introduction

Recently, i started working on a personal project where i want to build an application which is cloud agnostic - i.e. it can be deployed on any cloud provider with minimal/no code changes. Primary requirement is to separate business logic with cloud provider specific logic.

In this post, i want to share the approach which was followed.

Create an interface which has abstract methods for performing operations in the cloud.
Create cloud provider specific classes which are subclasses of this interface and provide implemenation for the abstract methods.
Create a separate class/method which will return the cloud provider implementation based on some condition. Factory Pattern
The calling class will use the object from the above class/method and call its methods.

Code

Below code uses python

Interface with abstract methods

from abc import ABC, abstractmethod

class IObjectStorage(ABC):

    @abstractmethod
    def upload_object_to_bucket(self, file_name, file_content):
        _raise an error that method is not implemented_

Create cloud provider specific implementation

class AWSObjectStorageConnector(IObjectStorage):

    def __init__(self, bucket_name):
       _Initialize a s3 client using boto3 and initialize a variable using bucket name_

    def upload_object_to_bucket(self, file_name, file_content):
        _Implement the logic to upload the file to AWS S3 bucket_

Create a method to get the specifc cloud provider implementation class object - Factory method

This method takes a cloud provider variable which will be passed from the calling method

def get_object_storage(cloud_provider, bucket_name) -> IObjectStorage:

    if cloud_provider == 'aws':
        return AWSObjectStorageConnector(bucket_name=bucket_name)
    else:
        raise ValueError(f'Unsupported cloud provider: {cloud_provider}')

Call the factory method to get an instance of the object

cloud_provider variable will be read from an environment variable passed as input. This ensures that the same logic works fine with different cloud providers.

object_storage_connector = get_object_storage(cloud_provider=provider, bucket_name=bucket_name)

Please feel free to comment with any suggestions or feedback.

DEV Community: Rahul Kashyap