DEV Community: Samuel Arogbonlo

Dynamic HTTP Server Configuration in Go with gorealconf

Samuel Arogbonlo — Thu, 23 Jan 2025 09:55:35 +0000

Managing configuration changes safely and efficiently is a critical challenge in today's cloud-native world. When building distributed systems, we often need to update configurations without downtime, ensure type safety, and roll out changes gradually. These requirements led me to create gorealconf, a library that brings robust, type-safe configuration management to Go applications.

Key Challenges

Before diving into gorealconf's features, let's understand the core challenges in modern configuration management:

Applications often require restarts to apply configuration changes
Type safety is frequently sacrificed for flexibility
Rolling out changes safely across distributed systems is complex
Validating configuration changes and handling failures gracefully is difficult
Monitoring and tracking configuration changes becomes unwieldy at scale

gorealconf addresses these challenges through thoughtful design and powerful features that make configuration management safer and more efficient.

Core Features and Implementation

Type Safety Through Go Generics

One of gorealconf's fundamental strengths is its use of Go generics to ensure type safety at compile time:

type DatabaseConfig struct {
    MaxConnections int           `json:"max_connections"`
    ReadTimeout    time.Duration `json:"read_timeout"`
    WriteTimeout   time.Duration `json:"write_timeout"`
}

// Create a type-safe configuration instance
cfg := gorealconf.New[DatabaseConfig](
    gorealconf.WithValidation[DatabaseConfig](validateConfig),
    gorealconf.WithRollback[DatabaseConfig](true),
)

This approach eliminates runtime type errors while maintaining a clean, intuitive API.

Real-Time Configuration Updates

gorealconf enables zero-downtime configuration changes through its watch mechanism:

changes, _ := cfg.Watch(context.Background())
go func() {
    for newCfg := range changes {
        updateDatabaseConnections(newCfg)
    }
}()

Changes propagate automatically to all components, ensuring consistency across your application.

Gradual Rollouts with Safety Controls

One of gorealconf's most powerful features is its support for gradual rollouts:

rollout := gorealconf.NewRollout[FeatureConfig](cfg).
    WithStrategy(gorealconf.NewPercentageStrategy(10)).
    WithValidation(validateFeature).
    WithRollbackThreshold(0.01) // Rollback if error rate exceeds 1%

This allows you to safely test configuration changes with a subset of your instances before full deployment.

Real-World Applications

Dynamic HTTP Server Configuration

Here's how gorealconf can manage a production HTTP server:

type ServerConfig struct {
    Port         int           `json:"port"`
    ReadTimeout  time.Duration `json:"read_timeout"`
    WriteTimeout time.Duration `json:"write_timeout"`
}

func main() {
    cfg := gorealconf.New[ServerConfig]()

    // Initialize server with configuration
    server := createServer(cfg.Get(ctx))

    // Watch for configuration updates
    changes, _ := cfg.Watch(context.Background())
    go handleConfigChanges(changes, server)

    // Start server
    server.ListenAndServe()
}

Feature Flag Management

gorealconf excels at managing feature flags and A/B testing:

type FeatureFlags struct {
    NewUI          bool    `json:"new_ui"`
    BetaFeatures   bool    `json:"beta_features"`
    RolloutPercent float64 `json:"rollout_percent"`
}

rollout := gorealconf.NewRollout[FeatureFlags](cfg).
    WithStrategy(gorealconf.NewCompositeStrategy().
        Add(gorealconf.NewRegionBasedStrategy(regions)).
        Add(gorealconf.NewPercentageStrategy(20)),
    )

Getting Started

Install gorealconf using:

go get github.com/samuelarogbonlo/gorealconf

The library includes comprehensive examples in the examples/ directory, covering:

Basic usage
Multi-source configuration
Gradual rollouts
Complete application setups

What's Next?

The future roadmap for gorealconf includes:

Enhanced encryption support
Additional configuration sources
Advanced rollout strategies
Improved observability features
Others etc.

Join the Community

gorealconf is open source and welcomes contributions. Whether you fix bugs, add features, or improve documentation, your help makes the library better for everyone.

Check out the GitHub repository to get started, and join our community discussions to share your ideas and experiences. Also the package details here

Managing Kubernetes Objects: A Comprehensive Guide

Samuel Arogbonlo — Tue, 03 Dec 2024 10:53:44 +0000

Managing Kubernetes Objects: A Comprehensive Guide

Kubernetes, often abbreviated as K8s, is a powerful platform for orchestrating containers. At its core, Kubernetes operates on the concept of objects, representing your cluster's desired state. To manage these objects effectively, you need to understand their lifecycle, configuration, and the tools available for their manipulation.

Understanding Kubernetes Objects

Kubernetes objects are persistent entities that represent the state of your system. Each object is identified by a unique name and identifier (UID) assigned at creation. This UID remains consistent throughout the object's lifecycle, even if its name changes.

These objects are defined declaratively in YAML or JSON configuration files, with YAML being the preferred format due to its readability. Storing these files in a version-controlled repository is a best practice, enabling traceability and collaboration.

Key Components of a Kubernetes Object

Metadata: Includes essential information such as name, namespace, labels, and annotations.
Spec: Defines the desired state of the object.
Status: Represents the object's actual state, as Kubernetes observes.

Labels and Selectors

Labels are key-value pairs attached to objects. They provide a powerful mechanism for identifying and grouping related resources. For example, you can assign a label like app: nginx to group all Nginx-related pods. Using selectors, Kubernetes enables operations on subsets of objects matching specific label criteria.

Deploying Workloads with Kubernetes Objects

The Role of Pods

At the heart of Kubernetes workloads are pods, the smallest deployable units. A pod encapsulates one or more tightly coupled containers, along with shared storage, networking, and lifecycle. While managing individual pods is possible, it becomes impractical in large-scale systems.

Controller Objects: Scaling and Managing Pods

To simplify management, Kubernetes provides controller objects that maintain the desired state of pods. These include:

Deployments: Ideal for stateless applications, ensuring high availability and scalability by managing the lifecycle of pods. Deployments monitor the desired and current states, recreating pods if they fail.
StatefulSets: Designed for stateful applications, ensuring predictable deployment and scaling with persistent identifiers for each pod.
DaemonSets: Ensure that a copy of a pod runs on every node, commonly used for tasks like logging or monitoring.
Jobs and CronJobs: Handle batch processing and scheduled tasks, respectively.

Namespaces: Logical Isolation and Resource Organization

Namespaces provide logical isolation and resource organization within a cluster. They are particularly useful in multi-team environments, enabling better organization and resource control.

Use Cases: Different environments (e.g., dev, test, prod) or multi-tenancy setups.
Best Practice: Avoid hardcoding namespaces in YAML files. Specify them dynamically using the kubectl CLI to enhance flexibility and enable cluster-wide updates.

Services: Enabling Communication Between Pods

Kubernetes services abstract and expose pods to the network, enabling seamless communication between application components. Services work with label selectors to route traffic to the appropriate pods, bypassing the ephemeral nature of pod IPs.

Types of Services

ClusterIP: Exposes the service only within the cluster.
NodePort: Exposes the service on each node’s IP at a static port.
LoadBalancer: Integrates with cloud provider load balancers for external exposure.
ExternalName: Maps a service to an external DNS name.

Services also create endpoint resources and assign virtual IP addresses to maintain connectivity.

Managing Storage in Kubernetes

Given the ephemeral nature of pods, handling persistent data is a critical challenge. Kubernetes addresses this with volumes, which provide durable storage for containers.

Types of Volumes

EmptyDir: Temporary storage shared among containers in a pod. Erased when the pod is deleted.
PersistentVolumes (PV) and PersistentVolumeClaims (PVC): Abstract external storage systems (e.g., NFS, cloud disks) for durability beyond pod lifecycles.
ConfigMap and Secret: Store configuration data and sensitive information securely. These can be mounted as volumes.

When defining pod specifications, you must explicitly mount volumes for each container. Properly designed volume configurations ensure data integrity and simplify scaling.

Best Practices for Managing Kubernetes Objects

Version Control Your YAML Files: Use Git or similar tools for traceability and collaboration.
Leverage Declarative Configurations: Use tools like kubectl apply and GitOps frameworks to maintain desired states.
Monitor and Audit: Use tools like Prometheus and Grafana for observability and anomaly detection.
Use Namespaces Wisely: Organize resources logically while minimizing complexity.
Set Resource Limits and Quotas: Define CPU, memory, and storage limits to prevent resource contention.

Conclusion

By mastering Kubernetes objects and adhering to best practices, you can efficiently manage complex workloads, ensure high availability, and scale applications seamlessly. A strong understanding of these principles will make Kubernetes a reliable ally in your infrastructure journey.

How To Setup Password Hash Synchronization In Microsoft Azure

Samuel Arogbonlo — Sat, 30 Nov 2024 14:56:34 +0000

Azure is quickly becoming one of the most efficient cloud providers in the space and given that I am a DevOps engineer, I get to explore different cloud tools as related to projects over the years across different companies from FAANG to growing startups.

I have always held the idea that as long as IAM is sorted while dealing with the cloud, you can achieve awesome stuff meanwhile this piece will be on a simple IAM task via Hybrid Identity.

We are going to address how to synchronize an on-premise Azure Directory with Microsoft Azure, and the synchronization here is called Hybrid Identity. Some organizations do not have an on-premise use case and have to deal solely with the cloud and may not need this hybrid identity. Another possible scenario is with a team where there is an Office 365 setup coupled with the on-premise setup but there are different users that have unique login details which is crazy because as the client base increases then it’ll be difficult to maintain the list. So this is where the SSO and single authentication details come in handy to make the job way easier to implement.

NOTE: When you enter a password, it won't be saved as plain text, it goes through a special algorithm or hash process before it gives you access with the unique string. We will synchronize the password with the password hash synchronization method for easy access and control.

Prerequisites

Understand basic IAM analogies in Azure with Office 365 possibly.
Create an Azure tenant and verify IAM.
Connect a domain if necessary.
Have an Azure account.

Demo

Go to the AD portal via aad.portal.azure.com

Sync users and groups from the on-premise directory user AD by selecting the required text as shown below:
Download Azure AD connect and run the setup file with any necessary framework needed to be downloaded

Setup the Azure AD connect software with all required especially domain service credentials
Install the Azure AD software
There is a possibility that you encounter a configuration error, don’t panic, get access to your terminal and run set-execution policy remotesigned

Then go back to the portal and see the reflected users with the major one which is the administrator account.

Voila, you got it done!

The essence of authentication is to provide users with a set of credentials, such as a username and password, and to verify that they provide the correct credentials whenever they want access to the application. Hence, we need a way to store these credentials in our database for future comparisons. However, storing passwords on the server side for authentication is a difficult task.

Now, remember, this article is not only for experts in the software space, even newbies could hop in and learn a lot and that is why I try to make everything clear both in layman's and professional terms, so if you have any questions, shoot or you can also reach out to me on GitHub

Thanks for reading ❤️

Please leave a comment if you have any thoughts about the topic — I am open to learning and knowledge explorations.

I can imagine how helpful this post has been, do leave a clap 👏 below a few times to show your support for the author! Also, if you need a DevOps engineer for consulting and freelancing, I am the guy you are looking for; hire me, and let’s get that project done.

Five-Finger Ways on How To Deploy Application on App Engine - GCP

Samuel Arogbonlo — Sat, 24 Oct 2020 02:43:53 +0000

In using the Google Cloud Platform, it is all about deploying applications and migrating products data to the cloud. This article addresses how to deploy a simple Hello World Python application to the cloud. The major steps used in this process is shown below:

Create a new project: The project is created by navigating to the top bar after logging in to the platform. If you are a new user, Google has provided $300 for starting offer so as to get used to the tools on the platform.

Activate google cloud shell - The cloud shell is where the scripting and operations take place and it can be switched on by selecting the icon at the top of the screen as well. To check the list of projects, use this command gcloud projects list
to see the project details on the platform.
Enable app engine admin API - Navigate to the menu on Dashboard then select APIs and Services. After that, search “App Engine” and enable. The app engine helps to build apps; it is a serverless solution takes care of the backend compute and gives the URL of the app and you do not care about any much resources. Finally, you should navigate to the project from the cloud shell using the command “ “

Create an app engine by using “gcloud app create” then select regions as required and finally, you can check the app engine to check the status of the created application.

Deploy application: Here we first cloned a GitHub repo “https://github.com/GoogleCloudPlatform/python-docs-sample“ then do some authentication then to check the application you use “cat app.yaml” and the response is “runtime:python37” then you could check the main python file through “cat main.py” and the details of the application displays.

The last step is to deploy through “gcloud app deploy app.yaml” then a response of Y/N shows up while the user types “Y” then deploying continues. As soon as the process has been done, the application can be viewed via the URL and the deploying could take a minimum of 7 minutes and more depending on the intensity of the application.

After the application has been deployed, it could be monitored and maintained but you could delete the application from the app engine homepage so as to save unnecessary cost on the platform.

Understanding the roles in Cloud Engineering

Samuel Arogbonlo — Sat, 24 Oct 2020 01:51:51 +0000

Cloud Engineering in its entirety has several roles and sections that make the space most profitable. The job is quite easy when the roles are understood and could special career paths for any interested developer or engineer hoping to be part of the cloud computing learning process. There are several roles in cloud engineering as regarding computing but majorly there are three; Cloud Architect, Cloud Developers and Cloud Engineers. Each of these roles could evolve to other sub-roles and administrative functions.

Cloud Architect is one of the most lucrative aspects of the cloud computing world as this role allows the holder to structure the architecture of the applications or company’s products. The metrics that help the architect decide how the structure will be implemented include the number of users, privileges needed, type of products, data components, deployment patterns amongst many others. While structuring the architecture, the process involves all the product managers, application developers and any other role involved in building the product so every role is catered for in the diagram that shows what is to be done for the application to be successfully migrated to the cloud. In some organizations, a degree in CS may not be required but in detail, any cloud architect must have an experience building product architecture in a way that the developers and engineers will implement without hassles.

Cloud developers are developers and software engineers who specialize in building for the cloud and do the bidding pf the architect most of the time. It is expedient that the cloud developers understand cloud systems and how they respond to code and backend languages like Goland, Node.JS and others. They also help in deploying applications with very low downtime. After the migration to the cloud has been completed, the developers are tasked with maintenance and monitoring as regarding the product responses to the cloud architecture, infrastructure and structure. Apart from back-end applications, many cloud developers build full-stack applications, data integration and other deployment tasks. Regarding the skills and learning paths, cloud developers work with backups, storage, Platform as a Service (PAAS), microservices, full-stack languages and Software as a Service (SAAS).

The last very relevant role in the cloud is the engineer and this more or less with conjunction with the architect as this role demands planning, design and of course building fo webs services for the cloud. Some of the languages used in this role include Python, Java and some other scripting languages. Right after, the architect has done the structuring, the engineer assesses before the migration is confirmed to have a better business solution for the products and save lots of money in the process.

After understanding all these roles, there are some other interesting roles like System administrators and solutions architect where some interesting jobs are being done as regards the cloud. The world is getting more inclined to the cloud as many companies will be a data company in the coming years - after all, every product needs proper security and networking with proper data processing which is most affordable only through the cloud especially through platforms like Google Cloud Platform (GCP), Microsoft Azure or Amazon Web Services (AWS). In getting ready for these roles, any interested party should get certifications in any of the cloud services, understand application architectures, databases, Linux and possibly application migration processes.

In conclusion, while in some organizations, a person could function in the three roles, some other organizations ensure there are special persons assigned to each role to ensure better outputs and integrations - but definitely, it’s good to be acquainted with the several roles.

Overview of Compute Engine in Google Cloud Platform

Samuel Arogbonlo — Sat, 24 Oct 2020 00:53:49 +0000

In running workloads on physical hardware, several components are required to allow the success of this operation but the most relevant is the Google Compute Engine popularly referred to as GCE. In clear terms, it is defined as an infrastructure as a service platform that gives clients and engineers the ability to process workloads on Google’s physical hardware and has the ability to function as clusters. It is managed by several means but the most preferred is the RESTful API and it is advisable to be conversant with the API building so as to avoid hassles when working with this computing service (GCE). Other managing processes used in GCE includes the command line interface or console provided on the platform.

In compute engine options; the virtual machine (VM) instances needed networking functionality which is provided by the Virtual Private Cloud (VPC). The Virtual Private Cloud (VPC) is just a service in the cloud space used to create network functionality to other services like the App engine and even the Google Kubernetes Engine (GKE) clusters. Also, Kubernetes can be used in compute engine to run some container functions especially in cases of lift-and-shift migration solutions scenarios - but if you want your data running in containers while google handles the rest then use app engine and in the case of code, then cloud functions will do a better job due to the serverless compatibility as well satisfying business logic.

The most obvious advantages of using the compute engine and corresponding products include value increment from applications - lesser pay and better outputs, global scaling of products, migration for virtual machines and the last one which is the very reason for the platform is the ability to easily integrate the service. In migrating to the compute engine, the task could be done in minutes without rewriting the applications, working on images or switching the processes for management in the cloud. But in all these, migrations require that the cloud engineer does the following to be guided through the processes.
-Set up the environment
-Migrate from a previous platform (AWS, Azure or any other cloud
host)
-Migrate in batches using runbooks and jobs
-Testing of migration done in the previous step
-Virtual machines upgrading
-Monitoring
-Automating
Each of the steps stated above has to be thoroughly done based on the demand of the business organization but the application can keep running as long as the migration is done in real-time as it has no adverse effects on the application.

In concluding the overview of Compute engine, it is necessary to understand the requirements needed to go on with compute engine migrations. Some metrics used to set up the requirements include
-Bandwidth - Min between the source environment and nodes should
be the larger of 20Mbit/sec symmetric but long term it depends on
the virtual machines running in the cloud.
-Operating system
-Network connectivity and
-Permissions of the previous cloud host.

Some other details, qwiklabs and processes can be gotten via link https://cloud.google.com/compute