DEV Community: EricKaranja17

Getting Started with Docker and Docker Compose: A Beginner’s Guide

EricKaranja17 — Mon, 25 Aug 2025 14:03:25 +0000

Getting started with Docker doesn't have to feel overwhelming. Think of Docker as a tool that helps you "package once and run anywhere," making your applications more portable and reliable.

This beginner guide aims to provide an introduction to
docker and Docker Compose. I assume that you have docker on your Windows machine, if not, check this beautiful article by Joy Akinyi.

In this article i will provide a brief history,pro/cons and definitions of Docker and Docker Compose.

What is Docker ?

Docker is an open source platform that enables developers manage applications within containers. Containers are lightweight, isolated environment that package an application with its dependencies, providing consistent and reproducible deployments across different computing environments that have Docker installed. Docker allows developers to build, ship and run applications efficiently, regardless of the underlying infrastructure.

What is Docker Compose ?

Docker compose is a tool that allows you to define and run multi-container applications. With Docker Compose, you can create a YAML file which we will see ahead, that defines the services that make your application, and then use a single command to start all services.

docker-compose up

Docker and Docker Compose: A Brief History

Due to my deep passion for history lets briefly narrate the conception of Docker and Docker Compose,

Docker was first released in March 2013 by Solomon Hykes and his team at a company called dotCloud later renamed into Docker, Inc. In 2010 dotCloud was experimenting with Linux Containers (LXC) tech. They solved the teeth shattering statement "but it works in my machine" because applications behaved differently across environments.In March 2013 Docker was officialy released as an open source project at PyCon. In June 2013 its first version is released evolving to what it is today.

Docker Compose was introduced in 2014 as a separate project initially called “fig”, later becoming an official Docker tool. It was created to address the need for managing complex applications with multiple interconnected containers. Docker Compose emerged as a solution for orchestrating container-based applications, allowing developers to define and manage their dependencies easily.

Docker Key Terminologies

Image: A lightweight, standalone, executable package that contains everything needed to run an application, including the code, runtime, libraries and system tools.

Container: A container is a runnable instance of an image on the host machine. It is isolated from other containers and the host machine, and has its own filesystem, network process space.

Dockerfile: A docker file is a text file that contains the instructions for building a Docker image.
Registry: A centralized repository that stores Docker images, such as Docker Hub or private registries.

Docker Hub: This is the official registry for Docker images.

Volume: A persistent storage mechanism that allows data to be shared between containers or between a container and the host system.

Docker Compose Key Terminologies

Service: A definition that describes how to run a specific container, including its image, configuration, dependencies and networking requirements.

YAML Ain’t Markup Language (YAML): A human-readable data serialization format used by Docker Compose to define the application’s structure and configuration. I briefly talked about YAML files in my previous article, Getting Started with GitHub Actions: A Beginner’s Guide.

Volume: A mechanism to persist data across container restarts or between containers.

Network: A virtual network that allows containers within the same Docker Compose project to communicate with each other.

Docker Hub

Fight Night:Batch vs Stream Processing.

EricKaranja17 — Mon, 11 Aug 2025 13:05:25 +0000

Data-driven decision-making has become the heart of today's world, organizations generate information at an unprecedented pace from daily sales transactions to real time sensor reading in IoThttps://www.oracle.com/africa/internet-of-things/ devices.
But if this data are not collected,mined and processed properly then it has no value. This depicts the importance of data processing.

In this article, i will explain the types if data processing,use cases and differences.

It begins.

There are two ways in which data are processed:

The Batch Processing.
The Stream Processing.

What is Batch Data Processing?

Batch processing is when computer handle high
-volumes and repetitive tasks by grouping data into batches and processing it.

Batch processing is mainly automated, with minimal human interaction. tasks are predefined, and the system executes them according to a scheduled timeline.

There are a variety of ETL tools for batch processing. A common tool is Apache Airflow, which allows users to quickly build up data orchestration pipelines that can run on a set schedule and have simple monitoring.

Note: Data orchestration is the automated process of managing and coordinating the flow of data across various systems, applications, and platforms.

What is Stream Data Processing?

Also called real-time data processing, I personally believe that its alias is self explanatory, regardless let me delineate it.

Stream processing continuously ingests and analyzes data. Instead of waiting for data to accumulate, you can process it instantly. This is crucial for time-critical analysis.

One popular framework is Apache Kafka. Apache Kafka is a distributed, fault-tolerant, scalable, and high-throughput messaging system.

Stream Processing Use Cases

Stream processing is particularly beneficial in several key areas. Here are 4 prime examples:

Fraud detection: Stream processing allows financial institutions to monitor transactions in real time. This helps identify and flag suspicious activities immediately, which helps in preventing fraud effectively.
Network Monitoring: In network management, stream processing enables you to constantly monitor your network traffic. This real-time analysis helps in quickly detecting and addressing any anomalies or issues, ensuring smooth network operations.
Predictive Maintenance: Industries use stream processing to monitor equipment health in real time. As a result, potential issues can be detected and addressed before they lead to equipment failure, which saves costs and improves efficiency.
Intrusion Detection: In cybersecurity, stream processing helps in real-time detection of unauthorized access or activities within a network. The detection allows for swift action to mitigate potential security threats.

Batch Processing Use Cases

You should use batch processing in scenarios where data processing must be scheduled and does not require immediate results. The 3 best examples include:

End-of-day reporting: Financial institutions often use batch processing for end-of-day reports. Transactions and activities are accumulated throughout the day and processed in one go, generating comprehensive reports for analysis.
Data warehousing: Organizations use batch processing to update data warehouses periodically. Large volumes of data are collected and processed in batches, ensuring that the data warehouse is up-to-date with the latest information for analytical purposes.
Payroll processing: Companies process payroll data in batches, typically on a bi-weekly or monthly basis. This involves collecting timekeeping data, calculating salaries, and generating paychecks, all done in bulk to streamline operations.

The Difference between Batch and Stream Processing.

Data Size and Scope: Batch processing is best for handling large volume of data. it can process all the data while Stream processing is ideal for low volumes of data. It handles a process real time data eg: Mpesa Transactions.
Performance: Looking at Data Latency, the time taken from when data is collected and made available for processing, Stream processing takes a few milliseconds while batch processing takes hours, days, weeks, months, years.
Required Hardware: Sequential data handling in batch processing requires more resources and storage system to handle the large data ingestion. Continuous data handling in stream processing requires less resources data is processed real-time with no need for storage for later processing.

Note: Data ingestion is the process of collecting, importing, and loading data from various sources into a system for storage and analysis

Functions; what is it to python ?

EricKaranja17 — Mon, 28 Jul 2025 14:44:28 +0000

Python being one of the most popular programming language due to its relatively ease to use, while still extremely versatile and powerful. It is the go to object-oriented programing language for data gurus. If you are burning to learn Python, then understanding how to write functions is a good starting point.

In the context of programming, a function is a named sequence of statements that performs a computation. When you define a function, you specify the name of the sequence of statements. Later, you can 'call' the function by name.

Function calls

marks= 10.2,100.10,16.17
type(marks)

The name of the function is type. The expression in parentheses is called the argument of the function.
A function takes an argument and returns a return value/ result.

A Python functions consists of three components.

A def statement defining a function

def function_name(parameter1, parameter2):

Body: The block of code that executes when the function is called.
A return statement: used to send a value back from the function to the caller.

Below are various examples of the syntax above:

def add(a, b):
        result = a + b
        return result

def my_function():
  print("Hello from a function")

my_function()

Adding New Functions
A function definition specifies the name of a new function and the sequence of statements that run when the function is called:
Eg:

def lyrics():
        print("Back Bencher ndani ya S-Class.")
lyrics()

def is a keyword that indicates that this is a function definition. The name of the function is lyrics. The rules of function names are the same as for variable names. Here is a synopsis of those rules: letters,number and underscores are legal, but the first character can't be a number. Keywords cannot be used as a name of a function. Avoid having a variable and a function with the same name.
The def statement should end with a colon and the body has to be indented. Indentation is always four spaces.

The syntax for calling the new function is the same as for built-in functions. Once you have defined a function, you can use it inside another function.
For example, to repeat the previous refrain, we can write a function called repeat_lyrics:

def lyrics():
    print("Back bencher ndani ya S-class")

def repeat_lyrics():
    lyrics()

repeat_lyrics()

The program contains two functions definitions:lyrics and repeat_lyrics. Function definitions get executed just like other statements, but the effect is to create function objects.

Why functions?

Let me present to you a few reasons why functions are crucial in your program.

Creating a new function gives you an opportunity to name a group of statements,which makes your program easier to read and debug.
Functions can make a program smaller by eliminating repetitive code.Later, if you make a change, you only have to make it in one place.
Dividing a long program into function allows you to debug the parts one at a time and then assemble them into a working whole.
Well-designed functions are often useful for many programs. Once you write and debug one, you can reuse it.

[Boost]

EricKaranja17 — Sat, 26 Jul 2025 11:32:28 +0000

Benefits of OLAP and OLTP in Data Management.

EricKaranja17 — Fri, 25 Jul 2025 17:09:06 +0000

Explain how the separation of OLTP and OLAP systems benefits overall organizational data management strategy.

First lets understand what is OLTP and OLAP:

OLTP(Online Transaction Processing): is a type of data processing that consists of executing a number of transactions occurring concurrently.For example online banking, shopping,order entry etc.

Definition according to IBM:

OLTP: enables the real-time execution of large numbers of database transactions by large numbers of people, typically over the internet.

In lay man terms, it processes real time data.

NB:A database transaction is a change, insertion, deletion, or query of data in a database.

OLAP (Online Analytical Processing): a type of data processing that involves numerous real-time transactions executed concurrently by many users.

In lay man terms, it process historical data most for analysis.

Understanding the benefits of OLAP and OLTP systems separation on data management strategy

Now, that we have a basic comprehension of the systems we can now delve into their various individual prowess:

Data formatting: OLAP systems use multidimensional data models; in cube format, thus you can view the same data from different angles. Enabling it to handle complex queries, facilitating in-depth analysis for decision making.

OLTP systems uses uni-dimensional data models that organizes data into tables on one data aspect. Ensuring data accuracy and consistency, crucial for maintaining the reliability of organizational operations

Data Architecture: OLAP architecture prioritizes data read over data write operations. With this capability it is able to support reporting,data mining and other analytical tasks. Availability is a low priority concern ad the primary use case is analytics.

OLTP architecture prioritizes data write operations which helps it to handle simple, frequent transactions efficiently.
Availability is a high priority due to its significance in daily business operations.

Performance: OLAP processing times can vary from minutes, hours to days depending on the type and volume of data. Data updates in OLAP databases utilize batch processing , where by you periodically process data in large batches then upload the batch to the system all at once. Crucial for integrating data from multiple sources, providing a comprehensive view of the organization.

OLTP processing times in milliseconds or less updates are on real time and are initiated by you or your users. Stream processing is the most used where high volume data moves in a continuous incremental manner with the goal of low-latency processing.