DEV Community: Sospeter Mong'are

AI System Prompts for Businesses: From Chatbots to Digital Employees

Sospeter Mong'are — Thu, 20 Aug 2026 03:32:00 +0000

As businesses adopt AI, one of the most overlooked components is the system prompt.

Many organizations think of a prompt as simply an instruction such as:

"You are a customer support assistant. Answer customer questions."

That may work for a simple chatbot, but it is not enough for a business-critical AI system.

A production AI system needs to understand its role, its responsibilities, its boundaries, the information it can access, the actions it can perform, and when it should involve a human.

In other words, a good system prompt is not just a question to an AI. It is closer to an operating policy for an AI employee.

What is an AI system prompt?

A system prompt defines how an AI should behave throughout an interaction.

For a business, it can establish things such as:

What the AI is responsible for
What it is allowed to do
What it must never do
What information it can use
How it should make decisions
When it should ask for clarification
When it should escalate to a human
How it should handle sensitive information
How it should respond when a tool fails

For example, a customer support AI should not simply be told to "help customers."

It should know:

Which products it supports
Which policies it can reference
What refunds it can authorize
What customer information it can access
Which actions require approval
What happens when it cannot find an answer
When a human agent must take over

This is where system prompts become particularly valuable in business.

A business AI needs boundaries

One of the biggest mistakes businesses can make is focusing only on what an AI can do.

The more important question is:

What should the AI not do?

Imagine a finance AI connected to a company's accounting system.

If the AI can access transactions, generate reports, and initiate financial actions, simply telling it to "help the finance team" is dangerous.

The system needs explicit boundaries.

For example:

It can analyze transactions.
It can generate reports.
It can identify suspicious transactions.
It cannot approve payments.
It cannot change financial records without authorization.
It cannot invent financial figures.
It must escalate unusual transactions to a human.

The AI's capabilities should therefore be matched with clearly defined authority.

A practical structure for business system prompts

A useful business system prompt can contain several sections.

1. Role

Define who the AI is.

For example:

"You are the Customer Support AI for ABC Insurance."

This gives the model a clear operational identity.

2. Objective

Define what success looks like.

For example:

"Your primary responsibility is to resolve customer questions accurately while minimizing unnecessary escalation to human agents."

This is better than simply saying "help customers."

3. Scope

Define what the AI is responsible for.

For example:

Answer product questions
Explain policies
Check claim status
Create support tickets

And explicitly define what is outside its scope.

4. Rules

These are the business policies the AI must follow.

For example:

Never invent information.
Only provide information from approved sources.
Do not expose confidential information.
Do not make unauthorized decisions.
Do not claim an action succeeded unless the system confirms it.

5. Data handling

Businesses need to think carefully about what information an AI can access.

A system prompt can establish rules such as:

Only request information necessary for the task.
Treat customer information as confidential.
Never request passwords or authentication codes.
Never expose internal system information.

However, the prompt should not be the only security mechanism. Permissions and access controls should exist outside the prompt as well.

6. Tool usage

Modern AI systems increasingly interact with tools.

An AI might have access to:

CRM systems
Databases
Payment APIs
Email
Ticketing systems
ERP systems
Internal APIs

The prompt should explain how those tools should be used.

For example:

"Before modifying customer information, verify the customer's identity and confirm that the requested operation is authorized."

The actual authorization should still be enforced by the underlying system.

7. Escalation

This is one of the most important parts.

The AI should know when not to act.

For example, a customer support AI should escalate when:

The customer explicitly requests a human.
The issue involves a disputed financial transaction.
A refund exceeds its authorization limit.
There is a potential security incident.
The customer threatens legal action.
The AI cannot find sufficient information.
A required system is unavailable.

A mature AI system understands that escalation is sometimes a successful outcome.

8. Failure handling

What happens when the AI's tools fail?

Suppose an AI calls a payment API and the API times out.

The AI should not tell the customer:

"Your payment was successfully processed."

Instead, it should know that the operation could not be confirmed.

This is a critical principle for AI automation:

Never confuse an attempted action with a successful action.

The system should verify outcomes before reporting success.

Different businesses can build different AI employees

The same architecture can support many business functions.

Customer Support AI

Responsible for resolving customer questions and support requests.

Sales AI

Can qualify leads, answer product questions, recommend products, and schedule meetings.

Finance AI

Can analyze transactions, generate reports, identify anomalies, and assist finance teams.

HR AI

Can answer questions about company policies, leave procedures, benefits, and onboarding.

IT Helpdesk AI

Can troubleshoot common technical issues, create tickets, and guide employees through approved procedures.

Data Analyst AI

Can answer business questions using approved datasets and generate analytical summaries.

Operations AI

Can monitor workflows, identify failures, and notify responsible teams.

Document Processing AI

Can extract information from documents, validate fields, classify documents, and route exceptions.

The key is that each AI needs a specific responsibility and authority model.

But a system prompt is not the AI system

This is perhaps the most important distinction.

A company can write an excellent system prompt and still build a terrible AI system.

Why?

Because a production AI system is much more than a prompt.

A useful mental model is:

AI Business System = Prompt + Knowledge + Tools + Permissions + Workflow + Monitoring + Human Escalation

Prompt

Defines how the AI should behave.

Knowledge

Provides trusted information.

This could include:

Company policies
Product documentation
FAQs
Contracts
Procedures
Internal documentation

This is where technologies such as RAG become useful.

Tools

Allow the AI to interact with business systems.

For example:

AI
 |
 +-- CRM
 +-- Database
 +-- Payment API
 +-- Email
 +-- Ticketing System

Permissions

Determine what the AI is actually allowed to do.

The prompt can say:

"You cannot approve payments."

But the stronger control is for the payment system itself to reject unauthorized payment approval requests.

This leads to an important engineering principle:

Do not rely on prompts for security.

Prompts provide behavioral guidance. Applications, APIs, identity systems, and databases should enforce security.

Workflow

Determines when the AI runs and what happens before and after it runs.

For example:

Customer request
       |
       v
AI analyzes request
       |
       +---- Simple question ---> Answer
       |
       +---- Support issue -----> Create ticket
       |
       +---- Sensitive action --> Human approval
       |
       +---- Unknown ----------> Escalate

This is where AI moves from being a chatbot to becoming part of a business process.

Monitoring

Businesses also need to know when their AI is failing.

They should monitor things such as:

Failed tool calls
Escalation rates
Incorrect responses
Response latency
Token usage
Business outcomes
API failures
Human overrides

This creates an important question for every AI automation:

What happens when it breaks, and who knows first?

The three questions before automating AI

Before building an AI workflow, businesses should ask three simple questions:

Who owns it?

Someone must be responsible for the AI system.

What does good look like?

Define measurable success.

For example:

70% of common support requests resolved automatically
Less than 2% incorrect responses
All financial actions require approval
Average response time below 10 seconds

What happens when it breaks, and who knows first?

There should be an escalation path, monitoring, alerting, and an accountable person or team.

These questions apply to both traditional automation and AI automation.

The future is not "AI everywhere"

Businesses should not automate a process simply because AI can automate it.

The better question is:

Where does AI create measurable business value without introducing unacceptable risk?

A customer support AI might make sense because thousands of repetitive questions can be handled automatically.

An AI approving million-shilling payments without human oversight might not.

The difference is not the AI model.

The difference is system design.

Final thought

The next stage of AI adoption in businesses will not simply be about giving employees access to ChatGPT.

It will be about building AI systems that operate inside real business processes.

These systems will have roles, permissions, knowledge, tools, workflows, monitoring, and escalation mechanisms.

And the system prompt will become one of the components that defines how these AI employees operate.

But businesses should remember:

A prompt can tell an AI what it should do. A well-engineered system determines what it can actually do.

That distinction is what separates an AI demo from a production AI system.

How to Set Up a VPS Server From Scratch (Complete Beginner's Guide)

Sospeter Mong'are — Wed, 19 Aug 2026 07:25:30 +0000

Introduction

You just bought a VPS (Virtual Private Server). You have an IP address, a root password, and a blank Ubuntu server staring back at you. What do you do next?

This is the question most beginners struggle with. Tutorials online tend to jump straight into deploying applications without covering the foundational work that every production server needs before a single line of application code is deployed.

This guide covers everything you need to do after buying a VPS, in the correct order. By the end you will have a secure, hardened, production-ready server capable of hosting PHP/Laravel applications, Node.js APIs, Python apps, multiple domains, SSL certificates, and databases.

Everything in this guide is based on a real Ubuntu 24.04 LTS server setup. The same principles apply to any Ubuntu or Debian-based VPS from providers like Contabo, DigitalOcean, Linode, or Vultr.

What Is a VPS and Why Does the Setup Matter?

A VPS is a virtual machine running on shared physical hardware in a data centre. Unlike shared hosting where everything is managed for you, a VPS gives you full root access to a Linux server. You control everything.

That freedom comes with responsibility. A freshly provisioned VPS is:

Accessible to the entire internet on port 22 (SSH)
Running as root by default
Accepting password-based logins
Already being scanned by bots within minutes of going live

If you skip the setup and jump straight to deploying your application, you are building on an insecure foundation. A poorly configured VPS can be compromised within hours of going online.

The setup process is not optional. It is the difference between a server that gets hacked and a server that does not.

Phase 1: Inspect Your Server Before Touching Anything

The first rule of working on any server is to understand what you have before you change anything. When you first log in, run a series of inspection commands to build a picture of your server.

Connect via SSH

ssh root@YOUR_VPS_IP

Your VPS provider will give you the root password in a welcome email. Once connected you will see a prompt like:

root@vps-hostname:~#

You are now logged in as root on your server.

Check the Operating System

cat /etc/os-release && uname -r

This tells you the exact OS version and kernel. Everything else depends on this. Ubuntu 24.04 LTS is the recommended choice for production servers as it is supported until April 2029.

Check CPU, RAM, and Disk

lscpu | grep -E "^CPU\(s\)|^Model name|^Thread|^Core"
free -h
df -h

Note how many CPU cores you have, how much RAM is available, and how much disk space you are working with. These numbers affect how you configure PHP workers, database buffers, and swap space later.

Check What Is Already Running

ss -tlnp
systemctl list-units --type=service --state=running

On a clean VPS from a reputable provider you should see almost nothing running except SSH. If you see MySQL, Redis, or a web server already running, note it before proceeding.

Check the Firewall

ufw status verbose

On most fresh Ubuntu VPS instances the firewall is installed but inactive. This means the only thing protecting your server right now is that nothing unexpected is listening. We will fix this shortly.

Check SSH Configuration

sshd -T | grep -E "^permitrootlogin|^passwordauthentication|^pubkeyauthentication|^port"

On a fresh VPS you will typically see:

permitrootlogin yes
passwordauthentication yes
pubkeyauthentication yes
port 22

This means root can log in with a password over the public internet. This is the most dangerous default configuration on any server and the first thing we will fix.

Phase 2: Secure SSH Access

SSH (Secure Shell) is the gateway to your server. Securing it is the single most important step in this entire guide.

Why Root Login Is Dangerous

When you log in as root, every command you run has unlimited power. A typo, a mistake, or a compromised session can destroy your entire server instantly. Worse, root is a known username that attackers target specifically. Brute force bots attempt thousands of root password combinations every hour against any server with port 22 open.

The solution is to create a separate administrative user that uses SSH keys instead of passwords.

Create a Non-Root Admin User

adduser yourname
usermod -aG sudo yourname

Choose a username that is not obvious. Avoid admin, ubuntu, or user as these are common brute force targets.

Set Up SSH Key Authentication

SSH keys are cryptographic key pairs. Your private key stays on your local machine. Your public key is installed on the server. When you connect, SSH verifies the keys match without ever transmitting a password.

On your local machine, check if you already have a key:

ls ~/.ssh/

If you see id_rsa and id_rsa.pub or id_ed25519 and id_ed25519.pub you already have a key pair. If not, generate one:

ssh-keygen -t ed25519 -C "your-email@example.com"

Ed25519 is the modern recommended key type. It is faster and more secure than the older RSA format.

View your public key:

cat ~/.ssh/id_ed25519.pub

The output is a long string starting with ssh-ed25519. This is safe to share openly.

Install the public key on the server by running this as root on the VPS:

mkdir -p /home/yourname/.ssh
echo "YOUR_PUBLIC_KEY_HERE" >> /home/yourname/.ssh/authorized_keys
chown -R yourname:yourname /home/yourname/.ssh
chmod 700 /home/yourname/.ssh
chmod 600 /home/yourname/.ssh/authorized_keys

The permissions are critical. SSH will refuse to use the key if the permissions are too open. 700 on the directory means only the owner can access it. 600 on the file means only the owner can read it.

Test the New User Before Changing Anything

Open a second SSH session and test logging in as your new user:

ssh yourname@YOUR_VPS_IP

You should be logged in without entering a password. Then verify sudo works:

sudo whoami

It should return root.

Do not close your original root session until this test passes. This is your safety net.

Harden the SSH Configuration

Only after confirming your new user works, back up and edit the SSH configuration:

sudo cp /etc/ssh/sshd_config /etc/ssh/sshd_config.backup
sudo nano /etc/ssh/sshd_config

Change these settings:

Setting	Change to
`PermitRootLogin yes`	`PermitRootLogin no`
`PasswordAuthentication yes`	`PasswordAuthentication no`
`X11Forwarding yes`	`X11Forwarding no`
`#MaxAuthTries 6`	`MaxAuthTries 3`
`#LoginGraceTime 2m`	`LoginGraceTime 30`

On Ubuntu 24.04, also check the override files in /etc/ssh/sshd_config.d/. Files in this directory can override settings in the main config. Make sure PasswordAuthentication no is consistent across all files in that directory.

Test the configuration before applying it:

sudo sshd -t

Silence means the configuration is valid. Then reload:

sudo systemctl reload ssh

Test root login from a new terminal. You should see Permission denied (publickey). Root is now blocked.

What If You Get Locked Out?

Every major VPS provider offers a web-based console in their control panel. This gives you direct access to the server regardless of SSH configuration. If you ever lock yourself out, log into your provider's control panel and use the console to fix the SSH config.

Phase 3: Configure the Firewall

Ubuntu ships with UFW (Uncomplicated Firewall). It is inactive by default. Configure and enable it before installing anything else.

Why You Need a Firewall

As you install software, some services will listen on public ports by default. MySQL, Redis, and PostgreSQL have all been compromised on servers where they were accidentally left exposed. A firewall provides a second layer of protection even when a service is misconfigured.

Set Default Policies

sudo ufw default deny incoming
sudo ufw default allow outgoing

Deny everything coming in, allow everything going out. Then explicitly allow only what needs to be public.

Allow Only What Is Necessary

sudo ufw allow ssh
sudo ufw allow http
sudo ufw allow https

Three rules. SSH for administration, HTTP for web traffic and SSL certificate validation, HTTPS for encrypted web traffic.

Never open these ports publicly:

Port	Service	Reason to keep it closed
3306	MySQL	No application needs direct public database access
5432	PostgreSQL	Same reason
6379	Redis	Redis with no password and public access leads to immediate compromise
3000	Node.js	App servers go behind Nginx, never directly public
8000	Python	Same reason

Enable the Firewall

sudo ufw enable

Verify the rules:

sudo ufw status verbose

You should see only ports 22, 80, and 443 allowed. Everything else is blocked.

Phase 4: Harden the Server

With SSH secured and the firewall active, harden the server itself.

Add a Swap File

A swap file is disk space used as overflow when RAM fills up. Without it, the Linux OOM (Out of Memory) killer will terminate processes randomly when memory runs out. On a server this usually means your database or web server gets killed unexpectedly.

sudo fallocate -l 2G /swapfile
sudo chmod 600 /swapfile
sudo mkswap /swapfile
sudo swapon /swapfile
echo '/swapfile none swap sw 0 0' | sudo tee -a /etc/fstab

The last command makes swap permanent across reboots. Also reduce swappiness so the kernel prefers RAM over swap:

echo 'vm.swappiness=10' | sudo tee -a /etc/sysctl.conf
sudo sysctl vm.swappiness=10

Set a Meaningful Hostname

sudo hostnamectl set-hostname your-server-name

Then update the hosts file:

sudo nano /etc/hosts

Find the line with your old hostname and replace it with the new one. A clear hostname like production-01 or app-server makes logs easier to read, especially when managing multiple servers.

Enable Automatic Security Updates

sudo apt install unattended-upgrades -y
sudo dpkg-reconfigure --priority=low unattended-upgrades

Select Yes when prompted. This automatically installs security patches without requiring manual intervention. It applies security updates only, not major version upgrades, so it will not break your applications.

Disable Unnecessary Services

Ubuntu Server ships with services that serve no purpose on a VPS:

sudo systemctl stop ModemManager multipathd udisks2
sudo systemctl disable ModemManager multipathd udisks2

ModemManager handles mobile modems. multipathd manages multipath storage for enterprise SAN systems. udisks2 manages removable drives. None of these belong on a web server. Removing them reduces memory usage and attack surface.

Apply Kernel Hardening

sudo nano /etc/sysctl.d/99-hardening.conf

Add these settings:

# IP Spoofing protection
net.ipv4.conf.all.rp_filter = 1
net.ipv4.conf.default.rp_filter = 1

# Ignore ICMP redirects
net.ipv4.conf.all.accept_redirects = 0
net.ipv6.conf.all.accept_redirects = 0

# Block SYN flood attacks
net.ipv4.tcp_syncookies = 1
net.ipv4.tcp_max_syn_backlog = 2048

# Ignore ICMP broadcast requests
net.ipv4.icmp_echo_ignore_broadcasts = 1

# Hide kernel pointers
kernel.kptr_restrict = 2

Apply immediately:

sudo sysctl -p /etc/sysctl.d/99-hardening.conf

These settings protect against common network attacks including IP spoofing, SYN floods, and ICMP redirect attacks.

Phase 5: Install Nginx as Your Reverse Proxy

Nginx sits in front of all your applications. The internet talks to Nginx. Nginx decides where to send each request.

sudo apt install nginx -y
sudo systemctl enable nginx

The Nginx Architecture

Internet
    |
    v
Nginx (port 80/443)
    |
    +-- example.com        --> PHP-FPM (Laravel)
    |
    +-- api.example.com    --> Node.js on port 3000
    |
    +-- app.example.com    --> Python on port 8000

Your applications never talk to the internet directly. Only Nginx does. This gives you SSL in one place, security headers in one place, and the ability to run many applications on a single server.

Harden the Default Nginx Configuration

Open the main config:

sudo nano /etc/nginx/nginx.conf

Inside the http {} block, make these changes:

Hide the Nginx version number from response headers:

server_tokens off;

Enable gzip compression:

gzip on;
gzip_vary on;
gzip_min_length 1024;
gzip_types text/plain text/css application/json application/javascript text/xml;

Set sensible timeouts and upload limits:

client_max_body_size 64M;
client_body_timeout 30;
keepalive_timeout 65;
send_timeout 30;

Disable the Default Site

sudo rm /etc/nginx/sites-enabled/default
sudo nginx -t && sudo systemctl reload nginx

Create Your Application Directories

sudo mkdir -p /var/www/example.com/public
sudo chown -R youruser:youruser /var/www/example.com

Create a Server Block for Each Site

For a Laravel application:

server {
    listen 80;
    listen [::]:80;
    server_name example.com www.example.com;
    root /var/www/example.com/current/public;

    add_header X-Frame-Options "SAMEORIGIN";
    add_header X-Content-Type-Options "nosniff";
    add_header X-XSS-Protection "1; mode=block";

    index index.php;

    location / {
        try_files $uri $uri/ /index.php?$query_string;
    }

    location ~ \.php$ {
        fastcgi_pass unix:/var/run/php/php8.3-fpm.sock;
        fastcgi_param SCRIPT_FILENAME $realpath_root$fastcgi_script_name;
        include fastcgi_params;
    }

    location ~ /\.(?!well-known).* {
        deny all;
    }

    access_log /var/log/nginx/example.com.access.log;
    error_log  /var/log/nginx/example.com.error.log;
}

For a Node.js API:

server {
    listen 80;
    listen [::]:80;
    server_name api.example.com;

    location / {
        proxy_pass http://localhost:3000;
        proxy_http_version 1.1;
        proxy_set_header Host $host;
        proxy_set_header X-Real-IP $remote_addr;
        proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
        proxy_set_header X-Forwarded-Proto $scheme;
    }

    access_log /var/log/nginx/api.example.com.access.log;
    error_log  /var/log/nginx/api.example.com.error.log;
}

Enable each site and test:

sudo ln -s /etc/nginx/sites-available/example.com /etc/nginx/sites-enabled/
sudo nginx -t && sudo systemctl reload nginx

Always run nginx -t before reloading. If the test fails, Nginx keeps the old working configuration running.

Phase 6: Install Your Application Stack

PHP 8.3 and Composer

sudo apt install -y php8.3 php8.3-fpm php8.3-cli \
php8.3-mysql php8.3-pgsql php8.3-mbstring php8.3-xml \
php8.3-curl php8.3-zip php8.3-bcmath php8.3-opcache \
php8.3-intl php8.3-gd

curl -sS https://getcomposer.org/installer | php
sudo mv composer.phar /usr/local/bin/composer
sudo chmod +x /usr/local/bin/composer

Node.js LTS

Do not install Node.js from Ubuntu's default repositories. The version there is outdated. Use NodeSource:

curl -fsSL https://deb.nodesource.com/setup_22.x | sudo -E bash -
sudo apt install -y nodejs

Python and Virtual Environment Tools

Ubuntu 24.04 ships with Python 3.12. Add the tooling:

sudo apt install -y python3-pip python3-venv python3-dev

Always use virtual environments for Python applications. Never install application packages system-wide as this creates conflicts between projects.

Phase 7: Install and Secure Your Databases

MySQL

sudo apt install -y mysql-server
sudo mysql_secure_installation

During secure installation answer the prompts like this:

Validate password component: No
Remove anonymous users: Yes
Disallow root login remotely: Yes
Remove test database: Yes
Reload privilege tables: Yes

PostgreSQL

sudo apt install -y postgresql postgresql-contrib

Redis

sudo apt install -y redis-server
sudo sed -i 's/^bind 127.0.0.1 ::1/bind 127.0.0.1/' /etc/redis/redis.conf
sudo systemctl restart redis-server
sudo systemctl enable redis-server

The sed command restricts Redis to localhost only. Redis with no password and public internet access is one of the most common and damaging server compromises in existence. Always keep it bound to 127.0.0.1.

Accessing Databases Remotely

Your databases are not publicly accessible and should stay that way. To connect from your local machine using a tool like TablePlus, DBeaver, or HeidiSQL, use an SSH tunnel:

# MySQL tunnel
ssh -L 3306:127.0.0.1:3306 youruser@YOUR_VPS_IP -N

# PostgreSQL tunnel
ssh -L 5432:127.0.0.1:5432 youruser@YOUR_VPS_IP -N

Then connect your database client to 127.0.0.1 on the forwarded port. The connection travels through your encrypted SSH session. No database port is ever exposed to the internet.

Phase 8: Set Up Backups

A server with no backup strategy is a server waiting to fail. Set this up before deploying anything.

Create a backup script:

sudo mkdir -p /opt/backups/files
sudo nano /opt/backups/backup.sh

Add this content:

#!/bin/bash
DATE=$(date +%Y%m%d_%H%M%S)
BACKUP_DIR="/opt/backups/files"

# MySQL
mysqldump --all-databases --single-transaction | gzip > "$BACKUP_DIR/mysql_$DATE.sql.gz"

# PostgreSQL
sudo -u postgres pg_dumpall | gzip > "$BACKUP_DIR/postgres_$DATE.sql.gz"

# Nginx config
tar -czf "$BACKUP_DIR/nginx_$DATE.tar.gz" /etc/nginx/

# Remove backups older than 7 days
find "$BACKUP_DIR" -type f -mtime +7 -delete

Make it executable and schedule it:

sudo chmod +x /opt/backups/backup.sh
sudo crontab -e

Add this line to run every night at 2 AM:

0 2 * * * /opt/backups/backup.sh >> /var/log/backup.log 2>&1

Backups that exist only on the same server are not real backups. Set up offsite backup to Google Drive using rclone, to AWS S3, or to a separate server. If your VPS is compromised or the data centre has an incident, local-only backups go with it.

Phase 9: Monitor Your Server

Native Linux Tools

These are available immediately with no installation:

# Interactive process monitor
htop

# Disk usage
df -h

# Memory usage
free -h

# Real time Nginx access log
sudo tail -f /var/log/nginx/example.com.access.log

# Real time error log
sudo tail -f /var/log/nginx/example.com.error.log

# System logs
sudo journalctl -f

# Active listening ports
sudo ss -tlnp

Netdata Dashboard

Netdata gives you a real-time visual dashboard for CPU, RAM, disk, Nginx, MySQL, Redis, and PHP-FPM all in one place:

curl https://get.netdata.cloud/kickstart.sh > /tmp/netdata-kickstart.sh
sudo sh /tmp/netdata-kickstart.sh --stable-channel --disable-telemetry

Netdata runs on port 19999. Do not open this port publicly. Access it via SSH tunnel:

ssh -L 19999:localhost:19999 youruser@YOUR_VPS_IP -N

Then visit http://localhost:19999 in your browser for the full dashboard.

External Uptime Monitoring

Sign up for a free account at UptimeRobot. Add your domain and it checks every 5 minutes, sending you an email or SMS the moment your site goes down. This catches server crashes, Nginx failures, and network issues before your customers do.

The Pre-Deployment Checklist

Before pointing any domain at this server or deploying any application, run through this checklist:

Security

[ ] Root SSH login is disabled
[ ] Password authentication is disabled
[ ] SSH key authentication is confirmed working
[ ] Admin user has sudo access
[ ] Firewall is active with only ports 22, 80, and 443 open
[ ] No database ports are publicly accessible
[ ] Redis is bound to localhost only
[ ] Kernel hardening parameters are applied
[ ] Unnecessary services are disabled

Stack

[ ] Nginx is running and enabled on boot
[ ] PHP-FPM is running and enabled on boot
[ ] Composer is installed
[ ] Node.js and npm are installed
[ ] Python and venv tools are installed
[ ] MySQL is installed and secured
[ ] PostgreSQL is installed
[ ] Redis is installed and localhost-only

Infrastructure

[ ] 2GB swap file is active and permanent
[ ] Hostname is set correctly
[ ] Automatic security updates are enabled
[ ] Backup script is created and scheduled
[ ] Backup script has been tested manually at least once
[ ] Offsite backup destination is configured
[ ] Uptime monitoring is active

Nginx

[ ] Default site is disabled
[ ] Server blocks exist for all your domains
[ ] nginx -t passes with no errors
[ ] server_tokens off is set
[ ] Security headers are in all server blocks
[ ] Log files are configured per site

What Comes Next

Once your checklist is complete your server is production ready. Here is what to do next:

Point your DNS first. Update your domain's A record to point to your server IP at your registrar. DNS propagation takes anywhere from a few minutes to 48 hours depending on your provider.

Install SSL certificates. Once DNS is pointing to your server, run Certbot to get free Let's Encrypt certificates for each domain. This takes under two minutes per domain.

Deploy your applications. Each application type has its own deployment process. Treat each one as a separate focused task rather than trying to deploy everything at once.

Create application databases. For each app, create a dedicated database user with only the permissions that app needs. Never use the root database user for application connections.

Common Mistakes to Avoid

Skipping the firewall. Every day you run without a firewall is a day your databases could be accidentally exposed. Enable UFW before installing any software.

Deploying as root. Applications should run as dedicated low-privilege users. Running as root means a compromised application has unlimited access to your entire server.

Using root for database connections. Always create application-specific database users with minimal permissions. The MySQL or PostgreSQL root user should never appear in an application config file.

Storing secrets in Git. Your .env file contains database passwords, API keys, and application secrets. It must never be committed to version control. Add .env to .gitignore before your first commit.

Skipping backups until later. Later never comes. Set up backups before your first deployment. The worst time to realise you have no backups is after something breaks.

Opening database ports publicly. Use SSH tunneling for database access from your local machine. Ports 3306 and 5432 should never appear in your UFW allowed rules.

Summary

Setting up a VPS correctly is not complicated but it requires doing things in the right order. Here is the sequence that matters:

Inspect the server before changing anything
Create a non-root admin user with SSH key authentication
Test the new user, then disable root login and password auth
Configure and enable the firewall before installing any software
Harden the server with swap, kernel parameters, and automatic updates
Install Nginx as a reverse proxy
Install your application stack: PHP, Node.js, Python
Install and secure your databases: MySQL, PostgreSQL, Redis
Set up automated backups with offsite storage
Set up monitoring and uptime alerts

Only after completing all of these steps should you point a domain to the server and begin deploying applications.

The time you invest in this setup pays back every single day your server runs without incident. A server configured this way can host multiple production applications for years without requiring significant maintenance or firefighting.

For technical assistance whatsapp me here

This guide is part of a series on production server management. Other articles in the series cover Nginx as a reverse proxy in depth, setting up free SSL certificates with Let's Encrypt, and deploying Laravel applications with zero-downtime releases.

Let's Encrypt SSL: A Beginner's Guide to Free HTTPS for Your Website

Sospeter Mong'are — Sun, 16 Aug 2026 19:15:27 +0000

Introduction

A few years ago, adding HTTPS to your website cost money — sometimes hundreds of dollars per year for an SSL certificate. Many small websites and developers skipped it entirely, leaving their users exposed.

Then in 2016, Let's Encrypt changed everything.

Today, there is no excuse for a website without HTTPS. Let's Encrypt provides free, trusted, automatically renewing SSL certificates to anyone with a domain name. It powers over 300 million websites worldwide and is trusted by every major browser.

This guide explains what SSL is, why it matters, how Let's Encrypt works, and exactly how to install it on an Nginx server running on Ubuntu.

What Is SSL and Why Does It Matter?

SSL (Secure Sockets Layer) — more accurately called TLS (Transport Layer Security) today, though the term SSL stuck — is a protocol that encrypts the connection between a visitor's browser and your web server.

Without SSL, data travels across the internet in plain text. Anyone between the visitor and your server — their ISP, a coffee shop router, a malicious actor on the same network — can read that data. This includes:

Login credentials
Credit card numbers
Personal information
Form submissions
Session cookies

With SSL, all of that data is encrypted. Even if intercepted, it is unreadable without the decryption key.

The Padlock in the Browser

You have seen it thousands of times — the padlock icon in the browser address bar next to a URL that starts with https://. That padlock means:

The connection between the browser and server is encrypted
The server's identity has been verified by a trusted Certificate Authority
Data cannot be intercepted or tampered with in transit

Without SSL, modern browsers display a "Not Secure" warning in the address bar. This alone drives visitors away and destroys trust in your website.

SSL Is Not Optional Anymore

Beyond security, SSL affects your website in several concrete ways:

Search Engine Rankings: Google confirmed in 2014 that HTTPS is a ranking signal. Two otherwise identical websites — one with HTTPS, one without — the HTTPS site ranks higher.

Browser Warnings: Chrome, Firefox, and Safari all display prominent "Not Secure" warnings on HTTP sites, especially on pages with login forms or payment fields.

HTTP/2: The modern HTTP/2 protocol — which makes websites significantly faster — requires HTTPS in all major browsers. No SSL means no HTTP/2.

User Trust: Studies consistently show that users abandon websites that display security warnings. For any website handling user data, SSL is non-negotiable.

What Is a Certificate Authority?

Your browser doesn't just trust any SSL certificate. It maintains a list of trusted Certificate Authorities (CAs) — organisations that have been vetted and approved to issue certificates.

When you install an SSL certificate, you are essentially telling browsers: "A trusted organisation has verified that I own this domain. You can trust this connection."

Traditionally, Certificate Authorities charged for this service — anywhere from $10 to $1,000+ per year depending on the certificate type. The revenue model made sense: the CA did the work of verifying your identity and browsers trusted their signature.

Let's Encrypt disrupted this entirely. It is a non-profit Certificate Authority — backed by Mozilla, Google, Cisco, and others — that issues certificates for free, automatically, and without any manual verification process.

What Is Let's Encrypt?

Let's Encrypt is a free, automated, open Certificate Authority launched in 2016 by the Internet Security Research Group (ISRG).

Its certificates are:

Free — no cost, ever
Trusted — recognised by all major browsers and operating systems
Automatic — issuance and renewal are fully automated
Open — the process is transparent and publicly audited

The only meaningful difference between a Let's Encrypt certificate and a paid certificate is the validation level. Let's Encrypt issues Domain Validated (DV) certificates, which verify you control the domain but do not verify your organisation's legal identity. For the vast majority of websites — blogs, SaaS applications, APIs, portfolios — DV certificates are perfectly sufficient.

If you run a bank or a large e-commerce platform that wants to display your company name in the browser bar, you might want an Extended Validation (EV) certificate from a paid CA. For everything else, Let's Encrypt is the right choice.

How Let's Encrypt Works

Let's Encrypt uses a protocol called ACME (Automatic Certificate Management Environment) to automate the entire certificate lifecycle.

Here is the process, simplified:

Step 1 — Domain Ownership Verification

Before issuing a certificate for example.com, Let's Encrypt needs to verify you actually control that domain. It does this through a challenge.

The most common challenge type is the HTTP-01 challenge:

Let's Encrypt tells your server to place a specific file at a specific URL — for example http://example.com/.well-known/acme-challenge/randomtoken
Let's Encrypt then fetches that URL from its servers
If the file is there with the correct content, domain ownership is proven
The certificate is issued

This entire process happens in seconds and is fully automated by Certbot.

Step 2 — Certificate Issuance

Once domain ownership is verified, Let's Encrypt issues a certificate valid for 90 days. This short validity period is intentional — it limits the window of exposure if a certificate is ever compromised, and it forces automation of renewal.

Step 3 — Automatic Renewal

Certbot installs a systemd timer (or cron job) that runs twice daily. It checks whether any certificate is within 30 days of expiring. If so, it automatically renews it — re-running the challenge, obtaining a new certificate, and reloading Nginx — all without any manual intervention.

In practice, your certificates renew themselves every 60-70 days, long before the 90-day expiry.

What Is Certbot?

Certbot is the official client tool for Let's Encrypt, maintained by the Electronic Frontier Foundation (EFF). It is the software you install on your server that:

Communicates with Let's Encrypt's ACME servers
Handles the domain ownership challenge automatically
Obtains and installs the SSL certificate
Configures Nginx (or Apache) to use it
Sets up automatic renewal

Without Certbot, you would need to manually interact with Let's Encrypt's API. Certbot makes the entire process a single command.

Prerequisites Before Running Certbot

Before you can install an SSL certificate, three things must be true:

1. Your Domain Must Point to Your Server

Let's Encrypt verifies domain ownership by making an HTTP request to your domain. If your domain's DNS is not pointing to your server's IP address, the challenge will fail and no certificate will be issued.

Check your DNS records at your registrar. You need:

Record	Type	Value
`example.com`	A	Your server IP
`www.example.com`	A	Your server IP
`api.example.com`	A	Your server IP

DNS changes can take anywhere from a few minutes to 48 hours to propagate, depending on your registrar and TTL settings. You can verify DNS has propagated using:

dig example.com +short

Or use an online tool like dnschecker.org.

2. Nginx Must Be Running and Configured

Certbot needs Nginx to be running with a server block configured for your domain. The server block does not need SSL yet — Certbot adds that. It just needs to exist and be active.

3. Port 80 Must Be Open

Certbot uses port 80 (HTTP) to complete the domain challenge. Your firewall must allow incoming traffic on port 80. If you followed the server setup guide in this series, UFW already has port 80 open.

Installing Certbot on Ubuntu with Nginx

Step 1 — Install Certbot

sudo apt install -y certbot python3-certbot-nginx

The python3-certbot-nginx plugin allows Certbot to automatically modify your Nginx configuration to enable SSL — you do not need to edit any config files manually.

Step 2 — Obtain Your First Certificate

sudo certbot --nginx -d example.com -d www.example.com

What the flags mean:

--nginx — use the Nginx plugin to automatically configure SSL
-d example.com — issue certificate for this domain
-d www.example.com — include this as an additional domain on the same certificate

You can include multiple -d flags to cover multiple subdomains on one certificate:

sudo certbot --nginx -d example.com -d www.example.com -d api.example.com

Step 3 — Answer the Prompts

Certbot will ask you a few questions:

Enter email address: you@example.com
Agree to terms of service: Y
Share email with EFF: N (your choice)
Redirect HTTP to HTTPS: 2 (Always redirect — recommended)

Choose option 2 for the redirect. This automatically adds an Nginx rule that sends all HTTP traffic to HTTPS, so visitors who type http://example.com are seamlessly redirected to https://example.com.

Step 4 — What Certbot Does Automatically

After you answer the prompts, Certbot:

Contacts Let's Encrypt servers
Places a challenge file in your web root
Let's Encrypt verifies the file
Certificate is issued and saved to /etc/letsencrypt/live/example.com/
Certbot modifies your Nginx config to enable SSL on port 443
Certbot adds the HTTP to HTTPS redirect on port 80
Nginx is reloaded

Your site is now serving HTTPS. The entire process takes under 60 seconds.

What Your Nginx Config Looks Like After Certbot

Before Certbot, your Nginx server block looked like this:

server {
    listen 80;
    server_name example.com www.example.com;
    root /var/www/example.com/public;
    ...
}

After Certbot runs, it transforms it into:

server {
    listen 80;
    server_name example.com www.example.com;
    return 301 https://$host$request_uri;
}

server {
    listen 443 ssl;
    server_name example.com www.example.com;
    root /var/www/example.com/public;

    ssl_certificate /etc/letsencrypt/live/example.com/fullchain.pem;
    ssl_certificate_key /etc/letsencrypt/live/example.com/privkey.pem;
    include /etc/letsencrypt/options-ssl-nginx.conf;
    ssl_dhparam /etc/letsencrypt/ssl-dhparams.pem;

    ...your existing config...
}

The first server block catches all HTTP traffic and redirects it to HTTPS. The second handles all HTTPS traffic with your SSL certificate.

Where Certbot Stores Your Certificates

/etc/letsencrypt/
├── live/
│   └── example.com/
│       ├── fullchain.pem    ← Your certificate + intermediate chain
│       ├── privkey.pem      ← Your private key (keep this secret)
│       ├── cert.pem         ← Your certificate only
│       └── chain.pem        ← Intermediate certificates only
├── archive/
│   └── example.com/         ← Historical certificates (all versions)
└── renewal/
    └── example.com.conf     ← Renewal configuration

Important: The files in live/ are actually symbolic links pointing to the latest version in archive/. When Certbot renews your certificate, it creates new files in archive/ and updates the symlinks — so your Nginx config never needs to change.

Never delete anything in /etc/letsencrypt/ unless you know exactly what you are doing.

Automatic Certificate Renewal

This is where Let's Encrypt's real power shows. Certbot installs a systemd timer that runs twice daily:

# Check the timer status
sudo systemctl status certbot.timer

You will see it is active and scheduled to run at random times twice per day. The randomisation prevents all Certbot installations worldwide from hitting Let's Encrypt's servers simultaneously.

When the timer runs, Certbot checks each certificate. If any certificate expires within 30 days, it renews it automatically. In practice your certificates renew around the 60-day mark — well before the 90-day expiry.

Test Renewal Without Actually Renewing

sudo certbot renew --dry-run

This simulates the entire renewal process without actually changing anything. Run this after installation to confirm renewal will work when the time comes.

Expected output:

Congratulations, all simulated renewals succeeded:
  /etc/letsencrypt/live/example.com/fullchain.pem (success)

Managing Multiple Domains

If you host multiple websites on one server, you can have separate certificates for each domain:

# Certificate for your main app
sudo certbot --nginx -d example.com -d www.example.com

# Certificate for your API
sudo certbot --nginx -d api.example.com

# Certificate for another site
sudo certbot --nginx -d anotherdomain.com -d www.anotherdomain.com

Each domain gets its own certificate stored separately in /etc/letsencrypt/live/. Certbot manages renewal for all of them automatically.

List all your certificates:

sudo certbot certificates

Output:

Found the following certs:
  Certificate Name: example.com
    Domains: example.com www.example.com
    Expiry Date: 2026-11-14 (VALID: 89 days)
    Certificate Path: /etc/letsencrypt/live/example.com/fullchain.pem

  Certificate Name: api.example.com
    Domains: api.example.com
    Expiry Date: 2026-11-14 (VALID: 89 days)
    Certificate Path: /etc/letsencrypt/live/api.example.com/fullchain.pem

Testing Your SSL Installation

After installing your certificate, verify it is working correctly.

Test in Your Browser

Visit https://example.com and click the padlock icon. You should see:

Connection is secure
Certificate is valid
Issued by: Let's Encrypt

Test with SSL Labs

SSL Labs provides a free, detailed SSL analysis of your domain. It checks:

Certificate validity
Certificate chain
Protocol support (TLS 1.2, TLS 1.3)
Cipher strength
Known vulnerabilities

A properly configured Let's Encrypt certificate with default Certbot settings typically scores A on SSL Labs.

Test from the Command Line

curl -I https://example.com

Expected output:

HTTP/2 200
server: nginx
...

The HTTP/2 200 confirms both HTTPS and HTTP/2 are working.

Common Problems and Solutions

Challenge Failed — Domain Not Pointing to Server

Error:

Challenge failed for domain example.com

Cause: DNS is not pointing to your server yet, or DNS has not propagated.

Solution:

# Check what IP your domain resolves to
dig example.com +short

# It must match your server IP
curl ifconfig.me

Wait for DNS propagation and try again.

Port 80 Is Blocked

Error:

Connection refused on port 80

Cause: Your firewall is blocking HTTP.

Solution:

sudo ufw allow http
sudo ufw status

Nginx Configuration Error

Error:

nginx: configuration file test failed

Solution:

sudo nginx -t

Fix whatever error it reports before running Certbot again.

Certificate Already Exists

If you run Certbot for a domain that already has a certificate, it will ask whether to renew, expand, or reinstall. Choose:

Renew — get a fresh certificate for the same domains
Expand — add new domains to the existing certificate
Reinstall — reinstall the existing certificate without changes

Rate Limits

Let's Encrypt has rate limits to prevent abuse:

Limit	Value
Certificates per domain per week	5
Duplicate certificates per week	5
Failed validations per hour	5

In practice you will never hit these limits in normal use. They only matter if you are repeatedly requesting certificates for the same domain due to testing or errors.

If you do hit a rate limit, you must wait before trying again. To avoid this during testing, use Let's Encrypt's staging environment:

sudo certbot --nginx --staging -d example.com

Staging certificates are not trusted by browsers but have much higher rate limits. Use staging to test your setup, then run without --staging for the real certificate.

Revoking a Certificate

If your private key is ever compromised, revoke the certificate immediately:

sudo certbot revoke --cert-path /etc/letsencrypt/live/example.com/cert.pem

Then obtain a new certificate:

sudo certbot --nginx -d example.com

The Bigger Picture — HTTPS Everywhere

Let's Encrypt's mission is to make HTTPS universal. Before it existed, the friction and cost of SSL certificates meant that millions of websites transmitted user data in plain text. Today there is no technical or financial barrier to HTTPS.

If you are building a web application, an API, or any internet-facing service, HTTPS is not a feature — it is a baseline requirement. Let's Encrypt removes every excuse not to have it.

Quick Reference — Certbot Commands

# Install Certbot
sudo apt install -y certbot python3-certbot-nginx

# Obtain certificate for a domain
sudo certbot --nginx -d example.com -d www.example.com

# List all certificates
sudo certbot certificates

# Test renewal (dry run)
sudo certbot renew --dry-run

# Force renewal immediately
sudo certbot renew --force-renewal

# Revoke a certificate
sudo certbot revoke --cert-path /etc/letsencrypt/live/example.com/cert.pem

# Delete a certificate
sudo certbot delete --cert-name example.com

# Check renewal timer
sudo systemctl status certbot.timer

# Test staging (no rate limits)
sudo certbot --nginx --staging -d example.com

Summary

Here is everything covered in this guide:

SSL encrypts the connection between your visitor and server — it is mandatory for any modern website
Let's Encrypt is a free, trusted, automated Certificate Authority that issues 90-day certificates
Certbot is the tool that talks to Let's Encrypt, installs your certificate, configures Nginx, and renews automatically
Domain ownership is verified via the HTTP-01 challenge — Let's Encrypt checks a file on your server
Certificates auto-renew every 60 days via a systemd timer — zero manual work required
Always run certbot renew --dry-run after setup to confirm renewal works
Use --staging for testing to avoid rate limits
SSL Labs gives you an independent grade of your SSL configuration

When your domain is pointed to your server and Nginx is configured, running Let's Encrypt takes less than two minutes. There is genuinely no reason to run any website without it.

What Is Nginx? A Beginner's Guide to the Web Server Powering the Modern Internet

Sospeter Mong'are — Sun, 16 Aug 2026 16:54:22 +0000

Introduction

If you have ever set up a website, deployed a web application, or rented a VPS (Virtual Private Server), you have almost certainly encountered the name Nginx (pronounced engine-x). It powers some of the busiest websites on the internet — including Netflix, Dropbox, and WordPress.com — yet many beginners find it intimidating at first glance.

This guide will change that.

By the end of this article, you will understand exactly what Nginx is, why it exists, how it works, and how to use it to serve real websites on a Linux server. No prior experience with web servers is required.

What Is Nginx?

Nginx is a web server — software that listens for requests coming from the internet and responds with web pages, files, or data.

But Nginx is more than just a web server. It is also:

A reverse proxy — it sits in front of your applications and forwards requests to them
A load balancer — it can distribute traffic across multiple servers
An HTTP cache — it can store and serve cached responses to reduce server load
An SSL terminator — it handles HTTPS encryption so your applications don't have to

Think of Nginx as the reception desk of a large office building. Every visitor (web request) walks in through the front door (port 80 or 443). The receptionist (Nginx) greets them, figures out where they need to go, and directs them to the right department — the Laravel team, the Node.js team, or the Python team. The visitor never wanders the building themselves.

Why Not Just Use Apache?

Apache is the other major web server and has been around since 1995. For years it dominated the web. So why does Nginx exist?

The answer comes down to architecture.

Apache creates a new thread or process for every incoming connection. This works fine for low traffic, but under heavy load — thousands of simultaneous connections — Apache consumes enormous amounts of RAM and CPU.

Nginx was built in 2004 specifically to solve this problem. It uses an event-driven, asynchronous architecture. Instead of one thread per connection, a single Nginx worker process can handle thousands of simultaneous connections efficiently using very little memory.

The result:

	Apache	Nginx
Architecture	Thread per connection	Event-driven
Memory usage under load	High	Low
Static file serving	Good	Excellent
Reverse proxy	Possible	Native, highly optimised
Configuration style	`.htaccess` per directory	Centralised config files

For modern application servers running Laravel, Node.js, or Python, Nginx is the industry standard choice.

How Nginx Fits Into a Real Server

Here is the architecture that professional developers use on production servers:

Internet
    |
    ▼
[ Nginx — Port 80/443 ]
    |
    ├──► example.com        → PHP-FPM (Laravel app)
    ├──► api.example.com    → Node.js running on port 3000
    └──► app.example.com    → Python running on port 8000

Notice that none of the applications talk directly to the internet. Only Nginx does. This is the reverse proxy pattern and it is fundamental to how modern servers work.

Why is this pattern so powerful?

Security: Your Node.js app running on port 3000 is never exposed to the internet. UFW (the firewall) blocks port 3000 publicly. Only Nginx — on port 443 — is reachable, and Nginx decides what to forward.

SSL in one place: Instead of configuring HTTPS in your Laravel app, your Node.js app, and your Python app separately, you configure it once in Nginx. All apps automatically get HTTPS.

Multiple apps on one server: Without a reverse proxy, you could only run one application per server (one thing can listen on port 80 at a time). With Nginx routing by domain name, you can run dozens of applications on a single server.

Static files: Nginx serves static files (images, CSS, JavaScript) directly from disk at incredible speed — without involving PHP, Node.js, or Python at all. This dramatically reduces load on your application.

Understanding Nginx Configuration

Nginx configuration lives in /etc/nginx/ on Ubuntu/Debian systems. Here is the directory layout:

/etc/nginx/
├── nginx.conf              ← Main configuration file
├── sites-available/        ← All site configs (active or not)
│   ├── example.com
│   └── api.example.com
├── sites-enabled/          ← Symlinks to active sites only
│   ├── example.com → ../sites-available/example.com
│   └── api.example.com → ../sites-available/api.example.com
├── conf.d/                 ← Additional configuration fragments
└── snippets/               ← Reusable configuration pieces

The sites-available and sites-enabled pattern

This is an elegant system. You write your site configuration in sites-available/. To activate it, you create a symbolic link (a shortcut) in sites-enabled/. To deactivate a site without deleting its configuration, you simply remove the symlink.

# Enable a site
sudo ln -s /etc/nginx/sites-available/example.com /etc/nginx/sites-enabled/

# Disable a site (config is preserved)
sudo rm /etc/nginx/sites-enabled/example.com

Server Blocks — Nginx's Virtual Hosts

In Nginx, each website or application is configured using a server block. This is equivalent to Apache's Virtual Hosts. A server block tells Nginx: "When a request comes in for this domain name, here is how to handle it."

Example 1: Serving a Static HTML Website

server {
    listen 80;
    listen [::]:80;
    server_name example.com www.example.com;
    root /var/www/example.com/public;

    index index.html;

    location / {
        try_files $uri $uri/ =404;
    }

    access_log /var/log/nginx/example.com.access.log;
    error_log  /var/log/nginx/example.com.error.log;
}

Let's break this down line by line:

listen 80 — listen for HTTP traffic on port 80
listen [::]:80 — also listen on IPv6
server_name example.com www.example.com — this block handles requests for these domain names
root /var/www/example.com/public — files are served from this directory
index index.html — the default file to serve when a directory is requested
try_files $uri $uri/ =404 — try to find the requested file; return 404 if not found
access_log and error_log — where to write logs for this site

Example 2: Serving a Laravel (PHP) Application

Laravel is a PHP framework that requires PHP-FPM to process .php files. Here is how Nginx is configured to work with it:

server {
    listen 80;
    listen [::]:80;
    server_name example.com www.example.com;
    root /var/www/example.com/public;

    index index.php;

    location / {
        try_files $uri $uri/ /index.php?$query_string;
    }

    location ~ \.php$ {
        fastcgi_pass unix:/var/run/php/php8.3-fpm.sock;
        fastcgi_param SCRIPT_FILENAME $realpath_root$fastcgi_script_name;
        include fastcgi_params;
    }

    location ~ /\.(?!well-known).* {
        deny all;
    }

    access_log /var/log/nginx/example.com.access.log;
    error_log  /var/log/nginx/example.com.error.log;
}

The key addition here is the location ~ \.php$ block. This tells Nginx: "For any request ending in .php, don't serve it as a static file — pass it to PHP-FPM for processing."

PHP-FPM (FastCGI Process Manager) is a separate service that runs PHP code and returns the result to Nginx. They communicate via a Unix socket (php8.3-fpm.sock) — a fast, secure local communication channel.

The location ~ /\.(?!well-known).* block denies access to hidden files (files starting with a dot, like .env). This is a critical security rule — your .env file contains database passwords and application secrets, and it must never be publicly accessible.

Example 3: Reverse Proxy to Node.js

When your Node.js/Express application is running on port 3000, Nginx forwards requests to it like this:

server {
    listen 80;
    listen [::]:80;
    server_name api.example.com;

    location / {
        proxy_pass http://localhost:3000;
        proxy_http_version 1.1;
        proxy_set_header Upgrade $http_upgrade;
        proxy_set_header Connection 'upgrade';
        proxy_set_header Host $host;
        proxy_set_header X-Real-IP $remote_addr;
        proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
        proxy_set_header X-Forwarded-Proto $scheme;
    }

    access_log /var/log/nginx/api.example.com.access.log;
    error_log  /var/log/nginx/api.example.com.error.log;
}

The proxy_pass http://localhost:3000 directive is the core of reverse proxying. Every request that hits api.example.com is forwarded to your Node.js process running locally on port 3000.

The proxy_set_header directives pass important information to your Node.js app — particularly X-Real-IP and X-Forwarded-For, which tell your application the real IP address of the visitor (since from Node.js's perspective, all requests appear to come from localhost).

Important Security Headers

Every Nginx server block should include these security headers:

add_header X-Frame-Options "SAMEORIGIN";
add_header X-Content-Type-Options "nosniff";
add_header X-XSS-Protection "1; mode=block";

X-Frame-Options SAMEORIGIN — prevents your site from being embedded in an iframe on another website (protects against clickjacking attacks)
X-Content-Type-Options nosniff — prevents browsers from guessing the content type of a response (protects against MIME type confusion attacks)
X-XSS-Protection — enables the browser's built-in cross-site scripting filter

Hiding the Nginx Version Number

By default, Nginx tells the world exactly which version it is running. This is visible in HTTP response headers:

Server: nginx/1.24.0

This is a gift to attackers — they can look up known vulnerabilities for that exact version. One line in your nginx.conf fixes this:

server_tokens off;

After this, the header simply shows:

Server: nginx

Testing and Reloading Configuration

This is the most important operational habit with Nginx. Always test your configuration before reloading.

# Test configuration syntax
sudo nginx -t

# If the test passes, reload gracefully
sudo systemctl reload nginx

The difference between reload and restart is important:

reload — applies the new configuration without dropping existing connections. Zero downtime.
restart — stops and starts Nginx completely. Active connections are dropped.

Always use reload in production. Only use restart if Nginx is genuinely broken and needs a full restart.

A passing test looks like this:

nginx: the configuration file /etc/nginx/nginx.conf syntax is ok
nginx: configuration file /etc/nginx/nginx.conf test is successful

If you see any errors, fix them before reloading. A broken configuration with a running reload command will leave the old configuration running — Nginx is smart enough not to apply a broken config.

Understanding Nginx Worker Processes

When Nginx starts on a modern server, you will see multiple processes running:

nginx: master process
nginx: worker process
nginx: worker process
nginx: worker process
nginx: worker process

The master process manages the workers and handles configuration reloads. It runs as root so it can bind to ports 80 and 443.

The worker processes handle the actual connections. They run as the www-data user — a low-privilege account — so even if a worker is compromised, the damage is limited.

By default on Ubuntu 24.04, Nginx automatically sets the number of workers to match your CPU core count. On a 4-core server you get 4 workers. This is the correct production setting.

Nginx Logs

Every site configured in Nginx writes to its own log files:

# Access log — every request
/var/log/nginx/example.com.access.log

# Error log — problems and warnings
/var/log/nginx/example.com.error.log

To watch requests in real time:

sudo tail -f /var/log/nginx/example.com.access.log

To watch errors:

sudo tail -f /var/log/nginx/example.com.error.log

The access log shows you every request: the IP address, timestamp, requested URL, HTTP status code, and response size. The error log shows you configuration problems, upstream connection failures, and PHP-FPM errors.

Common Nginx Commands Reference

# Check if Nginx is running
sudo systemctl status nginx

# Start Nginx
sudo systemctl start nginx

# Stop Nginx
sudo systemctl stop nginx

# Reload configuration (zero downtime)
sudo systemctl reload nginx

# Test configuration syntax
sudo nginx -t

# Enable a site
sudo ln -s /etc/nginx/sites-available/example.com /etc/nginx/sites-enabled/

# Disable a site
sudo rm /etc/nginx/sites-enabled/example.com

# View access logs in real time
sudo tail -f /var/log/nginx/example.com.access.log

# View error logs in real time
sudo tail -f /var/log/nginx/example.com.error.log

What Comes After Nginx?

Once Nginx is installed and configured, the typical next steps in building a production server are:

PHP and PHP-FPM — to run Laravel and other PHP applications
Node.js — to run Express.js APIs and applications
SSL certificates with Let's Encrypt — to serve everything over HTTPS
MySQL or PostgreSQL — for your application databases
Redis — for caching and queues

All of these work behind Nginx. The internet sees only Nginx. Everything else is internal.

Summary

Nginx is the backbone of modern web server infrastructure. Here is what you have learned in this guide:

Nginx is a web server, reverse proxy, load balancer, and SSL terminator
It uses an event-driven architecture that handles thousands of connections efficiently
The reverse proxy pattern keeps your applications hidden from the internet
Server blocks define how Nginx handles requests for each domain
PHP applications use PHP-FPM, Node.js and Python apps use proxy_pass
Always test configuration with nginx -t before reloading
Use reload not restart in production to avoid downtime
Security headers and server_tokens off are essential hardening steps

Nginx rewards the time you invest in learning it. Once you understand its architecture, managing multiple applications on a single server becomes straightforward, predictable, and reliable.

Airflow vs Cron: Is Apache Airflow Just a Cron Job?

Sospeter Mong'are — Wed, 12 Aug 2026 11:31:46 +0000

If you're new to data engineering, you may come across Apache Airflow and wonder:

"Isn't Airflow basically just a fancy cron job?"

The short answer is not exactly.

Airflow can do what cron does - schedule tasks to run at specific times - but Airflow is designed to solve a much bigger problem.

A useful way to think about it is:

Cron is primarily a scheduler. Airflow is a workflow orchestrator that also provides scheduling.

Let's break that down.

What is Cron?

Cron is a time-based job scheduler available on Unix and Linux systems.

You can tell cron:

"Run this command every day at 2 AM."

For example:

0 2 * * * python process_data.py

This means:

Every day at 2:00 AM
        |
        v
Run process_data.py

That's the main job of cron.

It answers:

When should this command run?

Cron is excellent for simple scheduled tasks.

For example:

Backing up a database
Cleaning temporary files
Running a simple script
Sending a scheduled report
Running a maintenance task

But things become more complicated when your process has many steps and dependencies.

What is Airflow?

Apache Airflow is a platform for developing, scheduling, and monitoring workflows.

Instead of simply saying:

"Run this command at 2 AM."

you can define an entire workflow.

For example:

2:00 AM
   |
   v
Extract data
   |
   v
Load staging tables
   |
   v
Run dbt transformations
   |
   v
Validate data
   |
   v
Send notification

Airflow understands the relationship between these tasks.

It knows that:

The transformation shouldn't run until the data has been loaded.

And:

The validation shouldn't run until the transformation has completed.

This is where Airflow goes beyond a traditional cron job.

Airflow uses DAGs

The workflows you create in Airflow are called DAGs, which stands for Directed Acyclic Graph.

Don't let the name scare you.

A DAG is essentially a definition of:

What tasks should run, in what order, and under what conditions?

For example:

Extract Customers
       |
       v
Load Customers
       |
       v
Run dbt Customers
       |
       v
Validate Customers

You can also have multiple branches:

             Extract Data
                  |
          +-------+-------+
          |               |
          v               v
   Load Customers    Load Policies
          |               |
          v               v
   dbt Customers     dbt Policies
          |               |
          +-------+-------+
                  |
                  v
            Final Validation

Airflow understands these dependencies.

Airflow can schedule tasks like cron

This is where the comparison comes from.

You can tell Airflow:

Run every day at 2 AM

or:

Run every hour

or:

Run every Monday

So yes, Airflow can perform the scheduling role of cron.

For example:

Every day at 2 AM
        |
        v
Start DAG
        |
        v
Execute workflow

But scheduling is only one part of Airflow.

The biggest difference: Dependencies

Imagine you have this workflow:

Extract data
     |
     v
Load staging
     |
     v
Run dbt
     |
     v
Validate

You don't want Run dbt to execute before Load staging has completed.

Airflow understands this dependency.

Cron, on the other hand, would typically require you to manually coordinate the schedules.

For example, you might end up with:

2:00 AM -> Extract
2:30 AM -> Load staging
3:30 AM -> Run dbt
4:00 AM -> Validate

But what happens if extraction takes 2 hours instead of 30 minutes?

Now your timing assumptions break.

Airflow doesn't need you to rely solely on fixed times.

It can say:

Extract completed successfully
        |
        v
Start Load
        |
        v
Load completed successfully
        |
        v
Start dbt

That's a much more reliable approach for complex workflows.

Airflow can retry failed tasks

Another important feature is retries.

Imagine your pipeline is loading data from an external database and the connection temporarily fails.

Instead of requiring someone to manually restart everything, Airflow can be configured to retry the task.

For example:

Load data
   |
   X
Failed
   |
   v
Wait
   |
   v
Retry
   |
   v
Success

You can configure things such as:

Number of retries
Retry delay
Timeout
Failure behavior

This is particularly useful for data pipelines that depend on external systems.

Airflow provides monitoring

Cron doesn't give you a sophisticated workflow monitoring interface.

You might have logs scattered across servers and applications.

Airflow provides a UI where you can see your workflows and tasks.

For example:

DAG: kehealth_elt

Task                     Status
------------------------------------
extract_claims            SUCCESS
load_claims               SUCCESS
dbt_staging_claims        SUCCESS
dbt_fct_claims            RUNNING
validate_claims           QUEUED

You can immediately see:

What has completed
What is currently running
What failed
What is waiting
How long a task has been running

This becomes extremely valuable when workflows have many steps.

Airflow handles failure paths

You can also define what should happen when something fails.

For example:

                 Run Task
                    |
              +-----+-----+
              |           |
           SUCCESS       FAIL
              |           |
              v           v
        Next Task       Retry
                          |
                    +-----+-----+
                    |           |
                 SUCCESS       FAIL
                    |           |
                    v           v
                 Continue     Alert

This gives you much more control than simply running a command from cron.

Airflow can orchestrate different technologies

Another major advantage is that Airflow doesn't care that every task uses the same technology.

One DAG could potentially orchestrate:

Oracle
   |
   v
Python
   |
   v
SQL
   |
   v
dbt
   |
   v
API
   |
   v
Data validation

For example:

Extract data from Oracle
        |
        v
Run SQL transformation
        |
        v
Run dbt model
        |
        v
Call an API
        |
        v
Validate results

Airflow acts as the coordinator.

How Airflow fits with ADF and dbt

This becomes particularly interesting in modern enterprise data platforms.

You might have an architecture like:

                 SOURCE
                   |
                   v
                 Oracle
                   |
                   v
                  ADF
                   |
                   v
              STAGING TABLES
                   |
                   v
                AIRFLOW
                   |
                   v
                  DBT
                   |
             +-----+-----+
             |           |
             v           v
        Dimensions      Facts
             |           |
             +-----+-----+
                   |
                   v
             Data Consumers

Each technology has a different responsibility.

ADF

ADF may handle:

Connecting to source systems
Extracting data
Moving data
Loading staging tables
Initial orchestration

Think:

"Get the data from A to B."

Airflow

Airflow may handle:

Scheduling
Dependencies
Workflow orchestration
Retries
Monitoring
Error handling
Coordinating different processes

Think:

"Make sure the entire workflow happens in the right order."

dbt

dbt may handle:

Data transformation
Business logic
SQL models
Facts
Dimensions
Data tests

Think:

"Turn the raw/staged data into useful business data."

A real-world example

Imagine an insurance company wants to process claims every night.

The workflow might look like this:

              Oracle
                |
                v
               ADF
                |
                v
       Claims staging table
                |
                v
             Airflow
                |
                v
       Run dbt transformations
                |
                v
          fct_claims
                |
                v
          Data validation
                |
                v
            Reporting

At 2 AM, Airflow could start the workflow.

ADF extracts the claims data from Oracle and loads it into staging.

Once the staging process is complete, Airflow can trigger the dbt models.

dbt transforms the staged data into the final fact and dimension tables.

Airflow then runs validation tasks.

If everything succeeds:

SUCCESS

If something fails:

FAILURE
   |
   +--> Retry
   |
   +--> Log error
   |
   +--> Notify team

That's much more sophisticated than:

2:00 AM
   |
   v
Run script

So, is Airflow a cron job?

The best answer is:

Airflow includes scheduling capabilities similar to cron, but it is not simply a cron job.

Think about the difference this way:

Capability	Cron	Airflow
Schedule tasks	Yes	Yes
Run scripts	Yes	Yes
Manage dependencies	Limited	Yes
Retry failed tasks	Basic/manual	Yes
Workflow visualization	No	Yes
Task monitoring	Limited	Yes
Complex workflows	Difficult	Designed for it
Data pipelines	Not its primary purpose	Designed for it
dbt orchestration	Possible	Common use case
Failure handling	Basic	Advanced

A simple analogy

Imagine you're managing a restaurant.

Cron is an alarm clock.

It says:

"At 6 PM, start cooking."

Airflow is the restaurant manager.

It says:

"At 6 PM, start preparing the food. Once the ingredients are ready, start cooking. Don't serve until the food is ready. If the oven fails, retry. If the problem continues, alert the manager. Keep track of what has been completed."

That's the fundamental difference.

The mental model to remember

If you're working with ADF, Airflow, and dbt, a useful mental model is:

ADF
  |
  | Move/ingest data
  v
STAGING
  |
  v
Airflow
  |
  | Orchestrate workflow
  v
dbt
  |
  | Transform data
  v
FACTS / DIMENSIONS

So instead of thinking:

"Airflow is just another cron job."

think:

"Cron schedules commands. Airflow schedules and orchestrates workflows."

That distinction becomes very important once your data pipelines start having multiple dependencies, retries, validations, long-running tasks, and different systems that need to work together.

Understanding Azure Data Factory Pipelines: From Source Systems to Production

Sospeter Mong'are — Wed, 12 Aug 2026 11:26:21 +0000

If you're new to Azure Data Factory (ADF), pipelines can initially feel complicated. You may see triggers, linked services, datasets, integration runtimes, copy activities, staging tables, transformations, Airflow, dbt, and several other components working together.

The easiest way to understand ADF is to stop thinking of it as just a tool that "moves data" and instead think of it as part of a data journey.

A typical enterprise data flow might look like this:

Source System
     |
     v
ADF Trigger
     |
     v
ADF Pipeline
     |
     v
Integration Runtime
     |
     v
Extract / Read Source
     |
     v
Load to Staging
     |
     v
Validation
     |
     v
Airflow
     |
     v
dbt Transformations
     |
     v
Fact / Dimension Tables
     |
     v
Reporting / Applications

Let's break down what happens at each stage.

1. The Source System

Everything starts with a source.

The source is the system where the original data exists. It could be:

Oracle
SQL Server
MySQL
PostgreSQL
REST APIs
SFTP servers
CSV files
Azure Blob Storage
Other enterprise applications

For example, imagine an organization stores claims data in an Oracle database.

ADF needs to connect to that Oracle database before it can extract anything.

Conceptually:

Oracle Database
      |
      v
  ADF Pipeline

If ADF cannot establish a connection to Oracle, nothing else can happen.

For example, you might encounter an error such as:

ORA-12170: Cannot connect.
TCP Connect timeout of 20000
for host 10.10.10.30 port 1521

This tells you that the problem occurred while trying to establish the network connection to Oracle. It is not necessarily a problem with the SQL query or the target table.

2. The Trigger

A trigger determines when a pipeline should run.

There are several common ways a pipeline can be triggered.

Scheduled trigger

A pipeline might run every night:

Every day
     |
     v
02:00 AM
     |
     v
Start pipeline

Manual trigger

During development or testing, an engineer might manually start the pipeline.

For example:

ADF
 |
 +--> Trigger Now

Event-based trigger

A pipeline can also start when something happens.

For example:

New file uploaded
       |
       v
Event detected
       |
       v
ADF pipeline starts

So, the trigger essentially answers:

When should this pipeline start?

3. The Pipeline

Once the trigger fires, ADF creates a pipeline run.

The pipeline defines what should happen.

A pipeline can contain multiple activities:

Start
  |
  v
Extract data
  |
  v
Load staging table
  |
  v
Validate data
  |
  v
Complete

Each activity performs a specific task.

For example, a pipeline could contain:

Copy Activity
Lookup Activity
Stored Procedure Activity
Data Flow
If Condition
ForEach
Web Activity
Execute Pipeline

The pipeline is essentially the workflow that connects these activities together.

4. The Integration Runtime

This is one of the most important concepts to understand in ADF.

The Integration Runtime (IR) provides the infrastructure used by ADF to connect to data sources and move or process data.

There are different types of Integration Runtime, but one that frequently matters in enterprise environments is the Self-hosted Integration Runtime.

Imagine your Oracle database is inside a private network:

              Private Network

            Oracle Database
            10.10.10.30:1234
                   ^
                   |
                   |
          Self-hosted IR
                   ^
                   |
                   |
                  ADF

ADF may not be able to directly access the Oracle database.

The Self-hosted Integration Runtime provides the connection between ADF and the private data source.

This is why errors such as:

ORA-12170
TCP Connect timeout

can be caused by:

Network connectivity
Firewall rules
Oracle listener issues
Incorrect host or port
Self-hosted IR problems
Routing issues
Database availability

A useful troubleshooting question is:

Can the machine running the Self-hosted IR reach the Oracle server on port 1521?

For example, from the Self-hosted IR machine, you might test:

Test-NetConnection 10.10.10.30 -Port 1521

If the result is:

TcpTestSucceeded : True

the machine can establish a TCP connection to the Oracle server.

If it returns:

TcpTestSucceeded : False

you have a connectivity problem that needs to be investigated before looking at SQL or transformation logic.

5. Linked Services

A Linked Service defines how ADF connects to an external system.

Think of it as the connection configuration.

For example, an Oracle Linked Service may contain:

Host
Port
Service Name / SID
Username
Password
Integration Runtime

Conceptually:

Linked Service
      |
      +--> Oracle
      |
      +--> Host
      +--> Port
      +--> Credentials
      +--> Integration Runtime

The Linked Service answers:

How do I connect to this system?

6. Datasets

A Dataset represents the data you want to work with.

For example:

Oracle Linked Service
        |
        v
     Dataset
        |
        v
tableusers

There is an important distinction between a Linked Service and a Dataset.

Linked Service:

How do I connect to the system?

Dataset:

What data do I want to access?

For example:

Linked Service
      |
      v
Oracle Database

Dataset
      |
      v
tableusers

7. Copy Activity

The Copy Activity is one of the most commonly used ADF activities.

Its primary purpose is to move data from a source to a destination.

For example:

Oracle
   |
   | Copy Activity
   v
SQL Server
   |
   v
Staging Table

Imagine the Oracle source contains:

tableusers

ADF could copy that data into:

stgschema.stg_table

The flow becomes:

Oracle
   |
   v
ADF Copy Activity
   |
   v
stgschema.stg_table

This is often the first major data movement step.

8. Staging

Many enterprise data platforms use staging tables as an intermediate landing area.

Instead of transforming everything directly from the source, the organization first lands the data into staging.

For example:

Oracle
   |
   v
ADF
   |
   v
Staging
   |
   v
Transformation

The staging layer provides a controlled location where the incoming data can be stored before further processing.

For example:

Oracle:
tableusers

        |
        v

ADF

        |
        v

SQL Server:
stgschema.stg_table

9. Transformation

After data has been loaded into staging, it often needs to be transformed.

For example, you may need to:

Clean data
Rename columns
Remove duplicates
Join tables
Apply business rules
Convert data types
Create calculated fields
Build dimensions
Build fact tables

ADF itself can perform transformations using Data Flows and other activities.

However, many modern data platforms separate data movement from data transformation.

For example:

ADF
 |
 v
Staging
 |
 v
Airflow
 |
 v
dbt
 |
 v
Fact / Dimension Tables

This is where understanding the entire ecosystem becomes important.

10. Airflow

In some enterprise environments, ADF is not responsible for the entire data pipeline.

Instead, ADF handles part of the workflow, while Apache Airflow handles downstream orchestration.

You might have:

Oracle
   |
   v
ADF
   |
   v
Staging Tables
   |
   v
Airflow
   |
   v
dbt

Airflow can coordinate what happens after the data has landed.

For example:

ADF finishes
     |
     v
Airflow starts
     |
     v
Run dbt models
     |
     v
Run validations
     |
     v
Complete pipeline

This is why it is important to understand the dependency between ADF and Airflow in your organization's architecture.

11. dbt

dbt is commonly used for the transformation layer.

You might have staging tables such as:

stg_tbltableusers
stg_tblcustomer
stg_tblpolicy

Then dbt transforms them into analytical models:

staging
   |
   v
dbt
   |
   +--> dim_customer
   |
   +--> dim_policy
   |
   +--> fct_claims

This is where raw or staged data becomes structured, business-ready data.

For example:

stg_tbltableusers
            |
            v
       dbt model
            |
            v
        fct_claims

12. Validation

A good data pipeline should not simply say:

"The data moved successfully."

It should also verify that the data is correct.

Validation might include checking:

Record counts
Null values
Duplicate records
Referential integrity
Data types
Business rules
Source-to-target counts

For example:

Source count:
1,500,000

Target count:
1,500,000

Result:
PASS

But if you get:

Source count:
1,500,000

Target count:
1,200,000

Result:
FAIL

the pipeline may need to stop or trigger an alert.

13. Dependencies

ADF pipelines can contain dependencies between activities.

For example:

Extract Data
     |
     v
Load Staging
     |
     v
Validate
     |
     v
Continue

The next activity only executes when the previous activity meets its dependency condition.

You can also have different paths:

             Extract
                |
        +-------+-------+
        |               |
     Success           Failure
        |               |
        v               v
     Continue        Error Flow

Understanding these dependencies is extremely important when troubleshooting.

When a pipeline fails, don't simply look at the pipeline's overall status.

Look at:

Which activity failed?

That usually tells you where the problem actually occurred.

14. Monitoring Long-Running Pipelines

This is particularly important when you're working with enterprise pipelines.

Not every pipeline takes five minutes.

You might encounter pipelines that run for:

10 minutes
1 hour
6 hours
12 hours
Several days

If someone tells you:

"Run this pipeline. It may take several days."

you need to know how to monitor it.

In ADF, you can go to:

Monitor -> Pipeline runs

You can inspect things such as:

Pipeline
Status
Start Time
End Time
Duration
Trigger
Parameters

You can then drill down into the individual activities.

For example:

Activity              Status       Duration
------------------------------------------------
Extract Claims        Succeeded    2h 14m
Load Staging          Running      5h 32m
Validation            Pending      -

This helps you distinguish between:

"The pipeline is still running"

and:

"The pipeline is actually stuck or failing."

That distinction is especially important with long-running data processes.

15. QA and Production

In an enterprise environment, you generally don't make changes directly in production.

A simplified deployment flow might look like:

Development
     |
     v
QA ADF
     |
     | Test
     v
Production ADF
     |
     v
Production Execution

The QA environment allows you to test whether the pipeline behaves as expected.

Once the changes are validated, they can be promoted to production according to the organization's deployment process.

However, don't assume that because something works in QA it will automatically work in production.

The environments can have different:

Databases
Credentials
Integration Runtimes
Network configurations
Firewall rules
Permissions
Connection strings
Parameters

Therefore, a pipeline can succeed in QA and fail in production because of an environmental difference.

Putting Everything Together

A typical enterprise data flow might therefore look like this:

                 SOURCE SYSTEM
                      |
                      v
                   Oracle
                      |
                      v
                 ADF Trigger
                      |
                      v
                 ADF Pipeline
                      |
                      v
            Integration Runtime
                      |
                      v
                Copy Activity
                      |
                      v
                  STAGING
                      |
                      v
                   Airflow
                      |
                      v
                    dbt
                      |
             +--------+--------+
             |                 |
             v                 v
       Dimensions            Facts
             |                 |
             +--------+--------+
                      |
                      v
             Reporting / Apps

The important thing is that ADF may only be one part of the overall data platform.

ADF might handle:

Source connectivity + extraction + data movement + initial orchestration

Airflow might handle:

Workflow orchestration + scheduling + dependencies

dbt might handle:

Data transformation + business logic + analytical models

A Simple Mental Model

If you're learning this ecosystem, remember these questions:

Component	Question
Trigger	When should it run?
Pipeline	What should happen?
Linked Service	How do I connect?
Dataset	What data am I accessing?
Integration Runtime	Where/how does the connection execute?
Copy Activity	How do I move the data?
Data Flow	How do I transform data in ADF?
Staging	Where does the incoming data land?
Airflow	How do I orchestrate downstream processes?
dbt	How do I transform the data into business models?
Validation	Did the process produce correct data?
Monitor	What happened during execution?

The bigger picture is:

SOURCE
  |
  | Oracle / API / Files / etc.
  v
ADF
  |
  | Integration Runtime
  v
EXTRACT
  |
  v
STAGING
  |
  v
AIRFLOW
  |
  v
DBT
  |
  v
FACTS / DIMENSIONS
  |
  v
REPORTING / CONSUMPTION

Once you understand this flow, ADF errors become much easier to reason about.

For example, an error like:

ORA-12170
TCP Connect timeout

points you toward the connectivity layer.

An error like:

ORA-00942
table or view does not exist

points more toward the database object or permissions layer.

A dbt UNION column-count error points toward the transformation layer.

And an Airflow DAG failure points toward the orchestration layer.

The key skill is therefore not just learning how to click through ADF. It is learning to identify which layer of the data pipeline is failing and why.

Are Businesses Really Adopting AI? It Depends on What You Mean by "Adoption"

Sospeter Mong'are — Mon, 10 Aug 2026 12:54:36 +0000

Artificial intelligence has become one of the biggest conversations in technology and business.

Everywhere we look, businesses are being encouraged to "adopt AI", leaders are asking how AI can improve their operations, and employees are experimenting with tools such as ChatGPT, Microsoft Copilot, Claude, and AI-powered automation platforms.

But there is an important question we often overlook:

What exactly do we mean when we say a business has "adopted AI"?

If AI adoption means building a sophisticated AI product from scratch, then relatively few businesses can claim to have adopted it.

But if we consider the different ways AI can be applied across a business, the picture becomes very different.

AI adoption exists on a spectrum.

AI exists in six levels:

Everyday AI
AI-powered workflow automation
AI integration into existing systems
AI-powered products
AI agents and autonomous workflows
Agentic AI for software development

1. Using AI for everyday tasks

The most basic level of AI adoption is simply using AI as a productivity tool.

Employees can use tools such as ChatGPT or Copilot to:

Draft emails and proposals
Write and summarize reports
Analyze documents
Generate meeting notes
Research topics
Brainstorm ideas
Translate or improve content
Write and review code

This may not sound like "AI adoption" in the traditional sense, but it is.

If an employee previously spent two hours preparing a report and can now complete the first draft in 30 minutes with AI assistance, the business is already benefiting from AI.

The AI does not necessarily need to be integrated into the company's core systems for it to create value.

2. Automating business workflows with AI

The next level goes beyond individual productivity.

Businesses can integrate AI into workflows to automate repetitive tasks.

For example, a company could use AI to:

Categorize customer support requests
Qualify sales leads
Extract information from invoices
Process application forms
Summarize customer conversations
Generate business reports
Classify documents
Route requests to the appropriate department

Consider a business receiving hundreds of customer emails every day.

Instead of having employees manually read and categorize every message, an AI-powered workflow could identify the intent of each email, extract important information, categorize it, and route it to the right team.

That is a much deeper form of AI adoption because AI has become part of the business process itself.

3. Integrating AI into existing products and systems

Businesses don't always need to build completely new products to benefit from AI.

They can add AI capabilities to systems they already use.

For example:

A financial platform could introduce intelligent transaction analysis.

An e-commerce platform could provide personalized recommendations.

A customer service system could use AI to summarize conversations and suggest responses.

An internal enterprise system could allow employees to search company documents using natural language rather than navigating through folders and databases.

A business could also use AI to summarize large amounts of operational data and surface insights that would otherwise take employees hours to discover.

In this case, AI becomes another capability within an existing product.

4. Building AI-powered products

This is where many people traditionally think AI adoption begins.

Instead of simply using AI internally, a company builds a product where AI is a fundamental part of the customer experience.

Examples could include:

AI tutors
Financial assistants
Legal document analysis tools
Personalized learning platforms
AI-powered customer service platforms
Intelligent business analytics tools
AI content and marketing platforms

Here, AI is not just helping employees work faster. It is part of what the company is actually selling to its customers.

This is a significantly different level of adoption.

5. Using AI agents to handle business processes

The conversation becomes even more interesting when we move from AI that generates responses to AI agents that can take actions.

An AI agent can potentially receive a goal, reason about what needs to be done, use available tools, and execute multiple steps to achieve that goal.

For example, imagine a customer says:

"I made a payment but my order hasn't been updated."

An AI agent could potentially:

Identify the customer's account.
Check the payment status.
Check the order status.
Determine whether the payment was successful.
Update the relevant system if necessary.
Respond to the customer.
Escalate the issue if it cannot resolve it.

The important difference is that the AI is not simply answering a question. It is interacting with business systems and taking actions.

This is where AI starts moving from being a productivity tool to becoming an operational component of the business.

6. Using agentic AI for coding and software development

There is another form of AI adoption that is particularly interesting for technology companies: agentic AI for software development.

Developers are increasingly using AI coding agents to do more than autocomplete code.

These systems can help developers:

Understand unfamiliar codebases
Implement new features
Fix bugs
Write tests
Refactor existing code
Investigate errors
Review code
Update documentation
Run commands and tests
Create pull requests

This changes the software development workflow itself.

A developer can give an AI coding agent a task such as:

"Investigate why customers are receiving duplicate payment notifications, identify the root cause, implement a fix, write tests, and prepare the changes for review."

The agent can potentially work across multiple files, inspect the existing code, run tests, and make the required changes.

That does not mean developers are becoming irrelevant.

It means the way developers work is changing.

AI Adoption Is Not Binary

This is why I think asking whether businesses are "adopting AI" is too broad.

AI adoption isn't simply:

Adopted AI vs. Not adopted AI.

There are different levels.

A business might start with employees using ChatGPT for productivity.

Then it could move to AI-powered workflow automation.

Then integrate AI into its existing systems.

Eventually, it could build AI-powered products or deploy AI agents that interact directly with its business processes.

These are all forms of AI adoption, but they represent very different levels of maturity and impact.

So, Are we Adopting AI?

I think the more useful question is not simply:

"Are we adopting AI?"

It is:

"At what level are we as a business/individual adopting AI, and where is that adoption creating measurable value?"

Some businesses may be using AI quietly for internal productivity.

Others may be automating workflows.

Some may be integrating AI into their existing products, while others are experimenting with AI agents and more autonomous systems.

And some may still be trying to figure out where AI actually makes sense for their business.

The absence of a flashy "AI-powered" product does not necessarily mean a company isn't adopting AI.

AI adoption can happen behind the scenes.

The Real Question Should Be Value

Ultimately, adopting AI should not be about adopting AI simply because everyone else is doing it.

The better question is:

What problem are we solving with AI?

Does it reduce operational costs?

Does it save employees time?

Does it improve customer experience?

Does it increase revenue?

Does it reduce errors?

Does it help employees make better decisions?

Does it allow a small team to accomplish what previously required a much larger team?

If the answer is yes, then AI is already creating business value.

The future of AI adoption will probably not be defined only by companies building large AI products.

It will also be defined by thousands of businesses quietly integrating AI into the way they operate, serve customers, build software, analyze information, and make decisions.

So, when we ask whether businesses are adopting AI, perhaps we should first define what "adoption" actually means.

The level of adoption looks very different depending on the scope.

Stop Trading Time for Money: A Software Engineer's Guide to Building Wealth Through Code

Sospeter Mong'are — Fri, 31 Jul 2026 08:02:30 +0000

One piece of advice every software engineer should think about is this:

"Figure out how to use your skills to make money, not just trade time for money."

At first, it sounds like someone is saying jobs are bad. That's not the point.

The real message is about leverage.

The Difference Between Working In Software and Building With Software

Most developers begin their careers by trading time for money.

You write code for a company, complete freelance projects, or work as a consultant. Your income depends on the number of hours you work.

There is absolutely nothing wrong with this. In fact, it's how most successful engineers develop their skills, gain experience, and build financial stability.

The limitation is simple: if you stop working, the income usually stops too.

Now imagine a different model.

Instead of building the same solution repeatedly for different clients, you build it once and allow hundreds or even thousands of customers to use it.

That's leverage.

You're Not Just a Developer. You're a Problem Solver.

One of the biggest mistakes software engineers make is believing their value lies in writing code.

It doesn't.

Your real value lies in understanding problems well enough to build solutions that people are willing to pay for.

The code is simply the tool.

Every project you've worked on has exposed you to problems that businesses struggle with every day.

Ask yourself:

What tasks do companies repeat constantly?
What processes waste the most time?
What integrations are always painful?
What manual work could be automated?

Those are business opportunities disguised as technical problems.

Productize What You Already Know

Many developers chase the next trendy idea when their biggest opportunity is hiding in the work they're already doing.

Suppose you've built payment integrations for multiple companies.

Instead of manually implementing the same integrations over and over, you could build:

A unified payments API
A webhook management platform
A payment reconciliation tool
A payment testing environment
A subscription billing platform

Rather than getting paid once for every integration, you create a product that many businesses can subscribe to.

The same principle applies across every industry.

If you've solved a problem more than three times, there's a good chance that problem deserves a product.

Build Software That Works While You Sleep

The beauty of software is that it scales.

Imagine you've built an inventory management system for a retail shop.

If you sell it as custom software, every new client requires another implementation, customization, and deployment.

But if you build it as a cloud-based SaaS platform, every new retailer simply signs up, configures their business, and starts paying a monthly subscription.

The same product serves hundreds or thousands of businesses with minimal additional effort from you.

That's the power of recurring revenue.

Instead of asking:

"How much should I charge for this project?"

Start asking:

"How many businesses have this problem?"

Developers Have More Opportunities Than Ever

Today's software engineers can build businesses around almost any technical skill. Here are a few examples:

SaaS Applications

Inventory Management System
Property Management Platform
School Management System
HR & Payroll Software
Appointment Booking Platform
CRM for SMEs
Restaurant POS System
Hotel Management Software

APIs for Developers

Payment Aggregation API
SMS & Email API
WhatsApp Messaging API
Identity Verification API
Currency Exchange API
Address Validation API
Document Generation API
Authentication & Single Sign-On API

Automation Platforms

Invoice Processing Automation
Employee Onboarding Automation
Approval Workflow System
Procurement Automation
Customer Support Workflow Automation
Marketing Campaign Automation
Data Synchronization Platform

AI-Powered Tools

AI Customer Support Assistant
AI Resume Screening Platform
AI Meeting Notes Generator
AI Document Search & Q&A
AI Contract Analysis Tool
AI Sales Assistant
AI Knowledge Base Assistant

Developer Tools

API Testing Platform
Error Monitoring Service
Database Migration Tool
CI/CD Deployment Platform
Log Management System
Feature Flag Service
Code Quality Analyzer
API Documentation Generator

Industry-Specific Software

Healthcare Clinic Management
Insurance Claims Platform
Construction Project Management
Agriculture Farm Management
Logistics & Fleet Tracking
Manufacturing ERP
Legal Case Management
Sacco Management System

Integration Platforms

ERP Integration Hub
CRM Integration Platform
Accounting Software Connectors
E-commerce Integration Platform
Data Synchronization Engine
Enterprise API Gateway

Monitoring & Security Tools

API Monitoring Dashboard
Uptime Monitoring Service
Security Vulnerability Scanner
Audit Logging Platform
Fraud Detection System
Access Management Platform
Compliance Monitoring Dashboard

The internet has made it possible for a single engineer to serve customers across the world.

You no longer need thousands of employees to build a successful software company.

Sometimes, all you need is one valuable product solving one painful problem.

Your Job Is Not the Enemy

There's a common misconception that every developer should quit their job and become an entrepreneur.

That's terrible advice for most people.

Your job provides more than a salary.

It gives you:

Real-world experience
Access to business problems
Industry knowledge
Professional relationships
Capital to invest in your own ideas

Think of your job as a paid research lab.

Pay attention to the frustrations customers repeatedly face.

Observe the repetitive work your team performs.

Listen to the complaints people make every day.

Hidden inside those frustrations are business opportunities.

Think Like a Business Owner

Instead of asking:

"What app should I build?"

Ask:

"What expensive problem can I solve repeatedly?"

Instead of asking:

"How do I get more freelance clients?"

Ask:

"Can I build a product that hundreds of clients can use?"

Instead of asking:

"What programming language should I learn next?"

Ask:

"What industry do I understand well enough to improve with software?"

Those questions lead to businesses, not just careers.

The Journey

Most successful software entrepreneurs don't start by building billion-dollar companies.

They start by solving one real problem.

Then they solve it again.

Then they turn that solution into a product.

The journey often looks like this:

Developer → Problem Solver → Product Builder → Business Owner

The goal isn't to stop trading time for money overnight.

The goal is to gradually build assets that generate value beyond your own working hours.

Because the most powerful thing about being a software engineer isn't your ability to write code.

It's your ability to create systems that continue delivering value long after you've finished writing them.

Your AI Agent Is Not an Engineer

Sospeter Mong'are — Mon, 27 Jul 2026 08:44:59 +0000

Building Reliable Software in the Age of Agentic Coding

A few weeks into using coding agents seriously, not for autocomplete, but for actually driving multi-file changes, wiring up integrations, standing up agent pipelines, I noticed a pattern. The code that came out looked right. It compiled, it ran, the demo worked. And then, days later, something broke in a way that made it obvious the agent had never actually understood the system it was working in. It had pattern-matched its way to a plausible-looking answer.

That's the core thing to internalize: a coding agent is a very good pattern completer, not an engineer. Engineers hold intent, constraints, failure modes, and tradeoffs in their heads at the same time. Agents hold a context window. Confusing the two is where reliability problems start.

What coding agents are actually good at

To be fair to the tools, they've earned their place in the workflow:

Boilerplate and scaffolding. CRUD endpoints, config files, test skeletons, repetitive glue code across a codebase.
Translating a clear spec into a first draft. If you can describe the shape of what you want precisely, an agent will get you 80% of the way there fast.
Working across unfamiliar syntax. An agent that's seen a thousand times more Terraform or SQL dialects than you have can save real research time.
Mechanical refactors. Rename this across 40 files, migrate this API surface, update this dependency's usage pattern everywhere it appears.

These are real productivity gains. The mistake is extrapolating from "good at boilerplate" to "can be trusted with architecture."

Where they fail

The failures aren't random, they cluster in predictable places:

Silent scope narrowing. Ask an agent to "handle errors properly" and it will handle the errors it can see in the current file, not the ones three layers up the call stack that your actual production traffic hits.
Confident wrongness. Agents don't have a strong signal for "I'm not sure." They'll write a Daraja API callback handler that looks idiomatic and is subtly wrong about idempotency, with the same confidence as one that's correct.
Local optimization, global blindness. An agent fixing a bug in one service has no model of the three other services that depend on the behavior it just changed.
Security as an afterthought. Left to their own devices, agents will happily hardcode a secret, skip input validation, or generate SQL that's one crafted string away from an injection, because none of that breaks the immediate task.
Test theater. Agents will write tests that pass, sometimes because they wrote the test to match the implementation's actual (wrong) behavior rather than the intended one.

None of this is a knock on the tools. It's what you'd expect from a system optimizing for "produce plausible next tokens" rather than "hold this system accountable over time."

Why fundamentals matter more, not less

The tempting narrative is that agentic coding makes engineering fundamentals less relevant, that if the AI writes the code, you don't need to understand it as deeply. The opposite is true.

When you're the one writing every line, your understanding is forced on you by the act of typing it. When an agent writes it, understanding becomes optional, and skipping it is exactly how systems accumulate debt nobody can explain later. The engineer's job shifts from writing correct code to specifying, verifying, and bounding correct code. That's arguably a harder skill, not an easier one. It requires the same grounding in data modeling, failure modes, concurrency, and system boundaries, just applied at review time instead of write time.

If you don't already know what a race condition looks like, you won't catch one in a 400-line diff an agent generated in nine seconds.

Context management

Most agent failures I've traced back aren't reasoning failures, they're context failures. The agent didn't have the right information in front of it, so it filled the gap with something plausible.

Practical habits that help:

Keep the agent's working context scoped to the smallest slice of the system that's actually relevant. A bloated context window doesn't mean better decisions, it often means diluted attention.
Externalize constraints instead of assuming they're inferred: rate limits, idempotency requirements, data retention rules, compliance boundaries. Say them explicitly, every time, rather than trusting the agent remembers them from three turns ago.
Treat the agent's memory of your codebase as a snapshot, not a live view. If you changed something in another session, don't assume it knows.

Testing AI-generated code

Agent-written code needs a different testing posture than human-written code, because the failure distribution is different. Where a human engineer tends to make errors of omission under time pressure, an agent tends to make errors of false confidence, it will implement something that looks complete but quietly diverges from the spec.

That argues for:

Writing the test cases yourself, or at minimum reviewing them line by line, rather than letting the agent write both the implementation and its own tests.
Testing the boundaries the agent wasn't explicitly told about, the empty list, the network timeout, the malformed webhook payload, since those are exactly where pattern-completion breaks down.
Running generated code against real, messy data before trusting it against clean fixtures.

Security risks

Agentic coding introduces a security surface that didn't exist five years ago, and it's not hypothetical:

Secrets leakage. Agents pulling credentials into logs, commits, or prompts they shouldn't.
Dependency sprawl. An agent reaching for a package to solve a small problem, without anyone evaluating its maintenance status or supply-chain risk.
Injection through generated code. String-built queries, unsanitized inputs, permissive CORS, the classic mistakes, just produced faster and in more places at once.
Prompt injection in agent pipelines. If your agent reads external content (a webpage, a customer message, a file) as part of its task, that content can carry instructions the agent will follow unless you've explicitly hardened against it.

None of this is solved by "the agent is usually careful." It's solved by treating every agent-touched surface as untrusted until reviewed, the same discipline you'd apply to a contributor you'd never met.

Reviewing agent decisions

Code review of agent output has to be different from code review of a colleague's PR, because the failure mode is different. A colleague's mistake usually comes with a reasoning trail you can interrogate. An agent's mistake often doesn't, it just produced the output.

That means review has to reconstruct the "why," not just check the "what":

Ask the agent to explain its reasoning before accepting a nontrivial change, and read the explanation critically, not as a formality.
Diff against intent, not just against the previous version. Does this change actually do what was asked, or does it do something adjacent that happens to pass the visible tests?
Be especially suspicious of changes that touch more files than the task seemed to require. That's often where an agent "helpfully" over-scoped.

Keeping humans in control

The workflows that hold up under real production load share a shape: agents propose, humans dispose. Concretely, that looks like:

Agents work in tightly scoped tasks with clear, checkable success criteria, not open-ended "improve this system" mandates.
Every agent-generated change goes through the same review gates as human-generated change, with no shortcut for "the AI wrote it so it's probably fine."
Irreversible or high-blast-radius actions, deployments, schema migrations, anything touching payments or customer data, require an explicit human approval step, not just an agent's self-assessed confidence.
Someone on the team remains accountable for understanding the system end to end, even if they didn't type most of the code.

The bottom line

Coding agents are a genuine force multiplier for the parts of engineering that were always mechanical. They are not a substitute for the parts that were never mechanical: judgment about tradeoffs, understanding of failure modes, and accountability for what ships. Building reliable software with agentic tools doesn't mean trusting them more as they get better. It means getting sharper about exactly where the boundary of their competence sits, and refusing to let that boundary quietly drift because the code that came out looked right.

How to Check Whether Your Git Branch Is Up to Date With QA, Master, and Your Remote Branch

Sospeter Mong'are — Tue, 21 Jul 2026 12:31:15 +0000

When working with Git in a team environment, especially with a workflow such as:

feature branch -> qa -> master

it is important to know whether your branch is up to date before creating or completing a Pull Request.

A common mistake is assuming that this:

Your branch is up to date with 'origin/feature/smongare'.

means your branch is also up to date with qa and master.

It does not.

That message only confirms that your local feature branch is synchronized with its remote feature branch.

Let's look at how to properly check everything.

1. Understanding Local and Remote Branches

Suppose you have:

feature/smongare
qa
master

Your local repository may look like this:

feature/smongare
qa
master

But the remote repository has separate branches:

origin/feature/smongare
origin/qa
origin/master

You can visualize the relationship like this:

Local Repository                 Remote Repository

feature/smongare  <---------->  origin/feature/smongare
qa                <---------->  origin/qa
master            <---------->  origin/master

Your local branches may not automatically know about the latest changes made remotely.

That is why the first step is always to fetch the latest information.

2. Fetch the Latest Changes

Run:

git fetch origin

This does not modify your working files or merge anything into your current branch.

It simply updates your local knowledge of the remote branches:

origin/feature/smongare
origin/qa
origin/master

After running:

git fetch origin

your Git repository knows the latest commit positions of the remote branches.

3. Check Whether Your Feature Branch Is Up to Date With QA

Suppose you are currently on:

feature/smongare

Run:

git rev-list --left-right --count origin/qa...feature/smongare

You might get:

3 5

This means:

3 commits  -> exist in QA but not in your feature branch
5 commits  -> exist in your feature branch but not in QA

In other words:

QA:               A - B - C - D - E
                         \
Feature Branch:          F - G - H - I - J

Your feature branch is missing 3 commits from qa.

You should probably bring the latest changes from qa into your feature branch before completing your PR.

4. Understanding the Output

The command:

git rev-list --left-right --count origin/qa...feature/smongare

returns two numbers:

<commits in QA> <commits in feature branch>

For example:

Result:

0 5

Your feature branch contains everything currently in qa.

Your feature branch has 5 additional commits.

QA:      0 missing commits
Feature: 5 additional commits

This is generally what you expect when developing a feature.

Result:

3 5

Your feature branch is behind qa by 3 commits.

QA:      3 commits your branch does not have
Feature: 5 commits QA does not have

You should consider updating your feature branch.

Result:

3 0

Your feature branch is behind qa.

It has no commits that are not already in qa.

5. Check Your Feature Branch Against Master

You can perform the same check against master:

git rev-list --left-right --count origin/master...feature/smongare

For example:

0 5

This means your feature branch contains all commits currently in master.

However, if your Pull Request is targeting qa, then qa is the more important branch to check before completing the PR.

Your typical workflow might be:

feature/smongare
        |
        v
       QA
        |
        v
     MASTER

Therefore:

Before PR to QA:
Check feature/smongare against qa

Before PR to master:
Check qa against master

6. Check Whether Your Local Feature Branch Matches the Remote Feature Branch

Your git status already tells you:

Your branch is up to date with 'origin/feature/smongare'.

This means:

Local feature/smongare
        =
Remote origin/feature/smongare

You can also verify it manually:

git rev-list --left-right --count origin/feature/smongare...feature/smongare

If you get:

0 0

then both branches are identical.

7. See the Actual Commits Missing From Your Feature Branch

Numbers are useful, but sometimes you want to see the actual commits.

To see commits that exist in qa but not in your feature branch:

git log --oneline feature/smongare..origin/qa

Example:

a1b2c3d Fix Airflow connection
e4f5g6h Update dbt configuration
i7j8k9l Add new environment variables

These are commits that your feature branch does not currently contain.

To see commits in master that your feature branch does not have:

git log --oneline feature/smongare..origin/master

To see commits in your feature branch that are not in qa:

git log --oneline origin/qa..feature/smongare

8. A Complete Check for All Branches

You can run:

git fetch origin

echo "Compared with QA:"
git rev-list --left-right --count origin/qa...feature/smongare

echo "Compared with MASTER:"
git rev-list --left-right --count origin/master...feature/smongare

echo "Compared with remote feature branch:"
git rev-list --left-right --count origin/feature/smongare...feature/smongare

You might get:

Compared with QA:
0 5

Compared with MASTER:
0 5

Compared with remote feature branch:
0 0

This means:

Your feature branch contains all the latest changes from qa
Your feature branch contains all the latest changes from master
Your local feature branch is identical to the remote feature branch
Your feature branch has 5 commits that have not yet been merged into qa

9. The Most Important Check Before a PR

Suppose your Pull Request is:

feature/smongare -> qa

The most important thing is to ensure that your branch works with the latest version of qa.

You can merge the latest qa changes into your feature branch:

git fetch origin

git checkout feature/smongare

git merge origin/qa

If Git reports:

Already up to date.

then your feature branch already contains the latest changes from qa.

If there are conflicts, Git will show them.

You can then:

Resolve the conflicts
Stage the resolved files

git add .

Complete the merge

git commit

Push the updated feature branch

git push origin feature/smongare

Your Azure DevOps Pull Request will then be updated with the latest changes.

The Key Lesson

This:

Your branch is up to date with 'origin/feature/smongare'.

only tells you:

local feature branch = remote feature branch

It does not tell you:

feature branch = qa

or:

feature branch = master

To properly check your branch relationships:

git fetch origin

Then compare branches:

git rev-list --left-right --count origin/qa...feature/smongare

git rev-list --left-right --count origin/master...feature/smongare

And before completing a Pull Request to qa, the safest approach is usually:

git merge origin/qa

This is one of the simplest ways to discover potential conflicts before Azure DevOps blocks or complicates your Pull Request.

Airflow 3 Simple Auth Manager

Sospeter Mong'are — Wed, 15 Jul 2026 10:03:46 +0000

If you've recently upgraded to Apache Airflow 3 and logged into the webserver, you may have noticed something different about how login and permissions work. Gone is the Flask-AppBuilder (FAB) based login system that powered Airflow 2. In its place is the Simple Auth Manager, the new default authentication and authorization system for Airflow 3.

This guide explains what Apache Airflow is for readers who are new to it, what an auth manager does, how the Simple Auth Manager works, how to configure it, and what you need to know before relying on it, especially in production.

What Is Apache Airflow?

If you're brand new to Airflow, here's the short version: Apache Airflow is an open-source platform used to author, schedule, and monitor workflows. A "workflow" in Airflow is called a DAG (Directed Acyclic Graph), which is essentially a series of tasks with dependencies between them, written as Python code. Data engineers commonly use Airflow to orchestrate things like ETL pipelines, machine learning training jobs, and routine data syncs between systems.

Because Airflow has a web-based UI where you can view, trigger, and monitor these workflows, it needs a way to control who can log in and what they're allowed to do once they're in. That's where an auth manager comes in.

What Is an Auth Manager?

An auth manager is the component in Airflow responsible for two things:

Authentication — verifying who a user is (usually via username and password, or an external identity provider)
Authorization — determining what that user is allowed to see and do once logged in

Airflow's auth manager system is pluggable, meaning you can swap in different implementations depending on your needs. You could use one built around your company's identity provider, for example, or write a custom one. The Auth Manager documentation covers the full interface and the available options.

What Is the Simple Auth Manager?

The Simple Auth Manager is the auth manager that comes by default in Airflow 3. As its name suggests, its logic and implementation are intentionally simple. It replaces FabAuthManager, the default in Airflow 2, as part of a broader effort to remove the FAB dependency from Airflow's core and make the authentication layer pluggable.

The intended usage of the Simple Auth Manager is only for development and testing purposes, and it should not be used in production. It's deliberately limited in scope: it doesn't support flexible role definitions or custom mappings between roles and capabilities. If you need a production-grade authentication setup, Airflow supports other pluggable auth managers, including ones built around cloud identity providers, and you can write your own using the auth manager interface.

The full official reference for this feature lives in the Simple Auth Manager documentation.

How User Management Works

Unlike FAB, which stored users and roles in Airflow's metadata database, the Simple Auth Manager is fully config-controlled and doesn't touch the database at all. Users are defined directly in your Airflow configuration file (airflow.cfg), or via environment variables.

Here's the basic format:

[core]
simple_auth_manager_users = "bob:admin,peter:viewer"

The list of users is comma-separated, and each user is a username/role pair separated by a colon. In the example above:

bob is assigned the admin role
peter is assigned the viewer role

Passwords Are Auto-Generated

You'll notice the configuration only specifies usernames and roles, not passwords. That's intentional. Passwords are auto-generated for each user and printed in the webserver logs. Once generated, they're saved to a file set by core.simple_auth_manager_passwords_file, which defaults to $AIRFLOW_HOME/simple_auth_manager_passwords.json.generated. You can open that file and read (or manually update) the passwords if needed.

If you're running Airflow through Breeze (Airflow's local development environment) for local development, two users come predefined out of the box: admin and viewer, each with a password matching their username.

A common gotcha: setting environment variables like AIRFLOW__SIMPLE_AUTH_MANAGER__USERNAME and PASSWORD won't reliably let you pre-seed a specific password. Since Airflow generates a random password on startup by default, this trips up a lot of people trying to force a known password this way. If you need deterministic credentials, editing the generated passwords file directly after startup is currently the more dependable route.

Roles and Permissions

The Simple Auth Manager doesn't let you create custom roles or fine-tune permission mappings. There's no option to manage roles and permissions; they're defined as part of the implementation and can't be modified. There are four fixed roles:

Role	Permissions
Viewer	Read-only access to Dags, assets, and pools
User	Viewer permissions, plus full edit, create, and delete permissions on Dags
Op	User permissions, plus full permissions on pools, assets, config, connections, and variables
Admin	All permissions

This is a deliberate tradeoff. Rather than giving you the granular role-based access control (RBAC) that FAB offered, the Simple Auth Manager sticks to four broad tiers that are easy to reason about and hard to misconfigure, at the cost of flexibility.

Multi-Team Support

If you're running Airflow for multiple teams sharing a single environment, the Simple Auth Manager supports basic team-based resource isolation. When multi-team mode is enabled, users can be associated with one or more teams, and teams restrict which resources (DAGs, connections, variables, pools) a user can access.

Enable it with:

[core]
multi_team = True

Then assign teams to users by adding a third colon-separated field, with multiple teams separated by a pipe character:

[core]
multi_team = True
simple_auth_manager_users = "bob:admin:team1|team2,peter:viewer:team1,alice:op:team2"

In this example, bob is an admin with access to both team1 and team2, peter is a viewer restricted to team1, and alice is an op restricted to team2.

A couple of details worth flagging:

Resources that aren't explicitly assigned to a team are treated as global and remain accessible to everyone, including team-restricted users.
Admins bypass team restrictions entirely. You can still assign an admin to a team, but it has no practical effect since admins already have access to everything.

Disabling Authentication Entirely

For fast local development or throwaway testing environments, the Simple Auth Manager offers an option to skip authentication altogether. This lets anyone who accesses the Airflow UI automatically log in as an admin with full permissions.

You enable it like this:

[core]
simple_auth_manager_all_admins = "True"

This is convenient when you're spinning up a local instance just to test a DAG, but it should never be used anywhere reachable outside your own machine.

Generating a JWT Token for the API

Airflow 3's public REST API is JWT-based, and the Simple Auth Manager provides the endpoint that issues those tokens. To generate one, send a POST request with your username and password:

ENDPOINT_URL="http://localhost:8080"
curl -X 'POST' \
  "${ENDPOINT_URL}/auth/token" \
  -H 'Content-Type: application/json' \
  -d '{
    "username": "<username>",
    "password": "<password>"
  }'

You can then include the returned token in subsequent API requests. If simple_auth_manager_all_admins is enabled, you can generate a token without providing any credentials at all. For more detail, see the Simple Auth Manager token guide and the token API reference.

Key Things to Keep in Mind

It's not built for production. This is the single most important thing to understand. The Simple Auth Manager exists to give Airflow 3 a working, dependency-light default so the core project could drop its hard reliance on FAB. If you're deploying to production, you should either configure a different auth manager or make sure access to your Airflow environment is locked down through other means (network isolation, a reverse proxy with its own auth layer, VPN access, and so on).

No database, no dynamic role editing. Everything lives in your configuration file. That makes it simple to version-control and reason about, but it also means you can't add or edit roles without changing the auth manager implementation itself.

Passwords are ephemeral by default. Since Airflow generates them fresh unless you intervene, plan for how you'll retrieve or fix credentials, particularly in containerized or ephemeral environments like Docker Compose or Kubernetes, where logs might not be easy to inspect after the fact.

Multi-team isolation isn't full multi-tenancy. It's resource-level filtering based on team assignment, not a hard security boundary. Global (unassigned) resources are visible to everyone, so don't rely on it as your only access control layer.

Switching auth managers later is a heavy lift. If you start with the Simple Auth Manager and later move to a production-grade option, all users, roles, and permissions need to be recreated in the new system. It's worth deciding early if the Simple Auth Manager is only a placeholder for you or a long-term fit.

Frequently Asked Questions

Is Simple Auth Manager safe to use in production?
No. It's explicitly intended for development and testing only. For production, use a different auth manager and control access through additional layers like network isolation or a reverse proxy.

Where do I find my auto-generated password?
Check the webserver logs on startup, or look in the file set by core.simple_auth_manager_passwords_file (default: $AIRFLOW_HOME/simple_auth_manager_passwords.json.generated).

Can I create custom roles in Simple Auth Manager?
No. Only four fixed roles exist: Viewer, User, Op, and Admin. Custom roles require a different auth manager.

How do I disable login entirely for local testing?
Set simple_auth_manager_all_admins = "True" under [core] in your Airflow configuration. Everyone who accesses the UI will be logged in as an admin automatically.

Wrapping Up

The Simple Auth Manager is Apache Airflow's answer to needing a lightweight, dependency-free authentication system out of the box for Airflow 3. It's easy to configure, requires no database, and gets you up and running in minutes with config-defined users and four fixed roles. But its simplicity is also its boundary: no custom roles, no dynamic user management, and an explicit warning against production use. If you're evaluating Airflow 3 for a real deployment, treat the Simple Auth Manager as your development sandbox and plan your production authentication strategy separately.

For more on Airflow's authentication options, see the official Auth Manager documentation and Airflow Security guide.

What Is a Proxy Server? Types, Setup, and How It Differs From a VPN

Sospeter Mong'are — Wed, 15 Jul 2026 05:59:40 +0000

If you've ever looked into web scraping, bypassing geo-restricted content, or just browsing more privately, you've probably run into the term "proxy." But what exactly is it, and how does it stack up against a VPN? This guide breaks down everything you need to know before buying or setting one up.

What Is a Proxy?

A proxy server acts as a middleman between your computer and the internet. Instead of connecting directly to a website, your traffic first passes through the proxy server, which forwards your request and relays the response back to you. In the process, it can mask your real IP address, filter content, cache data, or help you get around geographic restrictions.

Types of Proxies

Proxies aren't one-size-fits-all. They vary by protocol, anonymity level, IP source, and intended use case.

By Protocol

HTTP/HTTPS proxy: Handles web traffic only, making it a good fit for browsing. It's simple and fast, but limited strictly to web protocols.
SOCKS proxy (SOCKS4/SOCKS5): Handles any type of traffic, including web browsing, torrents, email, and gaming. More flexible than HTTP proxies, though slightly slower since it lacks built-in caching.

By Anonymity Level

Transparent proxy: Doesn't hide your IP address; the destination server can see you're using a proxy and knows your real IP. Commonly used by companies and schools for content filtering.
Anonymous proxy: Hides your IP address but still identifies itself as a proxy to the destination server.
Elite/high-anonymity proxy: Hides your IP address and gives no indication that a proxy is being used at all.

By Source or Ownership

Datacenter proxy: Uses IP addresses from cloud or hosting providers. Fast and inexpensive, but easier for websites to detect and block.
Residential proxy: Uses IP addresses assigned by ISPs to real households. Harder to detect and typically more expensive, often used for web scraping or ad verification.
Mobile proxy: Uses IP addresses from mobile carriers (4G/5G networks). Very difficult to block, and usually the most expensive option.

By Use Case

Reverse proxy: Sits in front of a server (rather than your device) to distribute load, cache content, or add a layer of security. Nginx and Cloudflare are common examples.
Rotating proxy: Automatically switches IP addresses at set intervals, which is especially useful for web scraping at scale.

How to Connect a Proxy on Your Computer

Setting up a proxy varies slightly depending on your operating system.

On Windows:

Go to Settings → Network & Internet → Proxy
Under "Manual proxy setup," toggle on "Use a proxy server"
Enter the proxy's IP address and port number
Save your settings

On macOS:

Go to System Settings → Network → select your connection → Details (or Advanced)
Open the "Proxies" tab
Check the relevant protocol (Web Proxy HTTP, Secure Web Proxy HTTPS, or SOCKS Proxy)
Enter the server address and port
Apply the changes

At the browser level: Firefox allows you to configure proxy settings independently under Settings → Network Settings, without affecting the rest of your operating system. Chrome, by contrast, defaults to your system-wide proxy settings, though extensions like FoxyProxy can manage rules on a per-browser basis.

Via command line or apps: Many applications, including curl, git, and torrent clients, accept proxy settings directly through config files or environment variables such as http_proxy and https_proxy.

Whichever method you choose, you'll need the proxy's IP address, port number, and possibly a username and password, all of which are provided by whoever supplies the proxy, such as a paid service or your workplace.

How to Confirm Your Proxy Is Connected

Once you've bought a proxy and entered your settings, it's worth double-checking that it's actually working before you rely on it.

1. Check whether your IP address changed. Before connecting, visit a site like whatismyipaddress.com or ipinfo.io and note your current IP. Then enable the proxy and reload the same site. If the IP shown is now different, and matches the proxy provider's location (a different city or country), it's working.

2. Check the proxy status in your OS or browser settings.

Windows: Settings → Network & Internet → Proxy should show "In use" next to your manual proxy entry.
macOS: System Settings → Network → your connection → Details → Proxies should show your configured address under the checked protocol.
Firefox: Settings → Network Settings will show your saved proxy configuration; there's no separate "connected" indicator, but if pages load and the IP check above confirms it, it's active.

3. Use command-line tools. If you set proxy environment variables (http_proxy/https_proxy), test with:

curl -x http://your-proxy-ip:port https://ipinfo.io

If it returns the proxy's IP instead of your own, the connection is live.

4. Watch for authentication prompts. If your proxy requires a username and password, your browser or OS should prompt you for credentials the first time you try to browse. If you're never prompted and pages fail to load, the proxy likely isn't applied correctly.

5. Check for DNS leaks. Even with a proxy active, your DNS requests might still go through your ISP, which can reveal your real location indirectly. Sites like dnsleaktest.com can confirm whether your DNS queries are also routed through the proxy or leaking.

A few signs your proxy is not working: your IP address stays the same as before, pages time out or fail to load entirely (often a wrong port or protocol mismatch), or you see "proxy refused connection" errors (usually wrong credentials or an IP not whitelisted by the provider).

The quickest gut-check: change your settings, then check your IP on a site like ipinfo.io. If it shows the proxy's location instead of yours, you're connected.

Proxy vs. VPN: What's the Difference?

While proxies and VPNs both reroute your internet traffic, they serve different purposes and offer very different levels of protection.

	Proxy	VPN
Encryption	Usually none	Encrypts all traffic
Scope	Often app/browser-specific	Covers the entire device
Speed	Generally faster (no encryption overhead)	Slightly slower due to encryption
Security	Low, traffic can be intercepted	High, encrypted tunnel
Use case	Bypassing geo-blocks, scraping, simple IP masking	Privacy and security, especially on public Wi-Fi
DNS leaks	Common, since only some traffic is routed	Typically routes DNS through the encrypted tunnel too

The Bottom Line

A proxy simply reroutes your traffic and hides your IP address for specific apps or browsers, but it doesn't encrypt anything. That means anyone snooping on the same network, like public Wi-Fi, could potentially still see your data. A VPN, on the other hand, encrypts everything at the operating-system level, protecting all of your device's traffic rather than just what passes through your browser.

If your goal is something like web scraping or accessing region-locked content, a proxy is usually sufficient and more affordable. But if privacy and security are your priority, especially on public networks, a VPN is the better choice.