DEV Community: Sospeter Mong'are

Premature Optimization in Software Engineering: When Trying to Be Clever Becomes the Problem

Sospeter Mong'are — Fri, 23 Oct 2026 09:22:36 +0000

There is a mistake developers make that often looks like good engineering.

You start building a feature, and before it even works, you are already thinking about caching, database indexes, microservices, asynchronous processing, memory allocation, and how the system will handle a million users.

You haven't even shipped the first version.

But somehow, you're already optimizing for a million users.

Welcome to premature optimization.

What Is Premature Optimization?

Premature optimization is the practice of making code faster or more resource-efficient before you have evidence that the optimization is necessary.

It happens when developers optimize based on assumptions rather than measurements.

For example, you are building an API that retrieves customer records. Instead of implementing a straightforward database query, you immediately introduce Redis caching, a complex caching invalidation strategy, background workers, and multiple layers of abstraction.

All this effort goes into solving a performance problem you haven't established exists.

Perhaps the query takes 15 milliseconds. Perhaps the application has only 50 users. Perhaps the real bottleneck is an external API that takes two seconds to respond.

You have optimized the wrong thing.

And the worst part? You now have more code to maintain.

The Origin of "Premature Optimization Is the Root of All Evil"

The phrase is commonly associated with Donald Knuth, one of the most influential computer scientists.

In his 1974 paper, Structured Programming with Go To Statements, Knuth wrote:

"We should forget about small efficiencies, say about 97% of the time: premature optimization is the root of all evil. Yet we should not pass up our opportunities in that critical 3%."

Notice something important about this quote.

Knuth wasn't arguing that performance doesn't matter. He was arguing that developers should distinguish between meaningful optimization and unnecessary attempts to make code more efficient.

There is a critical 3% where performance decisions matter enormously. The challenge is identifying that 3% instead of treating every line of code as a performance emergency.

Why Premature Optimization Is Dangerous

1. You waste time solving imaginary problems

Engineering time is limited. Every hour spent optimizing code that doesn't need optimization is an hour you could have spent improving functionality, fixing bugs, strengthening security, or delivering value to users.

A developer who spends two days optimizing an operation that executes once per day might achieve an impressive benchmark and almost zero business impact.

2. You introduce unnecessary complexity

Simple code is often easier to understand, test, debug, and maintain.

Premature optimization can introduce custom caching mechanisms, complicated concurrency patterns, unnecessary abstractions, and clever algorithms that future developers will struggle to understand.

The code might become marginally faster, but significantly harder to maintain.

And remember, software doesn't end when you write it. Someone has to operate it, debug it, extend it, and eventually replace it.

3. You optimize the wrong bottleneck

This is perhaps the most expensive mistake.

Imagine your API takes three seconds to return a response. You assume your Python code is slow, so you spend hours optimizing loops and rewriting functions.

After profiling the application, you discover that 2.7 seconds are spent waiting for a poorly indexed database query.

Your Python optimization was never going to make a meaningful difference.

Without measurement, optimization is often just educated guessing.

4. You create technical debt before delivering value

Every additional component introduces decisions and potential failure points.

Caching requires invalidation strategies. Background jobs require retry mechanisms. Distributed services require observability and communication management.

These are valuable engineering tools when the problem warrants them. Introduced unnecessarily, they become liabilities.

The goal is not to avoid sophisticated architecture. The goal is to earn that complexity by solving a real problem.

A Practical Example: Building a Payment API

Suppose you are developing an API that processes mobile money payments.

Your first implementation validates a request, calls the payment provider, stores the transaction, and returns a response.

It works correctly, but you want to make it production-ready.

A premature optimizer might immediately introduce:

Redis caching across transaction-related endpoints.
Multiple microservices for payment processing.
A distributed message broker before there is a demonstrated need.
Complex concurrency mechanisms to handle hypothetical traffic.
Custom database optimizations without examining query performance.

A more disciplined engineer asks different questions.

Are payment requests processing correctly?
Are duplicate callbacks causing duplicate transactions?
Are database queries slow under realistic traffic?
What happens when the payment provider times out?
Can the system handle the expected transaction volume?
What do the logs, metrics, and traces reveal?

These questions matter because payment systems have correctness, reliability, and security requirements that cannot be sacrificed for speed.

For example, implementing idempotency to prevent duplicate payment processing is not pointless premature optimization. It is a correctness and reliability requirement.

Likewise, adding indexes to address a measured slow query is a justified optimization.

The distinction is simple: build for known requirements, measure actual behavior, and optimize based on evidence.

So, When Should You Optimize?

A practical approach is to follow four steps.

1. Make it work.

Implement the feature using a clear, maintainable design. Focus on correctness, readability, security, and the actual business requirement.

2. Establish a baseline.

Measure response times, throughput, memory consumption, database query duration, and error rates where relevant.

You cannot confidently claim an improvement if you don't know where you started.

3. Profile the application.

Use tools appropriate to your stack, such as Python profilers, database query plans, application performance monitoring, distributed tracing, and load-testing tools.

Find out where the time and resources are actually going.

4. Optimize, then measure again.

Address the bottleneck with the greatest impact. Rerun your tests and compare the results against the baseline.

If the optimization adds significant complexity for negligible improvement, reconsider whether it is worth keeping.

Does This Mean We Should Ignore Scalability?

Absolutely not.

This is where the conversation often goes wrong.

Good engineering requires anticipating future needs. Security, data integrity, accessibility, and architectural constraints should not be ignored until something breaks.

If you know your application must process thousands of transactions per second, you should design and test for that requirement from the beginning.

If you are building a multi-tenant SaaS platform, tenant isolation and authorization must be part of the architecture from day one.

If you are developing an AI application, you should consider token costs, latency, rate limits, and model availability early because they directly affect the product.

But there is a difference between designing for known constraints and optimizing for imaginary ones.

You should make sound architectural decisions early without unnecessarily implementing every possible performance improvement upfront.

The Lesson for Software Engineers in the AI Era

This principle is even more relevant today.

AI coding assistants can generate caching layers, database abstractions, queues, complex design patterns, and entire service architectures in minutes.

But the fact that an AI tool can generate a solution quickly doesn't mean that solution is necessary.

A developer who asks an AI assistant to optimize code without profiling it may end up with more complexity, new bugs, and little or no measurable improvement.

The responsibility of the engineer is to understand the problem, evaluate the proposed solution, verify the results, and determine whether the additional complexity is justified.

AI can help write the code. Engineering judgment determines whether the code should exist in the first place.

Final Thoughts

Premature optimization is not about writing simple code forever. It is about knowing when simplicity is sufficient and when complexity becomes necessary.

Don't spend days optimizing a feature before understanding its performance characteristics. Don't introduce distributed systems because they sound impressive. Don't confuse architectural sophistication with engineering excellence.

Build the right thing. Measure how it behaves. Identify the bottleneck. Optimize where it matters.

And always remember: the fastest code to maintain is often the code you never needed to write.

PHP2Shell: Two Critical PHP Remote Code Execution Flaws (CVE-2026-109618 and CVE-2026-109619) and How to Patch Them

Sospeter Mong'are — Thu, 22 Oct 2026 08:08:58 +0000

Introduction

On October 21, 2026, the PHP project shipped emergency security releases for two critical vulnerabilities nicknamed PHP2Shell. They let an unauthenticated attacker run code on a server through an ordinary HTTP request, and they live in the PHP engine itself, so a perfectly written application can still be exposed. Because PHP powers WordPress, Magento, Drupal, countless control panels and most shared hosting, the reach is enormous.

The confusing part is the patching. Ubuntu and other distributions fix PHP without changing the version number, so a server reporting an old version such as 8.3.6 might be vulnerable or might be fully patched. Shared hosting customers and WordPress owners also can't simply "run an update". This article explains what the flaws are, which versions are affected, which are fixed, and exactly what to do whether you run your own Ubuntu server, a cPanel server, shared hosting, or a WordPress site.

What happened

On October 21, 2026, the PHP team released emergency security updates for PHP2Shell, two critical vulnerabilities (CVE-2026-109618 and CVE-2026-109619) that could let attackers execute code remotely on web servers without logging in. The PHP security team rated each 9.2 on CVSS v4.0, and security firm watchTowr rated it a full 10.0.

The flaws are in the PHP engine itself, not in any one application. Magento, Shopware and WooCommerce stores are exposed even when their own code is clean.

The two vulnerabilities

CVE-2026-109618, "RCE via OOM corruption." Out-of-memory handling can interrupt PHP while internal request data is only partially initialized, and request cleanup may then treat the invalid data as valid. JFrog's analysis says it can also cause Heartbleed-style memory disclosure.
CVE-2026-109619, "RCE via stale global state." A PHP worker can reuse stale state from a previous request.

Researchers describe the internals differently. Several tie both bugs to PHP's session upload-progress feature, which is why disabling that feature is the main stopgap (see below).

Affected versions

Versions as early as PHP 5.4 through PHP 8 are affected, so the vulnerable code is about 14 years old. Every PHP installation that serves web requests is affected, including PHP-FPM, Apache with mod_php, php-cgi, FrankenPHP and LiteSpeed.

Fixed versions

Branch	Vulnerable	Fixed
8.5	before 8.5.12	8.5.12
8.4	before 8.4.27	8.4.27
8.3	before 8.3.36	8.3.36
8.2	before 8.2.35	8.2.35
8.1 and earlier	End of life	No standard release

If you run an end-of-life branch, moving to a supported branch is the real fix. PHP 8.2 also reaches end of life on 31 December 2026, so don't treat 8.2 as a long-term home.

Is it being exploited?

Reports differ. Amazon Threat Intelligence recorded an exploitation attempt on October 17, four days before public disclosure, and Cloudflare has also reported attack traffic, including attempts to evade its protections. However, as of October 21, PHP had not reported in-the-wild exploitation, and CISA had not added either CVE to its catalog. Treat it as actively targeted either way.

Do you really need to upgrade?

Yes, you need to upgrade, but not to 8.3.36 if you installed PHP from Ubuntu's own repositories. Staying on the Ubuntu package is fine and is the recommended route. Ubuntu backports fixes without changing the version number, so you need the right build of 8.3.6, not a new PHP version.

For Ubuntu 24.04 (Noble), the fixed package is 8.3.6-0ubuntu0.24.04.13. That build is fixed even though it says 8.3.6. Anything older, such as .9, .10 or .11, is still vulnerable, even though php -v looks identical. Ubuntu's advisories are USN-8948-1 and USN-8948-2.

The rule of thumb: the version string alone doesn't tell you whether you're patched. If PHP came from your distribution, check the package build and changelog. If it came from upstream or a third-party repository (such as Ondřej Surý's PPA), you need 8.3.36 or later.

How to patch your own server

Ubuntu and Debian (distribution packages)

# What's installed?
dpkg -l | grep -E '^ii\s+php8.3'

# Upgrade
sudo apt update
sudo apt upgrade        # or: sudo apt install --only-upgrade 'php8.3*' 'libapache2-mod-php8.3'

# Confirm: on Ubuntu 24.04 the version should end in .13 or higher
dpkg -l php8.3-cli | tail -1
apt changelog php8.3-cli | grep -i CVE-2026

# Restart so running workers pick up the fix
sudo systemctl restart php8.3-fpm     # PHP-FPM
sudo systemctl restart apache2        # mod_php

Note the command is apt changelog, not apt-changelog. Restarting is essential: updated packages do not affect already-running workers. If you added a third-party PPA, check which repository supplies the package with apt-cache policy php8.3-cli and make sure you reach 8.3.36 or later.

On Debian, check the Debian security tracker for the fixed package version of your release, as I could not verify it at the time of writing.

RHEL, Rocky and AlmaLinux

sudo dnf upgrade 'php*'
sudo systemctl restart php-fpm     # or httpd, depending on your setup
rpm -q --changelog php-fpm | grep CVE-2026

cPanel/WHM servers (root access)

cPanel ships its own PHP builds (ea-php), separate from the operating system's PHP. Fixed builds were backported at the same version for older branches: ea-php74 7.4.33-19, ea-php80 8.0.30-12 and ea-php81 8.1.34-5. Update with /scripts/upcp, or dnf upgrade 'ea-php*' (yum on EL7), then restart Apache or PHP-FPM. The fixed build numbers for the newer ea-php branches (8.2 and up) were not confirmed, so check cPanel's EasyApache 4 changelog. If you resell hosting, check each account's PHP version in MultiPHP Manager, because accounts pinned to different versions can be patched at different times.

Shared hosting (no root access)

You can't patch PHP yourself on shared hosting. Your host has to. Here is what you can do:

Check your PHP version in cPanel under MultiPHP Manager, or put <?php phpinfo(); in a temporary file and delete it right after. Remember a backported build can still show an old version number.
Ask your host in writing whether PHP2Shell (CVE-2026-109618 and CVE-2026-109619) is patched, and when.
Apply the stopgap. In cPanel's MultiPHP INI Editor, or in a .user.ini file in your site root, set session.upload_progress.enabled = Off. Some hosts lock these settings, so confirm it took effect. Updating is still the only complete fix.
Don't rely on a lower-numbered PHP version as a workaround. Everything from PHP 5.4 up is affected.

WordPress sites

The vulnerability is in PHP, not in WordPress, so updating WordPress core, themes or plugins does not fix it. Because the flaws are in the PHP engine, sites are exposed even when their own code is clean. Do this:

Get PHP patched (by your host or on your server), then restart PHP.
Disable session.upload_progress in the meantime.
Put a WAF in front of the site if you can. Cloudflare has reported attack traffic targeting these flaws.
Look for signs of compromise: unfamiliar PHP files in wp-content/uploads, new admin users, modified .htaccess, and unusual file modification dates. Check logs back to at least October 17, since exploitation was recorded four days before disclosure.
Take a backup before and after patching.

If you can't patch immediately

Updating is the only complete fix. As a stopgap, disable the session upload-progress feature:

session.upload_progress.enabled = Off

Put it in your php.ini or a conf.d file, then restart PHP. Also consider WAF rules and request-size limits at your proxy. After patching, review logs for unusual multipart POST requests and unexpected PHP worker crashes or memory errors, going back to at least October 17.

Common mistakes to avoid

Assuming php -v showing 8.3.6 means you are vulnerable (on Ubuntu it may be patched) or that it means you are safe (it may not be). Check the package build.
Upgrading the package but not restarting PHP-FPM or Apache.
Updating WordPress and assuming that closes the hole.
Using apt-changelog, which is not a standard command. Use apt changelog <package>.
Staying on an end-of-life PHP branch with no fix available.

Conclusion

PHP2Shell is serious because it needs no login, sits in PHP itself, and has been probed in the wild since before disclosure. The good news is that fixes are out for every supported branch: 8.5.12, 8.4.27, 8.3.36 and 8.2.35 upstream, and backported builds from Ubuntu (8.3.6-0ubuntu0.24.04.13 on 24.04) and cPanel. If you manage your own server, update, restart PHP and verify the package changelog. If you are on shared hosting, press your provider for a patch date and apply the session.upload_progress stopgap. If you run WordPress, remember the fix lives at the PHP layer, so check for compromise as well as patching. Either way, act now rather than waiting on a version number to tell you whether you are safe.

Sources

Help Net Security: https://www.helpnetsecurity.com/2026/10/21/php2shell-php-vulnerabilities-patched-cve-2026-109618-cve-2026-109619/
Ubuntu: https://ubuntu.com/blog/php2shell-vulnerability-fixes-available
Sansec: https://sansec.io/research/php2shell
JFrog Security Research: https://research.jfrog.com/post/php2shell-all-you-need-to-know-about-cve-2026-109618-and-cve-2026-109619/
Datadog Security Labs: https://securitylabs.datadoghq.com/articles/php2shell-critical-vulnerabilities-in-php/
SOCRadar: https://socradar.io/blog/php2shell-cve-2026-109618-cve-2026-109619/
Brandefense: https://brandefense.io/blog/php2shell-php-rce-cve-2026-109618-109619/
Qualys: https://blog.qualys.com/vulnerabilities-threat-research/2026/10/20/php-announces-critical-security-release-for-two-vulnerabilities
Upwind: https://www.upwind.io/feed/initial-assessment-php2shell-two-critical-unpatched-php-rces-cve-2026-109618-cve-2026-109619
Abstract Security: https://www.abstract.security/blog/php2shell-critical-php-rce-vulnerabilities-cve-2026-109618-and-cve-2026-109619
php-fix (cPanel and Ubuntu build tables): https://github.com/ersinkoc/php-fix

Understanding WhatsApp Pricing: What Individuals, Businesses, and Developers Need to Know

Sospeter Mong'are — Sat, 10 Oct 2026 09:36:30 +0000

WhatsApp is one of the most popular messaging platforms in the world. Millions of people use it every day to chat with friends, make voice and video calls, share files, and stay connected with family. For ordinary users, these activities are generally free, provided they have an internet connection.

But things work differently when businesses use WhatsApp to communicate with customers at scale.

A business sending order confirmations, payment notifications, verification codes, appointment reminders, or marketing messages may incur messaging costs. The amount depends on the type of message, the customer's country, and whether the business uses Meta's official WhatsApp Business Platform directly or through a third-party provider.

Understanding these differences is important for businesses planning to use WhatsApp for customer communication, developers building WhatsApp integrations, and entrepreneurs offering WhatsApp messaging services.

Let's break it down.

WhatsApp Is Free for Everyday Users

For individuals, WhatsApp does not charge a fee for every message sent or received. You can send text messages, share photos and documents, and make voice or video calls without paying WhatsApp a per-message charge.

However, WhatsApp still uses your internet connection. If you are using mobile data, your network provider may charge you for data usage. If you are connected to Wi-Fi, the internet service itself may have a cost.

WhatsApp also offers the WhatsApp Business app, which small businesses can use to communicate with customers, organize conversations, create product catalogs, and set up automated greetings and away messages.

The app is generally free to use, although optional business services and related tools may have their own charges.

The important distinction is that using WhatsApp as an individual or through the standard WhatsApp Business app is not the same as using the official WhatsApp Business Platform, commonly called the WhatsApp API.

What Is the WhatsApp Business Platform?

The WhatsApp Business Platform is designed for businesses that need to communicate with customers programmatically or at scale.

For example, an e-commerce business might want to send a WhatsApp message whenever a customer places an order. A fintech company might need to send payment confirmations. A bank might send transaction alerts, while a software application might send one-time passwords during account registration.

Instead of manually sending each message from a phone, the business connects its systems to WhatsApp through an API.

This allows applications to send messages automatically in response to events such as:

A customer completing a payment.
An order being placed or shipped.
A user requesting a verification code.
An appointment being booked.
A customer asking for support.
A business launching a promotional campaign.

Meta charges businesses under its WhatsApp Business Platform pricing model. Depending on how the business connects to the platform, additional software or provider charges may also apply.

Understanding the Four WhatsApp Message Categories

WhatsApp Business Platform messaging has four main categories: marketing, utility, authentication, and service.

Each category serves a different purpose and has different pricing rules.

Marketing Messages

Marketing messages are used to promote products, services, offers, or customer engagement campaigns.

Examples include:

"Get 20% off your next purchase this weekend."
"We have new products available. Check them out."
"Come back and complete your shopping."
"Your favourite items are now on sale."

Marketing messages are generally more expensive than utility messages because they are promotional in nature.

For businesses, this means that sending marketing messages to thousands of customers can become costly if the campaign is not carefully planned.

Utility Messages

Utility messages communicate specific information about a transaction, account, order, or service that a customer has requested or is expecting.

Examples include:

"Your order has been confirmed."
"We have received your payment of KSh 2,500."
"Your package is out for delivery."
"Your appointment is scheduled for tomorrow at 10:00 AM."
"Your monthly statement is ready."

These messages are useful for businesses that need to keep customers informed without requiring manual communication.

Utility messages generally cost less than marketing messages. However, the message must meet Meta's classification requirements. A business cannot simply label a promotional message as utility to obtain a lower rate.

Authentication Messages

Authentication messages help verify a customer's identity or confirm that they control a particular account or phone number.

Examples include:

One-time passwords (OTPs).
Login verification codes.
Account registration codes.
Identity verification codes.

These messages are particularly useful for fintech applications, e-commerce platforms, and applications that require secure account access.

Authentication pricing is determined by the applicable market rate and authentication rules. It should not automatically be assumed to match the utility rate.

Service Messages

Service messages are replies businesses send when customers initiate conversations or request assistance.

For example, a customer might ask:

"Have you received my payment?"
"Where is my order?"
"How do I reset my password?"
"Can someone help me with my account?"

The business can respond through its customer support team or an integrated messaging system.

Service conversations are particularly important because they allow businesses to provide customer support without treating every interaction like a marketing campaign.

How Much Does WhatsApp Business Messaging Cost?

There is no single worldwide price for sending a WhatsApp Business Platform message.

Meta's rates depend primarily on the recipient's country or market and the message category. Authentication messages may also be subject to additional pricing rules, including international authentication rates where applicable.

The following figures from the initial research are illustrative estimates, not guaranteed current prices.

Message category	Estimated US rate	Estimated Kenya rate
Marketing	$0.0250	KSh 3.20-3.30
Utility	$0.0040	Around KSh 0.52
Authentication	Varies	Verify the applicable rate
Service	Often free within the applicable window	Often free within the applicable window

These figures should be checked against Meta's current official rate card before budgeting or publishing a price list. Rates can change, and the effective cost depends on the recipient's market and the applicable billing rules.

For official information, consult the WhatsApp Business Platform pricing page.

A Simple Cost Example

Imagine a business sends 10,000 marketing messages to customers in Kenya.

Using an illustrative rate of KSh 3.25 per delivered message:

10,000 × KSh 3.25 = KSh 32,500.

The estimated Meta messaging charge would be KSh 32,500, before any provider fees, applicable taxes, or other service charges.

Now imagine the same business sends 10,000 utility messages at an illustrative rate of KSh 0.52 each.

10,000 × KSh 0.52 = KSh 5,200.

That is a substantial difference.

This example shows why understanding message categories is important. Businesses should design their messaging workflows around the actual purpose of each message rather than assuming all WhatsApp messages cost the same.

The 24-Hour Customer Service Window

One of the most important concepts in WhatsApp pricing is the 24-hour customer service window.

Whenever a customer sends a message to a business, a 24-hour window opens or resets. During this period, the business can respond using free-form messages without requiring an approved template, subject to WhatsApp's messaging policies.

However, the pricing rules have changed over time, so businesses need to pay attention to the date of the applicable pricing policy.

Important pricing update: October 2026

As of October 1, 2026, Meta introduced changes to service and utility message charges. This means older articles stating that all service replies and utility messages within the 24-hour window are free may no longer be accurate.

Under the updated rules:

Service messages: Businesses receive an allowance of 1,000 free service messages per business phone number per month. Additional service messages are charged at the applicable rate.
Utility messages: Utility template messages sent within the 24-hour customer service window are now subject to the applicable per-message charge.
Marketing messages: Marketing template messages remain chargeable per delivered message.
Authentication messages: Authentication template messages remain chargeable at the applicable authentication rate.

These changes make it especially important to check current pricing before designing a customer communication system. See the WhatsApp Business Platform pricing page and this report on the October 2026 pricing changes in Kenya.

What Is a WhatsApp Message Template?

A message template is a predefined message that a business submits to Meta for approval before using it in situations where templates are required, particularly when initiating conversations outside the customer service window.

Templates can contain text, images, buttons, and variables that are replaced with customer-specific information.

For example, a payment notification template might look like this:

"Hello {{1}}, we have received your payment of KSh {{2}}. Your transaction reference is {{3}}. Thank you for using our service."

The placeholders allow the same approved template to be reused for thousands of customers without manually composing each message.

Templates are generally categorized as marketing, utility, or authentication. Meta determines whether the content qualifies for the requested category.

A critical point is that approval does not mean a message is free. A template can be approved and still incur a charge every time it is successfully delivered.

Are Incoming WhatsApp Messages Free?

Businesses are not charged by Meta simply because a customer sends them an inbound message.

However, receiving a message can trigger a customer service window, during which the business can respond according to the applicable messaging rules.

Businesses should distinguish between the cost of receiving a message and the cost of sending a response. They should also check whether their messaging provider charges separately for inbound or outbound traffic.

For example, Twilio charges its own fee of $0.005 per WhatsApp message, including inbound and outbound messages, in addition to applicable Meta template fees. Other providers have different pricing arrangements.

This is why a business can incur messaging costs even when Meta does not charge for a particular interaction.

Meta's Charges Versus Third-Party Provider Charges

This is another area that businesses often overlook.

There are two main cost components when using the WhatsApp Business Platform through a third-party provider.

1. Meta messaging fees

These are the charges imposed by Meta for eligible delivered template messages and other billable message categories.

2. Provider or platform fees

These are charges imposed by the software provider that connects the business to WhatsApp. Depending on the provider, they may include:

A fee per message.
Monthly subscription charges.
A fee per WhatsApp number or connected account.
Charges for team inboxes and customer support tools.
Additional fees for automation, analytics, or integrations.

For example, a business using Twilio's WhatsApp service may pay both Meta's applicable template fees and Twilio's separate per-message fee.

Other providers may offer monthly plans, usage-based billing, or a combination of the two.

Consequently, two businesses sending the same number of WhatsApp messages can pay different total amounts because they use different providers.

Understanding WhatsApp API Pricing for SaaS Businesses

Suppose you are building a WhatsApp messaging platform for other businesses. Your customers might use your software to send OTPs, payment confirmations, order updates, or group announcements.

Your pricing model must account for more than Meta's messaging rates.

Your costs may include:

Meta messaging fees.
WhatsApp infrastructure and hosting.
API servers, databases, and background jobs.
Payment processing fees.
Customer support.
Monitoring, maintenance, and security.
Provider fees, where applicable.

If you charge customers a flat monthly subscription, you need to estimate how much messaging each customer is likely to consume.

Consider a simplified example in which your platform charges a customer KSh 1,000 per month.

If the customer sends 200 marketing messages at an illustrative cost of KSh 3.25 each, the Meta messaging cost alone would be KSh 650.

That leaves KSh 350 before accounting for hosting, support, payment processing, and other operational expenses.

If the customer sends 1,000 marketing messages at the same illustrative rate, the Meta messaging cost would be KSh 3,250. A KSh 1,000 subscription would no longer cover the messaging cost.

This is why messaging platforms need to understand usage patterns and establish sustainable pricing models.

Possible approaches include:

Monthly subscriptions with reasonable usage limits.
Pay-as-you-go pricing.
Prepaid messaging credits.
Subscription plans with included message allowances and additional usage charges.

The right model depends on your target customers, expected message volumes, and operating costs.

How Businesses Can Reduce WhatsApp Messaging Costs

WhatsApp can be a valuable communication channel, but poorly designed messaging workflows can create unnecessary expenses.

Here are several practical ways to manage costs.

Send messages that serve a clear purpose

Avoid sending unnecessary notifications. A customer may need a payment confirmation, but they might not need several messages repeating the same information.

Choose the correct message category

Use utility messages for genuine transactional updates, marketing messages for promotions, and authentication messages for verification.

Do not misclassify messages to obtain cheaper rates. Meta may reclassify templates that do not meet its requirements.

Monitor delivered messages

Meta's fees generally depend on billable messages being delivered, not simply on attempts to send them. Monitor delivery-status webhooks and your billing reports to understand actual usage.

Track costs by customer and use case

If you operate a messaging SaaS, measure how much each customer spends on OTPs, notifications, and marketing campaigns.

This helps you identify expensive accounts and adjust subscription limits before your margins become negative.

Use customer-initiated conversations appropriately

Customer support remains an important use case for WhatsApp. However, because the pricing rules can change, do not assume every reply within the 24-hour window is free.

Check the current rules for the message category and your provider's fees.

Take advantage of volume-based pricing where available

Meta offers volume-based pricing tiers for certain utility and authentication messages. Eligible businesses may receive lower rates as their qualifying message volumes increase.

The applicable thresholds and discounts vary by market and category, so consult the current rate card.

Common Misconceptions About WhatsApp Pricing

Myth: All WhatsApp messages are free.

Reality: Ordinary personal messaging is generally free of WhatsApp per-message charges, but businesses using the official platform may pay for delivered messages.

Myth: Every WhatsApp API message costs the same.

Reality: Rates vary by message category, recipient's market, and applicable pricing rules.

Myth: All customer support replies are free.

Reality: The rules changed in October 2026. Businesses should account for the monthly free service allowance and applicable charges beyond it.

Myth: The WhatsApp Business app and WhatsApp Business Platform are the same product.

Reality: The Business app is designed for managing customer conversations through an app, while the Business Platform supports programmatic messaging and integrations at scale.

Myth: Using an official API provider eliminates Meta messaging charges.

Reality: An official provider does not automatically eliminate Meta's fees. Provider fees may be added on top of them.

Myth: A message template is free once approved.

Reality: Approval allows the template to be used, but billable messages can still incur charges.

Final Thoughts

Understanding WhatsApp pricing is essential for anyone building a business around WhatsApp messaging or integrating it into an existing application.

The most important thing to remember is that there is no single universal price for WhatsApp messages. Costs depend on how you use the platform, the type of message you send, the customer's country, the applicable pricing rules, and any additional charges imposed by your provider.

For developers, the lesson is to design systems that track message categories, delivery statuses, and usage costs from the beginning.

For business owners, it is to understand the total cost of communicating with customers before choosing a subscription or launching a large messaging campaign.

And for SaaS founders, it is to make sure that the price customers pay covers both messaging costs and the operational expenses of running the platform.

Before setting a budget or publishing a price list, always check Meta's latest rate card and your provider's pricing terms. WhatsApp pricing changes over time, and a pricing model that worked last year may not be profitable today.

AI Is Generally Not What the Average Person Thinks It Is

Sospeter Mong'are — Thu, 08 Oct 2026 11:38:46 +0000

Artificial Intelligence is everywhere.

It writes emails, generates images, writes code, recommends what we watch, detects fraud, translates languages, answers questions, drives vehicles, analyzes medical images, and increasingly interacts with software on our behalf.

And yet, there is a good chance that many people have the wrong mental model of what AI actually is.

Ask the average person what AI is, and you may hear descriptions of a machine that thinks like a human, understands everything, has its own intentions, or is slowly becoming conscious.

That picture has been shaped by science fiction, movies, sensational headlines, and increasingly aggressive AI marketing.

The reality is both less magical and, in some ways, more impressive.

AI is not one thing

One of the first misconceptions is treating "AI" as if it were a single technology.

It is not.

Artificial Intelligence is a broad field of computing focused on building systems that can perform tasks that traditionally require some form of human intelligence.

Machine learning is one approach to building AI systems. Instead of explicitly programming every rule, we train models to identify patterns from data.

Deep learning is a subset of machine learning that uses neural networks with many layers.

Generative AI is another category of AI systems that can generate new content such as text, images, audio, video, and code.

Large Language Models, such as the models behind ChatGPT, are specifically designed to work with language and related modalities.

So when someone says "AI," they could be talking about a recommendation algorithm, a fraud detection system, a computer vision model, a voice recognition system, a self-driving system, or a large language model.

They are not all doing the same thing.

The biggest misconception: AI is a digital human

This is probably the biggest gap between perception and reality.

People often interact with an AI system and think:

"It understands me."

"It knows what I mean."

"It thinks about the problem."

"It wants to help me."

"It knows that it is wrong."

The interaction can certainly feel that way.

But human-like interaction does not necessarily mean human-like cognition.

Modern AI systems can produce remarkably convincing outputs without possessing human consciousness, emotions, personal experiences, or desires.

A language model, for example, processes representations of language and calculates probabilities over possible outputs based on patterns learned during training and the context provided to it.

That is very different from a human sitting down and consciously thinking through an answer.

And this distinction matters.

AI does not have to think like us to be useful

There is another misconception hidden inside the first one.

People sometimes assume that if AI does not think exactly like a human, then it is somehow fake or not truly intelligent.

That is the wrong comparison.

An aircraft does not fly like a bird.

A calculator does not calculate like a mathematician.

A database does not remember like a human.

Yet all of them perform useful functions extremely well.

AI does not need to reproduce the human mind to perform tasks that previously required human intelligence.

In some tasks, it can already outperform humans.

It can process enormous amounts of information, identify patterns across large datasets, generate thousands of variations, analyze documents at scale, and perform repetitive operations far faster than a person.

The important question is therefore not:

"Does AI think exactly like a human?"

The more useful question is:

"What can this system reliably do?"

AI is not always "learning" while you talk to it

Another common misconception is that an AI is continuously learning from every conversation in the same way a human learns from experience.

Usually, that is not what is happening.

A model is trained using large amounts of data during a training process. The training process adjusts the model's parameters so that it becomes better at recognizing patterns and producing useful outputs.

When you send a prompt, the model generally uses its existing parameters together with the context available in that interaction to generate a response.

That is different from a human who continuously forms memories, develops beliefs, and accumulates personal experiences.

Some AI products can have memory features, retrieval systems, tools, or mechanisms that allow them to use information from previous interactions. But that still does not mean the underlying model is spontaneously developing a human-like consciousness.

The "autocomplete" explanation is useful - but incomplete

You have probably heard the statement:

"AI is just autocomplete."

There is some truth in this explanation, particularly when describing how language models generate text.

At a fundamental level, a language model predicts what tokens are likely to come next based on the context it has received.

That sounds simple.

But "just autocomplete" can also be misleading.

Modern models learn extremely complex statistical relationships from enormous datasets. Those relationships can allow models to perform translation, summarization, coding, reasoning-like tasks, classification, information extraction, and many other activities.

So comparing an advanced language model to the autocomplete on your phone is useful for explaining the basic mechanism, but it dramatically understates the complexity and capabilities involved.

It is better to think of it as:

Autocomplete at an enormous scale, with learned representations that enable surprisingly sophisticated behavior.

And even that describes only one part of AI.

AI can be extremely capable and still be wrong

This is another important misconception.

Many people assume that if an AI sounds confident, it must know what it is talking about.

It doesn't.

AI systems can produce incorrect information while presenting it in a very convincing way.

Large language models can "hallucinate" - generating information that sounds plausible but is factually incorrect.

This happens because generating a convincing answer and retrieving objective truth are not necessarily the same thing.

The model's objective is not simply:

"Always tell the truth."

Its behavior depends on how it was trained, the information available to it, the instructions it receives, and the tools it can access.

This is why AI output should not automatically be treated as authoritative.

The more important the decision, the more important verification becomes.

AI can also be biased

AI does not magically remove human bias.

AI systems are created, trained, evaluated, deployed, and used by humans.

Their data comes from human-created information.

That means AI systems can inherit biases, gaps, errors, and assumptions from the data and processes used to build them.

This does not mean every AI system is equally biased or that AI is inherently unreliable.

It means that "AI said it" is not a substitute for asking:

Where did this information come from?

What data was used?

How was the system evaluated?

What assumptions are built into it?

What happens when it encounters something outside its experience?

Those questions become increasingly important as AI moves into areas such as healthcare, finance, employment, security, and government.

AI is not necessarily AGI

Another misconception is that today's AI systems are already Artificial General Intelligence.

They are not.

AGI generally refers to a hypothetical level of AI capable of performing a broad range of intellectual tasks with a level of flexibility comparable to humans, rather than being highly capable in a narrower set of domains.

Today's AI systems can be astonishingly capable while still having significant limitations.

A model might write excellent code but struggle with a seemingly simple real-world task.

It might summarize a 200-page document in seconds but confidently misunderstand an important detail.

It might solve a difficult mathematical problem but fail at something that a child can do effortlessly.

This is one of the strange characteristics of modern AI:

Capability can be extremely high in one area and surprisingly weak in another.

That is very different from the flexible intelligence of a human being.

The most interesting part of AI is not consciousness

The discussion around whether AI is conscious gets enormous attention.

But from a practical perspective, I think we sometimes focus on the wrong question.

You do not need an AI system to be conscious for it to fundamentally change how businesses operate.

Consider what happens if software can:

Read thousands of documents.
Extract structured information from them.
Classify incoming requests.
Write and send responses.
Call APIs.
Query databases.
Generate reports.
Monitor systems.
Trigger workflows.
Analyze images.
Write and review code.
Interact with business applications.

None of that requires a robot with feelings.

It requires software that can perceive information, make useful predictions or decisions, use tools, and execute actions within defined boundaries.

That is already powerful enough to transform entire workflows.

The real AI revolution may be less dramatic than movies - and more important

Hollywood gives us robots, humanoids, autonomous machines, and superintelligent systems taking over the world.

The actual AI revolution may look much more ordinary.

It might be an employee who previously spent six hours preparing a report now doing it in thirty minutes.

It might be a developer generating a first implementation, then spending more time reviewing architecture, security, tests, and business logic.

It might be a customer support system handling routine requests while humans deal with complex cases.

It might be an insurance company processing documents automatically.

It might be a small business using AI to analyze its customers, automate follow-ups, generate content, and operate workflows that previously required several employees.

The transformation may not look like science fiction.

It may simply look like software becoming much more capable.

AI is a tool, not magic

Perhaps the healthiest mental model is to stop thinking of AI as magic.

AI is technology.

Very sophisticated technology, but still technology.

It has capabilities.

It has limitations.

It can amplify human productivity.

It can also amplify human mistakes.

It can automate tasks.

It can create new risks.

It can produce remarkably useful results.

It can also confidently produce nonsense.

The people who understand this distinction will probably use AI more effectively than those who either worship it or dismiss it.

You do not need to believe that AI is conscious to take it seriously.

And you do not need to believe that AI is going to replace everyone to recognize that it will change how many people work.

The better question to ask

Instead of asking:

"Is AI actually intelligent?"

A more useful set of questions is:

What is this AI system capable of?

What data does it rely on?

Where does it fail?

How can I verify its output?

What tasks should I give it?

What tasks should remain under human control?

What happens when the AI makes a mistake?

These questions move the conversation away from science fiction and toward engineering.

And that is where the real value of AI is.

AI does not have to be a conscious digital person to be transformative.

It only needs to be capable enough to change what software can do, how businesses operate, and how humans work.

And we are already seeing that happen.

Getting started with Automations

Sospeter Mong'are — Wed, 07 Oct 2026 07:09:26 +0000

Automation can sound like something reserved for software engineers, large companies, or people building complicated robots.

It isn't.

If you have ever created an email filter, scheduled a message, used a shortcut on your phone, or had one application automatically send information to another, you have already experienced automation.

At its simplest, automation means getting technology to perform a task automatically instead of doing it manually every time.

Add AI to the picture, and automation becomes even more powerful. Instead of simply following fixed instructions, an automated system can use AI to understand text, classify information, summarize documents, extract data, answer questions, and make decisions based on defined rules.

This guide is for beginners who want to understand automation and AI without getting overwhelmed by technical jargon.

First, what exactly is automation?

Imagine that every time someone fills out a contact form on your website, you manually:

Open the form submission.
Copy the person's name and email.
Add them to a spreadsheet.
Send them a welcome email.
Notify your sales team.
Create a follow-up task.

If this happens once, it is not a big deal.

If it happens 50 times a day, it becomes repetitive work.

Automation allows you to tell a computer:

"Whenever someone submits this form, do these things automatically."

That is the fundamental idea.

A simple automation can often be described as:

When X happens, do Y.

For example:

When a customer submits a form, add them to Google Sheets and send them a welcome email.

This introduces two concepts that you will see everywhere in automation:

Trigger - the event that starts the automation.
Action - what the automation does after the trigger happens.

For example:

Trigger: New customer submits a form
Action: Save customer details to a spreadsheet

You can then add more actions:

Trigger: New customer submits a form
Action 1: Save their details
Action 2: Send a welcome email
Action 3: Notify the sales team
Action 4: Create a follow-up task

This is called a workflow.

Why should you learn automation?

Most people don't spend their working day doing difficult tasks.

A surprising amount of time goes into things like:

Copying information between systems
Sending repetitive emails
Updating spreadsheets
Creating reports
Downloading and renaming files
Moving data from one application to another
Sending reminders
Checking whether something has happened
Creating the same documents repeatedly

These tasks may be necessary, but they don't always require a human.

Automation allows you to move from:

"I do this every day."

to:

"The system does this for me."

That can save time, reduce human error, and allow people to focus on work that actually requires judgment and creativity.

The basic building blocks of automation

Before learning any particular automation tool, understand these concepts.

1. Trigger

The trigger starts your workflow.

Examples:

A new email arrives
Someone submits a form
A payment is received
A new row is added to a spreadsheet
A customer creates an account
A file is uploaded
It becomes 8:00 AM
An API receives a request

Think of the trigger as:

"Something happened."

2. Action

An action is something the automation does after the trigger.

Examples:

Send an email
Create a database record
Send a WhatsApp notification
Add a row to Excel
Create a calendar event
Upload a file
Call an API
Generate a report

Think of it as:

"Because that happened, do this."

3. Condition

Sometimes you don't want an automation to run the same way every time.

You can introduce conditions.

For example:

When a new customer signs up, check their country.

If country = Kenya:

Send the Kenyan onboarding message.

If country = Uganda:

Send the Ugandan onboarding message.

Conditions allow your automation to make decisions based on information available to it.

4. Data

Automations are usually moving or transforming data.

For example, a form submission might contain:

Name: Jane Doe
Email: jane@example.com
Country: Kenya
Plan: Premium

Your automation might take that information and:

Store it in a database
Send an email
Create a CRM contact
Notify a salesperson
Send a WhatsApp message

Understanding how data moves between systems is one of the most important skills in automation.

The major types of automation

There are many ways to categorize automation, but beginners can think about them in a few broad areas.

Workflow and integration automation

This is probably the easiest place to start.

You connect different applications so they can work together automatically.

For example:

New website lead -> Google Sheets -> CRM -> Email notification

Or:

Payment received -> Update database -> Send confirmation -> Notify business owner

Popular tools include:

Zapier
Make
Microsoft Power Automate
n8n

The important thing is not memorizing the tools.

Understand the pattern:

Trigger -> Process -> Action

Personal productivity automation

Automation doesn't have to be for a company.

You can automate your own digital life.

For example:

Automatically filter newsletters from your inbox
Rename downloaded files
Create reminders
Automatically organize documents
Use keyboard shortcuts for frequently typed text
Schedule recurring tasks
Automatically back up files

Tools such as Apple Shortcuts, Windows Task Scheduler, Alfred, and built-in email rules can help with this.

The best automation is often a small one that removes an annoying task you repeatedly perform.

Marketing and customer automation

Businesses use automation heavily for marketing and customer communication.

For example:

A customer signs up for your product.

Your system can automatically:

Create the customer account.
Send a welcome email.
Add them to your CRM.
Send onboarding information.
Wait three days.
Check whether they have used the product.
Send another message if they haven't.
Notify a sales representative if they become highly engaged.

Tools such as HubSpot, Mailchimp, ActiveCampaign, and other CRM and marketing platforms provide these capabilities.

This is particularly useful when you have hundreds or thousands of customers.

Robotic Process Automation

Sometimes a business doesn't have an API or easy integration between two systems.

This is where Robotic Process Automation (RPA) can be useful.

Instead of integrating directly with an application, an RPA bot can interact with the application's interface.

It can:

Open an application
Click buttons
Read information
Copy data
Fill in forms
Download files
Upload documents

Tools such as UiPath and Power Automate Desktop are commonly used for this type of automation.

However, don't automatically choose RPA just because it can imitate a human.

If a reliable API exists, an API integration is often more robust than automating clicks on a screen.

Where does AI fit into automation?

This is where things get interesting.

Traditional automation is usually good at following explicit instructions.

For example:

If payment status is "successful", send a confirmation message.

There isn't much ambiguity.

AI becomes useful when the system needs to understand something.

For example:

Read this customer email and determine whether the customer is asking for a refund, reporting a technical problem, or asking about pricing.

A traditional automation may struggle with this because emails can be written in thousands of different ways.

An AI model can interpret the text and classify it.

The automation can then continue based on the result.

For example:

New email arrives

↓

AI reads the email

↓

Classify the request

↓

If refund request -> send to finance

If technical issue -> create support ticket

If sales inquiry -> notify sales

This combination is often called AI automation.

Automation vs AI automation

It helps to understand the difference.

Traditional automation

The system follows predefined rules.

If X happens, do Y.

Example:

If an invoice is paid, send a receipt.

AI automation

AI handles a task that requires some level of interpretation.

Understand X, determine what it means, then do Y.

Example:

Read the customer's message, determine what they need, and route it to the correct department.

The two can work together.

In fact, some of the most useful systems combine both.

Automation provides the workflow.

AI provides intelligence within the workflow.

A simple AI automation example

Imagine a company receives 200 customer emails every day.

A basic automation could detect new emails.

AI can then analyze each email.

For example:

New email
   |
   v
AI reads email
   |
   v
Classify request
   |
   +---- Refund ------> Finance
   |
   +---- Technical ---> Support
   |
   +---- Sales -------> Sales team
   |
   +---- General -----> Customer service

You could then automatically create tickets, notify employees, summarize the conversation, or draft a response.

The human can still remain in control.

AI does not necessarily have to replace the human.

It can simply remove the repetitive work around the human.

You don't need to start with AI

This is important for beginners.

Don't start by trying to build an autonomous AI agent that controls your entire business.

Start with automation.

Learn:

Trigger -> Data -> Condition -> Action

Then introduce AI when you encounter a problem that requires understanding or judgment.

For example:

"I can automatically collect these emails, but I need something to understand what each email is about."

That's a good place for AI.

No-code, low-code, or code?

There are three common ways to build automations.

No-code automation

You use a visual interface instead of writing code.

Examples include:

Zapier
Make
Microsoft Power Automate
IFTTT

You might build:

New Gmail email
       |
       v
Check attachment
       |
       v
Save to OneDrive
       |
       v
Send notification

This is an excellent way for beginners to understand automation concepts.

Low-code automation

Low-code tools allow you to use visual workflows while adding expressions, scripts, or custom logic when necessary.

This gives you more flexibility without requiring you to build everything from scratch.

For example, you might use Power Automate to create the workflow but write an expression to transform a date or calculate a value.

Code-based automation

Once you need more control, you can build automations using code.

Python is an excellent language for automation because it has a huge ecosystem of libraries and is relatively easy to learn.

You can use code to:

Call APIs
Process files
Clean data
Interact with databases
Scrape websites where permitted
Send emails
Process spreadsheets
Integrate AI models
Build scheduled jobs
Create custom automation services

For example, a Python program could retrieve transactions from an API, analyze them, store the results in a database, and send a daily report.

Three common ways to automate with code

1. API integrations

APIs allow software applications to communicate with each other.

For example:

Your Python application
        |
        v
Payment API
        |
        v
Transaction data
        |
        v
Your database

You can use Python libraries such as requests, json, and official SDKs to communicate with APIs.

If you understand APIs, you already have one of the most valuable foundations for building automations.

2. Browser automation

Sometimes a website does not provide an API that you can use.

Browser automation tools can control a browser programmatically.

Popular tools include:

Playwright
Selenium

For example, a permitted internal workflow could:

Open a website.
Log in.
Search for a record.
Extract information.
Save the information.
Generate a report.

However, browser automation is usually more fragile than using a proper API. Websites can change their interface and break your automation.

Always check a website's terms and access rules before automating it.

3. File and computer automation

You can also automate tasks directly on your computer.

Python can work with files, folders, CSV files, Excel files, and other data.

For example:

Downloaded files -> identify file type -> rename -> move to correct folder

Useful Python libraries include:

os
pathlib
shutil
pandas
openpyxl

You don't need complicated AI for this.

Sometimes a 30-line script can eliminate hours of repetitive work.

Scheduling your automation

An automation isn't very useful if you still have to remember to start it every morning.

You can schedule your code to run automatically.

Windows

Use Task Scheduler.

Linux and macOS

You can use cron or other scheduling tools.

Cloud

You can run automation using services such as:

GitHub Actions
AWS Lambda
Azure Functions
Google Cloud Functions
Cloud Run

You can also use workflow platforms that handle scheduling for you.

The important concept is:

Build it once, then let the system run it repeatedly.

How to choose your first automation

Don't start by asking:

"What cool automation can I build?"

Start with:

"What repetitive problem do I have?"

For one week, pay attention to the things you repeatedly do on your computer or phone.

Write them down.

For example:

Task	Frequency	Can it be automated?
Copy data from email to spreadsheet	10/day	Yes
Send daily report	1/day	Yes
Rename downloaded files	20/day	Yes
Decide whether a customer deserves a refund	2/week	Maybe, needs rules/human review
Design a new product	Occasionally	Not fully

Look for tasks that are:

Repetitive
Predictable
Rule-based
Time-consuming
Prone to human error

Those are excellent automation candidates.

Start with one small win

Don't try to automate your entire business.

Pick one small problem.

For example:

"Whenever I receive an invoice by email, save the attachment to a specific folder."

That's enough.

Once it works, improve it.

Maybe you add:

Extract invoice details -> save to database -> notify finance.

Then:

Use AI to extract the invoice number, supplier, amount, and due date.

Now you have progressed from a simple automation to an AI-powered workflow.

A practical learning path for beginners

If you are completely new, I would learn automation in this order.

Step 1: Learn the basic concepts

Understand:

Triggers
Actions
Conditions
Data
Workflows
APIs
Webhooks

Don't worry about advanced AI yet.

Step 2: Build a no-code automation

Use something like Power Automate, Make, Zapier, or n8n.

Build something simple.

For example:

Form submission -> spreadsheet -> email notification

Step 3: Learn APIs

Understand:

HTTP
GET
POST
PUT/PATCH
DELETE
JSON
Authentication
API keys
OAuth
Webhooks

This is where automation starts becoming much more powerful.

Step 4: Learn some Python

You don't need to become a Python expert immediately.

Learn enough to:

Read and write files
Work with JSON
Make HTTP requests
Work with APIs
Process data
Connect to databases
Handle errors

Step 5: Introduce AI

Learn how to use AI for tasks such as:

Classification
Summarization
Data extraction
Text generation
Document processing
Information retrieval

Then combine AI with your workflows.

Step 6: Learn how to deploy and monitor

A real automation needs more than just working code.

Learn about:

Logging
Error handling
Retries
Authentication
Secrets management
Monitoring
Scheduling
Notifications

This is what separates a small experiment from a reliable production automation.

A simple project you can build today

Here's a beginner-friendly project.

Automated expense tracker

Imagine you receive payment or expense notifications by email.

You could build:

New email
    |
    v
Extract transaction information
    |
    v
Check transaction type
    |
    v
Save to spreadsheet/database
    |
    v
Send confirmation

Later, add AI:

New transaction message
        |
        v
AI extracts:
- Amount
- Merchant
- Category
- Date
        |
        v
Save structured data
        |
        v
Generate weekly summary

Now you are learning automation, APIs, data processing, and AI in one project.

The most important lesson

Don't learn automation by memorizing tools.

Tools will change.

The concepts will remain.

Learn to look at a manual process and ask:

What starts this process?

What information is available?

What decisions need to be made?

What steps are repetitive?

What can a computer do automatically?

Where does the process require human judgment?

Could AI help with the parts that involve understanding unstructured information?

That way of thinking is more valuable than knowing how to use one particular automation platform.

Automation is not about removing humans

A common misconception is that automation means:

"Replace the human."

A better way to think about it is:

Automate the repetitive parts so humans can focus on the valuable parts.

A human might still approve a payment, review an important customer complaint, or make a business decision.

The automation simply handles the work around that decision.

For example:

Before automation:

Employee reads 100 emails -> identifies categories -> copies information -> creates tickets -> notifies teams.

After automation:

AI classifies emails -> automation creates tickets -> relevant teams are notified -> employee reviews the important cases.

The human is still involved, but their time is being used much more effectively.

Where you should start

If you are completely new, don't try to learn everything at once.

Start with this mental model:

Trigger -> Get data -> Process data -> Make a decision -> Take action

Then ask yourself where AI can help.

For example:

Trigger: Customer sends a message

Get data: Retrieve the customer's account information

Process: AI understands the message

Decision: Determine what the customer needs

Action: Create a ticket, send a response, or escalate to a human

Once you understand this pattern, you can build surprisingly sophisticated systems.

And you don't have to start with a complex AI agent.

Start with one annoying repetitive task.

Automate it.

Then make it better.

That's how you get started with automation.

MCP vs API: What They Are, How They Work, and When to Use Each

Sospeter Mong'are — Tue, 06 Oct 2026 17:11:57 +0000

If you work in backend engineering, integrations, or AI development, you have probably started hearing about MCP alongside something you already know well: APIs.

At first glance, MCP can look like another way of building an API. It is not.

Both MCP and APIs allow software systems to communicate, but they solve different problems. An API is primarily a general-purpose interface between software systems, while MCP is designed to give AI applications a standardized way to discover and use tools, resources, and capabilities.

Understanding this distinction is important because MCP does not replace APIs. In many practical systems, MCP actually sits on top of existing APIs.

What is an API?

API stands for Application Programming Interface.

An API defines how one piece of software can interact with another.

For example, suppose you are building an e-commerce application and need to retrieve a customer's orders.

Your backend might call:

GET /api/customers/123/orders
Authorization: Bearer <token>

The server could respond:

{
  "customer_id": 123,
  "orders": [
    {
      "id": 1001,
      "amount": 2500,
      "status": "paid"
    }
  ]
}

The API defines things such as:

What endpoint to call
What HTTP method to use
What parameters are required
How authentication works
What data is returned
What errors mean
What operations are available

This gives developers a predictable contract for integrating systems.

A simple way to think about an API

An API is like a service counter.

You know what services the counter provides, you know how to request them, and you know what kind of response you will receive.

Your application
       |
       | HTTP request
       v
     API
       |
       v
   Backend system
       |
       v
    Database

APIs are therefore fundamental to modern software architecture.

What is MCP?

MCP stands for Model Context Protocol.

It is a protocol designed to standardize how AI applications interact with external tools, resources, and prompts.

Instead of building a custom integration for every AI application, MCP provides a common protocol that an AI client can use to discover and interact with capabilities exposed by an MCP server.

Conceptually:

              AI application
                    |
                   MCP
                    |
          +---------+---------+
          |         |         |
        Tools    Resources   Prompts
          |         |         |
       Database    Files     Templates
       API         Docs
       SaaS

For example, an MCP server might expose tools such as:

get_customer()
create_invoice()
search_transactions()
send_payment()
get_account_balance()

An AI agent can discover these capabilities and decide when to use them.

This is one of the important differences between MCP and a traditional API.

The fundamental difference

The simplest distinction is:

APIs are primarily designed for software-to-software communication. MCP is designed to standardize AI-to-tool interaction.

Consider a payment system.

You might have an API such as:

POST /payments

A traditional application knows how to call this endpoint because a developer explicitly programmed that integration.

With MCP, an AI agent could be given a tool such as:

create_payment

with a defined schema:

{
  "customer": "string",
  "amount": "number",
  "currency": "string"
}

The AI application can discover that tool and determine that it should use it based on the user's request.

So the distinction is not simply:

API vs MCP

It is closer to:

Application -> API -> System

AI Agent -> MCP -> Tools/Resources -> Systems

And in many real systems:

AI Agent
    |
   MCP
    |
 MCP Server
    |
   API
    |
Backend System

How a traditional API works

Let's take a simple banking example.

A mobile application wants to retrieve an account balance.

The developer might implement:

GET /accounts/123/balance

The API server authenticates the request and returns:

{
  "account_id": "123",
  "balance": 125000,
  "currency": "KES"
}

The application already knows:

Which endpoint to call
When to call it
What parameters to provide
How to interpret the response

The developer wrote that logic.

The API is therefore a contract between systems.

How MCP works

MCP introduces another layer.

An AI application connects to an MCP server.

The MCP server exposes capabilities that the AI can discover.

For example:

MCP Server
|
+-- Tools
|   +-- get_customer
|   +-- search_orders
|   +-- create_invoice
|
+-- Resources
|   +-- customer documentation
|   +-- product catalog
|
+-- Prompts
    +-- customer support prompt
    +-- reporting prompt

The AI application can discover what is available.

For example:

Tool:
search_orders

Input:
{
  customer_id: string,
  status: string
}

The model can then determine:

"The user wants to know whether customer 123 has any unpaid orders. I have a tool called search_orders, so I should use it."

It can invoke the tool with structured arguments.

The MCP server then performs the operation.

MCP does not magically replace your backend

This is one of the most important things to understand.

If you already have a well-designed API, you do not necessarily need to throw it away because you are adopting MCP.

Instead, MCP can act as an AI-friendly interface over your existing systems.

For example:

                 AI Agent
                    |
                   MCP
                    |
              MCP Server
                    |
        +-----------+-----------+
        |           |           |
       REST        SQL        SaaS API
       API       Database       API

Suppose your company already has:

GET /customers/{id}
POST /payments
GET /transactions

You could build MCP tools around those capabilities:

get_customer
create_payment
search_transactions

The MCP server translates the AI's tool invocation into the appropriate backend operation.

This is similar to building an adapter layer.

API vs MCP: A practical comparison

Feature	API	MCP
Primary purpose	Software integration	AI-to-tool integration
Main consumer	Applications/developers	AI models/agents
Interface	Endpoints/functions	Tools/resources/prompts
Discovery	Documentation/specification	Protocol-based capability discovery
Typical interaction	Request/response	AI discovers and invokes capabilities
Authentication	API keys, OAuth, JWT, etc.	Depends on implementation and environment
Data access	Yes	Yes, through exposed resources/tools
Tool execution	Yes	Yes
AI-specific	No	Yes
Replaces APIs?	N/A	Usually no
Best use	Application integration	Giving AI controlled access to systems

APIs are deterministic interfaces

Imagine you have a weather API:

GET /weather?city=Nairobi

A developer explicitly decides:

response = get_weather("Nairobi")

The program knows exactly what it is doing.

The API does not need to understand natural language.

It simply receives a valid request and processes it.

This makes APIs excellent for:

Mobile applications
Web applications
Microservices
Payment integrations
Third-party integrations
Data pipelines
System-to-system communication
Event-driven architectures

MCP introduces model-driven tool selection

Now imagine a user tells an AI assistant:

"Check my outstanding invoices and send reminders to customers who are overdue."

That could require several operations:

1. Find invoices
2. Determine which are overdue
3. Retrieve customer information
4. Generate reminder messages
5. Send messages

The AI agent could potentially use several MCP tools:

search_invoices()
get_customer()
send_message()

The model decides which tools are relevant based on the task.

This is where MCP becomes particularly useful.

The developer does not necessarily have to create a separate hard-coded workflow for every possible natural-language request.

MCP is especially useful for AI agents

Traditional software generally follows predetermined workflows.

For example:

Receive payment
      |
      v
Validate payment
      |
      v
Update database
      |
      v
Send notification

An AI agent may instead operate more dynamically:

User request
      |
      v
AI Agent
      |
      +----> Search customer
      |
      +----> Check payment
      |
      +----> Query documentation
      |
      +----> Create response

The agent can select tools based on context.

This makes MCP particularly interesting for:

AI agents
Coding assistants
Enterprise copilots
Customer support agents
Research agents
Data analysis agents
Internal automation
AI-powered operations

MCP vs API is not really an either/or decision

This is where many discussions about MCP become misleading.

You don't necessarily choose:

MCP OR API

You often use:

MCP + APIs

For example, imagine a Kenyan fintech platform.

It already has:

Payment API
Customer API
Transaction API
Notification API

Now the company wants an AI operations agent.

Instead of giving the AI direct database access, you could create an MCP server:

                    AI Agent
                       |
                      MCP
                       |
                MCP Server
                       |
        +--------------+--------------+
        |              |              |
    Payment API    Customer API   Transaction API
        |              |              |
        +--------------+--------------+
                       |
                    Database

The APIs remain the core integration layer.

MCP provides the interface through which the AI agent interacts with those capabilities.

When should you use an API?

Use an API when you need predictable, programmatic integration.

For example:

A mobile application

Your Android or iOS application might communicate with your backend through REST or GraphQL APIs.

A payment integration

Your application
      |
      v
Payment API
      |
      v
Payment provider

A microservice architecture

Orders Service
      |
      v
Payments Service

A third-party integration

Your system
    |
    v
CRM API
    |
    v
CRM platform

These are classic API use cases.

When should you use MCP?

MCP becomes valuable when an AI model needs to interact with external capabilities.

For example:

AI customer support agent

The agent could use:

get_customer
search_orders
check_payment
create_ticket
send_message

AI data analyst

The agent could use:

query_database
get_schema
run_report
export_csv

Coding assistant

The agent could interact with:

read_file
search_code
run_tests
create_branch
inspect_logs

Enterprise AI assistant

The agent could access:

SharePoint
Databases
CRM
ERP
Internal APIs
Documentation
File systems

MCP provides a standardized way to expose these capabilities to the AI application.

Security changes when AI enters the picture

This is an area developers should take seriously.

With a normal API, the developer controls when the API is called.

With an AI agent, the model may decide when a tool should be invoked based on the user's request and the context it receives.

That introduces additional security considerations.

Suppose you expose:

delete_customer()

as an MCP tool.

You don't want the model to have unrestricted authority to execute destructive operations.

You need controls around:

Authentication
Authorization
Input validation
Tool permissions
Least privilege
Audit logging
Rate limiting
Human approval
Sensitive data access
Destructive operations
Prompt injection
Tool misuse

The fact that an operation is exposed through MCP does not make it safe.

In fact, giving an AI agent access to powerful tools makes authorization even more important.

Don't give an MCP server unnecessary access

Suppose your AI assistant only needs to check transaction status.

It probably doesn't need:

delete_transaction
refund_payment
create_payment
update_customer

A better design is:

MCP Server
|
+-- get_transaction
+-- search_transactions

This follows the principle of least privilege.

The AI should have access to the smallest set of capabilities required to perform its job.

This is especially important for financial systems, healthcare systems, enterprise systems, and production infrastructure.

MCP can provide a better abstraction for AI

Imagine an API exposes:

GET /v1/transactions

with dozens of parameters:

customer_id
account_id
status
start_date
end_date
page
limit
sort
include_metadata

A developer can read the API documentation and construct the request.

But an AI agent benefits from a more semantic tool:

search_transactions

with a clearly defined schema and description.

For example:

{
  "customer_id": "123",
  "status": "failed"
}

The MCP layer can hide some of the underlying implementation complexity.

That makes MCP useful as an AI-facing abstraction layer.

Think of MCP as a bridge

A useful mental model is:

                Human
                  |
                  v
              AI Agent
                  |
                  v
                 MCP
                  |
        +---------+---------+
        |         |         |
       API       DB       SaaS
        |         |         |
        +---------+---------+
                  |
               Systems

The API remains valuable.

The database remains valuable.

The SaaS integrations remain valuable.

MCP provides a standardized way for the AI layer to interact with those capabilities.

An example from a real business

Imagine an insurance company.

It has:

Claims API
Policy API
Customer API
Payments API
Document repository

The company wants an internal AI claims assistant.

A traditional application could use APIs directly:

Claims UI
   |
   +--> Claims API
   +--> Policy API
   +--> Customer API

An AI assistant could instead have:

Claims AI Agent
       |
      MCP
       |
   MCP Server
       |
       +--> get_claim()
       +--> get_policy()
       +--> get_customer()
       +--> search_documents()
       |
       v
Existing enterprise systems

Now an employee can ask:

"What is the current status of claim 45821 and what documents are still missing?"

The agent can determine which tools it needs, retrieve the information, and formulate the response.

What MCP does not solve

MCP is not a replacement for good software architecture.

It does not automatically solve:

Authentication
Authorization
Data modeling
Database design
API design
Security
Observability
Reliability
Business logic
Transaction management

You still need to build those properly.

MCP is a protocol for connecting AI applications to capabilities. It does not eliminate the engineering underneath.

A simple decision framework

When deciding between MCP and an API, ask:

"Who is consuming this interface?"

If the answer is:

Another application

Use an API.

If the answer is:

An AI application or agent

MCP may be appropriate.

But if your AI agent needs to interact with an existing backend, you may want:

AI Agent
   |
  MCP
   |
Existing APIs

rather than replacing those APIs.

API first, MCP when AI needs it

For many organizations, a sensible architecture is:

                Applications
                     |
                     v
                   APIs
                     |
              Business Logic
                     |
              Data / Services


                AI Applications
                     |
                     v
                    MCP
                     |
                     v
                   APIs
                     |
              Business Logic
                     |
              Data / Services

This separates concerns.

Your APIs remain your general integration layer.

MCP becomes your AI integration layer.

That architecture also means your existing systems do not have to be redesigned simply because you are introducing AI.

The key takeaway

MCP and APIs are not competitors in the way they are sometimes presented.

APIs answer:

"How can software interact with this system?"

MCP answers:

"How can an AI application discover and interact with tools, resources, and capabilities?"

That distinction matters.

If you are building a payment gateway, mobile backend, microservice, SaaS integration, or enterprise integration, you still need APIs.

If you are building an AI agent that needs to interact with those systems, MCP can provide the standardized interface between the agent and those capabilities.

The architecture may ultimately look like:

                 User
                   |
                   v
                AI Agent
                   |
                  MCP
                   |
              MCP Server
                   |
             Business Logic
                   |
          +--------+--------+
          |        |        |
         API      DB      Services
          |        |        |
          +--------+--------+
                   |
              Enterprise
                Systems

So rather than asking:

"Should I use MCP or an API?"

A better question is:

"What system am I integrating, who is consuming the interface, and does an AI agent need to discover and invoke these capabilities?"

If it is application-to-application communication, start with an API.

If it is AI-to-tool interaction, consider MCP.

And when you already have APIs, MCP can sit on top of them rather than replacing them.

Azure Data Factory vs Apache Airflow: Two Different Ways to Build Data Pipelines

Sospeter Mong'are — Fri, 02 Oct 2026 12:35:23 +0000

When people compare Azure Data Factory (ADF) and Apache Airflow, the conversation often becomes a comparison of features.

Both can orchestrate data pipelines. Both can call APIs. Both can move data between systems. Both can schedule jobs, monitor executions, and integrate with data platforms.

But that can be misleading.

ADF and Airflow solve similar problems using fundamentally different approaches.

Azure Data Factory is primarily a visual, configuration-driven data integration and ETL/ELT service.

Apache Airflow is primarily a programmatic, code-driven workflow orchestration platform.

Understanding this distinction is more important than simply comparing their feature lists.

First, what is Azure Data Factory?

Azure Data Factory (ADF) is a cloud-based data integration service from Microsoft Azure.

Its main purpose is to help you build pipelines that move and transform data between different systems.

Instead of writing an entire pipeline as a Python program, you can construct it using visual activities.

For example, an ADF pipeline might look like:

Get Configuration
       |
       v
Get Access Token
       |
       v
Call API
       |
       v
Store JSON in Azure Data Lake
       |
       v
Transform / Copy Data
       |
       v
Load Azure SQL / Synapse

Each box represents an activity configured through the ADF interface.

You can configure things such as:

URLs
HTTP methods
Headers
Parameters
Authentication
Source and destination datasets
Column mappings
Retry policies
Dependencies
Expressions
Conditional logic

ADF also integrates with other Azure services such as Azure Key Vault, Azure Data Lake Storage, Azure SQL, Azure Synapse and Azure Monitor.

The important idea is that you primarily describe what should happen through configuration and visual activities rather than implementing the entire workflow in application code.

What is Apache Airflow?

Apache Airflow is an open-source platform for programmatically authoring, scheduling and monitoring workflows.

Instead of building a pipeline primarily through a visual interface, you define it in code.

An Airflow workflow is usually represented as a DAG - Directed Acyclic Graph.

For example:

from airflow import DAG
from airflow.operators.python import PythonOperator

with DAG(
    dag_id="zoho_pipeline",
    schedule="@daily",
) as dag:

    extract_data = PythonOperator(
        task_id="extract_data",
        python_callable=extract_from_zoho
    )

    transform_data = PythonOperator(
        task_id="transform_data",
        python_callable=transform_data
    )

    load_data = PythonOperator(
        task_id="load_data",
        python_callable=load_to_database
    )

    extract_data >> transform_data >> load_data

The code defines the workflow and its dependencies.

Conceptually:

extract_data
      |
      v
transform_data
      |
      v
load_data

Airflow does not primarily exist to perform every data transformation itself. Its core strength is orchestration - coordinating tasks and systems.

Those tasks could involve Python, SQL, dbt, Spark, APIs, cloud services, containers or other external systems.

ADF vs Airflow: The Core Difference

The simplest way to understand the difference is:

ADF asks you to configure a data pipeline. Airflow asks you to program a workflow.

This distinction affects almost everything else.

Area	Azure Data Factory	Apache Airflow
Core paradigm	Visual, configuration-driven	Programmatic, code-driven
Primary interface	Web UI	Python code + UI for monitoring
Workflow definition	Pipelines and activities	Python DAGs
API integration	Activities, REST connectors, datasets	Python libraries, operators, hooks
Transformations	Mapping Data Flows, Copy Activity, SQL, etc.	Python, SQL, dbt, Spark and external tools
Version control	Possible through Git integration	Natural fit because DAGs are code
Infrastructure	Microsoft-managed service	You manage or use a managed Airflow service
Azure integration	Deep native integration	Strong, but usually through operators/providers
Multi-cloud	Possible	Strong fit
Custom Python logic	Possible, but often indirect	Native
Monitoring	Azure portal	Airflow UI
Scaling	Managed by Azure services	Depends on Airflow deployment
Maintenance	Low infrastructure management	Higher infrastructure responsibility

Neither approach is universally "better". They optimize for different engineering models.

Example: Connecting to the Zoho API

Suppose you need to extract data from Zoho.

The API requires:

Obtaining an access token
Calling an API endpoint
Passing authentication headers
Handling pagination
Saving the response
Transforming the JSON
Loading the data into a warehouse

In ADF

You can construct this visually.

For example:

Web Activity
    |
    | Get OAuth token
    v
Set Variable
    |
    | Store token
    v
Web Activity / Copy Activity
    |
    | Call Zoho API
    v
Azure Data Lake
    |
    v
Copy Activity
    |
    v
Azure SQL / Synapse

Credentials such as the Zoho client secret should be stored in a secure service such as Azure Key Vault rather than hardcoded in the pipeline.

ADF allows you to configure URLs, headers, parameters, expressions and activity dependencies through its interface.

In Airflow

The same workflow might be implemented in Python:

DAG
 |
 +-- Get Zoho Token
 |
 +-- Extract Zoho Data
 |
 +-- Transform JSON
 |
 +-- Load Warehouse

The API interaction could use Python's requests library or an appropriate Airflow provider/operator.

For example:

response = requests.get(
    api_url,
    headers={
        "Authorization": f"Zoho-oauthtoken {access_token}"
    }
)

The difference becomes more noticeable when the API becomes complicated.

If you need custom retry logic, unusual pagination, token refresh behavior or complicated response processing, Python gives you considerably more freedom.

What About JSON Transformations?

APIs rarely return perfectly flat data.

You might receive:

{
    "data": [
        {
            "id": "123",
            "name": "John",
            "activities": [
                {
                    "type": "call",
                    "date": "2026-09-30"
                }
            ]
        }
    ]
}

You may need to flatten:

data
  -> customer
       -> activities

ADF approach

ADF can handle many transformations through Copy Activity mappings and Mapping Data Flows.

You can configure mappings and expressions through the UI.

This works well when the transformation can be expressed using ADF's available activities and expressions.

Airflow approach

With Airflow, you can process the response using Python.

For example, libraries such as Pandas can help normalize nested JSON:

from pandas import json_normalize

df = json_normalize(response.json()["data"])

You can then apply arbitrary Python logic before loading the result.

This illustrates a broader difference:

ADF gives you configurable transformation capabilities. Airflow gives you a programming environment from which you can call virtually any transformation tool.

Where Does the Processing Happen?

Another important distinction is execution architecture.

With ADF, data movement and transformation can be executed through Azure Integration Runtime or other configured compute resources.

You generally do not manage the underlying servers yourself.

Airflow, on the other hand, executes tasks through its workers or connected execution environments.

For example:

Airflow Scheduler
       |
       v
Airflow Worker
       |
       +-- Python
       +-- SQL
       +-- dbt
       +-- Spark
       +-- API

This gives engineers considerable control, but it also introduces infrastructure considerations.

If your Python task loads a very large dataset into memory, for example, the worker's resources become relevant.

With Airflow, you therefore need to think about:

Worker sizing
Concurrency
Memory
CPU
Queues
Executors
Containers
Kubernetes
Task isolation

That operational responsibility is one of the biggest differences between the two platforms.

ADF: When Does It Make Sense?

ADF is particularly attractive when your environment is heavily invested in Azure.

For example:

API
 |
 v
ADF
 |
 v
Azure Data Lake
 |
 v
ADF / Transformation
 |
 v
Azure SQL / Synapse / Fabric

The ecosystem fits together naturally.

ADF can also be a good choice when your team prefers visual development.

A team consisting primarily of SQL developers, analysts or database professionals may find it easier to understand:

Activity A
    |
    v
Activity B
    |
    v
Activity C

than a Python-based orchestration framework.

ADF also removes much of the infrastructure burden.

You do not have to build and maintain an Airflow cluster just to execute your pipelines.

Airflow: When Does It Make Sense?

Airflow becomes particularly useful when your organization wants workflows to behave more like software.

Your pipeline is code.

Therefore you can naturally apply software engineering practices such as:

Git
Branching
Pull requests
Code reviews
Automated testing
CI/CD
Reusable Python functions
Package management
Code quality checks

For example:

Git Repository
      |
      v
Pull Request
      |
      v
Code Review
      |
      v
CI/CD
      |
      v
Airflow
      |
      v
Production DAGs

Airflow is also useful in environments that are not exclusively Azure.

You could have:

Airflow
  |
  +-- Azure
  +-- AWS
  +-- GCP
  +-- Snowflake
  +-- Databricks
  +-- PostgreSQL
  +-- APIs

This makes it attractive for organizations with heterogeneous or multi-cloud data platforms.

What About Complex Pipelines?

Imagine a Zoho API that requires:

Multiple authentication steps
Token refresh
Cursor-based pagination
Custom retry logic
Rate-limit handling
Nested JSON processing
Data validation
Custom error handling

In ADF, these requirements can sometimes be implemented using activities, expressions, variables, loops and conditions.

But the pipeline can become increasingly configuration-heavy.

In Airflow, you can implement the logic directly in Python.

That might look conceptually like:

def extract_zoho_data():
    token = get_token()

    while True:
        response = fetch_page(token)

        process(response)

        if not has_next_page(response):
            break

This is one of the strongest arguments for a code-based orchestrator.

When workflow logic becomes software, treating it as software can make the implementation easier to reason about.

But Airflow Is Not Automatically Better for Everything

There is an important misconception to avoid.

Using Python does not automatically make a pipeline better.

A poorly designed Airflow deployment can introduce significant complexity.

You now have to think about:

Airflow Scheduler
Airflow Workers
Metadata Database
Executor
DAG Deployment
Secrets
Logs
Monitoring
Scaling
Upgrades
Networking

With ADF, much of this infrastructure is abstracted away.

Therefore, the question should not simply be:

"Can Airflow do this?"

It almost certainly can.

The better question is:

"Does the additional flexibility justify the additional engineering and operational responsibility?"

A Practical Decision Framework

Think about the pipeline and the team building it.

Choose ADF when:

Your data platform is primarily Azure
Your destination is Azure SQL, Synapse, Fabric or Azure Data Lake
You want a managed service
Visual pipeline development is valuable
Most transformations can be handled through existing ADF capabilities
Your team is comfortable with configuration and SQL
You want minimal infrastructure management

Consider Airflow when:

Your workflows require substantial custom Python logic
You operate across multiple clouds or platforms
Your data engineers are comfortable with Python
Git-based workflow development is important
You need complex orchestration
You need to integrate many different processing technologies
You want workflows managed using software engineering practices

The Most Important Distinction

ADF and Airflow can both orchestrate a pipeline such as:

API
 |
 v
Extract
 |
 v
Transform
 |
 v
Load
 |
 v
Data Warehouse

But they approach that pipeline differently.

ADF is closer to:

Configure the pipeline
        |
        v
Microsoft manages the platform

Airflow is closer to:

Write the workflow
        |
        v
Deploy and operate the orchestrator

That difference becomes increasingly important as your data platform grows.

If your primary goal is Azure-native data integration with minimal infrastructure management, ADF provides a natural model.

If your primary goal is flexible, code-first workflow orchestration across different systems, Airflow provides a different model.

Neither tool eliminates the need for good data engineering.

You still need to think about:

Authentication
Secrets management
Idempotency
Retries
Rate limits
Data quality
Observability
Schema changes
Error handling
Incremental loading
Backfills
Dependency management

The orchestration tool is only one part of the architecture.

The real decision is not simply ADF vs Airflow. It is configuration-driven orchestration vs code-driven orchestration, and which model fits your team's skills, infrastructure and operational requirements.

Stop Giving AI Coding Agents Your Passwords: A Practical Guide to Secure Credentials

Sospeter Mong'are — Fri, 25 Sep 2026 19:31:56 +0000

AI coding agents such as Cursor, Claude Code, GitHub Copilot and other agentic development tools are changing how developers build and deploy software.

They can read your code, modify files, run terminal commands, execute tests and, depending on how they are configured, interact with your servers and deployment infrastructure.

That power is useful - but it also introduces a security problem:

How do you give an AI coding agent enough access to deploy and troubleshoot your application without giving it the keys to your entire infrastructure?

The answer is not simply "put your password in a .env file."

The better approach is to use least privilege, scoped credentials, SSH keys, CI/CD secrets and separate deployment accounts.

Here is a practical approach.

1. Never Paste Your VPS or cPanel Password Into an AI Chat

This should be your first rule.

Avoid prompts like:

My VPS password is MySuperSecretPassword123.

SSH into the server and deploy my application.

Or:

cPanel username: myuser
cPanel password: MyPassword123

Even if you trust the AI tool, you are unnecessarily exposing a credential through a conversation.

That credential could potentially appear in:

Chat history
Terminal output
Agent logs
Debug output
Screenshots
Project files
Shell history
Deployment logs

More importantly, you are giving the AI agent a secret that may provide far more access than the task actually requires.

Instead of asking:

"How do I give Cursor my server password?"

ask:

"How do I let Cursor perform this task with the minimum access required?"

That change in mindset is the foundation of secure AI-assisted development.

2. Use Environment Variables - But Understand Their Limitation

For local development and deployment scripts, environment variables are much better than hardcoding credentials into your application.

For example:

CPANEL_FTP_HOST=ftp.example.com
CPANEL_FTP_USER=my_deployment_user
CPANEL_FTP_PASS=your_password

Your deployment script can then read these values.

For example:

import os

host = os.getenv("CPANEL_FTP_HOST")
username = os.getenv("CPANEL_FTP_USER")
password = os.getenv("CPANEL_FTP_PASS")

And make sure .env is ignored by Git:

.env
.env.*
!.env.example

You can safely commit a template:

CPANEL_FTP_HOST=
CPANEL_FTP_USER=
CPANEL_FTP_PASS=

But here is the important part

.env protects your secrets from being committed to Git. It does not necessarily protect them from an AI coding agent.

If Cursor can read your project directory, it may be able to read .env too.

Therefore, don't think:

"It is in .env, so the AI cannot see it."

Instead think:

"The secret isn't in my source code or Git history, but I still need to control who and what can access the environment."

3. Prefer SSH Keys Over Password Authentication

For VPS access, SSH keys are generally preferable to passwords.

Generate an Ed25519 key:

ssh-keygen -t ed25519

You will typically have:

~/.ssh/id_ed25519
~/.ssh/id_ed25519.pub

The public key goes to the server.

The private key stays on your machine.

Never share the private key:

id_ed25519

The public key:

id_ed25519.pub

is the one intended to be installed on the server.

You can then connect using:

ssh deploy@your-server.com

This eliminates the need to paste a server password into an AI conversation.

But there is another important issue

An SSH key is only as safe as the account behind it.

If your key connects as:

root@production

you have potentially given the AI agent full control over the server.

That is not much better than giving it the root password.

4. Create a Dedicated Deployment User

Instead of giving an AI agent root access, create a dedicated user:

deploy

Give that user access only to what it needs.

For example:

/var/www/myapp

rather than the entire server.

Your architecture becomes:

AI Agent
   |
   v
SSH Key
   |
   v
deploy user
   |
   v
/var/www/myapp

Instead of:

AI Agent
   |
   v
root credentials
   |
   v
Entire VPS

This is the principle of least privilege.

If the deployment credential is compromised, the potential damage is limited.

5. For cPanel, Create a Dedicated Deployment Account

The same principle applies to cPanel.

Your main cPanel account may have access to:

Websites
Databases
Email
DNS
FTP
SSL
Cron jobs
Backups
File Manager

There is usually no reason for a deployment script to have access to everything.

Instead, create a dedicated FTP/SFTP account for the application.

For example:

Username:
myapp-deploy

Directory:
/public_html/my-app

Now the deployment credential is associated with the application rather than your entire hosting account.

If the credential is compromised, the attacker does not automatically get your entire cPanel account.

Where supported, prefer secure protocols such as SFTP or SSH over plain FTP.

6. Separate Development, Staging and Production Credentials

Don't use the same credentials everywhere.

Avoid:

Development -> Production credentials
Staging     -> Production credentials
Production  -> Production credentials

Instead:

Development
   |
   +-- Development credentials

Staging
   |
   +-- Staging credentials

Production
   |
   +-- Production credentials

This becomes especially important when working with AI agents.

Give your development environment access to test credentials whenever possible.

For example, if you are integrating a payment API, use sandbox credentials during development instead of your live production secret.

The same principle applies to:

Databases
Cloud accounts
SSH
Payment APIs
Email providers
Third-party APIs

7. Keep Production Secrets Out of the AI Agent Where Possible

For production deployments, a better architecture is to let the AI agent work on the code while your CI/CD system handles the production credentials.

For example:

Cursor
   |
   | git push
   v
GitHub
   |
   v
CI/CD
   |
   | Deployment secrets
   v
Production VPS

Your GitHub repository contains the code and deployment workflow, but not the actual production password.

Secrets can be stored in your CI/CD platform's secret store.

For example:

DEPLOY_HOST
DEPLOY_USER
DEPLOY_SSH_KEY

The workflow uses those values during deployment without putting them directly into the repository.

This creates a useful separation:

AI agent -> code

CI/CD -> production deployment

That is often safer than allowing the AI agent to directly administer production.

8. Give AI Agents the Minimum Access They Need

Before giving an AI agent access to anything, ask:

What does it need to do?

For example:

Deploy my application.

What does it need to access?

/var/www/myapp

What permissions does it need?

Read + write application files

How long does it need access?

Only during deployment

Then create credentials around those requirements.

Don't start with:

"Here is my root password. Figure it out."

Start with:

"This deployment user can only modify this application's directory."

9. Watch Your Terminal Output

Credentials can leak even when you don't explicitly paste them into a chat.

Avoid commands such as:

echo $CPANEL_PASSWORD

or:

env

if your environment contains secrets.

Also be careful with shell debugging such as:

set -x

because commands containing sensitive values may appear in terminal or CI/CD logs.

Your rule should be:

Secrets should not appear in source code, terminal output, application logs or CI/CD logs.

10. What If You Already Shared a Credential?

If you have already pasted a production password, API key or private key into an AI conversation, don't simply delete the message and assume the problem is solved.

Treat the credential as potentially exposed.

Rotate it.

A simple response procedure is:

1. Identify the exposed credential
2. Revoke it
3. Generate a replacement
4. Update the application or deployment system
5. Review authentication and server logs
6. Check for suspicious activity
7. Review how the credential was exposed

For an SSH key, generate a replacement key and remove the old public key from the server.

For an API key, revoke the old key and generate a new one.

For a password, change it.

Credential rotation is more important than simply deleting the leaked secret from the conversation or code.

A Practical Secure Setup

For a small VPS or SaaS project, you don't need a complicated enterprise security architecture.

A practical setup could look like this:

                  +----------------+
                  |  Cursor / AI   |
                  |     Agent      |
                  +-------+--------+
                          |
                          v
                  +----------------+
                  |  Source Code   |
                  +-------+--------+
                          |
                       Git Push
                          |
                          v
                  +----------------+
                  |    GitHub      |
                  +-------+--------+
                          |
                          v
                  +----------------+
                  | CI/CD + Secrets|
                  +-------+--------+
                          |
                     SSH / SFTP
                          |
                          v
                  +----------------+
                  | Deploy User    |
                  +-------+--------+
                          |
                          v
                  +----------------+
                  | Application    |
                  | Directory      |
                  +----------------+

The AI agent can help you write, test and prepare the application.

The deployment system handles the production credentials.

The deployment account has only the permissions required to deploy the application.

Security Checklist

Before giving an AI coding agent access to your infrastructure, check:

[ ] I am not pasting my master password into an AI chat.
[ ] Production secrets are not committed to Git.
[ ] .env is included in .gitignore.
[ ] I understand that .env can still be read by an agent with filesystem access.
[ ] I use SSH keys where practical.
[ ] Production uses a dedicated deployment user.
[ ] The deployment user does not have unnecessary root access.
[ ] cPanel credentials are scoped to the application.
[ ] Development, staging and production credentials are separate.
[ ] Production secrets are stored in CI/CD or a secret manager where appropriate.
[ ] Database users have only the permissions they need.
[ ] Secrets do not appear in terminal or CI/CD logs.
[ ] I can quickly revoke and rotate credentials.
[ ] Destructive production operations require appropriate human approval.

Final Thought

AI coding agents are becoming increasingly capable of interacting with real infrastructure.

The goal should not be to prevent them from doing useful work.

The goal is to limit the damage they can cause if something goes wrong.

Don't ask:

"How can I give Cursor my VPS password?"

Ask:

"How can I allow Cursor to perform this task without giving it unnecessary access to my infrastructure?"

Use environment variables to keep secrets out of source code, SSH keys instead of passwords where practical, dedicated deployment users instead of root, scoped cPanel accounts instead of master credentials, and CI/CD or secret managers for production secrets.

The safest credential is not simply the one that is hidden.

It is the one that has the least power necessary to get the job done.

A Beginner's Guide to Learning RPA: Where to Start and What to Learn

Sospeter Mong'are — Thu, 24 Sep 2026 11:43:38 +0000

Robotic Process Automation, commonly known as RPA, is an area of technology that focuses on using software robots to automate repetitive, rule-based tasks that people normally perform on computers.

If you have ever copied information from an email into Excel, downloaded files from a website, entered data into another system, or generated and sent the same report every day, you have probably encountered a process that could potentially be automated with RPA.

The good news is that you do not need to be an advanced programmer to start learning RPA.

You can begin with visual automation tools and gradually learn programming, APIs, databases, cloud services and AI as you become more advanced.

This guide explains a practical path for someone starting from zero.

What is RPA?

Imagine an employee receives an email containing an Excel file every morning.

They have to:

Open the email.
Download the Excel file.
Read the information.
Open another application.
Enter the information.
Save the changes.
Generate a report.
Send the report to their manager.

If these steps happen repeatedly and follow predictable rules, an RPA bot can potentially perform them automatically.

A simple RPA workflow could look like this:

Email received
      ↓
Download attachment
      ↓
Read Excel file
      ↓
Validate information
      ↓
Open business application
      ↓
Enter data
      ↓
Generate report
      ↓
Send email

The goal of RPA is not simply to replace people.

It is to automate repetitive work so that people can spend more time on tasks that require judgment, communication, creativity and decision-making.

What makes a good RPA process?

Not every process should be automated.

RPA works particularly well when a process is:

Repetitive
Rule-based
Predictable
High-volume
Time-consuming
Based on structured data
Performed using existing computer applications

For example:

Every morning, download a report from a website, process it, update an Excel spreadsheet and email the results.

This is a good candidate for RPA.

A process such as:

Decide whether a complex customer complaint deserves compensation.

may require human judgment and may not be suitable for traditional RPA alone.

Do I need to know programming?

No.

Many RPA platforms provide visual interfaces where you build workflows by dragging and configuring activities.

However, learning programming will give you a significant advantage as you become more advanced.

Useful programming skills include:

Python
JavaScript
SQL
REST APIs
JSON
Regular expressions

You do not need to master all of these before starting.

You can learn them gradually as your automation projects become more complex.

The main RPA tools to learn

There are many RPA platforms, but beginners do not need to learn all of them.

I recommend starting with two major tools:

1. Microsoft Power Automate

Power Automate is part of Microsoft's automation ecosystem.

It allows you to automate processes across services such as:

Outlook
Excel
SharePoint
Teams
OneDrive
SQL databases
HTTP APIs
Microsoft 365
Other third-party services

It also includes Power Automate Desktop, which allows you to automate tasks performed directly on a Windows computer.

For example:

Open browser
    ↓
Go to website
    ↓
Login
    ↓
Download report
    ↓
Open Excel
    ↓
Process data
    ↓
Save file

Power Automate is a good starting point, especially if you work in organizations that already use Microsoft 365 and Azure.

2. UiPath

UiPath is another major RPA platform.

It provides tools for building, deploying and managing software robots.

Important UiPath concepts include:

UiPath Studio
Activities
Variables
Arguments
Selectors
DataTables
Workflows
Exception handling
Logging
Queues
Orchestrator
Robots

UiPath is worth learning if you want to pursue RPA as a dedicated career path.

What should you learn first?

Do not try to learn everything at once.

A good learning path looks like this:

RPA Fundamentals
       ↓
Power Automate Desktop
       ↓
UiPath
       ↓
Web Automation
       ↓
Excel & Data Processing
       ↓
APIs
       ↓
SQL
       ↓
Python
       ↓
Enterprise RPA
       ↓
AI + RPA

Let's break this down.

Step 1: Learn RPA fundamentals

Before learning a specific tool, understand the basic concepts.

Learn:

What RPA is
What RPA is not
RPA use cases
Attended automation
Unattended automation
Rule-based processes
Process discovery
Workflow design
Exceptions
Logging
Monitoring

One important concept is process discovery.

Before building a bot, you should understand the process you are trying to automate.

For example:

What starts the process?
        ↓
What information is required?
        ↓
What applications are used?
        ↓
What decisions are made?
        ↓
What can go wrong?
        ↓
What should happen when something fails?

This is an important skill for an RPA developer.

Step 2: Learn Power Automate Desktop

Once you understand the basics, build simple desktop automations.

Start with:

Automation 1 - File management

Build a bot that:

Find files in a folder
       ↓
Read filenames
       ↓
Create folders
       ↓
Move files
       ↓
Generate a summary

Automation 2 - Excel

Build an automation that:

Open Excel
     ↓
Read data
     ↓
Filter records
     ↓
Calculate values
     ↓
Create a report

Automation 3 - Email

Build:

Check email
    ↓
Find messages matching a condition
    ↓
Download attachments
    ↓
Process attachment
    ↓
Send response

Automation 4 - Browser

Build:

Open browser
     ↓
Navigate to website
     ↓
Login
     ↓
Read information
     ↓
Save information

These simple projects will teach you many of the concepts used in real RPA work.

Step 3: Learn UiPath

Once you are comfortable with basic automation, start learning UiPath.

Focus on understanding:

Activities

Activities are the building blocks of UiPath workflows.

Examples include:

Open browser
Click
Type
Read text
Read Excel
Write Excel
Send email
Create file
Move file

Variables

Learn how to store information inside your automation.

For example:

customerName
invoiceNumber
amount
emailAddress

Conditions

Learn how to make the bot behave differently depending on the data.

IF amount > 100000
    send for approval
ELSE
    process automatically

Loops

Learn how to process multiple records.

For each customer
    Validate customer
    Update system
    Generate result

Selectors

Selectors are particularly important in UI automation.

They allow an RPA tool to identify elements on a webpage or application.

For example:

Username field
Password field
Login button
Submit button

Understanding selectors will help you build more reliable automations.

Step 4: Learn Excel and data processing

A lot of business automation involves data.

You should become comfortable with:

Excel
CSV
JSON
XML
DataTables
Basic data validation
Filtering
Sorting
Transformations

For example:

Excel
   ↓
Validate records
   ↓
Remove duplicates
   ↓
Transform data
   ↓
Send to API

You should also learn basic SQL.

At minimum, understand:

SELECT
WHERE
JOIN
GROUP BY
ORDER BY
INSERT
UPDATE

Step 5: Learn APIs

This is one of the most valuable skills you can add to RPA.

Modern automation does not always need to interact with a website through clicks.

If a system provides an API, you can often integrate with it directly.

Instead of:

RPA Bot
   ↓
Open browser
   ↓
Click login
   ↓
Click menu
   ↓
Click customer
   ↓
Enter information

you may be able to do:

RPA
 ↓
REST API
 ↓
Business System

Learn:

REST APIs
HTTP methods
GET
POST
PUT/PATCH
DELETE
Headers
Authentication
OAuth 2.0
API keys
JSON
Webhooks

This is where RPA starts connecting with traditional software engineering.

Step 6: Learn Python

Python is not mandatory for RPA, but it can make you much more capable.

You can use Python for:

Data processing
File manipulation
API integration
Complex business logic
Data transformation
Automation scripts
AI integration

Useful libraries include:

pandas
requests
openpyxl
Playwright
Selenium
PyAutoGUI

For web automation, Playwright is particularly useful to learn.

You can eventually build workflows such as:

RPA
 ↓
Python
 ↓
Process data
 ↓
REST API
 ↓
Database
 ↓
RPA
 ↓
Email notification

Step 7: Learn enterprise RPA

After learning the basics, start thinking about how automation works in real organizations.

This introduces concepts such as:

Attended automation

A user starts or interacts with the automation.

Example:

Employee
   ↓
Starts bot
   ↓
Bot processes task
   ↓
Employee continues

Unattended automation

The automation runs without someone sitting in front of the computer.

For example:

2:00 AM
   ↓
Robot starts
   ↓
Processes 5,000 records
   ↓
Generates report
   ↓
Sends notification

You should also learn:

Queues
Scheduling
Credentials
Secrets
Logging
Monitoring
Retry mechanisms
Exception handling
Audit trails
Deployment
Version control

These concepts become important when automations are running in production.

Step 8: Learn AI + RPA

This is an exciting area because AI can extend what traditional RPA can do.

Traditional RPA is generally good at structured, predictable tasks.

AI can help with less structured information.

For example, consider invoices.

A traditional RPA workflow might expect:

Invoice number = cell A1
Amount = cell B1
Date = cell C1

But invoices can have different layouts.

AI can help extract information from documents:

Invoice
   ↓
OCR / Document AI
   ↓
Extract information
   ↓
Validate
   ↓
RPA
   ↓
Update accounting system

You can eventually learn technologies such as:

OCR
Intelligent Document Processing
Azure AI Document Intelligence
Large Language Models
Structured AI output
AI agents
Human-in-the-loop workflows

This combination is often referred to as intelligent automation.

Build projects instead of only watching tutorials

One of the biggest mistakes beginners make is spending months watching tutorials without building anything.

After learning a concept, build something.

Here are some beginner projects.

Project 1 - Employee onboarding

Build:

New employee information
        ↓
Validate information
        ↓
Create folder
        ↓
Generate documents
        ↓
Send welcome email
        ↓
Notify HR

Project 2 - Invoice processing

Invoice arrives by email
        ↓
Download attachment
        ↓
Extract information
        ↓
Validate invoice
        ↓
Store information
        ↓
Send notification

Project 3 - Daily reporting

Scheduled automation
        ↓
Retrieve data
        ↓
Process data
        ↓
Generate Excel report
        ↓
Send email

Project 4 - Customer data synchronization

System A
    ↓
API
    ↓
Transform data
    ↓
System B
    ↓
Log result

Project 5 - AI document processing

Document
   ↓
AI extraction
   ↓
Validate information
   ↓
Human approval if necessary
   ↓
RPA/API
   ↓
Business system

RPA skills checklist

As you progress, you can use this checklist.

Beginner

[ ] Understand RPA
[ ] Identify automation opportunities
[ ] Understand workflows
[ ] Power Automate Desktop
[ ] Basic UiPath
[ ] Excel automation
[ ] File automation
[ ] Email automation
[ ] Browser automation
[ ] Conditions
[ ] Loops
[ ] Variables

Intermediate

[ ] UiPath selectors
[ ] Exception handling
[ ] Logging
[ ] Debugging
[ ] DataTables
[ ] SQL
[ ] REST APIs
[ ] JSON
[ ] OAuth
[ ] Python
[ ] Playwright
[ ] Power Automate Cloud

Advanced

[ ] UiPath Orchestrator
[ ] Queues
[ ] Unattended automation
[ ] Credentials and secrets
[ ] Monitoring
[ ] CI/CD
[ ] Version control
[ ] Enterprise architecture
[ ] Intelligent Document Processing
[ ] AI integration
[ ] AI agents
[ ] Human-in-the-loop automation

Do you need to learn everything?

No.

You don't need to become an expert in UiPath, Power Automate, Python, APIs, SQL and AI before building your first bot.

Start small.

A practical progression is:

Learn RPA concepts
       ↓
Build a simple desktop automation
       ↓
Automate Excel
       ↓
Automate email
       ↓
Automate a website
       ↓
Learn APIs
       ↓
Learn SQL
       ↓
Learn Python
       ↓
Build enterprise workflows
       ↓
Add AI

The most valuable skill is not knowing how to click around an RPA platform.

It is learning how to look at a business process and answer:

"What part of this process can be automated, how should it be automated, and how can I make the automation reliable?"

Once you develop that ability, learning another RPA platform becomes much easier.

Final thoughts

RPA sits at an interesting intersection between software engineering, business processes and automation.

You can start without being a programmer, but programming, APIs, databases and cloud technologies can significantly expand what you are able to build.

A strong long-term skill set could look like:

                Automation Engineer
                        |
        +---------------+---------------+
        |               |               |
       RPA             APIs            AI
        |               |               |
   UiPath / PA      REST / OAuth      LLMs
        |               |               |
   Desktop/Web       Python/SQL      Agents
        |               |               |
        +---------------+---------------+
                        |
                 Business Automation

If you are starting today, don't try to learn ten tools simultaneously.

Start with Power Automate Desktop or UiPath, build a few real automations, then add APIs, SQL and Python. Once you understand those foundations, explore AI-powered automation.

That progression can take you from building simple bots to designing complete business automation solutions.

Understanding Retrieval-Augmented Generation (RAG)

Sospeter Mong'are — Fri, 18 Sep 2026 15:40:46 +0000

Introduction

Large Language Models (LLMs) have changed how we interact with software. They can answer questions, summarize documents, generate code, analyze information, and perform many other tasks using natural language.

However, LLMs have an important limitation: they do not automatically know everything about your organization, your internal documents, or information that has changed since their training.

This is where Retrieval-Augmented Generation (RAG) comes in.

RAG is an AI architecture that allows an LLM to retrieve relevant information from an external knowledge source before generating a response. Instead of relying entirely on what the model learned during training, the system first searches a collection of trusted information, retrieves relevant content, and provides that content to the LLM as context.

A simple way to think about RAG is an open-book exam.

Instead of asking the AI to answer a question entirely from memory, you give it access to the relevant documents first. The AI searches those documents, finds the information it needs, and then uses that information to formulate an answer.

For example, imagine an employee asks:

"What is our company's Q3 remote work policy?"

A general-purpose LLM may not know the company's internal policy. With RAG, the system can search the organization's policy documents, retrieve the relevant section, provide it to the LLM, and generate an answer based on that information.

This makes RAG particularly useful for organizations working with proprietary, domain-specific, or frequently changing information.

The Two Major Pipelines of RAG

A complete RAG system can be thought of as having two major pipelines:

The Data Preparation Pipeline, also called ingestion
The Runtime Pipeline, which happens when a user asks a question

Understanding this distinction is important because RAG does not simply mean "put documents into an AI model."

There is significant work that happens before the user ever asks a question.

1. The Data Preparation Pipeline

The first step is preparing your data so that an AI system can efficiently search and retrieve relevant information.

Your source data could come from:

PDF documents
Word documents
Internal company wikis
Websites
Databases
SharePoint
Emails
Knowledge bases
API responses
Internal documentation

The general process looks like this:

Documents -> Chunking -> Embeddings -> Vector Database

Chunking

Large documents are usually too big to retrieve as a single piece.

Imagine having a 200-page employee handbook. If someone asks about the company's leave policy, you don't want to retrieve the entire 200-page document.

Instead, the document is divided into smaller pieces called chunks.

A chunk might contain:

A paragraph
Several paragraphs
A section
A group of related sentences

For example:

Employee Handbook
        |
        v
+-------------------+
| Chunk 1           |
| Introduction      |
+-------------------+
| Chunk 2           |
| Leave Policy      |
+-------------------+
| Chunk 3           |
| Remote Work       |
+-------------------+
| Chunk 4           |
| Benefits          |
+-------------------+

The goal is to create chunks that contain enough information to be useful while remaining small enough for efficient retrieval.

Chunking is more important than it might initially appear. Poorly chosen chunks can result in the system retrieving incomplete or irrelevant information, which can ultimately affect the quality of the generated answer.

Embedding

Once the documents have been divided into chunks, each chunk is passed through an embedding model.

An embedding model converts text into a numerical representation called a vector embedding.

For example, a sentence such as:

"Employees can work remotely three days per week."

is converted into a mathematical representation containing many numbers.

The important idea is that the vector represents the semantic meaning of the text.

This allows a system to identify relationships between concepts rather than simply matching exact words.

For example, a user might ask:

"How many days can I work from home?"

The document might contain:

"Employees can work remotely three days per week."

Even though the wording is different, the system can recognize that the two pieces of text are semantically related.

Storing the Embeddings

The generated embeddings are stored in a vector database.

Examples include:

ChromaDB
Pinecone
Weaviate
pgvector
Qdrant
Milvus

The vector database acts as a searchable index for the knowledge base.

A simplified representation might look like:

Document Chunk
      |
      v
Embedding Model
      |
      v
Vector
[0.021, -0.183, 0.492, ...]
      |
      v
Vector Database

The database can then perform similarity searches to identify which pieces of information are most relevant to a user's question.

2. The Runtime Pipeline

The second part of RAG happens when a user actually interacts with the application.

This is the part most people associate with RAG.

The basic process is:

User Query
     |
     v
Retrieval
     |
     v
Retrieved Context
     |
     v
Augmentation
     |
     v
LLM
     |
     v
Generated Answer

There are three core stages:

Retrieval
Augmentation
Generation

Step 1: Retrieval

The user submits a question.

For example:

"What is our company's Q3 remote work policy?"

The system converts the question into an embedding using the same or a compatible embedding model used during ingestion.

It then searches the vector database for chunks that are semantically similar to the query.

The vector database might return something like:

Chunk 1:
"Employees are required to work from the office..."

Chunk 2:
"Employees may work remotely up to three days per week..."

Chunk 3:
"Remote employees must remain available during..."

The system selects the most relevant chunks to provide as context to the LLM.

This is the retrieval part of Retrieval-Augmented Generation.

Step 2: Augmentation

The retrieved information is then combined with the user's original question.

The application typically uses a prompt template to tell the LLM how to use the retrieved information.

Conceptually, it could look like:

Use the following information to answer the user's question.

Context:
Employees may work remotely up to three days per week.
Remote employees must remain available during
normal working hours.

Question:
What is our company's Q3 remote work policy?

The LLM now has access to information that was retrieved specifically for the user's question.

This is the augmentation part of RAG.

Step 3: Generation

The augmented prompt is sent to the LLM.

The model uses the retrieved context, together with its language understanding and reasoning capabilities, to generate the final response.

For example:

"According to the company's policy, employees may work remotely up to three days per week while remaining available during normal working hours."

This is the generation part of RAG.

Putting everything together:

                   DATA PREPARATION
                   -----------------

Documents
    |
    v
Chunking
    |
    v
Embedding Model
    |
    v
Vector Database
    |
    |
    |                    RUNTIME
    |                    -------
    |                       |
    |                  User Question
    |                       |
    |                       v
    +-------------->  Query Embedding
                            |
                            v
                     Similarity Search
                            |
                            v
                    Relevant Chunks
                            |
                            v
                       Prompt + Context
                            |
                            v
                           LLM
                            |
                            v
                     Generated Answer

Why Organizations Use RAG

RAG is particularly useful for organizations that need AI systems to work with their own data.

1. Grounding AI Responses

LLMs can sometimes generate information that sounds convincing but is incorrect. This is commonly referred to as hallucination.

RAG can reduce this problem by providing the model with relevant source information before it generates an answer.

However, RAG does not magically eliminate hallucinations.

If the retrieval system returns irrelevant information, incomplete information, or outdated information, the LLM can still produce a poor answer.

This is why the quality of the retrieval system is just as important as the LLM itself.

2. Easier Knowledge Updates

Suppose a company changes its leave policy.

With a traditional approach, you might consider retraining or fine-tuning a model to incorporate the new information.

With RAG, you can update the underlying knowledge base.

The new document can be processed, chunked, embedded, and added to the vector database.

The LLM itself does not need to be retrained simply because the organization's documentation changed.

This makes RAG particularly useful for information that changes frequently.

3. Working With Private Data

Organizations have large amounts of internal information that general-purpose AI models were not trained on.

This could include:

Internal policies
Product documentation
Customer support information
Financial reports
Technical documentation
Employee handbooks
Business processes
Internal knowledge bases

RAG provides a way to build an AI interface over this information while keeping the organization's knowledge source separate from the underlying LLM.

Of course, privacy and security still need to be designed carefully. Access controls, authentication, authorization, data isolation, logging, and appropriate handling of sensitive information remain important.

4. Cost and Maintenance

Continuously retraining a large language model whenever an organization's information changes is generally impractical.

RAG separates the knowledge layer from the language model.

The LLM handles language understanding and generation, while the retrieval system provides the relevant organizational knowledge.

This separation makes it easier to maintain and update the knowledge source independently.

RAG Is More Than a Vector Database

One common misconception is that building a RAG system simply means:

"Put documents into a vector database and connect it to an LLM."

In practice, there are several components involved.

A production RAG system may include:

Document ingestion
Text extraction
Chunking
Embedding models
Vector databases
Metadata filtering
Keyword search
Semantic search
Hybrid search
Reranking
Prompt engineering
LLMs
Access control
Observability
Evaluation
Caching
Data freshness mechanisms

This means RAG is better understood as an architecture rather than a single technology.

The Importance of Retrieval Quality

One of the most important lessons when building RAG systems is:

A better LLM cannot always compensate for poor retrieval.

Imagine a user asks:

"What is the company's maternity leave policy?"

But the retrieval system returns documents about annual leave.

The LLM now has the wrong context.

Even if the LLM is extremely capable, it may generate an answer based on irrelevant information.

A simplified way to think about the process is:

Good Retrieval
      +
Relevant Context
      +
Good LLM
      =
Useful Answer

But:

Poor Retrieval
      +
Wrong Context
      +
Good LLM
      =
Potentially Wrong Answer

This is why RAG evaluation needs to look beyond the final response. You also need to evaluate whether the system retrieved the right information in the first place.

RAG and Traditional LLM Applications

A standard LLM application might look like:

User
  |
  v
LLM
  |
  v
Response

The model relies primarily on the information encoded in its parameters during training, plus whatever information is included in the current conversation or prompt.

A RAG application adds an external knowledge layer:

                  +----------------+
                  | Knowledge Base |
                  +-------+--------+
                          |
                          v
User -> Retrieval -> Context -> LLM -> Response

This architectural difference is what makes RAG useful for organization-specific knowledge.

A Practical Example

Consider a healthcare organization with thousands of documents.

An employee asks:

"What is the process for approving a medical claim?"

Without RAG, the LLM may have general knowledge about medical claims, but it does not necessarily know the organization's specific process.

With RAG:

User Question
     |
     v
Search Knowledge Base
     |
     v
Retrieve Claims Procedure
     |
     v
Add Procedure to Prompt
     |
     v
LLM
     |
     v
Answer Based on Retrieved Procedure

The same architecture can be applied to many other domains:

Banking
Insurance
Healthcare
Education
Legal documentation
Customer support
Enterprise IT
Human resources
Government services

Anywhere an organization has a large collection of documents or structured knowledge, RAG can potentially provide a useful interface for accessing that information.

Conclusion

Retrieval-Augmented Generation is one of the most practical architectures for connecting LLMs to external knowledge.

The fundamental idea is simple:

Retrieve relevant information first, then let the LLM generate an answer using that information.

A RAG system generally consists of two major phases.

During data preparation, documents are collected, chunked, converted into embeddings, and stored in a searchable knowledge store.

During runtime, a user's question is converted into a searchable representation, relevant information is retrieved, that information is added to the prompt, and the LLM generates the final response.

The important thing to remember is that RAG is not just about the LLM. The quality of the entire pipeline matters - from how documents are processed and chunked, to how information is retrieved, filtered, ranked, and finally presented to the model.

As organizations look for practical ways to use AI with their own data, understanding RAG provides an important foundation for building AI applications that are connected to real, domain-specific knowledge rather than relying solely on what a model learned during training.

What is Grok Bot? The Complete Beginner's Guide and Use Cases

Sospeter Mong'are — Fri, 18 Sep 2026 06:57:56 +0000

Imagine having a personal assistant who doesn't just type out answers, but actually sits down at a computer, opens a web browser, fills out forms, and organizes files for you. That is exactly what Grok Bot does.

As artificial intelligence evolves, we are moving away from simple chatbots and entering the era of AI "agents"—programs that can actively execute multi-step tasks. In this guide, we will break down what Grok Bot is, how it works, and how you can use it to automate your daily digital chores.

What is Grok Bot?

At its core, Grok Bot is an autonomous AI assistant designed to interact with a computer just like a human would. Unlike traditional AI that only generates text or code, Grok Bot is equipped with a dedicated, persistent Linux virtual machine in the cloud.

This cloud computer gives the bot its own:

Web Browser: To navigate websites, click buttons, and log into portals.
File Manager: To download, organize, and edit documents or spreadsheets.
Terminal: To run commands and execute lightweight scripts.

Because it operates on an independent cloud machine, it is always on. You can give Grok Bot a complex assignment, close your laptop, walk away, and it will keep working in the background until the job is done.

Top Use Cases for Grok Bot

Because Grok Bot can use the internet and manage files, its practical applications are almost limitless. Here are the most common ways people use it:

1. Automating Boring Workflows

We all have repetitive digital tasks that eat up our time. Grok Bot can take over these chores entirely. For example, it can log into your company’s vendor portal every Friday, download invoice data, and manually type that information into an accounting spreadsheet. It is perfect for moving data between two software platforms that do not easily talk to each other.

2. Deep Web Research and Monitoring

Instead of spending hours searching the web, you can deploy Grok Bot to do the heavy lifting. It can scan competitor websites to track pricing changes, gather contact details from public directories to build lead lists, or monitor retail sites for hard-to-find inventory, instantly alerting you when an item comes back in stock.

3. File and Data Management

If you deal with large volumes of data, Grok Bot can act as your digital organizer. It can look through a folder containing thousands of downloaded files, rename them based on specific rules, convert them from PDFs to text files, and filter out duplicate entries using its built-in terminal tools.

4. Software and QA Testing

For developers and businesses, Grok Bot can act as a real user. You can instruct it to visit your newly built website, click through the checkout process, and report back if any links are broken or if the system crashes.

What Else Should Beginners Know?

Before you dive in, there are a few essential things to keep in mind to get the most out of your AI assistant:

A Shared Environment: If you build or deploy multiple bots under your account, they all share access to the same single cloud machine, file system, and browser sessions. This makes it easy for different bots to collaborate on the same set of files.
Security and Credentials: Because Grok Bot needs to log into websites to do its job, you will need to manage security credentials. Always practice safe data habits by using secure credential managers or dedicated account permissions for your bot.
Beginner-Friendly Controls: You don't need to be a computer programmer to use Grok Bot. You can guide it using plain, natural language instructions, and it will translate your goals into clicks, typing, and commands.

Conclusion

Grok Bot represents a massive leap forward in personal productivity. By pairing advanced artificial intelligence with a fully functional cloud computer, it transforms from a simple conversation partner into a capable digital worker. Whether you are trying to escape repetitive data entry, track market trends, or manage messy file directories, Grok Bot can handle the heavy lifting—leaving you free to focus on the work that actually matters.

How to deploy a Laravel app to a VPS

Sospeter Mong'are — Wed, 16 Sep 2026 18:57:04 +0000

This is a beginner guide for a Laravel app where the pages and the API live in the same project. Blade is the frontend. Laravel routes under /api are the backend. There is no separate Node server.

Replace YOUR_SERVER_IP, YOUR_DOMAIN, YOUR_APP, and STRONG_PASSWORD with your own values. Do not copy a real password, APP_KEY, or a private key into a chat, a screenshot, or git.

Introduction

A VPS is a Linux computer in a data center. Visitors type a domain. An A record points that domain at the server's IP. Nginx receives HTTPS and hands every URL to PHP.

One Laravel app answers both kinds of request. They share public/index.php, the same .env, and the same MySQL database.

Request	Who answers	Why
`https://YOUR_DOMAIN/` and `/login`	Laravel `routes/web.php`	Blade pages
`https://YOUR_DOMAIN/api/...`	Laravel `routes/api.php`	The API in the same app
MySQL	`127.0.0.1:3306` only	Laravel's own database. It is not on the public internet

There is no second port to open. PHP-FPM is a local socket, not a public port. Nginx is the only door.

GitHub Actions cannot upload code until the server already has a folder, a database, .env, Nginx, and an SSH key. Actions logs in as deploy, not as your sudo account. A leaked GitHub key can update the app, but it cannot install packages or change the firewall.

Two accounts, two jobs:

User	Job
`YOUR_SUDO_USER`	The account you SSH in as. It has `sudo`. On the Contabo server used for MtandaoBilling this account is `sos`. A lot of VPS images do not give you a root password.
`deploy`	Owns `/var/www/YOUR_DOMAIN`, runs Composer and Artisan, and is the GitHub Actions login. It has no password.

PHP itself runs as www-data. That user must be able to write storage/ and bootstrap/cache even though deploy owns the files.

1. Log in, and do not SSH as deploy yet

From your PC:

ssh YOUR_SUDO_USER@YOUR_SERVER_IP

You are on the server. The prompt looks like YOUR_SUDO_USER@your-server:~$.

Do not run ssh deploy@YOUR_SERVER_IP from that prompt. deploy has no password and, at this point, no SSH key. The server answers Permission denied (publickey). You are already on the machine. Switch user instead:

sudo -u deploy -i

sudo means "run this as another user." -u deploy -i opens a login shell as deploy. Type exit to go back to your sudo account.

apt, Nginx, and MySQL stay on the sudo account. Files inside the app folder are edited as deploy, after that user owns the folder.

2. Install what Laravel needs

You do not need PM2. PHP-FPM already stays running. You do need Node once, on the machine that builds CSS and JavaScript. On the server, Composer installs PHP packages. Vite assets are usually built in GitHub Actions and uploaded, so the server does not have to run npm if that build already happened.

Package	Why
`nginx`	Receives HTTPS and sends PHP files to PHP-FPM
`php8.3-fpm` and PHP extensions	Runs Blade pages and the `/api` routes. `fpm` is the PHP process Nginx talks to
`composer`	Installs Laravel's PHP libraries
`mysql-server`	Laravel's database
`certbot`	Gets a free HTTPS certificate from Let's Encrypt
`acl`	Lets `www-data` write logs and sessions in a folder owned by `deploy`
`ufw`	Firewall. Only SSH, HTTP, and HTTPS should be open

As YOUR_SUDO_USER:

sudo apt update && sudo apt upgrade -y
sudo apt install -y nginx mysql-server certbot python3-certbot-nginx \
  git curl unzip ufw acl \
  php8.3-fpm php8.3-cli php8.3-mysql php8.3-mbstring php8.3-xml \
  php8.3-curl php8.3-zip php8.3-bcmath php8.3-tokenizer

curl -sS https://getcomposer.org/installer | sudo php -- --install-dir=/usr/local/bin --filename=composer

If php8.3-* is not in your Ubuntu version, install PHP 8.2 or newer and later point Nginx at that socket (ls /run/php/).

Check that the commands exist:

php -v
composer -V
nginx -v
mysql --version

Create deploy once, if it does not exist. --disabled-password is deliberate. GitHub will log in with a key, not a password that can be guessed.

sudo adduser --disabled-password --gecos "" deploy

Open only SSH and the web ports. Do not open MySQL port 3306.

sudo ufw allow OpenSSH
sudo ufw allow 'Nginx Full'
sudo ufw enable

OpenSSH keeps your current login alive. Nginx Full is ports 80 and 443.

3. Make the app folder, and make deploy the owner

Nginx will not use this folder as the website root. It uses public/ inside it. The rest of Laravel (.env, app/, vendor/) must stay outside the web root so visitors cannot download it.

sudo mkdir -p /var/www/YOUR_DOMAIN
sudo chown -R deploy:deploy /var/www/YOUR_DOMAIN

chown means "this user owns these files."

A folder created earlier as root stays owned by root. mkdir -p as deploy does not change a folder that already exists, and nano then fails with Permission denied. Check with:

ls -ld /var/www/YOUR_DOMAIN

The owner must be deploy deploy. If it says root root:

sudo chown -R deploy:deploy /var/www/YOUR_DOMAIN

4. Create the MySQL database

The Linux user deploy is not the database user. Create a separate MySQL user. In MySQL, localhost and 127.0.0.1 are two different accounts.

localhost means "connect through a Unix socket." 127.0.0.1 means "connect with TCP." If .env says DB_HOST=127.0.0.1, a user created only @localhost is invisible. ALTER USER for the TCP account then fails with ERROR 1396 because that account does not exist yet.

PHP's MySQL driver can use MySQL 8's normal caching_sha2_password plugin. You do not need mysql_native_password unless this same database is also used by Prisma. This guide uses the normal plugin.

As YOUR_SUDO_USER:

sudo mysql

sudo mysql works without a MySQL root password because Ubuntu lets the system admin in through the socket. Invent a long password. Call it STRONG_PASSWORD here. Use the same string later in .env. Do not reuse a password from a chat log.

CREATE DATABASE your_app CHARACTER SET utf8mb4 COLLATE utf8mb4_unicode_ci;
CREATE USER 'your_app_user'@'127.0.0.1' IDENTIFIED BY 'STRONG_PASSWORD';
GRANT ALL PRIVILEGES ON your_app.* TO 'your_app_user'@'127.0.0.1';
FLUSH PRIVILEGES;
SELECT user, host FROM mysql.user WHERE user = 'your_app_user';

The host column must include 127.0.0.1. Then type exit.

5. Write `.env` on the server only

There is one .env, at the Laravel project root, not a separate frontend file. GitHub must not upload it.

As deploy:

nano /var/www/YOUR_DOMAIN/.env

Start from your project's .env.example and set at least these. Leave APP_KEY empty. Artisan fills it after Composer has run.

APP_NAME=YOUR_APP
APP_ENV=production
APP_KEY=
APP_DEBUG=false
APP_URL=https://YOUR_DOMAIN

DB_CONNECTION=mysql
DB_HOST=127.0.0.1
DB_PORT=3306
DB_DATABASE=your_app
DB_USERNAME=your_app_user
DB_PASSWORD=STRONG_PASSWORD

SESSION_DRIVER=database
CACHE_STORE=database
QUEUE_CONNECTION=database

APP_DEBUG=false so a crash does not print secrets in the browser. The real error still goes to storage/logs/laravel.log.

SESSION_DRIVER=database is fine for a Laravel app that owns MySQL, but only after php artisan migrate has created the sessions table. If you want the first page to work before that migrate, set SESSION_DRIVER=file and CACHE_STORE=file instead. A 500 that says no such table: sessions means the driver is database and migrate has not run.

As the sudo user, lock the file:

sudo chown deploy:deploy /var/www/YOUR_DOMAIN/.env
sudo chmod 600 /var/www/YOUR_DOMAIN/.env

600 means owner read/write, everyone else nothing.

6. Install Nginx and the free certificate

The website root is public/, not the project root. If you point Nginx at /var/www/YOUR_DOMAIN, visitors can request .env.

As YOUR_SUDO_USER:

sudo nano /etc/nginx/sites-available/YOUR_DOMAIN

Paste this. Read the comments. They are the reason each line exists.

server {
    listen 80;
    listen [::]:80;
    server_name YOUR_DOMAIN;

    # Only public/ is on the internet. app/, .env, and vendor/ are not.
    root /var/www/YOUR_DOMAIN/public;
    index index.php;

    access_log /var/log/nginx/YOUR_APP.access.log;
    error_log  /var/log/nginx/YOUR_APP.error.log;

    charset utf-8;
    client_max_body_size 20m;

    # Real files (CSS, images, Vite /build) are sent as files.
    # Every other URL, including /api/..., goes to public/index.php.
    # Laravel then chooses web.php or api.php. Nginx does not split them.
    location / {
        try_files $uri $uri/ /index.php?$query_string;
    }

    location = /favicon.ico { access_log off; log_not_found off; }
    location = /robots.txt  { access_log off; log_not_found off; }

    error_page 404 /index.php;

    # PHP-FPM socket. If this file does not exist, run: ls /run/php/
    location ~ \.php$ {
        include snippets/fastcgi-php.conf;
        fastcgi_pass unix:/run/php/php8.3-fpm.sock;
        fastcgi_param HTTPS $https if_not_empty;
    }

    # Block .env and .git if they ever land inside public/.
    # /.well-known/ stays open so Certbot can prove you own the domain.
    location ~ /\.(?!well-known).* {
        deny all;
    }
}

Enable the site and test it before you reload. nginx -t stops a typo from taking every site on the server down.

sudo ln -s /etc/nginx/sites-available/YOUR_DOMAIN /etc/nginx/sites-enabled/
sudo nginx -t && sudo systemctl reload nginx

A 404 or a PHP error now is normal. The app is not uploaded yet.

Then ask Let's Encrypt for a certificate. Certbot is free. The prompt "agree to register with the ACME server" is their terms, not a paid account. Type yes, and enter an email so they can warn you before a certificate expires.

sudo certbot --nginx -d YOUR_DOMAIN

Certbot edits the Nginx file on the server. It adds port 443 and redirects HTTP to HTTPS. Do not copy those certificate paths into git.

7. Give GitHub a key that can log in as deploy

A password in GitHub is worse than a key you can delete. The key has two halves:

File	Where it goes
`YOUR_APP_deploy.pub`	The server. This is public. It says "this key is allowed."
`YOUR_APP_deploy`	Your PC, then the GitHub secret. This is private. Never commit it, and never paste it into a chat.

On your PC, not inside the SSH session:

ssh-keygen -t ed25519 -C "github-actions-YOUR_APP" -f ~/.ssh/YOUR_APP_deploy -N ""
scp ~/.ssh/YOUR_APP_deploy.pub YOUR_SUDO_USER@YOUR_SERVER_IP:~/

In PowerShell, if ~ is not your user folder, use the full path, for example C:\Users\ADMIN\.ssh\YOUR_APP_deploy.pub.

On the VPS, as YOUR_SUDO_USER:

sudo mkdir -p /home/deploy/.ssh
sudo tee -a /home/deploy/.ssh/authorized_keys < ~/YOUR_APP_deploy.pub
sudo chown -R deploy:deploy /home/deploy/.ssh
sudo chmod 700 /home/deploy/.ssh
sudo chmod 600 /home/deploy/.ssh/authorized_keys

authorized_keys is the list of public keys allowed to log in as that user. 700 and 600 are not optional. SSH ignores a key file if other users can read it.

Test from your PC. You should land in /home/deploy with no password:

ssh -i ~/.ssh/YOUR_APP_deploy deploy@YOUR_SERVER_IP

8. Add the GitHub secrets

In the repository: Settings, then Secrets and variables, then Actions, then New repository secret.

Secret	Value
`SSH_HOST`	`YOUR_SERVER_IP`
`SSH_USERNAME`	`deploy`
`SSH_PRIVATE_KEY`	The whole private file, including the `BEGIN` and `END` lines
`SSH_PORT`	`22`
`DEPLOY_PATH`	`/var/www/YOUR_DOMAIN`

Show the private key on your PC only, then paste it into GitHub. Use the file without .pub.

cat ~/.ssh/YOUR_APP_deploy

A small workflow you can put in .github/workflows/deploy.yml of that Laravel project. It builds the Vite files in GitHub, uploads the app, and does not upload .env or node_modules.

name: Deploy Laravel

on:
  push:
    branches: [main]

jobs:
  deploy:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4

      - uses: shivammathur/setup-php@v2
        with:
          php-version: "8.3"
          tools: composer:v2

      - uses: actions/setup-node@v4
        with:
          node-version: "20"

      - name: PHP dependencies and Vite build
        run: |
          composer install --no-interaction --prefer-dist --optimize-autoloader --no-dev
          npm ci
          npm run build

      - name: Upload
        uses: appleboy/scp-action@v0.1.7
        with:
          host: ${{ secrets.SSH_HOST }}
          username: ${{ secrets.SSH_USERNAME }}
          key: ${{ secrets.SSH_PRIVATE_KEY }}
          port: ${{ secrets.SSH_PORT || 22 }}
          source: ".,!node_modules,!.env"
          target: ${{ secrets.DEPLOY_PATH }}
          overwrite: true

      - name: Migrate and refresh caches
        uses: appleboy/ssh-action@v1.2.0
        with:
          host: ${{ secrets.SSH_HOST }}
          username: ${{ secrets.SSH_USERNAME }}
          key: ${{ secrets.SSH_PRIVATE_KEY }}
          port: ${{ secrets.SSH_PORT || 22 }}
          script: |
            cd "${{ secrets.DEPLOY_PATH }}"
            composer install --no-dev --optimize-autoloader --no-interaction
            php artisan storage:link || true
            php artisan migrate --force
            php artisan config:cache
            php artisan route:cache
            php artisan view:cache
            sudo systemctl reload php8.3-fpm

deploy cannot run sudo systemctl reload unless you allow that one command. If the reload line fails, SSH in as YOUR_SUDO_USER and run sudo systemctl reload php8.3-fpm yourself after the first deploy. PHP keeps the old cached config until that reload.

public/build must be in the upload. If GitHub skips npm run build, Laravel pages that use @vite return 500 with "Vite manifest not found."

9. First boot on the server

After the first upload, as deploy:

cd /var/www/YOUR_DOMAIN
composer install --no-dev --optimize-autoloader --no-interaction
php artisan key:generate --force
php artisan migrate --force
php artisan storage:link
php artisan config:cache
php artisan route:cache
php artisan view:cache

key:generate writes APP_KEY. Without it, every page is 500. Do not run it again on later deploys. A new key logs everybody out and can make old encrypted cookies unreadable.

migrate --force creates tables, including sessions if you use the database session driver. --force is required because Laravel refuses to migrate when APP_ENV=production unless you say so.

storage:link makes public/storage point at uploaded files. Skip the error if the link already exists.

config:cache bakes .env into one PHP file. After this, editing .env does nothing until you run config:cache again.

PHP must be able to write logs and sessions. As YOUR_SUDO_USER:

sudo mkdir -p /var/www/YOUR_DOMAIN/storage/framework/{cache/data,sessions,views} \
  /var/www/YOUR_DOMAIN/storage/logs \
  /var/www/YOUR_DOMAIN/bootstrap/cache
sudo chown -R deploy:www-data /var/www/YOUR_DOMAIN/storage /var/www/YOUR_DOMAIN/bootstrap/cache
sudo chmod -R ug+rwX /var/www/YOUR_DOMAIN/storage /var/www/YOUR_DOMAIN/bootstrap/cache
sudo systemctl reload php8.3-fpm

www-data is the PHP user. The group write bit is what lets PHP save sessions and log a 500 instead of dying with Permission denied.

10. Confirm the site

Open https://YOUR_DOMAIN/ and one API URL, for example https://YOUR_DOMAIN/api/user or whatever your routes/api.php exposes. A 401 on a protected API route is fine. It means Laravel answered. A 404 from Nginx, with no Laravel styling, means the request never reached index.php.

If the page is 500:

sudo grep "production.ERROR" /var/www/YOUR_DOMAIN/storage/logs/laravel.log | tail -1

The first production.ERROR line is the cause. The long stack under it is not.

What the log says	What to do
`No application encryption key`	`php artisan key:generate --force`, then `config:cache`, then reload PHP-FPM
`no such table: sessions`	`php artisan migrate --force`, or set `SESSION_DRIVER=file` and cache config again
`Permission denied` under `storage/`	Repeat the `chown` and `chmod` in step 9
`Vite manifest not found`	`npm run build` in GitHub (or on the server) so `public/build/manifest.json` exists
`Access denied for user`	`DB_USERNAME`, `DB_PASSWORD`, and `DB_HOST=127.0.0.1` must match the MySQL user from step 4

After any .env change:

cd /var/www/YOUR_DOMAIN
php artisan config:cache

Then as the sudo user: sudo systemctl reload php8.3-fpm.

Conclusion

Prepare the server before GitHub deploys. The order is: PHP, Composer, Nginx, and MySQL, a deploy user, a folder that user owns, a MySQL user at 127.0.0.1, one .env, Certbot, an SSH key, then a push.

Blade pages and /api are the same app. Nginx sends both to public/index.php. You do not run a second process and you do not open an API port.

When it fails, look here first:

storage/logs/laravel.log for a 500 page. Read the production.ERROR line.
Confirm Nginx root is .../public, not the project root.
Confirm DB_HOST is 127.0.0.1 and that SELECT user, host FROM mysql.user shows that host.
After an .env edit, php artisan config:cache and reload PHP-FPM.

Do not put the database password, APP_KEY, or the private key in git. deploy is the only account GitHub should log in as.