DEV Community: Antony Nyagah

I Tried to Learn RAG and Got Stuck on API Costs. Turns Out My CPU Could Do the Expensive Bit.

Antony Nyagah — Mon, 14 Sep 2026 06:00:00 +0000

A few months ago, I started building a Retrieval-Augmented Generation application because I wanted to understand what was actually happening underneath all the buzzwords. The project I built was appropriately named simple_rag.

The idea was simple enough: upload a PDF, ask questions about it, and have an LLM answer using information from the document. I got it working. Then I mostly abandoned it.

The thing I couldn't figure out how to handle cleanly was running the embedding model locally.

I accidentally made a simple project expensive

My original implementation used Gemini for both embeddings and generation. Later, I switched it to Mistral. Both worked, but the more I played with the project, the more I disliked one part of the architecture: every time I wanted to turn some text into embeddings, I was calling somebody else's API, with its own rate limits and its own costs.

That meant another API key, another service to depend on, another rate limit to think about, and potentially another bill. For a production application where the trade-off makes sense, fine. For a tiny side project whose entire purpose was basically:

Antony would like to understand RAG.

...it felt ridiculous.

Embeddings are fundamental to how this implementation works. When you upload a document, the text gets split into chunks, and each chunk gets converted into an embedding: a numerical representation of its meaning. That embedding gets compared against the question's embedding, and whichever chunks are most semantically similar get sent to the LLM as context. That is basically the retrieval part of Retrieval-Augmented Generation.

So if creating embeddings depended on an API I didn't want to keep paying for, the whole experiment got a lot less interesting. The repository sat there.

Then I discovered I was overcomplicating the problem

Recently I came back to the project and discovered fastembed, and had one of those slightly embarrassing engineering moments where you realize the problem that made you abandon something months ago never actually needed the solution you'd assumed it needed.

I could generate the embeddings locally, on my CPU, without touching an embedding API, a GPU, or a heavy PyTorch install, and without a per-embedding bill.

I'm currently using BAAI/bge-small-en-v1.5 through FastEmbed. It runs on ONNX Runtime and is small enough that my completely ordinary laptop handles it without breaking a sweat.

That changed the architecture quite a bit. The current flow is basically:

PDF / EPUB
    ↓
Extract text
    ↓
Split text into chunks
    ↓
Generate embeddings locally on my CPU
    ↓
Store the vectors in memory
    ↓
User asks question
    ↓
Embed question locally
    ↓
Compare it with document embeddings
    ↓
Retrieve the most relevant chunks
    ↓
Send those chunks + question to the LLM
    ↓
Answer

That's it. And weirdly, stripping things away helped me understand RAG much better than adding more AI tooling ever did.

I don't even have a vector database

This was another thing I had mentally associated with RAG.

Embeddings mean vectors. Vectors mean vector databases. Therefore RAG means I need Pinecone, Qdrant, Chroma, pgvector, or something similar.

Except... not necessarily. My application currently stores embeddings as plain Python data and uses NumPy to calculate cosine similarity. For every question, it compares the question's vector against the chunk vectors, sorts by similarity, and grabs the best matches.

Is that how I'd build a system with millions of documents? Absolutely not. Is it enough for a PDF, an EPUB, or a few hundred pages while I'm trying to understand retrieval? Absolutely.

That distinction has become increasingly important to me when learning new technology. There's a huge difference between asking what do I need to understand this, and what would I need to operate this at serious production scale. The internet has a habit of giving you the second architecture while you're still trying to answer the first question.

The LLM is now the only remote part

I still needed a model to generate the final answer, and this is where I found Groq particularly interesting.

For my current side-project level of usage, their free tier is enough to experiment with. I created an API key without even adding a credit card, pointed the OpenAI-compatible client at Groq, and that was basically it. I'm currently using openai/gpt-oss-120b.

So now the parts have a much cleaner separation.

My machine handles:

Extracting the document text.
Chunking it.
Creating embeddings.
Storing those embeddings.
Embedding the question.
Retrieving the relevant chunks.

The LLM handles:

Taking the question plus retrieved context.
Producing a useful, human-readable answer.

That separation made another thing click for me: RAG isn't really about making the LLM know your data, since the model doesn't suddenly learn my PDF. I'm building a retrieval system that finds useful information first, then handing that information to the model at generation time.

The retrieval is doing a lot more of the work than I originally appreciated.

I wanted the implementation to stay boring

The core RAG logic is only a few hundred lines of Python. That's deliberate.

There's a FastAPI API around it and a small browser UI, plus a CLI because apparently I cannot make a Python project without eventually wanting to run it from the terminal. But the actual interesting bit is separate from all that.

The RAG module handles:

Text extraction.
Chunking.
Embedding.
Similarity search.
Retrieval.
Generation.

The interfaces just call it. I also added an evaluation script using a paper I already had lying around, Attention Is All You Need, with questions of varying difficulty.

Again, the point was not to build some enterprise RAG platform. I wanted something small enough that I could open rag.py, read it top to bottom, and understand what every part was doing. I learn better that way.

Local AI is becoming much more interesting to me

The embedding discovery also changed how I think about adding AI features to my own applications. My default assumption used to be that "adding AI" meant sending everything to some massive remote model. That assumption is becoming less useful. There are increasingly parts of the pipeline I can run locally or self-host:

Embeddings.
Classification.
Reranking.
Transcription.
Smaller language models.
Various forms of semantic search.

That leaves paid inference for the bits that actually benefit from a larger model, which matters for side projects, and matters even more when building software in places where throwing $50 or $100 a month at every interesting API isn't a particularly attractive development strategy.

I already like self-hosting normal infrastructure. Apparently I'm slowly becoming the person who wants to self-host chunks of the AI stack too. This was probably inevitable.

More importantly, I want to actually use this stuff

I have zero interest in adding an AI button to every application just because investors have collectively decided that everything now needs AI. What I am increasingly interested in is finding places where these models genuinely improve an existing workflow.

RAG is interesting because there are obvious uses for it. Documentation is one: imagine an internal application with years of manuals, procedures, policies and reports. Instead of manually searching through them, you ask a question and the relevant sections get retrieved before the answer is generated. That's useful.

Another thing I want to experiment with next is databases. I've spent years building systems where somebody eventually asks:

Can you give me a report showing X, grouped by Y, for these dates?

...and then somebody has to write SQL, add a report screen, export something to Excel, or build yet another dashboard. I want to experiment with a safer natural-language interface where somebody can ask:

How many registrations did we receive per month this year?

and the system understands the database schema, generates a read-only query, validates it, executes it, and explains the result.

That's not necessarily the same problem as RAG. Structured databases need different techniques, and I definitely do not want an LLM firing random DELETE statements at production. But it's exactly the kind of AI experimentation I'm interested in now: giving the application another interface to capabilities it already has.

The useful lesson wasn't actually about RAG

I started this project because I wanted to understand embeddings, retrieval and RAG. I did learn those things, but I think the more useful lesson was about how easily we confuse the infrastructure around a technology with the technology itself.

You do not need a vector database to understand vector search.

You do not need an embedding API to create embeddings.

You do not need an enormous framework to understand RAG.

And you definitely do not need to start by building the architecture meant to serve ten million users.

Sometimes you need a Python file, NumPy, a small model running on your CPU, and enough curiosity to keep poking at the thing until it stops looking like magic. That is what simple_rag has become for me.

The repository is still called simple RAG. This time, I think it actually deserves the name.

The project is on GitHub: tony-nyagah/simple_rag.

I am going to keep experimenting with practical AI integrations and documenting what I learn, especially the boring, useful stuff between "call an LLM" and "build an autonomous superintelligence."

Self-Hosting is easier than you think: A guide to open source PaaS

Antony Nyagah — Wed, 24 Jun 2026 09:30:00 +0000

What I Was Looking For
What Dokploy Actually Is
What I Actually Run on It
The Trade-offs
Ten Minutes to Your Own Platform
Should You Switch?

A while back I built an email verifier using Go while learning the language and wanted to deploy it. Sadly, Vercel did not support Go at the time. None of the other platform-as-a-service providers supported it either at the time. Most of them supported only JavaScript, TypeScript and maybe Python.

So I went looking.

In the process of looking for something similar to the PaaS I was used to, I came across various self-hosted solutions.

What I was looking for

I wanted three things:

A nice UI + dashboard. Managing apps and services on the terminal is fun and all but I am a more visual person. I wanted a dashboard that would let me see all my apps and services at a glance.
Git-push deploys. I got used to that flow from Vercel and I wasn't giving it up.
No lock-in. If the platform died tomorrow, I want to move on quickly to another solution.
Ease of use. I wanted something that I could get up and running with little issue.

I tried a few things:

Coolify is the most mature. It supports something like 50 different services and is very feature-rich. But I found myself lost when trying to set it up on my VPS. This might have changed since I last tried it.

The UI was also a bit overwhelming in terms of what I had to configure to get my projects running.
CapRover is the opposite. Nice and simple. Does exactly what you want. This has greatly improved since I last tried it a few years ago. Very simple and to the point.
Kamal is interesting. It's 37signals' tool for deploying Basecamp and HEY. All CLI, no web UI. If you live in a terminal and you trust yourself with YAML at 2am, it's solid. I wanted something between "bare metal CLI" and "kitchen sink platform."
Docker Compose + GitHub/GitLab actions. I still use this on some projects. But I wanted a better way to monitor what's happening with my software.

I finally settled on Dokploy.

Dokploy / dokploy

Open Source Alternative to Vercel, Netlify and Heroku.

Join us on Discord for help, feedback, and discussions!

Dokploy is a free, self-hostable Platform as a Service (PaaS) that simplifies the deployment and management of applications and databases.

✨ Features

Dokploy includes multiple features to make your life easier.

Applications: Deploy any type of application (Node.js, PHP, Python, Go, Ruby, etc.).
Databases: Create and manage databases with support for MySQL, PostgreSQL, MongoDB, MariaDB, libsql, and Redis.
Backups: Automate backups for databases to an external storage destination.
Docker Compose: Native support for Docker Compose to manage complex applications.
Multi Node: Scale applications to multiple nodes using Docker Swarm to manage the cluster.
Templates: Deploy open-source templates (Plausible, Pocketbase, Calcom, etc.) with a single click.
Traefik Integration: Automatically integrates with Traefik for routing and load balancing.
Real-time Monitoring: Monitor CPU, memory, storage, and network usage for every resource.
Docker Management: Easily deploy…

View on GitHub

What Dokploy actually is

One binary. You SSH into a fresh VPS, run a single curl command, and you get a dashboard on port 3000. Behind the scenes it's Docker and Traefik doing the actual work. Dokploy is the UI layer on top.

curl -sSL dokploy.com/install.sh | bash
# That's it. Open port 3000.

What you get:

Git integration. Connect GitHub or GitLab, push to main, it builds and deploys. Same flow as Vercel.
Docker Compose support. I didn't have to learn a new config format. It reads my existing compose files.
Auto SSL. I have never once manually configured an SSL cert with Dokploy. Let's Encrypt via Traefik, zero touching.
Built-in monitoring. Logs, resource usage, healthchecks. I can see if my Telegram bot crashed without opening a terminal.

One of the most interesting parts is the Templates for common apps such as Anubis, AppSmith, Listmonk, ERPNext, Ghost, GitLab, Bugsink and many more. The list is massive.
The templates give you an editable Docker Compose file so you can customize the app to your needs. So you're not forced to use a specific setup.

Of note is that most of these solutions depend on Docker. Most of them are essentially a UI on top of a standard Docker + Docker Compose + Traefik configuration. This means you can seamlessly move from one self-hosted solution to another. No lock-in whatsoever.

What I actually run on it

I have been using Dokploy in production for a few month now to run some open source applications including:

Listmonk - email marketing campaigns for an SME
umami - website analytics to track user behavior on a new project
Bugsink - error tracking for a legacy project we're maintaining

So far the setup was simple and straightforward. Pick a template, point a domain at it, edit the configurations such as environment variables and secrets, and press the deploy button.

I also have some personal projects running on it. Mainly projects built while learning new technologies or experimenting with new ideas.

The trade-offs

Self-hosting isn't all upside. I'm not going to pretend it is.

You don't get a global edge network. Your server is in one region. If your users are in Australia and your VPS is in Germany, they'll feel it. Vercel's edge network is genuinely good at this. For most side projects this doesn't matter. But if you're building something global, keep it in mind.

You manage capacity. If you get posted on Hacker News and your traffic goes 10x in an hour, Dokploy won't auto-scale. You'll need to handle that. For most side projects? Won't happen. But be honest with yourself.

You're responsible for uptime. If the server goes down at 3am, nobody's fixing it but you. I use Uptime Kuma for monitoring and I get Telegram alerts. Works for me. Might not work for you.

Database backups are on you. Dokploy makes it easy to provision handle that though. Your Docker volumes can be backed up to an S3 bucket or equivalent.

But here's what you get: fixed monthly cost. No surprise bills. Run "unlimited" projects. Full control over your infrastructure. Data stays wherever you want it. And the DevOps skills you pick up? They transfer everywhere.

Ten minutes to your own platform

Get a VPS. I use Hetzner. DigitalOcean and Linode work too. Latest Ubuntu.
SSH in and run the install command. That's it. Dokploy sets up Docker, Traefik, and the dashboard.
Point your domain. A record to the VPS IP. SSL happens automatically.
Deploy something. Connect a GitHub repo, paste a compose file, or pick a template.

Ten minutes. Then you have a platform that costs less than lunch, runs "unlimited" apps, and you actually own.

And if there's any issues, the Dokploy documentation is fantastic.

Should you switch?

If you're happy on Vercel or Heroku or something similar and you don't hit their limits, stay. They're good products.

But if you've ever opened a surprise bill, or hit a timeout limit that broke your app, or wanted to run something that didn't fit the platform's mold — try Dokploy. Spin up a VPS for a month and deploy one or two or ten side projects. Worst case, you're out less than $10. Best case, you realize you don't need the closed source PaaS platforms.

You Don't Need Docker Desktop on Linux

Antony Nyagah — Wed, 17 Jun 2026 08:00:00 +0000

Docker Desktop was using 4GB of RAM on my machine. Sitting there. Doing nothing. No containers running. Just the daemon.

I don't know about you, but 4 gigs for a container runtime that isn't running any containers is a hard no from me.

I'm on Fedora, but this applies to Ubuntu, Debian, whatever. If you're running Linux, you don't need Docker Desktop.

So I started looking at what I actually needed. Turns out: not much.

What Docker Desktop actually is

Docker Desktop is a GUI wrapper around a VM that runs the Docker Engine. On Linux, the VM part is unnecessary. Docker runs natively on Linux. The VM exists because Windows and macOS don't have a native Linux kernel. But on Linux? You're running a VM to run something that already runs natively on your system. That's the bloat.

What you actually need on Linux:

Docker Engine — the daemon that runs containers (docker-ce)
Docker Compose — the plugin, not the old Python binary (docker compose, not docker-compose)
The CLI — docker commands, which come with the engine

That's it. No GUI. No VM. No 4GB idle footprint.

The auto-start nonsense

Here's what really pushed me over the edge. Every time I rebooted my Fedora machine, Docker Desktop wouldn't auto-start. I had to manually launch it. Every single time.

This is supposed to "just work." It's a desktop application. Starting on boot is table stakes. But nope — I'd boot up, try to run docker ps, get a "cannot connect to daemon" error, and have to go find Docker Desktop in my app launcher and start it manually. Like it's 2012 and I'm starting a service by hand.

Compare that to the Docker Engine installed natively:

sudo systemctl enable docker
sudo systemctl start docker

Done. Starts on boot. Every time. No manual launching. No GUI to hunt down. It just works. Because it's a system service, not a desktop app cosplaying as one.

Podman is right there

While I was figuring all this out, I also realized Podman Desktop exists. And it does basically the same thing Docker Desktop does — a GUI for managing containers, images, volumes — but it uses Podman under the hood. Which is daemonless. Which means even less overhead.

Podman Desktop on my machine: about 800MB idle. Docker Desktop: 4GB. Same machine. Same workload (nothing). Five times the memory for the same thing.

Now, I still use Docker for most things. My docker-compose files, my CI pipelines, my multi-arch builds — they're all Docker. I'm not saying everyone should switch to Podman. I'm saying if you're on Linux and you're using Docker Desktop, you're paying a VM tax for no reason. The engine is already there.

What I actually run

Here's my setup. Pick your distro:

# Fedora / RHEL
sudo dnf install docker-ce docker-ce-cli containerd.io docker-compose-plugin

# Ubuntu / Debian
sudo apt install docker-ce docker-ce-cli containerd.io docker-compose-plugin

# Then for both:
sudo systemctl enable --now docker
sudo usermod -aG docker $USER

That's it. No Docker Desktop. No VM. No 4GB idle. No manual launching. docker commands work. docker compose works. Everything I need.

(If Docker's repos aren't set up yet: Fedora guide or Ubuntu guide. It's three commands.)

And if I want a GUI for browsing images or checking logs? Podman Desktop. It connects to Docker's socket just fine. Best of both worlds: Docker engine under the hood (because that's what my CI and my compose files expect), Podman Desktop when I want a visual overview, and neither one is eating my RAM for breakfast.

Do you actually need Docker Desktop?

If you're on Windows or macOS: probably yes. Docker Engine needs a Linux kernel, and Docker Desktop provides that VM. It's doing real work on those platforms.

If you're on Linux: probably no. The engine runs natively. You don't need a VM. You don't need a GUI (the CLI is better once you learn 10 commands). And if you occasionally want a GUI, Podman Desktop is lighter and free.

Docker Desktop on Linux is like installing a virtual machine to run Firefox. Firefox already runs on Linux. Just install Firefox.

The 10 commands you actually need

If the CLI is what's holding you back, here's what covers 90% of daily use. It's not that many:

docker ps                 # what's running?
docker ps -a              # what exists (including stopped)?
docker logs <container>   # what's it saying?
docker compose up -d      # start my stack
docker compose down       # stop my stack
docker compose pull       # update images
docker compose up -d --build  # rebuild and restart
docker exec -it <container> sh  # get a shell inside
docker system prune -a    # clean up everything unused
docker stats              # what's eating my RAM?

Ten commands. You'll use maybe five of them regularly. The GUI is nice to have but not worth 4GB.

Just try it

Remove Docker Desktop. Install docker-ce and the compose plugin. Systemctl enable it. See if you miss the GUI. If you do, install Podman Desktop. You can always reinstall Docker Desktop — but I don't think you will.

I haven't opened Docker Desktop in over a year. My containers run fine. My RAM is happier. My machine boots and Docker is just... there. Working. Like it should.

How I Built a Real-Time DDoS Detection Engine That Learns What Normal Looks Like

Antony Nyagah — Wed, 29 Apr 2026 22:56:24 +0000

Imagine your web server suddenly starts getting hammered by thousands of requests from one IP address. Without any protection, your server slows down, legitimate users get blocked out, and you have no idea it's even happening until it's too late. That's the problem I set out to solve by building a real-time anomaly detection engine alongside a Nextcloud server. The tool watches every single HTTP request coming through Nginx, learns what "normal" traffic looks like over time, and automatically blocks attackers using iptables — the Linux firewall built into every server. No third-party tools, no Fail2Ban, just pure detection logic built from scratch in Python.

The core of the system is two ideas working together: a sliding window and a rolling baseline. The sliding window is a Python deque that holds the timestamps of every request in the last 60 seconds — one per IP, one globally. When a new request arrives, its timestamp is added to the back; anything older than 60 seconds is evicted from the front. Dividing the count by 60 gives the current requests-per-second rate. The baseline is a 30-minute rolling average of per-second rates, recalculated every 60 seconds, with separate slots for each hour of the day so the system knows that midnight traffic is naturally lighter than noon. When a new request comes in, the detector computes a z-score: (current_rate - mean) / stddev. If that score exceeds 3.0, or the rate is more than 5 times the baseline mean, the IP is flagged as an attacker. The response is immediate — the system runs iptables -I INPUT -s <attacker_ip> -j DROP, which tells the Linux kernel to silently drop every packet from that IP at the network level, before it even reaches the application. A Slack alert fires within seconds, and the ban is automatically lifted on a backoff schedule: 10 minutes for the first offense, 30 minutes for the second, 2 hours for the third, and permanent after that.

How to Leverage Git and AI to View Your Achievements as a Developer

Antony Nyagah — Wed, 30 Oct 2024 11:00:00 +0000

Introduction

As developers, we often move from one task to the next without much time to reflect. But periodically assessing what you’ve accomplished can be a great motivator and help you communicate your value to teams, managers, or potential employers.
Here’s a simple approach to using your Git commit history to track your achievements over time, with the help of AI to make sense of it all.

Step 1: Filter Your Git Logs

The first step is to look back at your commits over the past year (or a specific timeframe). Git makes it easy to search through your history by date and author. This command will give you a list of commits from the past year, authored by you:

git log --since="1 year ago" --author="your_email@example.com" --pretty=format:"%h - %an, %ar : %s"

Here’s a breakdown of this command:

--since="1 year ago" limits results to commits from the past year.
--author="your_email@example.com" filters commits to only those you made.
--pretty=format:"%h - %an, %ar : %s" displays commit hash, author, relative date, and message.

Here's an example of what my logs looked like:

8abe2e3 - Antony Nyagah, 5 days ago : feat(dashboard/*): add link for downloading packaging list
0409dac - Antony Nyagah, 5 days ago : feat(dashboard/*): add a link for downloading delivery notes from the frontend
0c199f9 - Antony Nyagah, 5 days ago : test(api/*): update tests for document_exporter app

Step 2: Export Your Commit Log

For easier analysis, you can export your commit history to a text file:

git log --since="1 year ago" --author="your_email@example.com" --pretty=format:"%h - %an, %ar : %s" > commit_log.txt

Step 3: Leverage AI for a Detailed Analysis

After exporting the commit log, I pasted the commits into ChatGPT(you can use any number of AI tool that are out there) to get a detailed summary and insights into the types of tasks I’ve accomplished.

The prompt I used was "Can you look at these commits and help me come up with a list of my achievements throughout this year?". The prompt itself can be improved 😅

This helped me uncover trends and group my work into categories like:

Infrastructure & CI/CD Enhancements
Frontend Development & Refactoring
Backend API Development & Maintenance
Database and Data Handling Improvements
Testing and Code Quality
Documentation & Project Management

ChatGPT’s analysis made it easy to see where I added value and what skills I honed over the past year, even bringing out details I might have overlooked. By using AI this way, you can turn your commit log into a meaningful narrative about your contributions.

Step 4: Keeping Commit Messages Meaningful

To make sure my commit messages are succinct and relevant, I personally use Commitizen. This tool helps maintain consistency by following a commit message convention that describes the "why" and "what" of each change.

Meaningful commit messages make logs easier to review, and there are other specifications and tools like Conventional Commits and Husky that can also keep your commit history clean and purposeful.

Step 5: Summarize and Reflect

Reviewing your contributions with AI insights can be both motivating and insightful. Write a brief summary for each category. This can serve as a reference for future performance reviews, portfolio updates, or simply as a reminder of your growth as a developer.

Final Thoughts

Using Git as a reflection tool is quick but effective. And with the added support of AI, you can see the big picture more clearly, connecting technical achievements with your overall development. Keep this record for yourself or share it in your performance reviews or job applications.