DEV Community: Solomon Aboyeji

If LLMs Were ATMs, Would You Still Count Your Money?

Solomon Aboyeji — Mon, 06 Apr 2026 08:09:43 +0000

There is this joke about some Nigerians still counting their money at the ATM. Even with an automated system, people in my home country, Nigeria in West Africa, perform this common practice of counting their cash the moment it comes out of the machine. I have never personally known anyone who got a different amount than what they asked for, but the habit carries on. And I think that habit has something to teach us about how we should be treating LLMs right now.

The buzz word everywhere you turn is AI, and many companies you know of want to quickly get on this new trend. As we rush to embrace this tool to become more productive and make our businesses more efficient, it falls on us as engineers to check what the output of this technology actually gives us. Do not vibe code to the extent that you no longer have control over what is being written. We should learn to do some extra checks, the same way people still count their notes at the ATM even when the machine has never shortchanged them. Trust is earned, and verification is cheap insurance.

So what does "counting your money" look like when the machine is an LLM?
First, what if I told you that agents are simply a set of continually prompting an LLM to do something, sending the output back to itself or to a different, more capable model as a prompt, and performing other actions such as criticism and review? Once you see it that way, it becomes obvious that every step in that chain is a place where things can quietly go wrong. You need checks at each step, not just at the end.

Second, LLMs are still unreliable in some vital areas. We need to put a system and structure in place, with a set of hard rules that the LLM's output is run against. These are not prompt rules, I mean a programmatic set of rules enforced in code. Take access checks as an example. Do not just prompt, "answer this question if the user is an admin". The LLM can fall into prompt injection attacks, where a user slips instructions into their input telling the model to ignore the admin check entirely. Access control belongs in your code, not in your prompt.

Third, evaluate the model's response regularly. When you change models or change any parameters in your pipeline, run the same checks each time to ensure the model has not degraded or drifted from expected or acceptable answers. This can be done with different evals.

And finally, if you vibe code a product, please get a Software Engineer to review it before you hand it over to customers, whether paying or non-paying. Many of the topics I mentioned above can be very technical, and you need someone who knows how code should behave to reliably implement them. Businesses should not be too eager to replace engineers.

The ATM in Nigeria has been around for years, and it still gives people the right amount almost every single time. And still, we count. That instinct is not paranoia, it is discipline. LLMs are newer, less predictable, and far more capable of being wrong in ways you will not notice at a glance. So, count the money, review the code!

The Developer Identity Crisis No One Is Talking About

Solomon Aboyeji — Sat, 28 Feb 2026 15:09:54 +0000

We have moved from actively writing code by hand, as this was arguably the fun thing to do in programming: the feeling that we were crafting things from scratch and watching it power engines and support communities around the globe.

However, in this transition where code is cheap, the goalpost has shifted to actively testing what was built by your AI agents: quality assurance, translating users' problems into well-defined tickets, more time spent on reviewing code. Most of these are things the average engineer dislikes.

Writing code by hand isn't going away entirely, as we still need engineers who understand code to debug and, importantly, architect systems, and you can only architect what you know how to build. This craft isn't dead, but evolving.

We now need to ensure that what we are building with our agents is reliable, maintainable even for human readers, and that performance is not degraded. I must say, with AI we should be able to do many of the checks and ensure things are in the right places.

The developers who only identify with code will struggle to adapt. Skills that matter are shifting, figure them out and learn them.

Going Local? 2,000+ Attackers Are Already Waiting

Solomon Aboyeji — Tue, 20 Jan 2026 04:55:34 +0000

Going local means hosting your own infrastructure instead of relying on managed cloud services. In this post, I talk about 2,331 login attempts in less than 30 Days, the reality of running your own VPS.

You resonate with the idea of having your own setup and deployment. It gives you control and, let's face it, it's cheap. You install Docker, set up UFW, open some ports to the web, block others. You think to yourself: now we're good.

Well, it takes one curious attacker to find the gaps in your checklist.

In less than a month, there have been 2,331 failed SSH login attempts and 235 banned IPs, all trying to gain access to the VPS server with bans occurring every 30-60 minutes, around the clock. These are automated, coordinated attempts.

I recently spun up a VPS and discovered that some ports were exposed to the public even with UFW configured. A MongoDB that was exposed on port 27017 for "internal use only"? It's been public this whole time. Turns out Docker bypasses UFW entirely. This isn't a bug. It's how Docker works and it has been known for years, however many developers either don't know or choose to ignore it due to the convenience of abstraction Docker provides.

What are they looking for?

The failed logins reveal what attackers expect to find on cheap VPS servers:

It seems they're not guessing randomly. They know what people run on $5 servers, and here it seems crypto infrastructure with hot wallets is a prime target.

Now that you are local, are you secured? In 2026, this is the question we all need to be asking. It's more than just having a presence online, it's about how secure your setup actually is. Whether that means a $3-5 VPS on Hetzner or an old laptop in your garage running your side projects.

If you're running Docker on an unmanaged VPS, you probably need to fix this. I put together a script that:

Prevents Docker from bypassing your firewall
Blocks all ports by default, exposing only what you explicitly allow
Hardens SSH to key-based authentication only
Installs fail2ban to stop brute-force attempts

How it works

The script does four things:

1. Fixes the Docker/UFW bypass
It installs ufw-docker, which modifies the iptables chain order so UFW rules are checked before Docker's. Without this, Docker punches holes through your firewall whenever you expose a port.

2. Blocks everything by default
UFW is configured to deny all incoming traffic. Only ports you explicitly specify (default: 22, 80, 443, 3000) are accessible from the internet. Your containers can still talk to each other internally.

3. Hardens SSH
Disables password authentication (key-only), disables root login, and removes cloud-init overrides that re-enable password auth (be careful with this as you might lock yourself out entirely if your ssh key is missing)

4. Installs fail2ban
Three failed SSH attempts results in a 24-hour ban. This stops brute-force attacks from hammering your server indefinitely. (You can adjust the ban duration based on your needs)

Here is the link to the Github repo, I would advise you run it on a throw-away server before making use of this in a more serious project: https://github.com/solomonaboyeji/secure-vps

Engineers who explore build better AI products

Solomon Aboyeji — Sat, 20 Dec 2025 17:11:10 +0000

Data is king, but not knowing what data you have, its nature and characteristics is what keeps businesses at a loss. I implemented a RAG system to answer questions over a lengthy PDF document, but before feeding it to the model, I spent most of my time doing Exploratory Data Analysis (EDA).

This helped me go back into my data and clean it better because I now know where most of the confidence scores lie, the areas I need to work on, and most importantly, where humans need to spend my time.

I believe in saving cost and time by ensuring we focus on the most important tasks and let computers handle the ones without nuances. How would you know what those nuances are if you don't dig into your data and uncover its relationships, attributes, strengths, and anomalies? In a world where everyone is rushing to "AI," do they really have an in-depth understanding of their data and how to leverage it to thrive?

Your data is trying to tell you something. Go in and listen to it, ask it questions, and you'll gain insights.

Here is the statistics of the chunking process (I explain it in the next paragraph)

I enriched the original chunks with more data:

Chunk Classification Report

Looking at the enrichment report:

I broke the document into 434 pieces. First, I ran rule-based classification. For chunks where the rules weren't confident, they were flagged for LLM processing. More than half (58.8%) fell into this category, requiring AI calls to provide their own confidence scores. Now imagine a business or individual with needs at 50x this scale. That's over 12,000 AI calls flagged before even running them. Even with batching, you're processing significantly more tokens through the LLM, and costs add up fast.

Without EDA, I might have fed all 434 chunks through the LLM, paying for processing that my rules already handled confidently. The 41.2% success rate from simple rules showed me what was already working, that's nearly half the workload I could have unnecessarily sent to AI processing. At 50x scale, that's avoiding 9,000+ wasteful LLM calls.

Here is another insight, two categories (system_section and spec_table_row) made up 51% of everything. With 41.2% already automated through rules, analyse why these types trigger LLM fallback and build better rules to handle them confidently. However, if that's difficult or genuinely going to be wasteful effort, let the LLM classify them and focus human review on the low confidence results where oversight catches nuances better.

Not all problems require LLM calls. Some genuinely just need better engineering. With EDA, we understand what data we have, what it can do, and deliver more value to users and stakeholders.

The Hidden Cost of Abstraction - Making an Informed Decision

Solomon Aboyeji — Sun, 23 Nov 2025 22:31:19 +0000

Abstraction promises convenience, but it often comes with hidden costs that only reveal themselves as time goes on.

Remember when you could download music to your phone and truly own it? You could buy CDs or DVDs, and they were yours, that means no subscriptions, no monthly fees. You had to deal with some issues though. You had to manage storage, worry about physical media (disc) breaking, and manually organise everything into folders jazz, afro etc. Today, the likes of Spotify, Youtube, Apple Music, and Netflix abstract all those concerns away. In exchange, you pay a subscription fee or pay with your attention by watching ads.

This same pattern plays out in software development. LangChain helps developers build LLM-based applications by abstracting away the manual orchestration of pipelines and replacing them with simple calls called chains. They released LangSmith, which helps you trace what each chain is doing and allows you to debug quickly. The first tier is free, but you can easily go over the limit set and will then need to jump on the paid tiers. While truthfully, you can write your own handler to help you trace, it comes with a cost of sitting down and building that logic, time you could spend on more productive work. Thankfully there is Langfuse, which is open source and has a more generous free tier when you use the cloud offering (or you can self-host it entirely).

The thing is, once you are onboard using an abstraction, you are now tied to whatever terms the provider gives in the future. This is what we call vendor lock-in in technical terms, where you cannot go elsewhere because you have been tied to the provider either through better service, through much more convenience, incompatibility with other providers in the same space or maybe through legal terms as with the Adobe subscription.

I have enjoyed many abstractions and not all abstractions are exploitative. Sometimes the convenience you or your team gets genuinely justifies the cost, and providers deliver real value. The key is being aware of the trade-off you're making.

So yes, abstraction brings convenience, but be ready to pay for it immediately or when the time comes, someone bears the costs of making it convenient for you. With that in mind, what abstractions do you enjoy using?

Call Them Customers: A Mindset Shift for Engineers in the AI Coding Era

Solomon Aboyeji — Sun, 05 Oct 2025 12:57:26 +0000

In 2025, when building products for paying customers, embracing AI-assisted coding can be a no-brainer because the productivity gains are undeniable.

With this power comes great responsibility. Engineers are now able to do so much in little time. Automatic documentation by just looking at the existing code, writing of unit test cases has improved if done, churning out code to build features or fix bugs in a matter of minutes, if not seconds.

While all of these are helpful, it is very easy for engineers to lose quality focus. You tend to ship bugs faster while also shipping products faster. We become lazier (not in the Bill Gates type of laziness).

A subtle yet powerful mindset shift can resolve this issue to a much greater extent. Start calling your users "customers"; the effect will be psychological. This will help you ensure you always provide quality work for them. Because every line of code, whether written by you or AI, can frustrate your customers. Take more time to review, document, and write unit test cases.

When you start adopting this, you should start seeing more quality work, as you wouldn't want customers to be frustrated. You also tend to communicate expectations more effectively, keeping your tech lead or product managers informed, who are then responsible for letting customers know what to expect.

Action Items

Review all code, whether written by AI or by you.
Write documentation on how things are built and some architecture decisions.
Communication is needed in this AI era; let other team members know what you are doing or what to expect from you.
Use AI to enhance quality, not just speed.

Remember, don't sacrifice speed for quality; your customers will thank you.