DEV Community: Okeke Chukwudubem

My AI agent was misreading bank balances 1 in 5 times. No crashes. No exceptions. Just quiet, dangerous wrong numbers. I built a three-layer verification system that caught the bug hiding in plain sight. Format validation. Double-read confirmation.

Okeke Chukwudubem — Sat, 18 Jul 2026 18:56:15 +0000

Summer Bug Smash: Smash Stories 🐛🛹

Okeke Chukwudubem

Jul 18

Smash Stories: The Bug That Whispered for Two Weeks Before I Heard It

#devchallenge #bugsmash #ai #gemma

4 min read

Smash Stories: The Bug That Whispered for Two Weeks Before I Heard It

Okeke Chukwudubem — Sat, 18 Jul 2026 18:51:38 +0000

This is a submission for DEV's Summer Bug Smash: Smash Stories powered by Sentry.

My AI agent was misreading financial data 1 in 5 times. No crashes. No exceptions. Just quiet, dangerous wrong numbers.

Some bugs scream. Null pointer exceptions. Segmentation faults. Crash logs that light up your screen like a Christmas tree.

This one whispered.

The Setup

I'm building an AI agent that controls Android phones using Gemma 4. Natural language commands get parsed into structured JSON, translated into ADB instructions, and executed on-device. No cloud. No API keys. Just a phone that does what you tell it.

By Day 16, I'd hit a milestone: the agent completed its first multi-app workflow. "Copy my bank balance and send it to Mom on WhatsApp."

Three apps. Banking → WhatsApp. Seven steps. Success.

I celebrated. Shipped the code. Published the build log. Went to sleep feeling like an engineer.

Then I tested it again the next morning. And again. And again.

The Whisper

Out of 50 test runs, the balance was wrong 10 times. Twenty percent.

The agent wasn't crashing. No exceptions were thrown. The logs showed success. But "₦15,000" was quietly becoming "₦15.000" or "₦15,00" or "₦15000." A currency symbol mistaken for a digit. A comma parsed as a decimal. A zero dropped from the end.

The wrong number was being sent to Mom. And I had no idea for two weeks because nothing was failing loudly enough to make me look.

The Root Cause

I traced the problem to a single bad assumption: I was treating OCR output as ground truth.

Banking apps scored F on my accessibility audit. Zero UI labels. The agent had to rely entirely on optical character recognition to read anything on screen. And banking apps are a hostile environment for OCR:

Condensed fonts designed for dense data, not readability
Grey text on slightly-darker-grey backgrounds
Currency symbols (₦) that OCR models frequently confuse with digits
Commas in large numbers that OCR sometimes parses as decimal points

One screenshot. One OCR pass. One number stored in memory. No verification. No double-check. No sanity check. I had built a pipeline that trusted a single, unreliable data source with financial information.

The Fix: Defense in Depth

I didn't try to make the OCR better. Tesseract and ML Kit are what they are. Instead, I built a three-layer verification system that treats every financial number as unverified until proven otherwise.

Layer 1: Format Validation

Before a number is accepted, it must match a valid currency pattern. The regex checks for currency symbols (₦, $, £, €) followed by properly formatted digits. Anything that doesn't match gets rejected instantly.


python
pattern = r'[₦$£€]?\s?\d{1,3}(?:,\d{3})*(?:\.\d{2})?'
matches = re.findall(pattern, extracted_text)
if not matches:
    return False, "", "No currency values found"

This alone catches about half the errors garbled text, partial reads, and non-numeric data that OCR sometimes returns.

Layer 2: Double-Read Confirmation

The agent captures two separate screenshots and runs OCR on both independently. If the numbers match, the value is accepted with high confidence. If they differ, a third screenshot is captured and the best-of-three wins.

# Read twice
text1 = extract_text(screenshot1)
text2 = extract_text(screenshot2)

num1 = extract_number(text1)
num2 = extract_number(text2)

if num1 == num2:
    return True, num1, "High confidence"

# Tiebreaker
text3 = extract_text(screenshot3)
num3 = extract_number(text3)

if num1 == num3:
    return True, num1, "Medium confidence (2/3)"
elif num2 == num3:
    return True, num2, "Medium confidence (2/3)"

This catches transient OCR errors—pixel-level variations between screenshots that produce different readings. The cost is 4-6 extra seconds per financial read. The benefit is catching errors that would otherwise silently corrupt data.

Layer 3: Range Validation

Even a correctly formatted, double-confirmed number can be nonsensical. A balance of "₦0" might mean an empty account—or it might mean the OCR failed completely and returned nothing. A balance of "₦999,999,999,999" is almost certainly an error.

if 1 <= value <= 1_000_000_000:
    return True
else:
    return False  # Suspicious. Re-read.

This catches the edge cases—numbers that pass format and double-read checks but are clearly wrong in context.

The Celebration

After implementing all three layers, I re-ran the 50-test battery. The error rate dropped from 20% to 6%. For the specific "₦15,000" test case, 20 out of 20 reads were correct with double-read confirmation.

The agent now succeeds 19 times out of 20 on multi-app financial tasks. The one failure is a timeout (banking app loads slowly on an old device), not a misread. And critically, when verification fails, the agent now knows it failed. It reports low confidence and aborts the task instead of silently sending wrong data.

What This Taught Me

The bug wasn't in the OCR engine. It was in my architecture. I had built a system that trusted a single, unreliable data source with financial information. That wasn't a Tesseract problem. That was a design problem.

The fix wasn't a better model. It was a better process. Three cheap, fast checks that together create confidence where no single check could. Defense in depth applied to data integrity.

This changed how I think about every feature I build. I now ask: "If this component silently returns wrong data, will I know? If not, where's the verification?"

The Code

The full verification system is open source on GitHub:

👉 github.com/Dexter2344/phone-agent

vision.py contains the verify_financial_data() function. agent.py calls it before storing any financial value in task memory.

The Build Log

I've been documenting this project publicly for 17 days on Dev.to. Every breakthrough, every dead end, every "why is this broken" debugging session at 1 AM.

This bug was Day 17. The one that almost shipped. The one that whispered.

It's not the loudest bug I've ever caught. But it's the one that taught me the most.

This is my entry for the Dev.to Smash Stories contest. The Gemma 4 model powers the agent's command parsing and reasoning. The verification layer is my own work.

Project Log #17: My Agent Misreads Bank Balances. Here's How I'm Fixing It.

Okeke Chukwudubem — Sat, 18 Jul 2026 18:21:54 +0000

Day 17. OCR on banking apps is unreliable. I built a verification layer that double-checks every number.

Day 16 was a milestone: multi-app workflows. The agent copied my bank balance and sent it to Mom on WhatsApp. Three apps. One task.

But behind that success was an uncomfortable truth: the agent misreads numbers about 20% of the time.

For a message to Mom, that's a typo. For a financial transaction, that's a disaster.

Today, I built the fix.

The Problem

Banking apps scored F on my accessibility audit. No UI labels. No content descriptions. The agent has to rely entirely on OCR to read anything on screen.

And banking apps have terrible OCR conditions:

Small, condensed fonts for account numbers and balances
Low contrast (grey text on slightly darker grey backgrounds)
Currency symbols (₦, $, £) that OCR often confuses with numbers
Commas in large numbers that OCR sometimes reads as decimals

The result? A balance of "₦15,000" sometimes gets read as "₦15.000" or "₦15,00" or "₦15000." One missing digit. One wrong decimal. And the entire task is compromised.

The Fix: Numeric Verification Layer

I built a verification step specifically for financial data. Before any number gets stored in task memory, it goes through three checks.

Check 1: Format Validation
The extracted text must match a valid currency format. It must contain a currency symbol (₦, $, £, €) followed by digits, optionally with commas and a decimal point. Anything that doesn't match this pattern is rejected immediately.

Check 2: Double-Read Confirmation
The agent reads the same number twice—two separate screenshots, two separate OCR passes. If both readings match exactly, the number is accepted. If they differ, the agent reads a third time. If two out of three match, that value wins. If all three differ, the task is aborted with an error message.

Check 3: Range Validation
The extracted number must fall within a reasonable range. A bank balance of "₦0" or "₦999,999,999,999" is probably an OCR error. The agent checks that the number is between ₦1 and ₦1,000,000,000. Anything outside that range triggers a re-read.

Today's Progress

Task	Status
Built numeric verification layer	✅ Done
Tested on banking app balance (50 attempts)	✅ 94% accuracy (up from ~80%)
Tested on "₦15,000" specifically (20 attempts)	✅ 100% accuracy with double-read
Added verification to `vision.py`	✅ Done
Updated `agent.py` to use verification for financial data	✅ Done

The Results

Before the verification layer: 20% error rate on financial data.
After the verification layer: 6% error rate. Still not perfect, but dramatically better.

For the specific task of reading a bank balance and sending it to Mom, the agent now succeeds 19 times out of 20. The one failure is usually a timeout (banking app loads slowly) rather than a misread.

What Still Breaks

The double-read confirmation adds about 4-6 seconds to any task involving financial data. For a 3-app workflow, that's an extra 10-15 seconds. Not a dealbreaker, but noticeable.

The range validation is crude. A legitimate bank balance of "₦0" (empty account) would be rejected as an error. I need a way to distinguish between "zero" and "OCR failed completely."

And the verification only works for numbers. Text extraction—names, addresses, transaction descriptions—still has no verification. The agent could send "Mom" the wrong transaction description and have no way of knowing.

What's Next (Day 18)

Add text verification for transaction descriptions
Test with real financial data (multiple transactions, not just balance)
Start building the demo video

The Repo

👉 github.com/Dexter2344/phone-agent

vision.py now includes verify_financial_data() with format validation, double-read confirmation, and range checking. agent.py calls it before storing any financial data in task memory.

This is Day 17. The agent is learning to double-check its own work.

Day 16: My AI agent now resets to the home screen between app switches. The first 3-app workflow just worked—banking app summarise WhatsApp. Reliability went from "sometimes" to "almost always." github.com/Dexter2344/phone-agent

Okeke Chukwudubem — Tue, 14 Jul 2026 15:08:45 +0000

Okeke Chukwudubem

Jul 14

Project Log #16: I Fixed the App-Switching Problem That Kept Breaking My Agent

#ai #webdev #programming #productivity

3 min read

Project Log #16: I Fixed the App-Switching Problem That Kept Breaking My Agent

Okeke Chukwudubem — Tue, 14 Jul 2026 15:08:12 +0000

Day 16. The agent now resets to home between apps. Three-app workflows are working.

Day 15 was a milestone: my agent completed its first multi-app task. It copied my bank balance and sent it to Mom on WhatsApp. Two apps. One task.

But behind the scenes, it was fragile. The agent sometimes opened apps to the wrong screen. The banking app would resume from where it left off instead of starting fresh. A notification from the previous app would cover the new one. The task worked—but only sometimes.

Today, I fixed the foundation.

The Home Screen Reset

Between every app switch, the agent now returns to the home screen first. It's a simple adb shell input keyevent KEYCODE_HOME command. But it solves a cascade of problems.

Before the reset, here's what would happen:

Agent opens Banking App. Reads balance. ✅
Agent opens WhatsApp. But WhatsApp was already open in the background on a different chat. ❌
Agent tries to find "Mom" but it's looking at the wrong screen. ❌
Task fails. ❌

After the reset:

Agent opens Banking App. Reads balance. ✅
Agent returns to home screen. ✅
Agent opens WhatsApp from scratch. It loads the main chat list. ✅
Agent finds "Mom" and sends the message. ✅

The fix is two lines of code. The reliability improvement is massive.

Today's Progress

Task	Status
Added `go_home()` between all app switches	✅ Done
Tested "Copy balance → send to Mom" 10 times	✅ 9/10 success
Tested "Copy address → paste into Maps" 10 times	✅ 10/10 success
Tested 3-app workflow: "Get last transaction from banking app → summarise → email to accountant"	✅ First success
Updated `agent.py` with home reset logic	✅ Done

The 3-App Workflow

The 3-app test was the real challenge. The command: "Get my last transaction from the banking app, summarise it, and send it to my accountant on WhatsApp."

The agent:

Opened the banking app. Read the last transaction: "₦15,000 debit — Electricity Bill — June 19." ✅
Stored it in task memory as last_transaction. ✅
Returned to home. Opened WhatsApp. ✅
Found "Accountant" in contacts. ✅
Retrieved last_transaction from memory. ✅
Typed: "Last transaction: ₦15,000 debit for Electricity Bill on June 19." ✅
Pressed send. ✅

Three apps. One task. Seven successful steps.

What Still Breaks

The banking app OCR is still unreliable. On 2 out of 10 tests, the agent misread a digit in the balance. A "₦15,000" became "₦15,000" in one test and "₦15,000" in another—but on the failures, it read "₦15,000" as "₦15,000" or missed a digit entirely. For sending a balance to Mom, that's a typo. For transferring money, that's a disaster.

I need a verification step specifically for financial data. Something that double-checks the OCR output before storing it in memory. That's the next wall to climb.

The home screen reset adds about 1.5 seconds between app switches. For a 3-app task, that's 4.5 extra seconds. Not a dealbreaker, but noticeable. I'm experimenting with using adb shell am force-stop instead of home reset—it's cleaner but takes longer.

What's Next (Day 17)

Add financial data verification step
Test 5-app workflows
Start building a demo video of the full pipeline

The Repo

👉 github.com/Dexter2344/phone-agent

agent.py now includes go_home() between app switches. Task memory handles multi-app data transfer. README updated.

This is Day 16. The foundation is solid. The walls are going up.

Day 15: My AI agent just completed its first multi-app task. It copied my bank balance from my banking app and sent it to Mom on WhatsApp. Two apps. One task. Fully autonomous. Task memory is the key. 👉 github.com/Dexter2344/phone-agent

Okeke Chukwudubem — Sat, 11 Jul 2026 22:21:45 +0000

Okeke Chukwudubem

Jul 11

Project Log #15: My AI Agent Just Completed Its First Multi-App Task

#ai #webdev #programming #productivity

3 min read

Project Log #15: My AI Agent Just Completed Its First Multi-App Task

Okeke Chukwudubem — Sat, 11 Jul 2026 22:21:17 +0000

Day 15. The agent switched between two apps, copied data from one, and pasted it into another.

For 14 days, my AI agent could do one thing well: operate within a single app. Open WhatsApp. Find a contact. Send a message. All within one ecosystem.

Today, that changed.

The Milestone

I gave the agent a command: "Copy my account balance from my banking app and send it to Mom on WhatsApp."

It opened the banking app. It read the balance from the screen using OCR (since banking apps have zero accessibility labels—F score, remember?). It copied the number. It switched to WhatsApp. It found Mom. It typed the balance. It hit send.

Two apps. One task. Completed autonomously.

What Made This Possible

The key was building a task memory system. The agent now stores values extracted from one app in a simple key-value store (a Python dictionary that persists for the duration of the task). When it switches to the next app, it retrieves that value and uses it.

The flow works like this:

Parse the command into a plan with Gemma 4.
Execute Step 1: Open banking app. Read the balance. Store it in memory as account_balance.
Execute Step 2: Open WhatsApp. Find the contact. Retrieve account_balance from memory. Type it. Send.

The memory system is simple—just a dictionary that gets passed between steps. But it's the bridge between single-app automation and true multi-app workflows.

Today's Progress

Task	Status
Built task memory system (key-value store)	✅ Done
Tested "Copy balance → send to Mom"	✅ Success
Tested "Copy address → paste into Maps"	✅ Success
Updated `agent.py` with multi-app support	✅ Done
Added task memory to README	✅ Done

What Still Breaks

The banking app is slow to load on my phone. Sometimes the agent times out waiting for the balance screen to appear. I added a retry loop, but it's not perfect.

App switching is clunky. The agent uses adb shell am start to launch apps, but if the app was already open in the background, it sometimes opens to the wrong screen. I'm working on a "reset to home screen" step between app switches.

And the OCR on banking app text is unreliable. The font is small. The contrast is low. The agent sometimes misreads the balance by a digit or two. For a task like "send my balance to Mom," that's embarrassing but not catastrophic. For a task like "transfer money," that's dangerous. I need higher accuracy before I can trust this with anything financial.

The Bigger Picture

Multi-app workflows are the difference between a cool prototype and a useful tool. An agent that can only operate in one app is a novelty. An agent that can move data between apps is an assistant.

The next step is chaining more complex workflows: "Find my last three transactions, summarise them, and email them to my accountant." That's three apps. That's the goal.

What's Next (Day 16)

Add a "reset to home screen" step between app switches
Improve OCR accuracy on banking apps
Test a 3-app workflow

The Repo

👉 github.com/Dexter2344/phone-agent

agent.py now includes a task memory system. Multi-app workflows are supported. README updated with Day 15 status.

This is Day 15. The agent is no longer trapped in one app.

Day 14: I doubled the accessibility audit to 30 apps. The public score list is now on GitHub. Every banking app scored F. WhatsApp, Google, and global apps score A. The pattern is undeniable.

Okeke Chukwudubem — Mon, 06 Jul 2026 19:38:32 +0000

Okeke Chukwudubem

Jul 6

Project Log #14: I Scored 30 Apps on Accessibility. Here's the Public List.

#ai #webdev #automation #agents

2 min read

Project Log #14: I Scored 30 Apps on Accessibility. Here's the Public List.

Okeke Chukwudubem — Mon, 06 Jul 2026 19:37:12 +0000

Day 14. The audit doubled in size. The accessibility score list is now public on GitHub.

Last week, I published my first accessibility audit. 15 apps. A simple A-to-F score. The response surprised me.

Developers reached out asking how their apps scored. A few wanted to know how to improve. One person told me they'd never thought about accessibility labels before reading my build log.

So I spent the next few days doubling the audit. 30 apps. Same methodology. Same scoring system. The results are now public on GitHub.

The Expanded Audit

I added 15 more apps across new categories. Health apps. Education apps. Government services. More local apps.

Category	Best Apps (A/B)	Worst Apps (D/F)
Messaging	WhatsApp, Telegram, Google Messages	—
Email	Gmail, Outlook	—
Productivity	Slack, Notion, Google Calendar	—
Navigation	Google Maps, Waze	—
Banking	—	5 out of 5 tested scored F
Health	—	3 out of 4 scored D or F
Government	—	4 out of 4 scored F
Food Delivery	Uber Eats, Deliveroo	Local apps scored D
Education	Duolingo, Khan Academy	Local education apps scored D

The pattern held. Global apps score high. Local apps score low. Every single banking app tested scored an F.

The Public List

I've published the full accessibility score list on GitHub. It's in the README under a new section called "Accessibility Audit."

Each app gets:

A letter grade (A through F)
A short explanation of why it scored that way
Whether the UI tree is navigable by screen readers
Whether my agent can automate it

This list is now the most detailed public resource I know of that connects AI agent capability to accessibility compliance.

What This Means for Developers

If you're building a mobile app, here's what you should know:

Accessibility labels are not optional. They're not a "nice to have." They determine whether millions of visually impaired users can use your app—and increasingly, whether AI agents can interact with it.
Adding labels is not hard. A single content-desc attribute on a button takes seconds to add. The reason most apps score F is not technical difficulty. It's that nobody prioritised it.
The gap is a business opportunity. An app that scores A is usable by more humans AND more machines. As AI agents become more common, accessible apps will have an automation advantage.

What's Next (Day 15)

Start reaching out to developers of F-scoring apps with specific, actionable feedback
Add a "How to Improve Your Score" guide to the repo
Begin testing whether my agent can now handle multi-app workflows (e.g., "Copy my bank balance from the banking app and send it to Mom on WhatsApp")

The Repo

👉 github.com/Dexter2344/phone-agent

The accessibility audit is now a permanent section of the README. 30 apps scored. Public. Free. For anyone to use.

This is Day 14. The list is out. The conversation is starting.

Day 12: I took a week off from coding to audit 15 Android apps for accessibility. WhatsApp and Google apps are fully automatable. Local banking and government apps? Completely invisible. My AI agent is becoming an accessibility audit tool.

Okeke Chukwudubem — Sat, 04 Jul 2026 23:14:38 +0000

Okeke Chukwudubem

Jul 4

Project Log #12: I Spent a Week Auditing Apps. The Results Changed This Project.

#ai #webdev #programming #productivity

3 min read

Project Log #12: I Spent a Week Auditing Apps. The Results Changed This Project.

Okeke Chukwudubem — Sat, 04 Jul 2026 23:14:06 +0000

Day 12. I tested 15 popular Android apps for accessibility. What I found revealed who gets left behind.

I took a week off from coding.

Not because I was stuck. Not because I was tired. But because I needed to understand something that had been nagging at me since Day 10.

Why does my AI agent work flawlessly on WhatsApp but completely fail on my banking app?

The answer took a week of testing, documenting, and thinking. And what I found changed the direction of this entire project.

The Week-Long Audit

I tested 15 popular Android apps. For each one, I asked the same question: "Can my agent find and tap a core button using the UI tree?"

I spent hours opening apps, dumping UI hierarchies, parsing XML, and documenting every content description I could find. I wasn't coding. I was investigating.

App	UI Tree Labels	Agent Can Automate?
WhatsApp	Excellent	✅ Fully
Google Messages	Excellent	✅ Fully
Gmail	Excellent	✅ Fully
Google Maps	Excellent	✅ Fully
Spotify	Good	✅ Mostly
Slack	Good	✅ Mostly
Banking App A	None	❌ Completely blind
Banking App B	None	❌ Completely blind
Banking App C	None	❌ Completely blind
Local Food Delivery App	Minimal	❌ Mostly blind
Government Service App	None	❌ Completely blind
Telecom App	Minimal	❌ Mostly blind

The pattern was impossible to ignore. Global apps had rich accessibility labels. Local apps had nothing.

What This Actually Means

Here's the truth I wasn't ready for.

Accessibility labels exist so that screen readers can tell blind and visually impaired users what's on their screen. When a developer adds content-desc="Send message" to a button, they're helping a blind person use WhatsApp.

WhatsApp invested in accessibility. Google invested in accessibility. Their apps are usable by millions of people with disabilities—and, accidentally, by my AI agent.

The local apps that skipped accessibility didn't just lock out my agent. They locked out every blind and visually impaired person who needs to use that banking app, that government service, that telecom platform.

My AI agent is not the victim here. It's a canary in a coal mine.

The Accessibility Divide

This divide follows a clear pattern. Global tech companies with large engineering teams and legal departments prioritise accessibility. They have dedicated accessibility engineers. They test with screen readers. They follow WCAG guidelines.

Local apps, often built by smaller teams with tighter budgets, deprioritise accessibility. It's seen as a "nice to have" rather than a core requirement. The result is an entire class of applications that are unusable by people with disabilities—and by AI agents.

What I'm Doing About It

I'm adding an accessibility score to every app my agent supports. A simple A-F rating.

A — Fully automatable. Rich labels everywhere. (WhatsApp, Google apps)
B — Mostly automatable. Good labels with some gaps. (Spotify, Slack)
C — Partially automatable. Some labels, mostly generic.
D — Mostly blind. Few labels. Heavy reliance on OCR fallback.
F — Completely inaccessible. No labels. Unusable by both AI and screen readers.

This score isn't just for my agent. It's a public signal. If your app scores an F, you're not just failing my project. You're failing every visually impaired person who tries to use your service.

What's Next (Day 13)

Expand the audit to 30+ apps
Publish the first version of the accessibility score list
Start reaching out to local app developers about their accessibility gaps

The Repo

👉 github.com/Dexter2344/phone-agent

The audit results are being added to the README. The scoring system is documented.

This is Day 12. I spent a week not writing code. And it was the most productive week of this entire project.

Day 12: My AI agent can control WhatsApp perfectly. But banking apps? Invisible. The difference is accessibility labels. Apps built for blind users are automatable. Apps that skip accessibility lock out both humans and ai.

Okeke Chukwudubem — Tue, 30 Jun 2026 13:49:34 +0000

Okeke Chukwudubem

Jun 30

Project Log #12: The Apps My AI Agent Can't See

#ai #webdev #programming #productivity

2 min read