DEV Community: bilal meziani

Why 90% of AI Automations Break in Production (Lessons From Building a Real AI Workflow)

bilal meziani — Mon, 20 Jul 2026 13:45:22 +0000

Building an AI demo today is easy.

Connect OpenAI to n8n or Make, write a prompt, click Run, and everything looks impressive.

That’s exactly how many of my workflows started while building AgenticScales.

The hard part wasn’t getting them to work.

The hard part was asking uncomfortable questions:

What happens when the API times out?
What happens when the AI doesn’t know the answer?
What happens when a customer asks for a refund?
What happens when someone spams the system with meaningless messages?

The more I worked on these workflows, the more I realized that production AI has very little to do with prompts and everything to do with handling failure.

Every lesson below came from finding a weakness in a workflow and redesigning it before it became a real problem.

1. Never Assume APIs Will Always Respond

One of my first mistakes was focusing on the happy path.

The workflow worked perfectly during testing, so I assumed it was ready.

Then I started asking: what happens if the LLM API times out, hits a rate limit, or returns an error mid-process?

Without retries and fallback logic, a single failure could break the entire workflow.

So I started treating API failures as something expected, not exceptional.

What I do now

Retry failed requests automatically (with a timeout so they never hang).
Log failures clearly.
When a call still fails after retries, degrade gracefully: route the ticket to a human queue instead of dropping it.

A workflow should never silently lose a task.

In my support flow, this is a built-in fail-safe: if classification fails, the ticket is marked needs_human and handed to a person instead of disappearing into the system.

2. AI Shouldn’t Handle Sensitive Requests Alone

While designing customer-support workflows, I quickly realized that not every request should get an AI-generated response.

A refund request, a security issue, or a legal complaint carries far more risk than a simple “How do I reset my password?”

Even if the model performs well most of the time, one bad response on a sensitive topic can create a serious problem.

I eventually realized that not every ticket should reach the AI.

Some requests carry too much risk.

Refunds.

Legal complaints.

Security issues.

Angry customers.

When the workflow detects one of these categories, it skips the AI entirely and hands the conversation to a human.

No generated response.

No guessing.

No unnecessary risk.

The customer simply gets a message that someone from the team will follow up.

Sometimes the smartest thing an AI can do is stay out of the conversation.

3. Automation Needs Boundaries

Early on, I treated automation as an all-or-nothing decision.

Either automate everything or approve everything manually.

Testing taught me the real answer is somewhere in between.

The key is deciding where automation ends and human judgment begins.

What I do now

I split actions into two lanes:

Low-risk and grounded

If the answer comes directly from a verified knowledge source and the topic is low-risk, the response can be sent automatically.

Sensitive or unverifiable

If the request touches anything important, unclear, or potentially risky, a human makes the final decision.

The question isn’t whether humans should approve everything.

The question is which actions are safe enough to automate and which actions should always remain under human control.

Getting that boundary right is what makes automation trustworthy.

4. Adopt a Zero-Invention Policy

One thing I noticed very quickly is that LLMs hate saying:

“I don’t know.”

If information is missing, they’ll often try to fill the gap.

That’s useful for brainstorming.

It’s dangerous for business processes.

What I do now

Every workflow is grounded in a knowledge source, and my prompts include a strict rule:

Never invent policies, account information, refund amounts, or facts that are not explicitly available in the provided data. If the information cannot be verified, say a human will follow up.

The payoff is real.

When I asked my own support bot about a refund policy that wasn’t documented yet, it didn’t invent an answer.

It simply said it didn’t have that information and that a team member would follow up.

That’s exactly the behavior I want in front of a customer.

5. Protect Your AI Budget From Abuse

This lesson caught me completely off guard.

While testing one of my workflows, I noticed something strange.

Every message triggered the full pipeline.

Even messages like:

“Hi”

or simply:

“?”

The workflow would still run classification, query the knowledge base, and make paid AI calls before realizing there wasn’t actually anything worth processing.

That’s when I saw the problem.

A bored visitor could send hundreds of meaningless messages and quietly generate a real bill.

A malicious user could do exactly the same thing.

Not because the workflow was broken.

Because I had never taught it the difference between a real request and noise.

What I do now

I run a cheap rule-based filter before any paid AI call.

Real requests go through the full AI workflow.
Vague messages trigger a clarification flow.
Repeated low-value messages get handled with predefined responses.

The expensive AI calls only happen when they actually add value.

One lesson I learned is that workflow nodes shouldn’t only be ordered by logic.

They should also be ordered by cost.

The cheapest filter should usually come first.

Order your nodes by cost, not just by logic.

Final Thoughts

Before building AgenticScales, I thought the difficult part of AI automation was making the model produce good answers.

Now I think that’s the easy part.

The difficult part is building systems that remain reliable when APIs fail, users behave unexpectedly, costs increase, or AI reaches the limits of what it knows.

Production-ready AI isn’t about maximizing automation.

It’s about designing clear boundaries:

When AI can act.
When AI should ask for help.
When a human must take over.

The workflows I trust the most today aren’t the ones that automate everything.

They’re the ones that know exactly when not to.

That’s the difference between a demo that looks impressive and a system a business can depend on.

Have you learned any production lessons the hard way while building AI workflows? I’d love to hear your stories in the comments.

What if 30,000 Ex-Oracle Employees Teamed Up?

bilal meziani — Fri, 03 Apr 2026 17:30:45 +0000

Recently, news broke that 30,000 Oracle employees were laid off because of AI. That’s a huge number and it got me thinking. What if these tech veterans came together in an open-source community?

Imagine this:

30,000 skilled minds in databases, cloud systems, and complex architectures.
Decades of combined experience solving some of the toughest engineering problems.
No corporate bureaucracy, no endless meetings, just pure creative innovation.

The idea is that they could build something bigger, faster, and more innovative than any traditional company, and maybe even rival Oracle itself.

This scenario shows that collective talent and collaboration can sometimes outperform even the largest corporations. Being laid off might feel like the end, but it could also mark the beginning of something entirely new.

What do you think? Could this really happen?

I Built an AI Agent That Forces You to Finish What You Start — Using Notion MCP

bilal meziani — Sun, 22 Mar 2026 01:49:35 +0000

OneTask uses Notion's Model Context Protocol to turn one weekly goal into a structured weekly execution plan, with an AI agent that protects your focus all week.

This is a submission for the Notion MCP Challenge


🔴 The Problem	Most productivity tools overload you with hundreds of tasks, creating distraction and stress.
💡 The Idea	What if an AI forced us to focus on only one task at a time, and guided your week around it?
✅ The Solution	OneTask is an AI-powered focus agent that reads your Notion workspace, plans your week in structured daily steps, and enforces focus using intelligent prompts.

What I Built

Every Monday I open Notion and see 15 tasks. By Friday, I've touched 8 and finished 0.

The problem isn't motivation. It's too many open loops at once. I start something, get pulled toward something else, and never build real momentum on anything.

So I built OneTask, a Notion MCP-powered AI agent that enforces radical weekly focus.

You pick one task. The agent reads your Notion workspace, plans your entire week inside Notion day by day, and then guards your focus. Want to switch mid-week? The agent reviews your reason and rejects weak excuses.

BEFORE OneTask:

□ Learn Python
□ Build landing page
□ Write business plan
□ Fix the API bug
□ Update portfolio
□ Reply to emails
□ ...7 more

AFTER OneTask:

✅ Learn Python
- Wednesday: Read chapters 1-3
- Thursday: Build first script
- Friday: Practice with exercises
- Saturday: Mini project

You choose one focus. OneTask plans the week, writes it to Notion, and guards your focus.

Key Features

📋 AI week planning: Gemini 2.5 Flash breaks your task into specific daily steps

✅ Real Notion checkboxes: Creates to_do blocks written directly into your task page

🔒 Focus Guardian: The AI agent rejects weak reasons for switching mid-week

🔄 Real-time Notion writes: Every action in the app updates Notion immediately

✏️ Adapt your plan: Rewrite the plan for your actual level mid-week

🌙 Dark mode: Auto-detects system preference

Live Demo

💡 Try it yourself: Duplicate the OneTask Notion Template and see the AI in action.

Show Us the Code

GitHub: github.com/bilalmez/onetask

Notion Template: Duplicate the OneTask workspace

Stack:

Backend: Python + FastAPI
Frontend: Vanilla HTML/CSS/JS (single file, zero build step)
AI: Gemini 2.5 Flash (native JSON output mode)
Integration: Notion REST API + Notion MCP tooling layer

The Focus Guardian

This is the feature that makes OneTask different. When you try to change your focus on Day 3:

You: "I want to work on something else"

Agent: "You're on Day 3 of 7. You've done 1 out of 5 steps. What's the real reason?"

You: "I feel like working on a different project"

Agent: "That sounds more like avoidance than necessity. I recommend staying with your current focus."

You: "My client moved a deadline to tomorrow morning"

Agent: "External deadline conflict detected. You're clear to switch."

Commitment enforced intelligently, not just suggested.

The agent uses Gemini to review the reason and decide: legitimate external conflict or procrastination? This transforms productivity from suggestion to enforcement.

How I Used Notion MCP

OneTask uses a Notion MCP integration layer alongside the Notion REST API. The MCP layer gives the AI agent tool-based access to the workspace. Instead of raw HTTP calls, the agent interacts with Notion through structured tools it understands natively.

Here's the hybrid engine at the center of the architecture:

def mcp_hybrid_execute(operation, tool_name, tool_args, rest_method, rest_path):
    # 1. Try MCP tool first (native tool-based access)
    result = mcp(tool_name, tool_args)
    if result and not result.get("isError"):
        return result  # MCP success

    # 2. Transparent REST API fallback
    return notion(rest_method, rest_path)

MCP tools used in OneTask:

Tool	Purpose
`API-retrieve-a-database`	Read the workspace schema
`API-get-block-children`	Read existing page content
`API-patch-block-children`	Write daily to-do blocks
`API-delete-a-block`	Clean up old plans before rewriting
`API-patch-page`	Update Status, Priority, and Notes

What MCP unlocks: The agent doesn't just call an API. It uses Notion as a set of tools it understands. This changes the mental model from "send HTTP requests" to "use workspace tools." The result is better agent behavior and cleaner integration logic.

Why hybrid? The MCP layer handles structured workspace actions reliably. The REST API handles database property updates where the MCP layer can hit validation edge cases. The hybrid strategy means the agent always works. MCP when possible, REST as fallback, unified result either way.

What Happens in Notion When You Set a Focus

Finds or creates your task in the database
Sets Status → In progress and Priority → Week Focus
Writes a personalized AI motivation note to the Notes field
Removes Week Focus from all other tasks (one focus enforced)
Writes to_do blocks inside the task page (real Notion checkboxes)

When all steps are completed, Status → Done and Priority → Medium update automatically. No manual cleanup needed.

Try OneTask Yourself

1. Duplicate the Notion Template

2. Clone the repo:

git clone https://github.com/bilalmez/onetask.git
cd onetask
pip install -r requirements.txt
cp .env.example .env

3. Add your Notion token & database ID in .env

4. Run:

start.bat

5. Open http://localhost:3001

Focus on one goal, let the AI guide you, and actually finish what you start.

Built for the Notion MCP Challenge, March 2026