DEV Community

Cover image for Designing an AI System: Where Do You Even Start?
Siddhartha Reddy
Siddhartha Reddy

Posted on

Designing an AI System: Where Do You Even Start?

"Most people start AI projects with models. That’s the wrong place to begin. Here’s how to think about designing AI systems correctly."

Most AI projects fail before they even start.

Not because of bad models but because of bad starting points.


🚨 The Default (Wrong) Approach

Most people start like this:

Problem → Model → Data → Deployment
Enter fullscreen mode Exit fullscreen mode
  • Pick a model
  • Train it
  • Hope it works

👉 This leads to:

  • Misaligned systems
  • Poor performance
  • Failed products

🧠 The Real Question

Before you touch a model, ask:

What problem are we actually solving?

Not:

  • “Can we use AI here?”
  • “Which model should we use?”

But:

  • Who is the user?
  • What decision are we supporting?
  • What does success look like?

👉 AI is not the goal.

👉 The system solving a real problem is.


⚙️ The Right Starting Point

Real AI systems start like this:

Problem → Data → Constraints → System → Model
Enter fullscreen mode Exit fullscreen mode

Let’s break this down.


🧩 1. Problem Definition

Be specific.

Bad:

  • “Build a recommendation system”

Good:

  • “Increase user retention by recommending relevant content within 200ms”

👉 Constraints and goals matter more than the model.


📊 2. Data Understanding

Ask:

  • What data do we have?
  • Is it reliable?
  • Does it reflect real-world usage?

👉 No data → no system

👉 Bad data → bad system


⚠️ 3. Constraints (Most Ignored Step)

Every system has constraints:

  • Latency (real-time vs batch)
  • Cost (compute limits)
  • Scale (number of users)
  • Accuracy requirements

👉 These decisions shape EVERYTHING.


🧱 4. System Design

Now you design:

  • Data pipeline
  • Processing layers
  • Model integration
  • Serving layer
  • Monitoring

👉 This is where most real engineering happens.


🤖 5. Model Selection (Finally)

Only now do you ask:

  • What model fits the constraints?
  • What tradeoffs are acceptable?

👉 Not:

“What’s the best model?”

But:

“What’s the right model for this system?”


🔁 Putting It Together

The correct mindset:

Start with the problem  
Design the system  
Then choose the model
Enter fullscreen mode Exit fullscreen mode

⚠️ Why Most Teams Get This Wrong

Because:

  • Models are exciting
  • Systems are complex
  • Data is messy

So people skip to:

The easiest visible part, The Model


🚀 Final Take

AI projects don’t fail because:

  • Models are bad

They fail because:

The system was never designed properly


🧠 If You Take One Thing Away

Don’t start with the model.

Start with the problem and constraints.


💬 Closing Thought

Anyone can train a model.

Very few can:

Design a system that actually works

👉 That’s the difference.

Top comments (0)