From Backend Engineer to Building AI Infrastructure at a Startup

#ai #startup #programming #career

What does it look like to go from a six-person startup team to running the infrastructure behind 1,000+ AI models?

Matteo and I have known each other for over 10 years, but I realized I'd never actually asked him about his engineering journey in depth. I knew he was doing infrastructure work, but I didn't know the full story of how he got there. This conversation filled in a lot of gaps for me.

In Episode 4 of Making Software, I talked to Matteo Ferrando, Platform and Infra Engineer at fal.ai, about exactly that.

What we covered

The pivot that changed everything. The company didn't start as an AI company. Matteo talks about what the original product was, how the pivot happened, and why letting go of something that's working is one of the hardest things you'll do at a startup as an engineer.
How a backend engineer ends up doing infra. There was no "switch." Matteo explains the reality of early-stage startups where in the beginning, there's no database, no Kubernetes cluster, no nothing. You just build it.
A decision-making framework worth stealing. One-way doors vs. two-way doors. Simple concept, but it changes how you think about MVPs, technical debt, and when to actually invest in doing things "the right way."
Optimizing a routing layer from 100ms to 5ms. The use case that forced this is wild. I'll let Matteo tell that story.
Do you still need computer science fundamentals? We talked about AI coding tools, what they're great at, and a very specific failure mode that Matteo keeps seeing on his team.

A couple of things Matteo said that stuck with me

"I'm not obsessed about technical debt like many engineers are. It's fine to have technical debt and we'll get to it."

This resonated a lot with me. I think a lot of us carry guilt about tech debt like it's something we did wrong. Matteo's framing is different: it's not a problem until it's actually a problem. That shift in perspective is freeing.

"A mistake introduced by an AI that you prompted is still your mistake. You still have to understand what you're shipping."

This one hit. Especially right now, when so many of us are leaning on AI tools to write code faster. Faster is great, but Matteo makes a really compelling case for why "I didn't write it" is not an excuse. He shares a specific example from his team that I think every developer using AI tools needs to hear.

Final thoughts

Talking to Matteo reminded me that there's no clean, linear path in engineering. You don't go from "backend engineer" to "infra engineer" because you planned it. You get there because something needs to exist and you're the one who builds it. I think that's something a lot of us can relate to, especially if you've ever worked at a startup.

Have you ever had to build something way outside your comfort zone because no one else was going to? Let me know in the comments!