DEV Community: Lilian Bernot

Mentoring Your First Intern

Lilian Bernot — Fri, 22 May 2026 09:19:51 +0000

Introduction

I started full-time in Paris in November 2025. Three months later, in early March, my manager told me I'd be mentoring an intern joining the team. I had finished my own internship at Datadog six months earlier. The chair I'd been sitting in — the one with the laptop that wouldn't quite configure, the standups I didn't understand, the standing question of "what am I going to say at defense?" — that chair was about to be filled by someone else, and I was the one supposed to help them sit comfortably in it.

This post is two things. First, the journey: what being an intern was actually like, and what changed when I became the mentor. Second, the part that matters more if you're about to mentor someone for the first time: the handful of things I liked as an intern, and the ones I tried to do once it was my turn.

My Internship Experience At Datadog

Two Kinds of Internships

There are roughly two shapes an internship can take.

The first is the solo niche project: you arrive, you're handed something specific, you have a mentor, and your job is to grind on that one thing for three or six months. Some schools love this shape — it sounds technically deep, easy to grade, easy to defend.

The second is being part of the team: you don't get a single big project. You take small tasks, you go to all the meetings, you talk to everyone, you learn how the company actually works.

I had teachers who were supportive of the second kind. I'd already done two internships before Datadog where I took part fully in what the company was doing, and that was the best preparation I could have asked for. I arrived at my third internship knowing what the industry looked like, with my own requirements for what made a company fit me.

The first shape sounds rigorous. In practice, if your niche project gets dropped because you were too slow, or because the company's direction shifted, you're left with a thin résumé and a defense full of caveats. The second shape gives you something more durable: a sense of how the work moves, who owns what, and what kind of company you want to be in next.

If you mentor an intern under a school constraint that pushes the first shape, you can still pull them toward the second. The deliverable on paper is one thing; the experience you give them is another.

First Days at Datadog

My first week at Datadog, I followed my manager around all day. Every meeting except hiring interviews. I had lunch with him and two other managers he was close with on day one. By the end of the week, I'd been introduced to most of the people who'd matter during my internship — the engineers I'd rely on, the product owners I'd talk to, the cross-team contacts I'd need.

I went to standup (we did it sitting around a table, which I never quite got over). I went to sprint planning, the monthly retro, every recurring meeting. I couldn't say anything for the first two weeks. The script went:

Week one: "Today I attended some internship presentations, set up my People Team account."
Week two: "I ran the service locally, opened my first one-line PR."
Week three: "I found this bug on prod and shipped a fix, deploy is running."

And without noticing it, I went from being a green plant staring at everyone to being someone with something to say.

My first project was so under-specified I didn't know where to start. "See that feature? The data is stored in some old database in a repo the company wants to deprecate." I had to chase the frontend code to find which backend it called, find that backend in another repo that nobody owned, find the database we didn't have access to. I made a lot of brain knots trying to figure out who owned what. One of the backends was so badly intricated that there was no way to test locally — I had to deploy straight to staging to run my database modifications, and I was happy not to fully destroy anything.

I learned more from the shape of that project than from finishing it. How the company was wired. Where the load-bearing people were. What "ownership" actually meant.

Becoming the Mentor

Six months later I was full-time. Three months after that, I was mentoring.

The thing that surprised me most was how recent the intern experience still felt — and how much that helped. I remembered the specific frustration of typing the same command for the fourth time because I hadn't yet aliased it. I remembered not knowing what to say in 1-1s with senior engineers.

What follows is the advice I'd give anyone about to mentor someone for the first time. It's not exhaustive. It's the handful of things that I think actually move the needle.

Some Opinionated Advices

The First Week Is Setup Week

The first week is for setup. All of it.

Laptop config. Tooling. Internal sites — the people directory, the codebase navigator, the dashboards, the docs wiki, the support channels. Going through the motions of finding things, even when nothing is broken. The better your intern does this in week one, the faster they'll be in week two, and the less embarrassing it will be in week three when they're still tweaking their shell aliases instead of shipping.

The other half of the first week: meeting the team. Get them in front of as many people as you can. Take notes during those introductions for future use. People at Datadog are — and I think this is true at most good companies — genuinely eager to explain what they do. Most engineers are nice to talk with. Use that.

If your intern leaves week one with a configured laptop and a list of names attached to faces, you've already done a lot.

One PR a Day

The most controversial piece of advice I give, and the one I keep coming back to: try to ship one PR a day.

I expected this to be obsolete by now. AI tools have changed how fast code gets written, how much context a single engineer can hold, how much can land in a single change. I assumed the one-PR-a-day rule would dissolve. It hasn't. The reasons are the same as they always were:

Show that you ship. Asking for PR reviews puts your name in channels. Your coworkers see what you're working on. They learn what they can ask you about. It also keeps you from drifting into a four-day investigation with nothing visible at the end.
Get daily feedback. Not just on code (which I hope you're not writing manually anymore). Feedback on PR descriptions, on naming, on how you communicate. The people you work with most often are the best loop for this, and a PR a day keeps that loop tight.
Practice moving forward in small steps. It's easy to slip into writing a 1000-line PR, especially when AI makes it cheap. Shipping in chunks is better in almost every dimension I can name: easier to find reviewers, easier to incorporate feedback, easier to roll back, easier for production bugs to surface gradually instead of all at once.

If your intern hits one PR a day for the first month, almost everything else takes care of itself.

Pair Programming Goes Both Ways

I had hour-long pair sessions every day for the first two weeks with my intern. We swapped configurations. I showed her shortcuts. She asked questions that made me realize how arbitrary some of my habits were.

This is one of those activities that sounds generous from the mentor's side but is genuinely two-sided.

I remember an "innovation week" during my own internship where I worked closely with a senior engineer who was mostly working on frontend. We built something together for a week. I walked out of that week with new tools, new ideas, and a noticeably smoother day-to-day workflow. That was someone choosing to spend a week pair-programming with an intern — and it visibly cost them nothing.

Two things I didn't expect about being on the mentor side of pair programming:

It makes you self-conscious about your own setup, in a good way. When you're describing what you're doing as you do it, you notice the friction. You can't find the alias. Your shell history is a mess. The repo organization you've internalized doesn't survive narration. You end up cleaning things up.
The intern just got out of school. They've been taught by many people, recently. They've read things you haven't. They've used languages or libraries you don't know. They'll mention bloggers, classes, tools. Some of that is genuinely useful to you. Don't dismiss it.

The framing I'd push back against is: "I'm helping the intern catch up." A better framing is: we're learning two different things from the same session.

When You're Not in the Same Office

If your intern is in a different office than you are, almost everything above gets harder. You can't grab them for a quick pair session. You won't run into them in the kitchen. The small, low-stakes interactions where most context actually transfers — those mostly disappear.

The single most useful thing I've seen for this: set up daily syncs. Even when there's nothing to talk about.

There will be days where the intern says they don't need it. Keep it on the calendar anyway. The point isn't your need; it's giving them a guaranteed slot to surface a question, a blocker, or something they noticed and didn't know what to do with. A standing fifteen minutes removes the "is this worth interrupting them?" hesitation. It also forces them to practice stating the problem clearly, out loud, every day.

When I was interning in New York, my mentor was in France. The time-zone gap was real but workable: there was a few-hour overlap in my morning and his afternoon. I learned to catch him before my day really started — check the unknowns and the complex bits of what I was about to do — and to catch him again later, just before he stopped working, to close the loop. That cadence saved me a lot of solo flailing.

For projects he was less involved in, I leaned on a couple of colleagues in my office who picked up part of the mentor role informally. Two of them became, in practice, partial mentors. They weren't on the org chart for it, but they were there when I needed them.

If you're mentoring someone in a different office, my strongest suggestion is: find someone in their office that you trust, and that they can rely on. Not officially. Just someone who'll grab lunch with them in the first week, answer a "stupid" question without making them feel stupid, and tell you privately if something's off. It's a quiet thing to set up, and it's worth a lot.

Delegate Real Work

A recent example: my manager was on-call and wanted to delegate part of it — at least the support questions — to the intern.

At first glance, this looks like classic top-down dumping. The senior person passes the annoying part to the most junior person. But look at what the intern actually gets:

Exposure to more products in the team than they'd otherwise see.
A reason to talk to engineers and product owners they wouldn't normally reach.
Real practice at writing clearly under time pressure (support questions don't wait).
A working sense of what being on-call actually feels like.

That last one matters more than it sounds. It's a great thing to talk about in an internship defense, and it's a real differentiator if you're trying to convert into a full-time offer.

More generally: give your intern small tasks that you could do yourself, probably much faster. It costs you some efficiency in the short term. In exchange, you see how they think, where they get stuck, what they get wrong. You get to be opinionated about how something should be done — which gives you both something concrete to talk about, and often makes you rethink your own habits when you hear yourself defend them. Before the internship helps, your mentee will be an autonomous colleague who you can work with side-by-side.

Explain the Why

The thing that's hardest to give an intern is context.

It's easy to tell them "go to this meeting" or "read this article" or "talk to this person." It's harder, and more valuable, to say why. Why does it matter that they go? What are they meant to take from it? Who should they look out for in the room?

This matters most for the meta-skill of knowing what to talk about with people more senior than you.

It's actually a hard problem. You know your life and theirs are different. You know your level of understanding is different. When you talk, they'll follow most of what you say. But when they talk, you'll often not even understand the point of their job — because you're trying to grasp a high-level description of something with cross-product, cross-team dependencies, or a technically obscure project that sits far from you in the stack.

This is one of the various things I stated here that I'm actively working on. I don't know how to teach it directly. But I know the antidote to the silence is context: "here's why I'm sending you to this person, here's what they care about, here's what would be interesting to ask them about." The mentor's job, more than handing out tasks, is handing out reasons.

Let Them Impress You

A short note to end on.

You're a few years ahead of your intern, not decades (well, my mentor was but you're probably not). The gap is real, but it's nowhere near as big as it might feel in the early days. Let yourself be impressed by them. They're freshly out of school, intelligent, taught by good people, and they're going to do something you don't expect.

Notice when it happens. Tell them.

Conclusion

The mentor I tried to be was, more than anything, the mentor I wished I'd had: someone who'd walk me around the office, sit me at the right tables, explain why the boring meeting mattered, give me real work, and quietly mention that one PR a day was worth aiming for.

Mentorship isn't a one-way transfer. The version of it that actually works runs in both directions — and the version that works best is the one where the mentor is honest enough to admit the intern is teaching them, too.

If you're about to mentor your first intern: configure their first week, push them to ship daily, pair with them, delegate real work, explain the why. Then watch them, and let them impress you.

Thanks to T.J., Fred, Kévin, Benoit and Tiina, who showed me what good mentorship looked like before I had to do it myself. And to Doaâ — most of what's in the second half of this post is something I figured out while working with you.

Build Your Harness While You Wait

Lilian Bernot — Fri, 15 May 2026 06:23:39 +0000

A few weeks ago, I sat down with a colleague for our weekly 1-1. He came in visibly exhausted — not from a late night debugging, not from an incident. He'd been on vacation and had come back to four active Claude sessions waiting for him, each one pulling in a different direction.

"I always told people not to do this".

I'd heard this before. But sitting in that room, I realized we were looking at a new kind of developer fatigue — one that didn't exist two years ago.

The New Speed Limit

In 2023, developers were tired from staying up until 3am reading through StackOverflow for a two-line fix, or building entire features from scratch to satisfy a PM's request. That kind of exhaustion had a shape to it: deep, slow, earned.

In 2026, the exhaustion looks different. It's not from doing too little — it's from doing too much, too fast. A new feature now takes three good lines of prompting. The bottleneck isn't execution anymore; it's context switching.

I walk into our office and hear more people complaining about feeling scattered than about being stuck. The challenge has shifted from "how do I build this?" to "how do I stay coherent while building five things at once?"

The Babysitting Trap

There's a mode I see a lot of colleagues still in, and I was there too not long ago: babysitting Claude. Accepting every file edit, rename, commit, and test run. Sitting in front of the screen, approving actions one by one.

This is a waste of the most valuable thing you have: your attention.

For a few weeks, in the repo I work in most, Claude was running complex Bazel queries that took forever. I kept multiple sessions open just to absorb the wait. At some point I actually looked at what was happening: it was running commands at the repository root level, which is catastrophically heavy in a monorepo. I spent an hour explaining Claude how to scope its commands — package level first, then service or domain, never the full repo.

That one hour saved me dozens.

Build Your Harness While You Wait

This is the reframe that changed how I work: use latency as leverage.

When an agent is doing something long — running tests, building a target, generating protobuf files — that's not dead time. That's configuration time. It's the 20% of my AI time I now deliberately protect. The remaining 5% I keep for research: what's new in the AI world, what are my colleagues trying, what's worth stealing for my own workflow?

Think of it like early career progression. As a newcomer, you have low privileges. You move carefully, ask for approval often. As you grow, you earn operational rights, architectural trust, autonomy. AI sessions are the same: untrained sessions are expensive to babysit. Trained sessions — with the right permissions, the right context, the right guardrails — run themselves.

We don't want to micro-manage. We want to configure, then delegate.

The Notifications Tax

One underrated part of this: notifications.

I turned off WhatsApp emoji reaction notifications months ago. Every Slack channel I'm in is muted except for PMs with my intern, my manager, and the handful of coworkers I might be on-call alongside.

This sounds antisocial. It's a trade: I respond more thoughtfully and less reflexively. The pings that reach me matter.

Boredom, it turns out, is useful. The moments where I step away from the screen — tidy something up, stare out the window — are where I do my best thinking. I anticipate what the agent will do next. I kill the process, refine the prompt, restart with a better question. The idle second is where the edit happens.

On Git Worktrees (and Why I Dropped Them for now)

I got excited about git worktrees for a while. The idea is elegant: while one agent works on branch A, spin up another one on branch B. No blocking, pure parallelism.

In practice: more context switching, not less.

Worktrees in a large monorepo are slow to spin up. You have to track which clone corresponds to which task, manage rebases across branches, stack PRs in ways that are genuinely hard to reason about. Some colleagues just maintain two clones of the repo — which is roughly the same problem. I tried letting the AI manage the worktrees for me. That was worse.

There are legitimate uses — deploying something while keeping the working tree open, running long bazel or yarn commands for auto-generation — but for most day-to-day work, the overhead isn't worth it. The overhead is cognitive, not computational.

The Comeback Sprint

Three weeks of sick leave. A half-finished feature I cared about. A week of on-call the moment I returned.

When I finally got two free days to work on it, I was four weeks out from the last time I'd touched it. I was also worried someone had moved on without me.

They hadn't. So I started.

I had notes. I had saved links, database queries, context scattered across docs and Slack threads. AI agents helped me reassemble all of it — regroup the data, spec the implementation step by step, proceed, test. By end of day one, I had shipped four PRs. One update on the database definition, one fixing multiple bugs in the existing code, one implementing part of the feature and the last one to build some tooling. That one hour of config work saved hours in the days that followed.

Day two: three more PRs. Done.

A year ago, that feature would have taken me two weeks. Minimum. When I sent it for review, I already knew it was clean. Approvals came fast. I moved on.

The feeling of finishing ahead of your own plan, when you had every reason to expect to be late — that's the feeling I'm chasing.

Flow, Not Orbit

I read a Forbes piece recently that argued "Orbit is the new Flow." The idea: agentic AI rewards context-switchers, people who keep many things in motion at once rather than diving deep into any single one.

I'm not convinced.

Flow — the uncomfortable, focused state where you stop noticing the room around you — is still where I do my best thinking. When I come out of it, complex things are in place, cleanly. I just have to document what I did, transfer context somewhere durable.

Orbit might scale better. But I'm not sure yet that it scales better for me. The risk is creating content and context faster than you can actually ingest or ship it.

The goal isn't to have many things moving at once. It's to have the right things moving — and to actually finish them.

Conclusion

The new developer fatigue is real, but it's manageable. It's not about doing less — it's about doing it with intention. Protect your focus. Use the wait to improve the harness. Mute the noise that doesn't matter. Ship things completely before starting the next one.

We're still engineers. We're just working at a different speed, with different constraints. Learning to configure the machine — including the AI — is the new leverage.

From Developer to Engineer: My Journey with AI Coding Tools

Lilian Bernot — Fri, 15 May 2026 06:09:00 +0000

Written: December 2025.

A recent blog post that resonated with me: The Year Coding Sped Up 100x:

The point is not that software engineering is dead.

Quite the opposite: the engineering in software engineering is thriving. Modeling, formal validation, higher-level languages—these were at the fringes of "traditional" software engineering for a long time because most of the brain power was spent writing code.

As a junior who started coding in 2023, I can deeply relate to that statement. This is the story of how AI tools transformed me from a struggling developer into a software engineer.

The Beginning: Learning the Hard Way (2022-2023)

When I started "learning code" at school in 2022, it was truly learning the hard way. Nothing comparable to formal textbooks—I had to grind through exercises, maintain a personal dictionary of vocabulary I struggled to remember, and piece together understanding bit by bit. At that time, GPT-3 was just starting out, mainly a playground for people asking it to tell jokes.

For a year, I took on the developer role in school projects because I wanted to learn more. I spent hours checking other people's code to copy-paste parts of it, reverse-engineering solutions without fully understanding them.

The Chile Semester: Coding in Three Languages

Then I spent the first semester of 2023 in Chile, taking classes like "web development fundamentals," "databases," and "algorithms." The real names were "Inicio Al Desarrollo Web," "Base de datos," and "Algoritmos"—all classes taught in Spanish, code in English, and my brain thinking in French. I couldn't describe this mess better than: "In my code, some comment lines were written in 3 languages."

During these classes, we worked with React and Koa, SQL, and C++, respectively. They were all supposed to be entry-level, but the students around me had already completed 2-3 semesters of similar courses and were far more advanced than me. I kept asking to see others' code, spent countless hours reading the easier classes' slides first to make my way through understanding the actual course material I was supposed to be following.

First Internship: Contradictions and Crisis (October 2023)

In October 2023, I started my first internship. GPT-3 was being fine-tuned to become what we now know as modern AI. At that time, I experimented a bit out of curiosity—asking it to explain pieces of code, which I would confirm with the classic StackOverflow questions and other sources. I also used it for repetitive tasks, like extracting just the keys from JSON into text format. Answers started to make sense. Others were using GitHub Copilot, which I didn't like because it was writing the code for me without giving me time to think about what I was doing.

The Identity Crisis

When I started my internship, I was full of contradictions: both happy to learn the new languages I was coding with (which I love to learn) and disappointed seeing people with such advanced skills that would take me years, maybe decades, to acquire. There I started questioning my career as a developer (vocabulary matters here).

In the meantime, the use of AI was rising and we kept hearing people say we would lose our jobs. What was I doing in a world where I was just a beginner, facing senior people with much more knowledge than me, and facing new AI tools that were becoming as efficient as an intern?

Back to Fundamentals (Fall 2024)

A year later, I returned to school to study "Fundamentals of Computer Science." I had briefly touched some C++ code before, but I was really lacking foundational knowledge. We learned Rust, which, coming from Python, really kicked my ass. We did Assembly and Haskell, and kept listening to our teachers who were so enthusiastic about low-level languages and optimizations.

I can painfully tell you that GPT had not been trained on Assembly books yet...

This was a great time—stepping back from speed-coding and "AI taking over the world," and trying to redefine what "software engineer" meant for us. It grounded me in the fundamentals and gave me perspective on what engineering actually entails.

The Turning Point: Datadog (Spring 2025)

After that semester, I started my internship at Datadog in New York. I was in a great team with excellent colleagues who taught me a lot. I started learning Go, using AI the best I could to accelerate my learning—helping me ship code while also teaching me core principles of the language. I had an excellent mentor who did really strict PR reviews that helped tremendously with my learning.

Innovation Week: AI Everywhere

One month in, we held an innovation week, trying to put AI everywhere in our product. At that time, all companies were releasing AI assistants—we were building Bits AI, others had their own solutions. The race had started.

In the following month, I started working on an NLQ (Natural Language Query) feature for which I had to read a lot of blog posts, study how others were approaching the problem, and experiment extensively. I learned a lot about how to work with AI and get these capricious little minds to respect what I was asking them to do.

The Hackathon Week: Meeting Cursor

After this period came a company-wide AI hackathon week, during which we attended numerous conferences and gathered for vibe-coding sessions where we built apps in an hour. At that time, I was pretty skeptical about AI, given my recent battles with it. I finally accepted to install Cursor (I didn't even have Copilot in VSCode as I didn't like the noise it brought).

And Cursor suddenly became my best colleague.

I was suddenly able to debug my code 10x faster, delegate boring tasks to AI, and read documentation or answer messages in the meantime. This was the moment everything changed.

My Current Setup (Late 2025)

Let's jump to what my setup and workflow are now: I run multiple Claude Code instances simultaneously to handle most of my AI tasks, combined with Cursor, which helps me read my code as an enhanced IDE. I use it to ask for clarifications on specific bits of code, no longer than a few lines.

Redefining the Role: Developer vs. Software Engineer

In a world where AI is, for some, a confidant to which they tell everything and ask to make important decisions in their lives, or use to justify bar discussions with a quick prompt, AI can also genuinely enhance some of our abilities.

I've had many interesting exchanges with quite a few engineers in the company now, which helped me gain perspective (I also had a month off between my internship and my full-time job—take holidays, devs!).

What "Software Engineer" Actually Means

And now I understand more clearly what my job is: I am a Software Engineer. My job requires developer skills—I need to be able to read code, understand how it works, know best practices in several different languages, and know how to deploy a service.

But as seniors were before me, I am more of an orchestrator than a code writer. My job is not to write lines of code. I can now respond to support messages, write fixes, deploy them, make sure they run in production, and manage the entire lifecycle of a feature.

The Transformation

As a junior, every part was scary, and the "write the fix" part was perhaps the worst: I could see myself fighting with language syntax for countless hours, reading again and again my personal notes on the language, realizing which patterns didn't actually apply in every situation, going back and forth with StackOverflow...

Now, every time I have to write code, I can stop worrying about that part. I focus more on:

Defining my needs clearly
Architecting solutions that AI will implement for me
Writing tests that aren't influenced by the code I wrote
Collaborating with other engineers and teams
Organizing projects end-to-end
Ensuring quality and maintainability

Altogether, this makes me more and more an Engineer, not just someone who writes code.

Conclusion

The transformation from developer to software engineer isn't about abandoning coding—it's about elevating your role. AI tools like Cursor and Claude Code haven't replaced me; they've liberated me to focus on what truly matters: engineering decisions, system design, collaboration, and delivering value.

The fear I felt in 2023—that AI would make me obsolete before I even started—was misplaced. Instead, AI accelerated my journey from a struggling beginner to a being-confident engineer who can architect, ship, and maintain production systems.

The engineering in software engineering is indeed thriving. And I'm thrilled to be part of it.

Choosing a Natural Language Query Architecture for Dynamic Data Systems

Lilian Bernot — Fri, 15 May 2026 06:06:17 +0000

When building a Natural Language Query (NLQ) interface for a complex, evolving data catalog, the choice of architecture can make or break your solution. This post explores the architectural decisions faced when implementing NLQ for a system with dynamic schemas, frequent changes, and complex query requirements.

The Challenge: Dynamic Data Systems

Our data architecture presented several unique challenges:

Atomicity: Data organized by resource types, with many type-specific fields
Dynamic Evolution: Rapid schema changes (50 → 140 tables in one year)
Complex Queries: Support for tags, date comparisons, and advanced operators (not just equality but >, <, etc.)
High Specificity: Domain-specific query syntax requirements

The key question: How do we build an NLQ system that adapts to these constant changes without requiring retraining or manual updates?

Solution 1: Fine-Tuned Static Models

The traditional approach involves:

Base Model: A foundation model trained on general query syntax
Fine-Tuning Layer: Product-specific training on historical data
Deployment: Static model serving predictions

Advantages

Well-understood approach with proven results
Can leverage existing infrastructure
Potentially lower per-query costs at scale

Limitations

Not Dynamic: Every schema change requires retraining
Team Dependency: Requires ML expertise and infrastructure
Training Cost: Significant upfront investment in data and compute
Lag Time: Delay between schema changes and model updates

For our use case with frequent schema evolution, this approach would create constant maintenance burden and potential staleness.

Solution 2: Model Context Protocol (MCP)

An alternative approach uses the Model Context Protocol to provide dynamic context to a general-purpose LLM:

User Query → LLM with MCP Tools → Query Syntax → Execution
                ↓
         [Dynamic Schema]
         [Available Fields]
         [Query Examples]

The MCP provides:

Real-time schema information
Available fields and types
Valid query patterns
Domain-specific constraints

Advantages

Fully Dynamic: Adapts instantly to schema changes
Team Ownership: No dependency on ML teams
Reduced Training: No model training or retraining required
Transparency: Tool usage provides explainability

Limitations

Per-Query Cost: Higher compute cost per query
Latency: Multiple LLM calls may be needed
External Dependency: Relies on external LLM providers

Solution 3: Hybrid Approach

Could we combine the best of both worlds?

The idea: Use a small, fine-tuned base model with MCP for dynamic context.

Current Reality

Most lightweight models (e.g., Llama 8B variants) don't support complex tool usage or reasoning required for MCP. They're trained for direct translation tasks.

Building a larger model with tool-use capabilities is:

Significantly more complex
Resource-intensive
Not a near-term option for most teams

Other Considerations

Direct SQL Generation

Why not skip the intermediate query syntax and generate SQL directly?

Complexity: Easy to generate syntactically correct but semantically wrong queries
Business Logic: Existing query layer encodes important domain knowledge
Maintenance: Duplicating logic in multiple places
Low Cost: If the execution layer is already efficient, why bypass it?

Agent SDK vs Custom Implementation

Modern agent frameworks (OpenAI Agents SDK, LangChain, etc.) provide higher-level abstractions. Why build custom?

Language Constraints: SDK may not be available in your stack (e.g., Python SDK when you need Go)

Performance: SDK adds:

Network overhead for API calls
Serialization/deserialization costs
Potential service-to-service latency

Duplication: Using external service means duplicating logic and data access

Control: Custom implementation provides:

Fine-grained optimization
Direct access to data
Single service boundary

For high-performance, low-latency requirements, a custom implementation integrated directly into your service may be preferable.

Decision Framework

When choosing an NLQ architecture, consider:

Schema Stability
- Frequent changes → Favor dynamic approaches (MCP)
- Stable schema → Fine-tuning becomes viable
Team Capabilities
- ML expertise available → Fine-tuning feasible
- Engineering-focused → MCP easier to maintain
Cost Profile
- High query volume → Fine-tuned model may be more economical
- Lower volume → MCP per-query costs acceptable
Latency Requirements
- Sub-second requirements → May need cached models
- Seconds acceptable → MCP works well
Ownership Model
- Centralized ML team → Fine-tuning with shared infrastructure
- Product team ownership → MCP with direct control

Our Decision: MCP

We chose the MCP approach because:

Dynamic First: Our schema changes frequently enough that retraining overhead would be significant
Independence: We wanted to move quickly without dependencies
Transparency: Tool usage provides clear audit trails
Cost Acceptable: Query volume doesn't justify training infrastructure
Experience Value: Even if we change approaches later, the work on query datasets and testing will transfer

Conclusion

There's no universal "best" architecture for NLQ systems. The right choice depends on your specific constraints:

Schema stability
Team structure and expertise
Query volume and latency requirements
Cost considerations
Ownership and maintenance preferences

For dynamic systems with evolving schemas, MCP-based approaches offer a compelling balance of flexibility, maintainability, and time-to-market. For stable, high-volume scenarios, the upfront investment in fine-tuned models may pay dividends.

The key is matching the architecture to your specific needs, not following a one-size-fits-all approach.