Technical Interview for an Agentic Technical Lead Role

The company name has been slightly modified.

(I will share the original company name after you subscribe to my DEV.to and https://www.linkedin.com/in/egor-f-a214b2411.)

Today I had a real technical interview for an Agentic Technical Lead position at 4nd-ever.

The interview focused on LangGraph, MCP, LangFuse, evaluation, observability, routing, tool calling, and the architecture of production-ready agentic systems.

---

2. Role Overview

2.1. Main Goal

The role was focused on building an internal platform that allows engineering teams to create, configure, evaluate, observe, and deploy agentic systems.

The goal was not to build one AI agent. The goal was to build reusable infrastructure for many teams and use cases.

2.2. Main Responsibilities

• Design an agentic systems framework.
• Build LangGraph-based orchestration.
• Create reusable MCP Server and MCP Client templates.
• Implement evaluation pipelines with LangFuse.
• Support datasets, experiments, and LLM-as-a-Judge.
• Add tracing, cost monitoring, latency metrics, and drift detection.
• Support RAG, tool calling, structured outputs, and prompt chaining.
• Provide reusable components through Backstage.
• Own production reliability, retries, fallbacks, releases, and rollbacks.
• Mentor engineers and define the platform roadmap.

2.3. Main Stack

• Python
• LangGraph
• LangFuse
• MCP Servers and Clients
• PostgreSQL
• Backstage
• AWS
• Kubernetes
• GitLab
• Vector databases
• RAG
• Tool calling

---

3. Theoretical Interview Questions

3.1. How did you use LangGraph, and how was the agent structured?

I used LangGraph as an explicit state graph.

Nodes perform individual steps, while edges define transitions and branches. The agent stores its current state, calls tools, supports retry and fallback paths, and terminates when a defined terminal condition is reached.

3.2. What were the inputs and outputs of the planner layer?

The planner received:

• the user goal;
• the current state;
• available tools;
• system constraints.

It returned a structured plan containing:

• execution steps;
• selected tools;
• tool arguments;
• completion criteria.

3.3. What does a typical LangGraph flow look like?

User request
-> classification and routing
-> retrieval or tool execution
-> result validation
-> final response

Failure branches may include:

retry
fallback
clarification
human escalation

3.4. Where should deterministic logic be implemented: in the MCP client or MCP server?

Both components require deterministic logic.

The MCP client manages:

• timeout;
• retry;
• routing;
• request format;
• response parsing.

The MCP server manages:

• schema validation;
• authorization;
• idempotency;
• safe tool execution.

3.5. How did you build observability and evaluation?

Observability answers:

What happened inside the workflow?

It includes:

• traces;
• prompts;
• tool calls;
• latency;
• cost;
• errors.

Evaluation answers:

How well did the agent complete the task?

It includes:

• offline datasets;
• metrics;
• regression testing;
• experiments;
• human review.

3.6. Did you use LangFuse or another tool?

LangFuse is a strong primary layer for:

• LLM tracing;
• datasets;
• experiments;
• evaluation.

Infrastructure monitoring still requires additional tools such as:

• CloudWatch;
• Grafana;
• Prometheus;
• structured application logs.

3.7. How do you create evaluation datasets when agents can follow different paths?

An agentic dataset should contain more than input and output.

It should include:

• user goal;
• initial state;
• available tools;
• allowed trajectories;
• forbidden actions;
• expected final state;
• edge cases.

I would begin with real production cases and then add synthetic edge cases.

3.8. Is evaluation context created automatically or manually?

The best approach is hybrid.

Humans define:

• taxonomy;
• rubric;
• quality criteria;
• safety boundaries.

Automation extracts:

• traces;
• tool calls;
• outputs;
• latency;
• errors;
• cost.

Important or ambiguous cases should pass human review.

3.9. What context should be provided to an LLM-as-a-Judge?

The judge should receive:

• the original task;
• the rubric;
• the expected result;
• constraints;
• the actual execution trace;
• tool calls;
• the final answer.

It should score specific criteria such as:

• correctness;
• safety;
• completeness;
• trajectory quality.

3.10. What should be included in an evaluation dataset?

For a simple LLM feature:

input + golden answer

For an agentic workflow:

• initial state;
• available tools;
• allowed tool calls;
• key intermediate steps;
• negative cases;
• success conditions;
• failure conditions.

3.11. Which metrics matter for agentic systems and tool calling?

• task-completion rate;
• tool-selection accuracy;
• argument accuracy;
• tool success rate;
• trajectory accuracy;
• latency per step;
• retries and fallbacks;
• cost;
• schema validity;
• safety violations.

3.12. What should the classifier predict?

The classifier should classify the user request and select an intent or route.

Example classes:

• RAG;
• action or tool use;
• support;
• billing;
• research;
• validation;
• unknown.

It may also estimate:

• risk;
• domain;
• confidence;
• escalation requirements.

3.13. What data does the classifier use?

• the current user request;
• recent conversation history;
• user and tenant metadata;
• current workflow state;
• previous tool results.

Training data may include:

• labeled real requests;
• production traces;
• synthetic rare cases.

3.14. How do you route arbitrary user requests?

I would use a router with a confidence threshold.

First, the system determines:

• intent;
• risk;
• confidence.

Then it routes the request to:

• RAG;
• action workflow;
• research;
• specialist agent.

Low-confidence or high-risk requests should trigger clarification or human review.

---

4. Practical Architecture Task

4.1. The Main Requirement

After the theoretical questions, I was asked to design a production-ready agentic architecture using LangGraph.

The key condition was:

The system does not know in advance what request the user will send or what context will be required.

This was the most important part of the task.

The interviewer clearly expected the architecture to introduce a classifier before full context collection and tool execution.

The expected insight was not simply:

User -> RAG -> LLM -> Answer

The expected insight was:

User -> Classifier -> Required Context -> Route -> Execution

The classifier is essential because the system must first determine:

• what the user wants;
• which domain the request belongs to;
• what context is required;
• which tools are allowed;
• whether the request is risky;
• whether human review is needed.

4.2. Architecture Diagram

Excalidraw Whiteboard

Excalidraw is a virtual collaborative whiteboard tool that lets you easily sketch diagrams that have a hand-drawn feel to them.

excalidraw.com

• suggested

[FIGURE 1 - LangGraph Agentic Architecture]

Figure 1 shows the proposed architecture with:

• Trigger or Cron;
• Pre-check node;
• Intent classifier;
• Context builder;
• Router;
• Execution node;
• Validation node;
• Checkpointer;
• HITL User;
• HITL Admin;
• Circuit Breaker;
• Redis, relational database, and vector database.

A critical detail in Figure 1 is that several nodes contain internal iterations.

These iterations are not decorative. They represent repeated processing inside the workflow.

Examples:

• build_context iterates over context sources;
• router iterates over candidate tools;
• execution_node iterates over planned tasks;
• validate_node iterates over evaluation metrics;
• checkpointer_node controls retry loops.

4.3. Main Flow

Trigger
-> Pre-check
-> Intent Classifier
-> Context Builder
-> Router
-> Execution
-> Validation
-> Final Response

Additional branches:

Retry
Fallback
HITL User
HITL Admin
Circuit Breaker

4.4. Pre-check Node

The pre-check node performs deterministic operations:

• authentication;
• input validation;
• data cleaning;
• language detection;
• metadata loading;
• rate limiting.

def pre_check(state):
    return {
        "query": clean(state["query"]),
        "status": "validated",
    }

4.5. Intent Classifier

The classifier is the central component of the architecture.

It receives minimal context and returns a structured routing decision.

class Intent(BaseModel):
    route: Literal[
        "rag",
        "action",
        "research",
        "support",
        "billing",
        "unknown",
    ]
    confidence: float
    risk: str
    required_context: list[str]
    allowed_tools: list[str]

def classify(state):
    result = llm.with_structured_output(Intent).invoke(
        {
            "query": state["query"],
            "history": state.get("history", [])[-5:],
        }
    )
    return result.model_dump()

This directly solves the unknown-context problem.

The system does not load everything first. It classifies the request and then retrieves only the required context.

4.6. Context Builder

The context builder may use:

• Redis;
• PostgreSQL;
• vector databases;
• conversation history;
• summaries;
• RAG;
• user metadata.

The internal iteration from Figure 1 can be implemented as:

def build_context(state):
    context = {}

    for source in state["required_context"]:
        context[source] = LOADERS[source](state)

    return {"context": context}

4.7. Router

The router selects a tool, specialist agent, or subgraph.

def route(state):
    if state["risk"] == "high":
        return "hitl_admin"

    if state["confidence"] < 0.5:
        return "hitl_user"

    return state["route"]

The router may internally evaluate several tools before selecting one.

4.8. Execution Node

The execution node processes the structured plan.

Figure 1 explicitly shows iteration over tasks.

def execute(state):
    results = []

    for task in state["tasks"]:
        results.append(TOOLS[task["tool"]].invoke(task["args"]))

    return {"results": results}

For production workflows, one task per graph transition is often safer because the state can be checkpointed after each external call.

4.9. Validation Node

The validation node checks:

• schema validity;
• correctness;
• safety;
• tool results;
• terminal conditions.

Figure 1 shows an internal loop over metrics.

def validate(state):
    scores = [metric(state) for metric in METRICS]

    return {
        "scores": scores,
        "passed": all(score["passed"] for score in scores),
    }

4.10. Retry, Fallback, and Circuit Breaker

A failed validation may trigger:

• retry;
• replanning;
• fallback;
• human review;
• circuit breaker.

def after_validation(state):
    if state["passed"]:
        return "final"

    if state["retry_count"] >= 3:
        return "circuit_breaker"

    return "retry"

4.11. Minimal LangGraph Assembly

graph = StateGraph(AgentState)

graph.add_node("pre_check", pre_check)
graph.add_node("classify", classify)
graph.add_node("context", build_context)
graph.add_node("execute", execute)
graph.add_node("validate", validate)

graph.add_edge(START, "pre_check")
graph.add_edge("pre_check", "classify")
graph.add_edge("classify", "context")
graph.add_edge("context", "execute")
graph.add_edge("execute", "validate")

graph.add_conditional_edges(
    "validate",
    after_validation,
    {
        "final": END,
        "retry": "execute",
        "circuit_breaker": END,
    },
)

---

5. Main Takeaway

The most important question was not:

How do we connect an LLM to a tool?

The real question was:

How do we design an agent when the required context is unknown in advance?

The interviewer wanted to hear that a classifier should appear before full context retrieval and tool execution.

The classifier determines:

• intent;
• risk;
• confidence;
• required context;
• allowed tools;
• execution route.

Only after that should the system retrieve data, build a plan, call tools, validate the result, and terminate the workflow.

The internal iterations shown in Figure 1 are also important because real agentic systems repeatedly process:

• context sources;
• candidate tools;
• execution tasks;
• evaluation metrics;
• retry attempts.

---

6. Interview Results

If you want to learn the final result of this interview, subscribe to my DEV.to and LinkedIn:

DEV.to: https://dev.to/efa

LinkedIn: https://www.linkedin.com/in/egor-f-a214b2411/

You can also message me directly, and I will reply privately.