DEV Community: PandeyC

Give Your Chatbot a Memory in Google Colab Before Your Next AI Interview

PandeyC — Wed, 15 Jul 2026 13:34:07 +0000

You don't need a vector database to understand LLM memory.

You don't need LangChain.

You don't need an API key.

You need:

Google Colab
Python
A conversation that keeps growing
About 15 minutes

In an earlier article, we built the retrieval half of RAG — chunking, embeddings, cosine similarity. This one builds the other half every multi-turn LLM app needs: memory. It's the exact mechanism behind the interview question, "how do you manage memory in an LLM application when conversations get long?"

Let's build it.

What Are We Building?

An LLM has no memory of its own. Every call only sees what you send it — the context window. Left alone, a growing conversation does this:

Turn 1 → Turn 2 → Turn 3 → ... → Turn 50
              ↓
   Send the full history every time
              ↓
   Context window fills, cost climbs

We'll build the fix: a token counter, a summarization trigger, and a remember() function that keeps a conversation coherent without sending everything, every time.

Step 1: A Conversation That Won't Stop Growing

Open a Colab notebook and simulate 50 turns of chat — no API needed yet.

def approx_tokens(text):
    return int(len(text.split()) / 0.75)  # a token is roughly 3/4 of a word

history = []
for i in range(1, 51):
    history.append({"role": "user", "content": f"Question {i} about our product."})
    history.append({"role": "assistant", "content": f"Answer {i}, referencing prior context."})

print("Total turns stored:", len(history))

Step 2: The Cost of "Just Send Everything"

The simplest memory strategy — an in-context buffer — resends the full history on every call. Let's measure what that actually costs.

def cost_of_turn(n_turns):
    sent = history[: n_turns * 2]
    text = " ".join(t["content"] for t in sent)
    return approx_tokens(text)

turn_2 = cost_of_turn(2)
turn_50 = cost_of_turn(50)

print("Turn 2 cost:", turn_2, "tokens")
print("Turn 50 cost:", turn_50, "tokens")
print("Turn 50 is", round(turn_50 / turn_2, 1), "x more expensive than turn 2")

Run it. Turn 50 costs roughly 25x what turn 2 costs — because turn 50 resends 49 prior turns as input. This is the token cost blowout, and it's a cost problem, not just a UX one.

Step 3: Add a Summarization Trigger

Once history crosses a token threshold, compress the old turns into a summary and drop the rest.

SUMMARY_THRESHOLD = 40  # tokens -- small on purpose, so it triggers a few times in 50 turns

def summarize(turns):
    goal = turns[0]["content"]
    facts = [t["content"] for t in turns if "plan" in t["content"].lower()]
    return f"Goal: {goal} | Facts: {facts or 'none stated'} | Compressed {len(turns)} turns"

def remember(user_msg, convo, summary):
    convo = convo + [{"role": "user", "content": user_msg}]
    full_text = summary + " ".join(t["content"] for t in convo)
    if approx_tokens(full_text) > SUMMARY_THRESHOLD:
        summary = summarize(convo[:-2])
        convo = convo[-2:]
    return convo, summary

A real summary can't just be "the last N words" — that's how you lose facts that fell in the middle of the conversation. LLMs already attend poorly to the middle of a long context (a property called "lost in the middle"); a lossy summary makes it worse. A summary that survives should always carry four things: the user's goal, key decisions made, open items, and any persistent facts (name, plan tier, stated preferences). Drop any of those and the assistant re-asks a question it already answered.

Step 4: Watch the Cost Curve Bend

Re-run the growing conversation, but through remember() this time.

convo, summary = [], ""
costs = []

for i in range(1, 51):
    convo, summary = remember(f"Question {i}", convo, summary)
    convo.append({"role": "assistant", "content": f"Answer {i}"})
    sent = summary + " " + " ".join(t["content"] for t in convo)
    costs.append(approx_tokens(sent))

print("Turn 2 cost:", costs[1])
print("Turn 50 cost:", costs[49])

Compare costs[49] here to turn_50 from Step 2 — roughly 28 tokens against roughly 666. It doesn't keep climbing — it stays bounded, because old turns keep getting folded into a fixed-size summary instead of piling up forever.

Make It Reusable

remember() is already the reusable piece. In a real app, summarize() is one extra LLM call instead of the string-matching stub above — same shape, smarter compression. Everything else — the threshold check, the token counter, the trigger — stays exactly as it is.

Five Colab Experiments to Try

Set SUMMARY_THRESHOLD = 100 and then 1000. Watch summarization trigger less and less — this threshold is a real production tuning knob.
Put a fact in turn 1 ("I'm on the Enterprise plan"), then run 30 turns and print summary after each compression. It survives the first compression — then quietly disappears on the second one, because summarize() only reads the current window, not the running summary. Watch it happen.
Now fix it: change summarize() to also accept the current summary string and fold its contents into the new one, instead of overwriting it. Rerun the same 30-turn test — the fact should survive indefinitely this time.
Print costs from Step 4 next to [cost_of_turn(n) for n in range(1, 51)] from Step 2. Chart both — that's the cost-vs-turn graph an interviewer is picturing when they ask about scaling memory.
Add a user_memory.json file that saves one fact outside history/summary entirely, and load it into a fresh convo/summary pair on a new run — that's semantic memory (persists across sessions), as opposed to episodic memory (resets with the conversation).

Interview Questions Hidden Inside This Notebook

Why not just use a bigger context window?

Cost and attention. A 200K-token context costs far more per call than a 4K one when full, and "lost in the middle" gets worse, not better, as the window grows.

What's the difference between an in-context buffer and summarization memory?

A buffer resends everything and is simplest but unbounded in cost. Summarization compresses old turns into a fixed-size summary once a token threshold is crossed, keeping cost roughly flat.

What is "lost in the middle"?

Even when a full context fits the window, models recall the beginning and end more reliably than the middle. Summaries have to explicitly preserve key facts — not just compress chronologically — or those facts vanish first.

Is memory design a cost decision or a UX decision?

Both, but cost drives the threshold. Turn 50 on a raw buffer can cost 25x turn 2 — that's a unit-economics problem before it's ever a UX one.

What's the difference between episodic and semantic memory?

Episodic memory is what happened this conversation and resets at session end. Semantic memory is persistent facts about the user (plan tier, preferences) that survive across sessions and get injected as context at the start of a new one.

The Memory Manager Is the Easy Part

The remember() function above is maybe fifteen lines. A production memory layer also has to handle:

context poisoning — a bad instruction from turn 1 surviving into every later summary
composite retrieval queries — for external/vector memory, a follow-up like "show me another one" has no retrieval signal on its own, so the query needs the summary plus the last couple of turns, not just the latest message
validating that summaries never overwrite system rules with user-turn content
knowing when to graduate: buffer for short sessions, summarization once you cross a few thousand tokens, external retrieval only once sessions span hours or multiple visits

Interviewers ask about these because they want to know you've operated a system like this, not just described one.

Want to Go Deeper?

If running this left you asking:

How do I pick a good summarization threshold for my actual traffic?

When do I need external memory instead of summarization?

How do I test that my summary isn't silently dropping facts?

Those are exactly the questions the full session covers — plus the 5-beat interview answer for "how do you manage memory in an LLM application":

https://confidentprep.com/courses/ai-ml-for-interview/4-context-and-memory-management/

And if you'd rather work through this with other developers and ask questions live, join one of the upcoming live sessions:

https://confidentprep.com/live/

Build the small version first. Watch it break at turn 50. Then learn how to explain why — that's a better way to prepare for an AI interview than memorizing another list of memory-management definitions.

AI Is Creating More Opportunities Than We Realize

PandeyC — Tue, 14 Jul 2026 16:35:44 +0000

The biggest fear around AI is simple: If two people with AI can do the work of ten people, what happens to the other eight?

It is a valid concern. AI will automate tasks. Some teams will shrink. Some skills will lose value.

But almost every discussion about AI replacing jobs makes one major assumption:

The amount of software the world wants to build will remain constant.

I don't think it will.

When building becomes dramatically cheaper, we don't just build the same things with fewer people. We build more things.

1. One Startup Needed 10 People. Now One Founder Can Build Four Startups.

Imagine that building a software company previously required ten people. Today, AI coding agents, cloud platforms, automation, and AI-powered support might allow two or three people to build the same product.

The obvious conclusion is: eight jobs disappeared.

But consider another possibility. What if the founder who could previously afford to build one startup can now launch four products in parallel?

Before: 1 startup × 10 people = 10 opportunities
After: 4 startups × 2–3 people = 8–12 opportunities

The individual companies became smaller, but the number of companies, products, and experiments increased.

We are already seeing early signs of this. In 2026, the startup JustPaid reportedly created a team of seven AI coding agents using OpenClaw and Claude Code. In one month, those agents built 10 major features.

Instead of eliminating every human role, the company redirected people toward higher-priority customer work and even hired a new developer who was trained largely by the AI agents.

AI doesn't only reduce the number of people required to build something. It increases the number of things people can afford to build.

2. We Have Seen This Before With Cloud Computing

Before cloud computing, companies needed people to buy and install servers, manage operating systems, configure networks, maintain storage and backups, provision infrastructure, and operate physical data centers.

Cloud computing automated or eliminated many of these responsibilities.

But the technology industry did not disappear. Instead, we created new careers:

Cloud engineers and solution architects
DevOps, platform, and Site Reliability Engineers
Cloud security specialists
Infrastructure automation and Kubernetes engineers
FinOps engineers and cloud consultants

Cloud computing removed work, but it also made technology cheaper and accessible to millions of additional companies.

AI could create the same transformation—much faster.

3. Millions of AI Agents Will Need Humans Around Them

We are moving from AI systems that answer questions to AI agents that take actions.

Agents can deploy infrastructure, modify production systems, process customer requests, approve transactions, access company data, and create cloud resources.

Now imagine an AI cloud agent making the wrong decision. It could delete production infrastructure or create millions of dollars in cloud costs.

Or imagine a support agent processing one million tickets with a 2% serious error rate. That is 20,000 incorrect decisions.

Someone still has to answer critical questions:

Is the agent safe enough to deploy?
Which actions can it perform autonomously?
Which actions require human approval?
How much money or infrastructure can it control?
Who monitors its behavior?
Who investigates failures?
Who stops it when something goes wrong?

The more autonomous AI becomes, the more valuable human judgment becomes.

4. The AI Evaluation and Security Economy Is Just Beginning

Traditional software testing will not be enough. AI systems are probabilistic, and agents increasingly have access to real tools, money, infrastructure, and data.

We will need people and platforms focused on:

AI evaluations and continuous testing
Agent observability and monitoring
Guardrails and permission boundaries
Human-in-the-loop approval systems
AI security and prompt-injection defense
Cost controls and anomaly detection
Audit trails, compliance, and governance
Red teaming and failure investigation
Rollback and recovery mechanisms

Cloud computing created cloud security. APIs created API security. Containers created container security. AI agents will create entirely new security, governance, and operational problems.

Automation does not eliminate responsibility. It increases the scale at which mistakes can happen.

Solving these problems will create companies, products, and careers that barely exist today.

5. The Biggest Advantage Will Belong to People Who Keep Learning

AI is moving incredibly fast. Skills that are valuable today may become automated tomorrow.

That creates an unusual opportunity because nobody has twenty years of experience building production AI agents, agent observability platforms, AI evaluation systems, or AI-native software companies.

The playing field has partially reset.

Young professionals can enter emerging fields before established career paths exist. Experienced professionals can combine decades of engineering and business judgment with powerful AI tools.

The advantage will not automatically belong to the youngest or most experienced person.

It will belong to the person willing to keep learning.

6. Don't Compete With AI. Become Excellent at Using It.

If AI becomes excellent at generating repetitive code, don't build your entire career around writing repetitive code. Move one level higher.

Learn how to:

Build real systems using AI
Evaluate and monitor AI agents
Secure autonomous systems
Design human-in-the-loop workflows
Integrate AI with real businesses
Identify valuable problems worth solving
Use AI to launch products that were previously too expensive to build

The most valuable technology professional of the AI era may not be the person who writes the most code.

It may be the person who knows:

What should we build? How can AI help us build it? How do we know it actually works? And how do we operate it safely at scale?

AI will eliminate some jobs. But it will also make thousands of new products economically possible.

Those products will need to be built, integrated, evaluated, secured, monitored, governed, improved, and turned into businesses.

The amount of opportunity in the world is not fixed.

AI is expanding what individuals and small teams can attempt to build. The biggest opportunity is not in competing with the tool.

It is in becoming exceptionally good at using it.

Want to Start Learning AI?

The best way to prepare for the AI era is to start building, experimenting, and learning how these systems actually work.

If you prefer learning interactively, join the upcoming live AI sessions:
https://confidentprep.com/live/

Or learn at your own pace with self-paced AI courses and practical assignments:
https://confidentprep.com/courses/

Don't wait until AI changes your role to start learning AI. Start learning how to use it now.

This RAG Interview Question Looks Simple. Many Candidates Still Get It Wrong.

PandeyC — Tue, 14 Jul 2026 16:18:15 +0000

You have converted 10,000 documents into embeddings and stored them in a vector database. A user asks a question. Explain exactly how the system finds the most relevant chunks to send to the LLM.

Could you answer this clearly in an interview?

Not just say:

"We perform semantic search."

But explain:

what happens to the user's question
why the question must be converted into an embedding
how it is compared with stored document embeddings
what cosine similarity is doing
how the chunks are ranked
what Top-K retrieval means
and what is finally sent to the LLM

You could easily answer if you read this article:

Build a RAG System in Google Colab Before Your Next AI Interview

The article makes you build the entire retrieval flow yourself:

Documents
    ↓
Chunks
    ↓
Embeddings
    ↓
User Question
    ↓
Query Embedding
    ↓
Cosine Similarity
    ↓
Ranked Chunks
    ↓
Top-K Results
    ↓
LLM Prompt

And this is exactly why building a small RAG system before an AI interview is useful.

Many candidates know the architecture diagram.

Far fewer can explain what actually happens between:

User Question → Vector Search → Retrieved Context

Once you implement it yourself, questions about embeddings, similarity search, Top-K retrieval, chunking, and context construction become much easier to answer.

But here is the follow-up question interviewers may ask:

If the most relevant chunk is ranked #6, but your system retrieves only Top-5 chunks, what happens? How would you improve the system?

Now the discussion moves into:

choosing top_k
retrieval recall
reranking
hybrid search
better chunking
metadata filtering
retrieval evaluation

That is where understanding RAG becomes more important than simply knowing how to call a vector database.

If you're preparing for AI/ML interviews, you can continue learning about embeddings, vector search, and RAG here:

Embeddings and RAG Interview Preparation

And if you'd like to practice these concepts through live discussions and interview questions:

Attend an Upcoming Free Live Session

Build a RAG System in Google Colab Before Your Next AI Interview

PandeyC — Sun, 12 Jul 2026 17:35:47 +0000

You don't need to study machine learning for six months to understand RAG.

You don't need Kubernetes.

You don't need a vector database.

You don't even need an OpenAI API key.

You need:

Google Colab
Python
A few paragraphs of text
About 20 minutes

By the end of this article, you will have built the core retrieval mechanism behind a RAG system.

More importantly, you will understand what is actually happening when someone says:

"We convert documents into embeddings, store them in a vector database, retrieve relevant chunks, and send them to an LLM."

Let's build it.

What Are We Building?

Imagine that we have a small knowledge base about AWS services.

A user asks:

"Which AWS service should I use to decouple applications?"

Our system will:

Break documents into chunks.
Convert the chunks into embeddings.
Convert the question into an embedding.
Compare the question with every chunk.
Retrieve the most relevant chunks.

That is the retrieval part of Retrieval-Augmented Generation.

The architecture looks like this:

Documents
    ↓
Chunking
    ↓
Embeddings
    ↓
Vector Store

User Question
    ↓
Question Embedding
    ↓
Similarity Search
    ↓
Relevant Chunks
    ↓
LLM
    ↓
Answer

In a production application, you might use Pinecone, OpenSearch, pgvector, Qdrant, or another vector database.

For learning, we don't need any of them.

We will store our embeddings in memory and use cosine similarity.

Step 1: Open Google Colab

Create a new Google Colab notebook.

Run:

!pip install -q sentence-transformers

We will use the sentence-transformers library to generate embeddings.

No API key is required.

Step 2: Create Our Knowledge Base

Let's create a tiny collection of documents.

documents = [
    """
    Amazon S3 is an object storage service designed for storing
    and retrieving files. It provides high durability and is
    commonly used for backups, static websites, data lakes,
    and application assets.
    """,

    """
    Amazon SQS is a managed message queue service.

    It allows applications to communicate asynchronously.

    Producers send messages to a queue and consumers process
    those messages independently.

    SQS is commonly used to decouple distributed applications.
    """,

    """
    AWS Lambda is a serverless compute service.

    Developers upload code and AWS executes the code in response
    to events.

    Lambda automatically manages servers and scales applications
    based on incoming requests.
    """,

    """
    Amazon DynamoDB is a managed NoSQL database.

    It provides low-latency access to data and automatically
    scales to handle large workloads.
    """
]

Obviously, real RAG systems contain thousands or millions of documents.

But the mechanism is exactly the same.

Step 3: Chunk the Documents

Why do we need chunks?

Because embedding an entire book or a 200-page PDF as one vector would produce a poor representation for individual questions.

Instead, RAG systems divide documents into smaller pieces.

Let's write a very simple chunker.

def chunk_text(text, chunk_size=250):
    words = text.split()

    chunks = []

    for i in range(0, len(words), chunk_size):
        chunk = " ".join(words[i:i + chunk_size])
        chunks.append(chunk)

    return chunks


chunks = []

for document in documents:
    chunks.extend(chunk_text(document))


print("Number of chunks:", len(chunks))

for index, chunk in enumerate(chunks):
    print(f"\nCHUNK {index}")
    print(chunk)

Production systems usually use more sophisticated strategies.

For example:

token-based chunking
overlapping chunks
recursive text splitting
semantic chunking

But our goal is to understand the mechanism first.

Step 4: Generate Embeddings

Now we need to convert text into vectors.

We will use a small embedding model.

from sentence_transformers import SentenceTransformer

model = SentenceTransformer("all-MiniLM-L6-v2")

Now generate embeddings for our chunks.

chunk_embeddings = model.encode(chunks)

print(chunk_embeddings.shape)

You should see something similar to:

(4, 384)

What does that mean?

We have four chunks.

Each chunk has been converted into a vector containing 384 numbers.

Something like this:

[0.023, -0.041, 0.087, ...]

Humans cannot interpret these numbers directly.

But mathematically, texts with similar meanings tend to have vectors that are closer together.

That is what makes semantic search possible.

Step 5: Ask a Question

Let's ask:

question = "Which AWS service can help decouple applications?"

Convert the question into an embedding.

question_embedding = model.encode([question])

Now we have:

Documents → Vectors

Question → Vector

We need to find which document vectors are closest to the question vector.

Step 6: Perform Similarity Search

We will use cosine similarity.

from sklearn.metrics.pairwise import cosine_similarity

scores = cosine_similarity(
    question_embedding,
    chunk_embeddings
)[0]

Let's inspect the results.

for index, score in enumerate(scores):
    print(round(score, 3), chunks[index][:100])

You should see the SQS document receive the highest similarity score.

Now retrieve the best chunk.

best_chunk_index = scores.argmax()

retrieved_chunk = chunks[best_chunk_index]

print(retrieved_chunk)

Congratulations.

You just built semantic retrieval.

This is the foundation of a RAG system.

Let's Make It Reusable

Let's wrap everything into a function.

def search(question, top_k=2):

    question_embedding = model.encode([question])

    scores = cosine_similarity(
        question_embedding,
        chunk_embeddings
    )[0]

    top_indices = scores.argsort()[::-1][:top_k]

    results = []

    for index in top_indices:
        results.append({
            "score": float(scores[index]),
            "text": chunks[index]
        })

    return results

Now try asking different questions.

questions = [
    "Where should I store application files?",
    "How can I run code without managing servers?",
    "Which database provides low latency access?",
    "How can microservices communicate asynchronously?"
]

for question in questions:

    print("\nQUESTION:", question)

    results = search(question)

    for result in results:
        print(
            round(result["score"], 3),
            result["text"][:150]
        )

You now have a tiny semantic search engine.

But Where Is the LLM?

This is where many developers misunderstand RAG.

The vector database does not answer the question.

The embedding model does not answer the question.

The retrieval system finds relevant information.

The retrieved information is then placed into the prompt sent to the LLM.

Conceptually:

def build_prompt(question, retrieved_chunks):

    context = "\n\n".join(retrieved_chunks)

    prompt = f"""
    Answer the question using only the context below.

    CONTEXT:

    {context}

    QUESTION:

    {question}
    """

    return prompt

Let's generate the prompt.

question = "Which AWS service should I use to decouple applications?"

results = search(question)

retrieved_chunks = [
    result["text"]
    for result in results
]

prompt = build_prompt(
    question,
    retrieved_chunks
)

print(prompt)

This prompt could now be sent to an LLM.

That completes the RAG pipeline:

Documents
    ↓
Chunks
    ↓
Embeddings
    ↓
Vector Search
    ↓
Relevant Context
    ↓
Prompt
    ↓
LLM
    ↓
Answer

Five Experiments You Should Try

Don't stop after running the notebook.

Change it.

First, add documents that use similar terminology.

For example:

SQS decouples applications.

EventBridge connects applications using events.

SNS distributes messages to multiple subscribers.

Does the correct document still rank first?

Second, change the chunk size.

Try:

chunk_size = 50

Then:

chunk_size = 500

What happens to retrieval quality?

Third, retrieve more documents.

Change:

top_k = 1

to:

top_k = 3

Would sending more context always produce a better answer?

Fourth, add irrelevant documents.

Does retrieval quality change?

Fifth, ask ambiguous questions.

For example:

Which service should I use for messaging?

Now you are starting to encounter the problems that real RAG systems must solve.

The Interview Questions Hidden Inside This Project

If you understand the notebook above, you should be able to discuss several common AI interview questions.

Why do RAG systems chunk documents?

Because retrieval usually needs to identify specific passages rather than entire documents.

What is an embedding?

A numerical representation of data that allows semantic relationships to be compared mathematically.

Why use cosine similarity?

Because it measures the direction between vectors and is commonly used to compare embedding similarity.

Does a vector database generate answers?

No.

It stores vectors and helps retrieve relevant information.

The LLM generates the answer.

Does retrieving more chunks always improve the answer?

No.

More context can increase cost, introduce irrelevant information, and make it harder for the model to identify the useful evidence.

What happens when the correct document isn't retrieved?

The LLM never receives the information required to answer correctly.

This is why evaluating retrieval quality is one of the most important parts of building a production RAG system.

The 20-Line Demo Is the Easy Part

You can build a RAG demo in twenty minutes.

Production RAG systems are harder.

You have to make decisions about:

chunk size and overlap
embedding models
metadata filtering
top-k retrieval
similarity thresholds
hybrid search
reranking
context-window management
hallucination control
retrieval evaluation
answer evaluation

And interviewers increasingly ask about these trade-offs.

Not because they expect you to memorize definitions.

They want to know whether you understand how the system behaves.

Want to Go Deeper?

If you ran this notebook and found yourself asking:

How should I choose chunk size?

When should I use a vector database?

How do I know whether retrieval is actually working?

What is the difference between semantic search, hybrid search, and reranking?

How would I design this system for thousands or millions of documents?

Those are exactly the questions worth exploring next.

I'm covering embeddings, vector search, RAG architecture, and the engineering decisions behind production AI systems on ConfidentPrep.

You can explore the deeper RAG learning material here:

https://confidentprep.com/courses/ai-ml-for-interview/3-embeddings-and-rag/

And if you'd rather learn by discussing these concepts live, asking questions, and working through interview scenarios with other developers, join one of the upcoming live sessions:

https://confidentprep.com/live/

Build the small version first.

Understand why it works.

Then learn how to explain the trade-offs.

That's a much better way to prepare for an AI interview than memorizing another list of 100 RAG questions.

Your New Prompt 'Feels' Better. That's Not an Eval.

PandeyC — Sat, 11 Jul 2026 14:59:57 +0000

You tweak the prompt. Run it against the three examples you always use to sanity-check. It looks better. Ship it.

That's not evaluation. That's vibes with extra steps — and it's the exact habit interviewers are trained to catch with one question.

🎯 TL;DR: One interview question, one real framework: don't trust "it looks better on my usual examples" — build a small eval set, score it the same way every time, and re-run it on every future change. Read the framework, then use it as the literal answer next time someone asks.

🎭 The question: "How do you know your new prompt is actually better — not just better on the five examples you tried by hand?"

This is one of the fastest ways an interviewer separates "I iterate on prompts" from "I've actually shipped prompt changes to production." Everyone iterates. Almost nobody evaluates.

How to answer it:

Name the trap first, out loud: eyeballing a handful of hand-picked examples is confirmation bias, not evaluation. You'll always find a few cases where the new version looks better — that's not evidence, that's cherry-picking with good intentions.
Describe a minimal eval set: 20-50 real or realistic input/expected-output pairs, run automatically against both the old and new prompt, scored the same way every time — not re-read by eye each round.
Say what "scored" actually means, because this is where most candidates go vague: exact match works for structured output (JSON fields, classifications); for open-ended text, you need a similarity or grounding score against a reference answer.
Close with the regression angle: the eval set isn't a one-time gate before this ship — it's something you re-run on every future prompt or retrieval change, so a "small tweak" three weeks from now doesn't quietly break something the old version got right.

🕳️ Why "it looked better to me" is losing you the room

Here's the uncomfortable part: this question isn't really testing whether you know what an eval set is. It's testing whether you've been burned by not having one — shipped a "better" prompt that regressed silently, found out from a user complaint instead of a dashboard. Candidates who've lived that answer differently than candidates who haven't, and interviewers can usually tell within one follow-up question.

Somewhere in your interview pool right now, another candidate already built that eval habit into their process — not because a course told them to, but because something broke on them once. That's the gap this question is actually measuring.

🔧 What the eval set looks like when it's not hypothetical

Skip the abstraction. In practice it's a spreadsheet or a JSON file with columns: input, expected/reference answer, prompt-A output and score, prompt-B output and score. Twenty rows is enough to start. The moment you can point to a number instead of a feeling, you've already answered this question better than most candidates do.

🔬 Next: the 12 lines of code that do the scoring

Saying "similarity score" in an interview is good. Being able to sketch the actual code is better — and next week's post is exactly that: a Colab-ready snippet that scores whether a generated answer is actually grounded in retrieved context, the same technique behind the eval-set scoring above.

The takeaway: "it looks better" is an opinion. A score you can re-run is an answer.

[See the paths that build that judgment →]
https://confidentprep.com/paths

🔴 Preparing for AI interviews? Don’t prepare alone. Join a live session where we break down real interview questions like this one, discuss how strong candidates structure their answers, and work through the follow-up questions interviewers actually ask. Bring your questions, test your answers, and learn live with other candidates preparing for AI roles.

👉 Join the next live AI interview prep session →
https://confidentprep.com/live/.

Stop Learning Machine Learning Before GenAI 🤖

PandeyC — Sat, 11 Jul 2026 04:25:49 +0000

Yes, you read that right.

If your goal is to understand Generative AI, build LLM-powered applications, or prepare for a GenAI interview, you don't need to finish learning Machine Learning first.

Yet many developers get stuck here.

Should I learn statistics first?
Then Machine Learning?
Then Deep Learning?
Then neural networks?
Then transformers?
And finally Generative AI?

This roadmap may make sense if your goal is to become an ML Engineer, Data Scientist, or AI researcher.

But for many software developers and technology professionals, it creates an unnecessary barrier.

You keep preparing to start.

But never actually start.

🚧 Don't Let the AI Roadmap Block You

Machine Learning is a large and valuable field.

But you don't need to master regression, classification algorithms, backpropagation, or the mathematics of neural networks before you can understand how modern GenAI applications work.

Start with a simpler question:

What do I need to understand to build and discuss a GenAI application?

The answer is much more approachable.

🧠 Start With Generative AI Fundamentals

Understand the basic concepts first.

What is Generative AI?

What is an LLM?

What is a prompt?

What are tokens and context windows?

Why do LLMs hallucinate?

How does an application communicate with an LLM?

What happens when you send a prompt and receive a response?

You don't need to understand every mathematical detail behind the model.

But you should be able to explain what these concepts mean, why they matter, and how they affect real applications.

That's a good starting point.

🛠️ Then Build Something Small

Call an LLM API.

Send a prompt.

Get a response.

Change the prompt and observe what happens.

Experiment with model parameters.

Then build a small application.

For example:

A document summarizer
A question-answering application
A chatbot
A structured data extraction tool
A basic RAG application

While building, you'll naturally encounter new questions.

How do I provide my own data to an LLM?

What are embeddings?

Why do I need a vector database?

How should documents be chunked?

How do I reduce hallucinations?

How do I evaluate the quality of responses?

How do I protect an application from prompt injection?

Now you have a reason to learn these concepts.

You're learning because you need to solve a problem—not because a massive AI roadmap told you to learn everything first.

🎯 Preparing for a GenAI Interview?

The same principle applies.

Don't wait until you know everything about Machine Learning before preparing.

Start with the questions that help you understand the GenAI application landscape.

Can you explain how an LLM-powered application works?

Can you explain tokens, context windows, and hallucinations?

Do you understand the difference between prompting, RAG, and fine-tuning?

Can you explain why an application might use embeddings and a vector database?

Can you discuss security, cost, latency, evaluation, and reliability?

Can you describe something you have built, even if it is small?

These questions give you a practical direction.

⚠️ Does This Mean Machine Learning Is Not Important?

No.

Machine Learning fundamentals become increasingly important depending on the role you are targeting.

If you want to train models, work deeply with model architectures, become an ML Engineer, or pursue AI research, you will need stronger foundations in Machine Learning, mathematics, and statistics.

But that's different from saying:

Everyone must learn Machine Learning before they can start learning Generative AI.

They don't.

For many developers, architects, testers, DevOps engineers, and other technology professionals, starting with GenAI applications is a perfectly reasonable path.

🌱 Start First. Go Deeper When You Need To.

The AI ecosystem is enormous.

You can spend months creating the perfect learning roadmap.

Or you can start.

Understand what Generative AI is.

Learn the fundamentals.

Build something small.

Prepare for practical interview questions.

Discover your knowledge gaps.

Then go deeper.

If you're preparing for a GenAI interview and don't know where to begin, I've put together a structured guide covering the concepts and questions worth exploring:

👉 Start Preparing for Your AI Interview

Don't let the size of Machine Learning stop you from starting with Generative AI.

Start small. Understand the fundamentals. Build something. Then go deeper. 🚀

→ For more details, see here.