DEV Community: confident_prep

How training and inference work: build and test a model by hand

confident_prep — Tue, 20 Oct 2026 02:23:51 +0000

You want to train your first model, but "training" and "inference" still sound like the same fuzzy thing.

This article argues that the fastest way to make those words concrete is to train and test a tiny model yourself, on paper, before you open any tool. You will run a learning procedure over six examples, get a model out of it, score it on examples it has never seen, and then break it on purpose so you can watch overfitting happen. Every no-code tool hides these steps behind buttons. If you have done them once by hand, you will know what each button is doing and what its scores mean.

You need a pen, a sheet of paper and nothing else

There is nothing to install, no account to open and no code to write. The exercise costs nothing, and cleanup means putting the sheet in a drawer. The aim is not a useful model. The aim is to watch four separate things happen in order.

Four words name those things, and the exercise depends on keeping them apart:

Algorithm: the learning procedure, the recipe that turns example data into a model.
Training: the stage where the algorithm runs over the data.
Model: the trained result, the thing that takes a new input and produces a prediction.
Inference: the stage where the trained model is used on new input.

Think of it as a bakery. The recipe is the algorithm, baking is training, the loaf is the model, and serving a slice is inference. The recipe is not the loaf. The loaf is not being baked while someone eats it.

An algorithm trained on data produces a model, and the model alone answers new input

Step 1: Write down labeled training data

Draw a small table. Each row is a delivery, with its distance and whether it arrived late. The figures are invented for this exercise and describe no real company.

Six labeled deliveries: the late column records the correct answer for each row

Notice two things about this table. First, it is labeled data: the correct answer, late or not, is recorded for every example. Because the answers are recorded, you can use supervised learning, which means learning from examples paired with their correct outputs. If nobody had written down which deliveries were late, this route would be closed, however many rows you had.

Second, it is structured data: it follows a fixed schema of rows and columns. A pile of scanned delivery notes would hold the same facts as unstructured data, with no set data model, and it would need far more preparation before any learning could start.

Below the table, draw a line and write four more rows you will not look at during training: 3 km, not late; 6 km, not late; 11 km, late; 14 km, late. This is your test data. Covering it up is the most important move in the exercise. It is the only way to find out whether your model learned the pattern or just learned the six rows.

Step 2: Run the algorithm and get a model

Here is your algorithm, written as plain instructions:

Try a cut-off distance between each pair of neighbouring distances.
For each cut-off, predict "late" for every delivery farther than it.
Count how many of the six training rows that prediction gets wrong.
Keep the cut-off with the fewest mistakes.

Now run it. A cut-off at 8.5 km predicts late for E and F only. It gets C wrong, because C was late at 5 km, and gets everything else right: one mistake. A cut-off at 4.5 km also makes exactly one mistake, on D. Neither does better, so choose 8.5 km as your tie-break and move on.

What you are holding now is a model: the single sentence "late if distance is over 8.5 km". It is not the four-step procedure. It is what the procedure produced. If you collect more deliveries next month and want a better cut-off, you run the algorithm again over the new data. That is retraining, and it is the only way this model changes.

Write the training score next to the model: 5 out of 6 correct.

Step 3: Uncover the test rows and score the model

Uncover the four test rows and apply your sentence to each one. At 3 km it predicts not late, which is correct. At 6 km it predicts not late, also correct. At 11 km and 14 km it predicts late, and both are correct. That gives 4 out of 4.

You now have two numbers: one from the data the model learned on, one from data it never saw. Reading those two numbers together is the core skill in judging a model. A good score on training data alone tells you very little. The model has already seen those answers.

Training happens once per build; inference repeats every time the model is used

Step 4: Use the model on a new delivery

A new delivery comes in at 9 km. Look at your sentence, and nothing else: over 8.5 km, so predict late. That is inference. You did not look at the training table. You did not rerun the four steps. The algorithm and the training data play no part at this stage. Only the model does.

This is the distinction that tends to stay abstract until you do it by hand. Training is effortful and happens when the model is built or rebuilt. Inference is quick and happens every time the model is used. A common mistake runs straight across that line: proposing to "retrain" when the real problem is how answers are being delivered to the people who need them.

Step 5: Break it on purpose by memorizing

Now build a second model with a lazier algorithm: "copy the training table exactly; for any distance not in the table, answer not late."

Score it on training data and it gets 6 out of 6, a perfect result. Every answer it gives was copied straight from that table. Score it on the test rows and the picture changes. None of 3, 6, 11 or 14 km appears in its table, so it answers "not late" four times. That is right twice and wrong twice: 2 out of 4.

This is overfitting: a model that scores well on its training data and poorly on new data, because it learned the examples, quirks and all, rather than the pattern behind them. Think of a student who memorized last year's answer sheet and is lost on a new paper. Notice that the memorizer even reproduces delivery C, the odd late one at 5 km. Your cut-off model sensibly got C wrong, and that one training mistake is part of why it held up on new data.

For completeness, try the opposite failure. Use a model that always says "not late". It gets 3 of 6 on training and 2 of 4 on test. That is low on both, which is underfitting: a model too simple to find the pattern at all.

The gap between training and test scores tells overfitting from underfitting

The rule to keep is a number test. High on training and low on new data means overfitting. Low on both means underfitting. Good on both means a good fit. The fix for overfitting is to make the model generalize better, for example by making it less flexible. Regularization is one name for techniques that do this. Your one-sentence cut-off is about as inflexible as a model can be, which is exactly why it beat the memorizer.

Step 6: Decide how the model would answer people

Your model works. The next practical question is how it would be put to use. There are four common ways to serve a trained model, and the deciding question is about the people asking, not about the model.

Real-time: someone is waiting for the answer right now, such as a dispatcher on the phone.
Serverless: requests come in bursts with idle gaps between them, and a slower first answer after a quiet spell is acceptable, because capacity scales down to nothing when idle.
Asynchronous: each request is large or slow, so it is queued, and the result is collected later from a storage location.
Batch: a whole dataset is scored in one job, such as every delivery on tomorrow's schedule, and nothing needs to stay running between jobs.

Think of it as a post office: a counter for people who are waiting, a sack of mail sorted overnight, and a large parcel left to be picked up later.

Who is waiting decides first, then dataset shape, then request size, then traffic pattern

The pairing that most often causes confusion is asynchronous against batch. A huge file that belongs to a dataset scored in one run is still batch. A huge file that arrives on its own and joins a queue is asynchronous. A second trap: data that arrives continuously is not the same as a person waiting for an answer. Continuous arrival on its own does not call for real-time serving.

Why the paper version transfers

In our view, any no-code tool you use next will ask you these same questions, in different words. Which column holds the answer you want predicted? How should the data be split for testing? What do the two scores look like? How will predictions be requested? If you know that the column you pick is the label, that "build" means training, and that "predict" means inference, then the tool's screens become easier to read.

The exercise also builds a habit worth keeping. Ask whether the outcome was recorded before you ask anything else. Asking "is it labeled?" closes off or opens up supervised learning. Asking "how much data is there?" settles nothing on its own. Then compare two scores before trusting either. Then decide serving from who is waiting.

Key takeaways

The algorithm is the procedure, the model is its output. Training builds the model, and inference only uses it, many times over.
Supervised learning needs labeled data. Whether the correct answer was recorded matters more than how many rows you have.
Always hold back test data. A model is judged by comparing its training score with its score on examples it never saw.
High-then-low means overfitting, low-on-both means underfitting. A model that memorizes its examples can look perfect and still fail on new cases.
Choose a serving mode from who is waiting, whether a whole dataset is scored at once, how large each request is, and how bursty the traffic is.

This article settles the concepts behind training and testing a first model, worked through by hand. It does not walk through the screens, settings or charges of any particular no-code tool. Those vary by product, and checking them against the tool's own current documentation is the honest next step.

How would you make sure an AI system is responsible?

confident_prep — Mon, 19 Oct 2026 07:06:58 +0000

An interviewer asks how you would make sure an AI system is responsible. Most candidates answer with values, but the credit goes to answers built on properties.

This article argues that responsible AI, as a widely used AI practitioner exam guide frames it, is a short list of things you can check. It names six features, and each one fails in a way you can see. A guardrail service can show evidence for two of them. Model selection has its own responsible practices. Five legal risks describe what the business pays when the features are missing. Learn the features and the risks as one list read from two ends, and you can answer both the interview question and the follow-ups. Those follow-ups are written to catch anyone who answers from good intentions.

The strong answer names properties, not good intentions

A weak answer says the system was built responsibly and points to a policy document. A strong answer runs in five beats, in this order:

Name the properties you claim: bias, fairness, inclusivity, robustness, safety and veracity.
Say what you would observe if each one failed.
Name how you enforce them and say plainly what that tooling cannot see.
Bring responsibility into model selection, not only into operation.
State the exposure in business terms, so a decision-maker can act on it.

The second beat separates a designed answer from a sincere one. A property you cannot detect is a value, not a control. Think of it as the difference between a smoke alarm and a fire-safety poster. Both say they care about fire, but only one tells you when there is a fire.

Each responsible AI feature is identified by the failure you would observe

Every option in a question on this topic describes something undesirable. That is why common sense fails here. Credit goes to the option that names the property the guide uses.

Follow-up: bias is the measurement, fairness is the standard

"Aren't bias and fairness the same thing?"

Model answer: No. Bias is the measurable skew: outcomes differ across groups in a way the individual records do not explain. You can compute it. Fairness is the standard you use to decide whether that skew is acceptable. You cannot compute fairness. You have to decide it. So a system can be biased before anyone has agreed it is unfair. A system can also be judged unfair over a small skew.

The weak candidate's mistake: treating the two words as synonyms, so that either one seems to fit. Use a simple test. A scenario that says "outcomes differ by group" describes bias. A scenario that asks "is this difference acceptable?" is asking about fairness.

For example, picture a lending assistant that gives more cautious eligibility guidance to customers who write in one language than to customers who write in another. The gap in outcomes is the bias. Whether the organisation accepts that gap is the fairness question.

Follow-up: safety, veracity and robustness fail in different ways

"The model gave a wrong answer. Is that a safety problem?"

Model answer: Not necessarily. Each of the three properties asks a different question:

Safety is about harm. The output is abusive, dangerous, or in a category the application should refuse.
Veracity is about truth. The output is wrong.
Robustness is about stability. The system behaves well on clean input and erratically on misspellings, mixed languages, odd formatting or deliberate attack.

Take a model that confidently states a drug interaction that does not exist. That is a veracity failure. If a user acts on it and is harmed, the harm is a separate consequence, and the guide gives it its own name: end user risk, which is one of the legal risks covered below.

The weak candidate's mistake: reasoning "it was wrong, so it was unsafe," or calling erratic output on messy input an accuracy failure. Instability under unexpected input is robustness, even when the answers are also wrong.

Inclusivity completes the six. It asks whether everyone the system is meant for can actually use it. A model that works well in only one language, or that assumes literacy or connectivity its users lack, has an inclusivity problem. That is true even if it is perfectly accurate for the users it does serve.

Three bad-output properties, separated by the question each one asks

Follow-up: guardrails check two points and show evidence for only two features

"Which tool would you use to show these properties?"

Model answer: The guide's named example is Amazon Bedrock Guardrails. It evaluates what goes into a model and what comes out against policies you configure, and blocks or filters anything a policy prohibits. It acts at two points:

On input, before the model is called. A prohibited request can be stopped before it ever reaches the model.
On output, before the user sees it. A response can be blocked or masked.

It has five policy types:

Content filters cover hate, insults, sexual content, violence and misconduct, at a strength you set.
Denied topics define subjects the application will not discuss at all.
Word filters block specific words and phrases.
Sensitive information filters detect personally identifiable information and mask or block it.
The contextual grounding check confirms that a response is supported by the source material provided and is relevant to the query.

The first four provide evidence for safety, and the sensitive information filters also cover privacy. The grounding check is the only policy that addresses truth rather than harm, which makes it the veracity policy.

Guardrails evaluate input before the model runs and output before the user sees it

The weak candidate's mistake: claiming more than the tool does. Guardrails do not measure bias across demographic groups, because no policy computes differences in outcomes between groups. They do not assess inclusivity or robustness. They do not retrain the model. They sit around the model and evaluate its traffic, and the model's weights are untouched. They do not document the model either. Recording intended uses and limitations is the job of a model card, a separate document.

Saying these limits without being asked is the third beat of the strong answer. It is also what makes the rest of the answer believable. One more point is worth adding. Retrieval-augmented generation supplies source material to the model, which makes a grounded answer possible. The grounding check is what verifies that the model actually used that material.

Follow-up: sustainability counts when you choose a model

"How does responsibility affect which model you pick?"

Model answer: The guide names environmental considerations and sustainability as responsible practices in model selection. In practice there are four:

Prefer the smaller model when it is sufficient. Energy and compute grow with model size, so a larger model that does the same job spends more for no gain.
Adapt an existing model rather than train a new one. Training is the largest energy cost in a model's life.
Consider the provider's position, because infrastructure efficiency and energy sourcing differ.
Write down the rationale. A choice you cannot explain is not a responsible practice, whatever you chose.

The order matters. The responsible screen runs after capability and cost, and it chooses only among models that already meet the requirement. Efficiency never rescues a model that cannot do the job.

The responsible screen chooses among models that already meet capability and cost

The weak candidate's mistake: dismissing sustainability as a soft, non-technical concern, or defending the bigger model as "headroom for the future." When a smaller model is enough, the larger one is the less responsible choice. One way to remember this: the usual ordering of customisation options, from prompting through retrieval and fine-tuning up to full pre-training, is ordered by cost, and it is ordered by energy too. "Don't fine-tune when prompting would do" is also the sustainable answer.

Follow-up: the legal risks are business exposures with three routes in

"What legal risks does generative AI create?"

Model answer: The guide names five: intellectual property infringement claims, biased model outputs, hallucinations, end user risk, and loss of customer trust. The way to frame them matters more than the list. These are exposures for the organisation, not bugs in the model.

It helps to sort them by where the exposure enters:

What went in. Infringement claims arise when training data or generated output reproduces protected work. One example is an assistant that copies sentences word for word from a competitor's brochure.
What came out. Biased outputs disadvantage a group. Hallucinations are confident, fluent and untrue.
What someone did with it. End user risk arises when a user relies on output in a setting where being wrong has consequences, such as medical, legal, financial or safety-critical decisions. Loss of customer trust is where all of these accumulate.

The five legal risks sorted by where the exposure enters

The weak candidate's mistake: offering a technical fault such as "overfitting" or "a robustness failure" as a legal risk. A second mistake is waving hallucination away as an evaluation issue. Hallucination is on the list on purpose. The technical event is a wrong answer. The legal event is that someone relied on it. That is why hallucinations and end user risk so often appear together: they are the two ends of a single incident.

Loss of customer trust differs from the other four. It is not a separate kind of incident. It is the consequence that outlasts the incident. You can fix a single hallucination, but fixing it does not repair the belief that the product cannot be trusted.

Follow-up: features and risks are the same list read from two ends

"How do the features relate to the risks?"

Model answer: The features name the properties a system should have. The risks name what it costs when they are missing. A missing veracity property shows up as hallucinations, which can lead to end user risk. Gaps in fairness and bias show up as biased model outputs. A safety failure becomes end user risk when someone acts on harmful output, and so does a robustness failure on unexpected input. An inclusivity gap becomes lost trust among the users who cannot use the system. Every one of these ends in loss of customer trust.

Each missing feature becomes a named legal risk, and all of them end in lost trust

The weak candidate's mistake: treating the features and the risks as two topics to memorise separately. Once you learn the mapping, a question that describes a harm and asks for the feature, and a question that describes a gap and asks for the risk, become the same question. In our view, this mapping is the single most useful thing to take into either an exam room or an interview.

Key takeaways

Responsible AI here means six observable features: bias, fairness, inclusivity, robustness, safety and veracity. Learn each one by the failure that reveals it.
Bias is the measurement and fairness is the standard. Harm is safety, truth is veracity, and stability under unexpected input is robustness.
Guardrails check input and output, and show evidence only for safety and veracity. The contextual grounding check is the veracity policy. Guardrails do not measure bias or change the model.
Sustainability is a real criterion for model selection. Apply it after capability and cost, and write down the rationale.
The five legal risks are business exposures. Hallucination and end user risk are two ends of one incident, and loss of customer trust is where all the risks accumulate.

This article settles how to name what is wrong with a system. It does not settle how to prove it. Showing that bias exists takes measurement, such as dataset analysis, subgroup comparisons and human audits, and none of that is covered here.

What is an agentic AI system actually made of?

confident_prep — Fri, 16 Oct 2026 09:39:42 +0000

The interviewer leans back and asks, "So what is an agent?" Most candidates say "an LLM that can use tools," and the interviewer nods without writing anything down. The same thing happens on foundational AI certification exams. A question offers four options that each sound plausible, and the wrong ones are built from that half-right answer.

This article claims that a strong answer to "what is an agentic AI system made of?" covers five parts in a fixed order: the loop, tools, connection, memory and coordination. It also ends by saying when not to add more agents. Each part has one confusion that examiners and interviewers probe on purpose. Most of those confusions come from not asking what sits on each end of a connection. If you can explain the five parts and their boundaries, you can handle the follow-up questions that come after the first one.

A strong answer names the loop before any technology

Here is the opening answer a strong candidate gives.

"An agent is a system that uses a foundation model to decide which actions to take toward a goal. It takes those actions, reads what came back, and decides again. It does not produce one answer in one pass. The defining property is that loop. If nothing can come back and change its mind, it is a single model call, however good the output is."

A foundation model is a large model trained on broad data that many applications build on. The loop is the important part of that answer. You could replace every technology in a system and it would still be an agent if the loop remained.

Think of it as the difference between a researcher with a library card and someone answering from memory. The researcher looks things up, and what they find changes their next question. Someone answering from memory can never be surprised.

An agent decides, acts, observes the result, then decides again or answers

The branch after the result is the agent: act again, or answer. The model itself gains no new ability. The new capability is that results come back in as context.

Follow-up 1: the model requests, the runtime executes

"When an agent uses a tool, what does the model actually do?"

Model answer: Tool usage means giving a model access to external functions, APIs or data sources. The model decides that a tool is needed and which arguments to pass. The runtime is the software around the model, and it makes the actual call and returns the result into context. The model then interprets that result and decides what happens next.

The weak-candidate mistake: saying "the model queries the database" or "the model books the slot." Any answer in which the model executes something has the mechanism backwards. To an interviewer, it signals that you have never watched one run.

Think of it as a doctor ordering a blood test. The doctor does not run the lab. They order the test, the lab runs it, the result comes back, and the diagnosis may change.

The model decides and interprets; the runtime executes and returns the result

Tools are worth this machinery because they let an agent be correct about things that changed after training. Suppose a user asks for the stock level of a part. Without a tool, the model generates a plausible number. With a tool, the runtime queries the real one. In both cases the model sounds equally confident, and that is exactly why the difference matters.

Follow-up 2: MCP connects an agent to external systems

"What is the Model Context Protocol, and what problem does it remove?"

Model answer: The Model Context Protocol (MCP) is an open protocol. It standardises how an AI application connects a model to external tools and data sources, so there is one common interface instead of a custom integration for every system. The problem it removes is multiplicative. Three applications that each connect to four systems need twelve separate integrations. With a shared protocol, each side is implemented once, which makes seven. Adding a fifth system then costs one implementation instead of three.

Think of it as USB-C. Every device used to need its own cable. With one connector standard, any device can meet any peripheral, and neither side has to know about the other in advance.

Bespoke integration grows with every pair; a shared protocol needs one implementation per side

The weak-candidate mistakes, in order of how often they appear:

"MCP lets agents talk to each other." It does not. It connects an agent to external systems. Agent-to-agent exchange is a separate concept called communication patterns.
"MCP is a product or managed service." It is an open protocol. Cloud services can speak it, which is a different claim. It never belongs in a list of services to select.
"You need MCP to use tools." Tool usage existed before MCP and works without it. With two or three connections, custom integration is tolerable. The case for a shared protocol grows as the number of pairs grows.

One question settles most of these: what is on each end of the connection? If it is an agent and an external system, that is MCP. If it is an agent and another agent, that is a communication pattern.

Follow-up 3: memory is a decision about what survives

"How does an agent remember things?"

Model answer: The model is stateless between calls. Earlier turns are re-sent each time, so memory is never automatic and never free. Memory management means deciding what the agent carries forward, for how long, and what it discards. Working memory holds the current task, such as recent turns and the last tool result, and is discarded when the task ends. Persistent memory holds durable facts, such as a customer's preferences or prior outcomes. It is stored outside the model and retrieved selectively.

The weak-candidate mistake: "Just give it the whole conversation history." Everything carried forward is re-supplied and counted again on every call. Cost and latency grow, and what matters gets diluted. Retention is also a risk: a one-off verification code kept in durable storage is a liability, not a feature.

Think of it as a hospital shift handover. Not everything from the shift is passed on. What matters carries over, and the handover is a summary, not a transcript.

Durable facts go to persistent memory; task-local material stays working and is discarded

So the useful question is not "how do we remember more?" It is "what must survive, and what must not?"

Follow-up 4: arrangement and message flow are separate questions

"Name some multi-agent patterns." Weak candidates answer this with one list. Strong candidates ask which kind of pattern is meant.

Model answer: A multi-agent system uses several specialised agents where one general agent would be stretched too thin. Two independent questions apply to it.

The system pattern is how the agents are structured:

Supervisor: a lead agent decomposes the task, delegates to specialists and assembles the result.
Sequential: each agent's output is the next agent's input.
Parallel: independent agents work at the same time and their results are merged.
Hierarchical: supervisors of supervisors.

The communication pattern is how the agents exchange information:

Direct (point-to-point): one agent calls another and waits.
Broadcast: one agent informs many at once.
Shared state: agents read and write a common store.
Message queue: messages are passed asynchronously.

The weak-candidate mistake: answering "supervisor" when asked how agents exchange information. That answers a different question.

Think of it as an organisation. The system pattern is the org chart. The communication pattern is how the memos move. A supervisor system can pass messages point-to-point or through shared state, and choosing one does not determine the other.

System patterns describe arrangement; communication patterns describe how information moves

Follow-up 5: orchestration owns the plan, including when to stop

"What is workflow orchestration, and who should own it?"

Model answer: Workflow orchestration is coordinating the steps of an agentic application. It covers sequencing, routing, handling failures and retries, and deciding when the task is finished. That last duty is easy to forget. The plan can be owned in one of two ways. In deterministic orchestration, the developer defines the workflow. It is predictable, testable and auditable, but it cannot adapt to the unforeseen. In model-driven orchestration, the model chooses each next step. It adapts to new situations, but its path varies between runs.

The weak-candidate mistake: treating model-driven orchestration as the sophisticated default. In a regulated or audited process, "the path varies between runs" is a defect.

Think of it as an air traffic controller. Each pilot flies their own aircraft, and the controller decides who moves, in what order, and when the sequence is complete.

Follow-up 6: one agent with good tools is the default

"When would you use several agents instead of one?"

Model answer: Use several agents only when a task spans genuinely distinct specialisms, and be able to name the reason. A task that is long but stays within one specialism is still one agent doing more steps. Every added agent multiplies four things: model calls, which each carry a cost; latency, from steps that wait on each other; failure modes, since any agent can stall or pass on a bad result; and debugging effort, because "which agent decided that?" becomes a real question.

The weak-candidate mistake: reaching for multiple agents because the scenario used the word "complex."

Here is an illustrative case. A shipment assistant answers "where is my parcel?" by reading a tracking database. It must remember each customer's preferred notification channel. When a shipment is disrupted, its recommendation needs pricing, customs and capacity expertise. The considered design has two paths. The plain lookup goes to one agent with tools. Only the disruption case goes to a supervisor that coordinates three specialists. Sending the simple query down the multi-agent path adds cost and latency and gains nothing.

Bounded single-specialism work stays with one agent; distinct specialisms justify several

Key takeaways

An agent is defined by its loop. If no result can come back and change the decision, it is a single model call.
The model requests; the runtime executes. Reject any answer in which the model itself queries, calls or books something.
Ask what is on each end of the connection. Agent to external system is MCP. Agent to agent is a communication pattern. MCP is a protocol, not a product, and tools work without it.
Memory is a choice about what survives. Separate working memory from persistent memory. "Keep everything" is a cost, not a safe default.
Structure and message flow are separate questions, and orchestration owns the plan, including knowing when to stop.
One agent with good tools is the default. Escalate only for distinct specialisms.

This article covers what the parts of an agentic system are. It does not settle what agents are for in business terms. It also does not cover how to choose between the specific managed services that implement these parts, or what each costs at scale. Those are separate questions, and exams typically test them under separate objectives.

🌡️ What Do Temperature and Top-p Really Do? Test Them on Amazon Bedrock From the CLI (Hands-on)

confident_prep — Sun, 11 Oct 2026 05:10:10 +0000

🎤 The interview question

"What do temperature and top-p actually do, and when would you change them?"

It's an AIF-C01 favourite, and most answers stop at "temperature controls creativity". A stronger answer: you've seen the difference yourself. Let's send the same prompt to Amazon Bedrock at different settings from the AWS CLI. About 10 minutes, under one cent. 👇

👉 Flow: Low temperature → High temperature → Low vs high top-p → Hit the output ceiling

🧰 Before you start

💻 A current AWS CLI v2. Old builds have no bedrock-runtime converse; update with curl -fsSL https://awscli.amazonaws.com/v2/install.sh | bash.
👤 Signed in (aws sts get-caller-identity works) as a user or role allowed bedrock:InvokeModel on the us.amazon.nova-micro-v1:0 inference profile and the amazon.nova-micro-v1:0 foundation model.
🌍 Region: us-east-1. Amazon models like Nova are enabled by default, so there is no model-access form.
💵 Cost: Nova Micro on-demand is $0.000035 per 1K input tokens and $0.00014 per 1K output tokens (AWS price list, us-east-1). The ~15 calls below cost well under a cent. On the Free plan this comes out of your credits.

Set two variables once, so every command below pastes as-is:

M=us.amazon.nova-micro-v1:0
P='[{"role":"user","content":[{"text":"Suggest one name for a new coffee shop. Reply with the name only."}]}]'

🧊 Step 1: Low temperature, three runs

Why: temperature controls how widely the model samples from its next-word probabilities. Low means it picks the most likely wording almost every time. That's what you want for extraction and classification.

for i in 1 2 3; do
  aws bedrock-runtime converse --region us-east-1 --model-id $M --messages "$P" \
    --inference-config '{"temperature":0.1,"maxTokens":20}' \
    --query 'output.message.content[0].text' --output text
done

✅ Expected: the same name (or nearly) three times. AWS promises "more deterministic", not identical.

⚠️ If you see AccessDeniedException → your identity lacks bedrock:InvokeModel on both ARNs above. Run aws sts get-caller-identity to check who you are, then fix that identity's policy.

🔥 Step 2: High temperature, same prompt

Why: high temperature lets less likely words through. Use it when variety is the point, such as subject lines or brainstorming.

for i in 1 2 3; do
  aws bedrock-runtime converse --region us-east-1 --model-id $M --messages "$P" \
    --inference-config '{"temperature":1,"maxTokens":20}' \
    --query 'output.message.content[0].text' --output text
done

✅ Expected: usually three different names. Sampling is random, so run it again if two match.

💡 Interviewers listen for this: low temperature makes output consistent, not correct. A wrong answer at low temperature is wrong the same way every time. 📚 The full chapter explains why that matters for the exam.

🎯 Step 3: Top-p, low vs high

Why: top-p limits the candidate words to the smallest set whose probabilities add up to p. With 0.1 only the very top words are allowed; with 1.0 every word stays in. AWS's Nova guide says to change temperature or top-p, not both, so temperature stays at Nova's default here.

for p in 0.1 1.0; do
  echo "topP=$p"
  for i in 1 2 3; do
    aws bedrock-runtime converse --region us-east-1 --model-id $M --messages "$P" \
      --inference-config "{\"topP\":$p,\"maxTokens\":20}" \
      --query 'output.message.content[0].text' --output text
  done
done

✅ Expected: under topP=0.1, repeats or near-repeats; under topP=1.0, more variety.

🧨 Break it on purpose

Set the output ceiling far too low and ask for a long answer:

aws bedrock-runtime converse --region us-east-1 --model-id $M \
  --messages '[{"role":"user","content":[{"text":"Explain what temperature does in a language model."}]}]' \
  --inference-config '{"maxTokens":5}' \
  --query '{text: output.message.content[0].text, stop: stopReason, out: usage.outputTokens}'

✅ Expected: a sentence cut off mid-way, "stop": "max_tokens" and "out": 5. maxTokens is a hard stop, not a request to be brief. If you want a short answer, ask for it in the prompt.

🧹 Cost and cleanup

💵 Under one cent in total, and every call above is capped at 20 output tokens or fewer
🧹 Nothing to delete: Converse calls create no resources.

💬 Say it in the interview

"Temperature and top-p both control how the model samples its next word. Low settings give consistent output, which suits extraction; high settings give variety, which suits brainstorming. I change one of them, not both, and I remember that low temperature makes a model consistent, not correct."

🎯 What you can now say (and the exam)

✅ "Low temperature = consistent, not correct"
✅ "High temperature or high top-p = more variety"
✅ "Change temperature or top-p, not both"
✅ "maxTokens truncates; stopReason: max_tokens is the tell"

📚 Go deeper

This is the hands-on cut of Chapter 8 of our free AWS AI Practitioner (AIF-C01) course: choosing a foundation model, inference parameters and agents, with practice questions.

👉 Read the full chapter on Confident Prep

💸 What Does One Amazon Bedrock Prompt Cost? Find Out From the CLI (Hands-on)

confident_prep — Sun, 11 Oct 2026 04:32:34 +0000

🎤 The interview question

"Your team wants to add an Amazon Bedrock chatbot. How would you work out what one prompt costs?"

It shows up in AIF-C01 prep and in real interviews. The strong answer: Bedrock charges per token, and every response tells you how many tokens it used. Let's prove that from the AWS CLI in 5 minutes. 👇

👉 Flow: Send one prompt → Read usage → Multiply by the price → Compare two prompts

🔧 Step 1: Set up

✅ AWS CLI v2, signed in (aws sts get-caller-identity works)
✅ Region us-east-1
✅ Model access for Amazon Nova Micro (Bedrock console → Model access)

💬 Step 2: Send one prompt with the Converse API

aws bedrock-runtime converse \
  --region us-east-1 \
  --model-id us.amazon.nova-micro-v1:0 \
  --messages '[{"role":"user","content":[{"text":"Explain overfitting in one sentence."}]}]' \
  --query '{answer: output.message.content[0].text, usage: usage}'

✅ Expected (your numbers will differ):

{
  "answer": "Overfitting is when a model learns the training data so closely ...",
  "usage": { "inputTokens": 8, "outputTokens": 24, "totalTokens": 32 }
}

👀 That usage block is your bill, in tokens.

🧮 Step 3: Turn tokens into money

Open Amazon Bedrock pricing, find Nova Micro, and copy its price per 1,000 input tokens and per 1,000 output tokens.

IN=8; OUT=24                      # from usage
P_IN=<price per 1K input tokens>  # from the pricing page
P_OUT=<price per 1K output tokens>
awk -v i=$IN -v o=$OUT -v pi=$P_IN -v po=$P_OUT \
  'BEGIN { printf "USD %.8f per prompt\n", i/1000*pi + o/1000*po }'

💡 Multiply by prompts per day to get a daily cost. That's the number your manager wants.

🧨 Break it on purpose

Ask for a long answer instead:

--messages '[{"role":"user","content":[{"text":"Explain overfitting in 500 words."}]}]'

📈 outputTokens jumps, and output tokens cost more than input tokens. Then add --inference-config '{"maxTokens":50}' and watch the cost come back down.

🧹 Cost and cleanup

💵 A few prompts on Nova Micro cost a fraction of a cent
🧹 Nothing to delete: Converse calls create no resources

🎯 What you can now say in the interview (and the exam)

✅ "Bedrock on-demand pricing is per input and output token"
✅ "Every Converse response returns inputTokens and outputTokens"
✅ "Output tokens usually cost more, so I cap them with maxTokens"
✅ "Cost per prompt × prompts per day = the daily bill"

📚 Go deeper

This is the hands-on cut of Chapter 5 of our free AWS AI Practitioner (AIF-C01) course: the full chapter explains tokens, why they drive cost, and how the exam asks about them.

👉 Read the full chapter on Confident Prep

When is machine learning the wrong answer to a business problem?

confident_prep — Sun, 11 Oct 2026 03:16:45 +0000

A manager asks for "an AI solution" to calculate shipping fees. The fees already come from a published rate table: weight band, destination zone, a fixed surcharge. Training a model would mean spending effort to produce an approximate version of a rule the business has already written down. The AWS Certified AI Practitioner exam (AIF-C01, Exam Guide version 1.1) checks whether you can spot this situation. One of the abilities it validates is to "identify the appropriate use of AI/ML and GenAI technologies to solve business problems".

The claim of this article: machine learning is the wrong answer in five situations:

the rule is already known;
the output must be exactly predictable;
the data is missing, poor or biased;
the business value cannot be measured;
a simpler, ready-made or narrower technique would do the job.

Each follows from how machine learning works. It infers a relationship it does not know in advance from examples, and its output is probabilistic rather than certain. The answer is written the way a strong candidate would give it in an interview, followed by five follow-up questions. Each follow-up has a short model answer and the mistake weaker candidates tend to make.

The strong answer: machine learning earns its place only when the rule is unknown and the data can teach it

Asked "When is AI or machine learning the wrong answer?", a strong candidate starts from the mechanism, not a list of slogans.

Machine learning starts from the idea that some mathematical relationship links inputs to outputs. The model does not know this relationship in advance. It can estimate it if it sees enough examples. The standard illustration is simple. Give an algorithm the pairs (2,10), (5,19) and (9,31), and it works out that the output is three times the input plus four. Asked about an input of 7, it predicts 25.

That example contains the whole argument. If you already knew "times three, plus four", you would write that line of code and skip the training. Machine learning helps when nobody can write the rule down but examples of it exist.

The strong answer therefore names the conditions that remove the case for machine learning:

a known rule;
a need for guaranteed, repeatable output;
too little data, or data of poor quality;
no measurable business goal;
a problem a simpler or ready-made technique already solves.

Machine learning is justified only after the cheaper questions are answered

Follow-up 1: a known rule beats a learned guess when output must be predictable

Interviewer: "Why not use machine learning anyway? It would learn the rule eventually."

Model answer: Because the two kinds of system make different promises. Most software responds predictably, so you can say "if the user does this, he gets that". Machine learning learns from observation, so it is probabilistic. The statement becomes "if the user does this, there is an X% chance of that happening".

Tax calculations, fee schedules, eligibility checks against fixed thresholds and access permissions all need the first kind of promise. Replacing an exact rule with a probability adds error and buys nothing.

The mistake weak candidates make: treating "learns automatically" as an advantage in every case. In practice, the giveaway is an answer that never mentions that a learned model can be wrong on inputs where the written rule would always be right.

Think of it as the difference between a printed timetable and a commuter's guess about when the bus usually arrives. The guess is useful only when no timetable exists.

Rule-based software promises an outcome; machine learning promises a likelihood

Follow-up 2: without enough clean, representative data there is no model worth building

Interviewer: "The rule is unknown, but we only have a few hundred messy records. Is machine learning still the answer?"

Model answer: Probably not yet. A model's performance depends on the quality of the data it was trained on. Missing values, inconsistent entries and noise can significantly degrade accuracy. A dataset that is not large enough can stop the model from learning effectively.

Data that looks plentiful can still be skewed. When some classes appear far more often than others, a model can do well on the majority class and fail on the minority class. For example, a hiring model trained on history that favoured one demographic may keep favouring it. Training an unbiased system needs large volumes of data and good data-quality processes behind them.

Supervised learning is the style trained on labelled examples, where each input is tagged with its correct output. Labelling at the scale it needs is itself a challenge.

The mistake weak candidates make: assuming the model will "clean up" the data, or that more training fixes bad inputs. A sensible default is to treat data readiness as a gate. If the data cannot be fixed, the project is not ready for machine learning.

Follow-up 3: a project with no measurable business goal cannot be justified

Interviewer: "Leadership wants AI on the roadmap. Where do you start?"

Model answer: With the problem, not the technology. Guidance on adopting machine learning puts "business goal" first in the lifecycle:

Identify the problem to solve and the value machine learning would add.
Check whether that value can be measured against specific success criteria.

Next comes problem framing. You decide what is observed and what should be predicted, and which performance and error metrics matter. Only after that do data processing, model development, deployment and monitoring follow.

If you cannot say what the model predicts, or how anyone would know it is working, the honest answer is that machine learning is premature.

The business goal and problem framing come before any data or model work

The mistake weak candidates make: answering with a service or a model type before naming a measurable outcome. On a scenario question, any option that skips "define the goal and success criteria" deserves suspicion.

Follow-up 4: when a decision must be explained, an opaque model carries a real cost

Interviewer: "A lender wants a model to approve loans. Any concerns?"

Model answer: Two concerns: explainability and bias.

Explainability. As models become more complex, and deep learning models especially, their decisions become harder to interpret. This affects usability, trust and the ethics of deploying them. Deep learning is the family of methods that passes data through many layers of artificial neurons.

Bias. Imbalanced historical data can lead a model to keep reproducing the patterns in that data.

Governance. Deploying AI means managing data quality, privacy and security, within regulatory restrictions and privacy laws. The organisation stays accountable for customer data. Responsible AI is the practice of attending to fairness, transparency and accountability in how AI is built and used.

None of this rules machine learning out automatically. It does mean that a decision that must be justified to the person affected is a weaker fit for a model whose reasoning is hard to read. The choice is between a more interpretable approach and accepting the cost of bias checks and oversight.

The mistake weak candidates make: treating responsible AI as a separate topic from "is this the right tool". In practice, the two often turn out to be the same question.

Four warning signs that machine learning may be the wrong answer

Follow-up 5: choosing the wrong type of AI is also a wrong answer

Interviewer: "Fine, AI fits here. Should we just use the biggest generative model available?"

Model answer: Not by default. Foundation models are large deep learning models trained on broad, unlabelled data that can perform many different tasks from a prompt. They are not automatically the best choice. Many organisations still use conventional machine learning models for many tasks, and those models can outperform foundation models for many use cases.

The same care applies within machine learning, because each learning style has a known limit:

Unsupervised learning, which finds structure in unlabelled data, cannot give precise predictions.
Reinforcement learning, which learns by collecting rewards, struggles because real-world environments change often and with little warning.

Pretrained AI services offer ready-made intelligence for common use cases. When one fits, it can remove the need to train anything.

Cost is the last factor. Training large models demands significant computing resources and can be time-consuming and costly. Processing power can be expensive and limit how far a system scales.

The mistake weak candidates make: equating "most capable" with "most appropriate". A sensible rule is to choose the narrowest technique that meets the success criteria. Choose a conventional classifier when the task is a fixed set of categories with labelled history. Choose a generative model when the output itself is new text or images.

How AIF-C01 turns this judgment into scored questions

The exam guide lists four things the exam validates:

describing AI, machine learning and generative AI concepts;
identifying the appropriate use of these technologies for business problems;
determining the correct types of technology for specific use cases;
using them responsibly.

The second and third map directly onto the follow-ups above.

The target candidate uses AI/ML solutions on AWS but does not necessarily build them. Coding models, feature engineering, hyperparameter tuning and statistical analysis of models are explicitly out of scope. The exam therefore asks you to judge fit, not to build the fix.

The question formats suit judgment. Multiple choice has one correct answer and three distractors. Multiple response needs every correct option selected. Matching pairs responses with three to seven prompts. Ordering asks you to place three to five responses in sequence.

In practice, a lifecycle ordering question or a use-case matching question is a natural home for "business goal first" and "right type for the task". That is an interpretation of the format, not a published statement of which questions appear.

Scored content is spread across five domains; judging fit draws on several of them

Weighting:

Domain	Share of scored content
Fundamentals of AI and ML	20%
Fundamentals of GenAI	24%
Applications of Foundation Models	28%
Guidelines for Responsible AI	14%
Security, Compliance, and Governance for AI Solutions	14%

Scoring:

50 questions count toward the score; 15 more are unscored and not identified.
Results are reported on a scale of 100 to 1,000, and 700 is the passing score.
Scoring is compensatory, so you need to pass only the overall exam, not each section.
Unanswered questions count as incorrect, and there is no penalty for guessing.

Key takeaways

Machine learning infers an unknown relationship from examples. If the rule is already known, ordinary code is the better answer.
Machine learning output is probabilistic. Where the outcome must be guaranteed and repeatable, a learned model adds error.
Poor, scarce or imbalanced data, or a goal with no measurable success criteria, means the project is not ready for machine learning.
Explainability, bias and governance are part of judging fit, not an afterthought.
"Most capable" is not "most appropriate". Conventional models and ready-made services often fit better than a large generative model.
AIF-C01 tests this as judgment, not building: identifying appropriate use and choosing the right type of technology for a use case.

The honest limit: this article draws on the exam guide's overview and two general explainers. It does not reproduce the detailed task statements inside each domain, so it cannot say exactly how many questions address inappropriate uses of AI or how they are worded.

Fine-tuning versus continued pre-training for Amazon Nova 2

confident_prep — Sun, 11 Oct 2026 03:16:20 +0000

Imagine an interviewer asks how to adapt Amazon Nova 2 Lite for a specialist domain: should you feed it a vast library of raw documents, or show it carefully prepared examples of the answers you want? Those choices correspond to continued pre-training and supervised fine-tuning, and they solve different problems. Continued pre-training develops deeper familiarity with a domain from large volumes of unlabeled text, while supervised fine-tuning teaches tasks and behaviors from labeled prompt-response examples; choosing correctly depends on whether the gap is primarily missing domain fluency or unclear behavior.

The strongest answer separates knowledge acquisition from behavior shaping

Interview question: What is the difference between fine-tuning and continued pre-training a foundation model on AWS, and what data does each need?

Model answer: Continued pre-training, or CPT, extends a foundation model’s pre-training by exposing it to additional raw, unlabeled text from a particular domain or collection. Its purpose is to help the model acquire deeper domain knowledge, terminology, writing patterns, and familiarity with particular content types.

Supervised fine-tuning, or SFT, instead trains the model on labeled input-output pairs. Each pair demonstrates a prompt and the response that the model should learn to produce, allowing the model to align with a specific task, instruction, workflow, format, tone, or style.

The simplest distinction is therefore knowledge acquisition versus behavior shaping. CPT answers, “What body of language and subject matter should the model become more fluent in?” SFT answers, “Given an input, what should the model do?”

The data requirements follow directly from that distinction. CPT needs large volumes of domain-specific raw documents, potentially tens of billions of tokens. SFT needs high-quality labeled examples in which each input is paired with the desired output. CPT generally needs subsequent instruction-tuning stages so that the model can use its newly acquired knowledge to complete useful tasks.

Think of it as preparing a specialist for work. CPT resembles giving the specialist an extensive professional library to absorb, while SFT resembles showing the specialist worked examples of how to respond to particular assignments. The analogy distinguishes broad exposure from explicit demonstrations; it does not imply that either method automatically guarantees correct answers.

Continued pre-training builds domain fluency, while fine-tuning demonstrates desired behavior

The mistake weak candidates make: In practical interview terms, a weak answer often treats both methods as interchangeable ways to “add company data.” That misses the decisive difference between learning from unlabeled documents and learning from labeled demonstrations.

The right choice begins with the gap you need to close

Follow-up question 1: When should you choose supervised fine-tuning?

Model answer: Choose SFT when you can express the desired behavior through high-quality input-output examples. It is suited to teaching a specific response format, tone, or style, as well as domain-specific instructions and workflows. It can also adapt the model for multimodal tasks involving text, images, video, or tool calling.

For example, suppose the model already understands the vocabulary in technical service records but does not consistently transform a record into the required structured response. A training set could pair representative records with correctly formatted outputs. That example illustrates SFT because every input is accompanied by a target response that demonstrates the expected task.

SFT is also the clearer choice when success can be shown directly: “For an input like this, produce an output like that.” This selection rule is a practical judgment, but it follows the structure of the required training data.

Supervised fine-tuning starts with examples that show the required response

The mistake weak candidates make: A weak answer may say that SFT merely gives the model more subject matter. In fact, its defining input is a set of labeled prompt-response demonstrations, and its stated purpose is alignment with tasks, instructions, or desired behaviors.

Raw domain corpora point toward continued pre-training

Follow-up question 2: When should you choose continued pre-training?

Model answer: Choose CPT when you have a very large volume of domain-specific text and want the model to develop deeper knowledge and more native fluency in that domain. Suitable material can include legal documents, medical literature, technical documentation, or proprietary business content. The documents do not need prompt-response labels because CPT trains on raw text.

CPT is particularly valuable at a scale of tens of billions of tokens. It can help the model learn specialized terminology, characteristic writing patterns, new subject matter, and particular content types. That makes CPT a different commitment from assembling a comparatively targeted collection of demonstrations: its stated use case assumes a large corpus rather than a list of correct answers.

For example, a collection of specialist manuals may contain the language and concepts the model should absorb without specifying a task for each passage. That is CPT-shaped data. By contrast, pairs such as “manual excerpt plus question” and “approved answer” are SFT-shaped data because they demonstrate how the model should respond.

Think of CPT as immersion in a domain’s literature. Immersion can build familiarity with the domain, but it does not by itself specify the useful task the model should perform with that familiarity.

The presence or absence of target responses determines the training-data shape

The mistake weak candidates make: A weak answer may recommend CPT simply because some proprietary documents exist. The stronger answer checks whether there is enough domain text for the documented large-volume use case and whether the real gap is domain fluency rather than response behavior.

Continued pre-training may be only the first adaptation stage

Follow-up question 3: Does continued pre-training make the model ready for a specific task?

Model answer: Not necessarily. After CPT, the model generally needs additional instruction-tuning stages so that it can use the newly acquired knowledge and complete useful tasks. CPT can deepen knowledge of a domain, but its raw documents do not directly provide the labeled demonstrations that define a desired response.

A sensible design is to treat CPT and SFT as potentially complementary rather than mutually exclusive. CPT can first expose the model to a large specialist corpus, after which instruction tuning can demonstrate how that knowledge should be applied. The first stage develops domain familiarity; the later stage shapes task performance.

For example, raw technical documentation could support CPT so the model encounters specialized terms and writing patterns. Labeled examples could then show how to answer a diagnostic question, follow a domain-specific workflow, or return a required response format. The example demonstrates why “knowing the material” and “performing the task” are separate adaptation goals.

Domain adaptation can continue from raw-text training into instruction-focused training

The mistake weak candidates make: In interview terms, weak candidates often imply that pouring raw documents into CPT automatically teaches question answering, formatting, or workflow compliance. The source makes the limitation explicit: additional instruction tuning is generally needed after CPT.

Data quality and data shape are different requirements

Follow-up question 4: What exactly should the training dataset look like for each method?

Model answer: A CPT dataset should be a large corpus of relevant, unlabeled domain text. Its useful signal lies in the documents themselves: their subject matter, terminology, writing patterns, and content types. The documented examples include legal, medical, technical, and proprietary business text.

An SFT dataset should contain labeled input-output pairs that demonstrate the correct behavior. Inputs are prompts or task examples, while outputs are the responses the model is meant to learn from. Those pairs can demonstrate a format, tone, style, instruction, workflow, or multimodal task.

“Unlabeled” does not mean “unrelated,” and “labeled” does not mean “automatically useful.” In practical terms, CPT material should represent the domain the model needs to absorb, while SFT examples should clearly represent the behavior it needs to reproduce. That is a judgment about dataset design rather than a separate sourced requirement.

For example, a folder of domain documents does not become an SFT dataset merely because the documents have filenames or categories. SFT requires the stronger relationship of an input paired with its intended output. Conversely, converting every raw passage into a demonstration is unnecessary for CPT because CPT is specifically designed to train on raw documents.

Choose the method by asking whether the dataset contains desired answers

The mistake weak candidates make: A weak answer discusses only the amount of data. Volume matters to CPT’s documented use case, while SFT is defined by high-quality labeled pairs; the structure and purpose of the data are therefore as important as its quantity.

Nova 2 Lite supports both paths, but model selection remains a tradeoff

Follow-up question 5: How does this distinction apply to Amazon Nova 2?

Model answer: Amazon Nova 2 is the current model generation in the question, and Nova 2 Lite supports both CPT and SFT. The documentation also lists Nova 1.0 Micro, Lite, and Pro for both techniques, but that is the older model generation.

Nova 2.0 is the documented choice when the use case needs enhanced reasoning with an explicit reasoning mode, broader multilingual performance, improved performance on complex work such as coding and tool use, or more accurate and stable handling of extended context. Nova 1.0 remains an option when standard language understanding is sufficient, lower training and inference costs are a priority, the focus is domain-specific knowledge and behavior rather than complex reasoning, or its performance has already been validated for the use case.

Model generation and adaptation method answer separate questions. Selecting Nova 2 Lite identifies the base model, while selecting CPT or SFT identifies how that model will learn from the available data. A sensible interview answer keeps those choices distinct and evaluates accuracy, speed, cost, and business requirements rather than assuming that a larger model is always preferable.

For example, choosing Nova 2 Lite because a task needs stronger reasoning does not establish whether CPT or SFT is appropriate. The data and adaptation goal still decide between raw-text domain learning and labeled behavior learning.

The mistake weak candidates make: A weak answer treats “use Nova 2” as a complete adaptation strategy. The model choice does not remove the need to determine whether the problem calls for domain knowledge from raw text, task behavior from demonstrations, or a sequence involving both.

Key takeaways

CPT extends pre-training with raw, unlabeled domain documents so the model can acquire deeper knowledge, terminology, writing patterns, and domain fluency.
SFT trains on labeled prompt-response pairs so the model can learn specific tasks, instructions, workflows, formats, tones, styles, or multimodal behaviors.
CPT is particularly relevant when the available corpus contains tens of billions of domain-specific tokens, while SFT depends on high-quality examples of the desired behavior.
CPT generally needs later instruction tuning before the adapted model can apply its new knowledge to useful tasks.
Nova 2 Lite supports both CPT and SFT, while the documentation also lists the older Nova 1.0 Micro, Lite, and Pro models.
The practical decision rule is to choose CPT for a domain-fluency gap, SFT for a demonstrated-behavior gap, and consider both in sequence when both gaps exist.

This distinction settles what each technique is meant to teach and what form of data it requires. It does not settle which method will meet a particular quality target, how much labeled SFT data is sufficient, or whether the cost-performance tradeoff favors adaptation over another design for a specific workload.

AWS AIF-C01: Bias, Variance & the AWS Tools That Detect Them

confident_prep — Thu, 01 Oct 2026 12:13:20 +0000

A model can perform well overall and still fail badly for a particular group. That is the central idea behind bias and variance questions on the AWS Certified AI Practitioner exam: training-versus-unseen performance reveals underfitting or overfitting, while differences between demographic groups reveal societal bias. To choose the right detection method, first identify which problem the evidence describes, then match the tool to when the check occurs—point-in-time, continuously after deployment, or on an individual prediction.

Bias has two meanings, and the scenario tells you which one applies

On the exam, “bias” can refer to either statistical bias or societal bias. These are separate problems, even though both can cause inaccurate predictions.

Statistical bias means a model is too simple to capture the underlying pattern. It performs poorly on the data used to train it and on data it has never seen. This failure is called underfitting.

Societal bias means outcomes differ systematically across demographic groups. A model might perform well in aggregate while producing much worse results for one group. This is a fairness failure, not necessarily a fitting failure.

The surrounding language tells you which meaning the question intends:

References to training accuracy, unseen-data accuracy, overfitting, or underfitting point to statistical bias.
References to fairness, demographic groups, or unequal outcomes point to societal bias.

Consider an illustrative example. A model scores poorly on both its training set and its test set. That is high statistical bias: the model never captured the pattern. Now suppose another model scores well on both sets overall but performs much worse for rural applicants. That is societal bias. The second model may generalize well in aggregate and still be unfair.

The same word points to different problems depending on the evidence in the scenario.

This distinction prevents a common exam mistake: calling every group disparity “overfitting.” Overfitting describes sensitivity to the training set. It does not mean that a model performs differently across groups.

Variance appears as a gap between training and unseen performance

Variance is a model’s sensitivity to the particular data on which it was trained. High variance appears as overfitting: the model performs extremely well on its training data but poorly on unseen data.

The model has learned details specific to the training set instead of learning a pattern that generalizes. The decisive evidence is the gap between training and unseen-data performance.

Underfitting has a different signature:

Condition	Training performance	Unseen-data performance
Underfitting, or high statistical bias	Poor	Poor
Overfitting, or high variance	Strong	Poor
Healthy aggregate fit	Strong	Strong

For example, suppose a model answers nearly every training example correctly but makes frequent errors on new examples. The strong training result does not show that the model is healthy. Combined with poor unseen performance, it shows overfitting.

Now change the example: the model performs well on both training and unseen data, but its unseen-data accuracy is much lower for one demographic group. Neither underfitting nor overfitting fully describes that result. The model can be well fitted in aggregate while remaining unfair.

Training and unseen performance separate underfitting from overfitting before fairness is assessed.

A useful exam rule follows: when a question gives two performance figures—one for training data and one for unseen data—it is usually testing fitting. When it gives an overall figure and mentions a demographic group, it is testing fairness.

Dataset characteristics explain where disparities begin

The exam separates four dataset characteristics because each answers a different question.

Inclusivity asks whether the groups the system will serve are present at all. If a group has no records, the model cannot learn from it, and there are no rows on which to calculate that group’s performance.

Diversity asks whether the data represents the real range of cases, conditions, and contexts. A dataset may include every named group yet cover only typical cases, leaving the model unreliable at the edges.

Balance asks whether represented groups appear in workable proportions. A group can be present but so heavily outnumbered that it has little effect on the training objective.

Curated sources have known, selected, and documented origins. Curation makes the data’s provenance inspectable; it does not guarantee neutrality.

Imagine a dataset containing 40,000 records from one group and 300 from another. The smaller group is present, so this is not an inclusivity failure. It is a balance failure: the group is represented but swamped.

By contrast, if the second group has no records, “rebalance the dataset” is not yet a sufficient answer. There are no records to reweight. Data must first be collected from the missing group.

Dataset checks move from whether groups exist to whether their representation can support reliable learning.

Curation is especially easy to misread. A carefully documented dataset collected entirely from one narrow population can be curated and skewed at the same time. Curation helps you identify what the dataset represents; it does not make the representation fair.

Bias may also enter through collection, labelling, training, deployment, or feedback. For example, inconsistent reviewers may assign different labels to similar cases. Rebalancing the dataset would not repair those prejudiced or inconsistent targets. The labels themselves require analysis.

Aggregate accuracy cannot establish fairness

An aggregate metric is an average. It can describe total performance without revealing how that performance is distributed.

Suppose a model reports 94% accuracy overall. That figure may be arithmetically correct while concealing much lower accuracy for a minority group. Nothing about the headline number answers the fairness question “for whom does the model work?”

The appropriate technique is subgroup analysis: calculate the same metric separately for each relevant group. Subgroup analysis is not an AWS product. It is the measurement habit that makes disparities visible.

This produces an important sequence for exam scenarios:

Refuse to treat overall accuracy as fairness evidence.
Identify the groups that need comparison.
Recalculate the relevant metric for each group.
Trace the disparity back through collection, labelling, training, deployment, and feedback.
Select a detection or monitoring tool based on timing.

Removing a demographic attribute is not a reliable fix. Other features may act as proxy variables, meaning they carry information correlated with the removed attribute. Removing the attribute can also eliminate the field needed to calculate per-group metrics, making the disparity harder to detect without removing its cause.

Subgroup analysis exposes disparities that remain invisible inside a healthy overall result.

Human judgment still matters because somebody must decide which groups to compare. A metric can only expose disparities it was configured to measure. Human audits review model behavior directly and can identify concerns that were never encoded in an automated check.

Timing selects Clarify, Model Monitor, or A2I

The AWS tools in these questions are not interchangeable. The simplest way to choose among them is to ask when the examination occurs.

Amazon SageMaker Clarify performs point-in-time bias measurement. It can assess bias in data before training and in a trained model after training. It also produces feature attributions, but in a bias question its relevant role is calculating bias metrics.

Choose Clarify when the scenario asks whether a training dataset or trained model is biased now, particularly before deployment.

SageMaker Model Monitor watches a deployed model continuously for changes in data, model quality, and bias relative to a baseline.

Choose Model Monitor when the scenario says the model was acceptable at launch but may have changed after deployment. Phrases such as “since launch,” “drift,” and “continually monitor” distinguish it from Clarify.

Amazon Augmented AI (Amazon A2I) routes individual predictions to human reviewers at inference time, often when confidence is low.

Choose A2I when a particular prediction or application needs human judgment. Do not choose it to calculate population-level disparity: A2I produces a human decision on a case, not a bias metric across groups.

The point in time determines whether the answer is Clarify, Model Monitor, or A2I.

Two other methods complete the exam’s detection set.

Analyzing label quality checks whether labels are correct and consistently applied. Choose it when the evidence points to reviewers using inconsistent standards or when bias may be encoded in the training target.

Human audits are appropriate when the concern requires judgment that no configured metric captures. They are not an inferior substitute for automation; they examine blind spots created by the choice of what to measure.

A compact selection rule is:

Scenario signal	Best match
Assess training data or a trained model now	SageMaker Clarify
Watch for change after deployment	SageMaker Model Monitor
Send one prediction for human review	Amazon A2I
Compare performance across groups	Subgroup analysis
Check inconsistent or prejudiced targets	Label quality analysis
Investigate concerns no metric encoded	Human audit

Common distractors fail because they answer the wrong question

Several plausible answers recur in bias and variance scenarios.

“Collect more data” does not fix missing coverage if the new records come from the same sources. More of the same population repeats the same skew. The corrective action must add relevant cases or groups.

“Remove the demographic column” does not prevent discrimination when proxy variables remain. It may instead destroy the ability to measure the disparity.

“Use Clarify for ongoing monitoring” ignores timing. Clarify measures at a point in time; Model Monitor handles continuous checks after deployment.

“Use Model Monitor on the training dataset” places a deployment tool before deployment. There is no endpoint behavior to monitor yet.

“Use A2I to measure bias” confuses case review with population analysis. A2I routes individual predictions to people; subgroup analysis or bias metrics reveal group-level disparity.

“Call the group failure overfitting” confuses demographic performance with generalization. Overfitting requires strong training performance and poor unseen-data performance. A group disparity requires per-group evidence.

Think of these distractors as category errors. Each proposed action may be useful somewhere, but it does not match the evidence, the level of analysis, or the point in the model lifecycle described by the question.

Key takeaways

Statistical bias means underfitting: the model performs poorly on training and unseen data.
High variance means overfitting: training performance is strong, but unseen performance is poor.
Societal bias means outcomes differ systematically across groups and can coexist with good aggregate performance.
Inclusivity is about presence; balance is about proportion; diversity is about range; curation is about documented provenance.
Overall accuracy cannot establish fairness. Calculate metrics separately for relevant groups.
SageMaker Clarify measures data or model bias at a point in time.
SageMaker Model Monitor watches a deployed model for drift in quality and bias.
Amazon A2I routes individual predictions to human reviewers; it is not a population bias metric.
Label quality analysis examines biased or inconsistent targets, while human audits catch concerns no metric was configured to detect.
Timing, not the general word “bias,” is the strongest clue when selecting an AWS tool.

These rules settle how to distinguish the concepts and select the exam’s named detection methods. They do not prove that a real model is fair: that still depends on which groups were examined, whether the labels and data reflect the deployment population, and whether human reviewers looked for harms the chosen metrics could not express.

Your LLM Prompt Costs More Than You Think — Build a Token & Cost Calculator in Colab

confident_prep — Thu, 01 Oct 2026 11:59:45 +0000

You don't need an API key to find out your AI feature is quietly expensive.

You don't need a production account.

You don't need to wait for the first bill.

You need:

Google Colab
Python
pip install tiktoken
About 10 minutes

The last article in this series, "Give Your Chatbot a Memory in Google Colab Before Your Next AI Interview," was about managing a conversation once it gets long. (Link it to that post's live Dev.to URL when you paste this in — it isn't in this repo.) This one rewinds further — before memory, before RAG, before agents — to the mechanism everything else in an LLM system is built on top of: tokens, and what they actually cost you. If you can't answer "how many tokens is this, and what does that mean in dollars," the rest of the stack doesn't matter yet.

Let's measure it.

What Are We Building?

Every LLM call is billed and bounded by tokens, not words or characters. We'll build three small tools:

A real tokenizer that counts tokens the way a production model actually would
A cost calculator that turns a token count into a daily and yearly dollar figure
A tiny simulation of what "temperature" actually does to next-token sampling

No fine print, no mocked numbers — every count below came from running the code, not estimating it.

Step 1: Count Tokens for Real

Skip guessing "about 4 characters per token." Use the same tokenizer family production APIs use.

import tiktoken

enc = tiktoken.get_encoding("cl100k_base")

text_a = "The meeting is scheduled for Thursday at 3pm. Please bring your notes from the last session and any open questions you want to discuss with the team."
text_b = "The rate-limiting factor in transformer inference is memory bandwidth during KV-cache reads, not FLOPs. Prefill is compute-bound; autoregressive decode is memory-bandwidth-bound."
text_c = "Here is the function signature: async def fetch_user(user_id: UUID, db: AsyncSession) -> Optional[UserModel]:. It should return None if the user does not exist, not raise an exception."

for name, t in [("A - plain English", text_a), ("B - technical jargon", text_b), ("C - mixed code", text_c)]:
    toks = enc.encode(t)
    print(f"{name}: {len(toks)} tokens, {len(t.split())} words -> {len(toks)/len(t.split()):.2f} tokens/word")

Output, run just now:

A - plain English: 31 tokens, 27 words -> 1.15 tokens/word
B - technical jargon: 37 tokens, 21 words -> 1.76 tokens/word
C - mixed code: 43 tokens, 27 words -> 1.59 tokens/word

Same rough word count across all three (21–27 words), but Text B costs 53% more tokens per word than Text A. Compound terms like "rate-limiting" get chopped into multiple sub-word pieces; UUIDs and type annotations in Text C do the same. Plain English is the cheapest register you can write in — technical prompts are a variable tax, not a fixed multiplier.

Step 2: Turn Tokens Into Dollars

A token count is abstract until it's attached to a query volume and a price sheet.

PRICING = {
    "claude-sonnet": {"input": 3.00, "output": 15.00},   # $ per 1M tokens
    "claude-haiku":  {"input": 0.80, "output": 4.00},
}

def estimate_cost(text, queries_per_day, model, days=365):
    tokens = len(enc.encode(text))
    daily_tokens = tokens * queries_per_day
    daily_cost = daily_tokens * (PRICING[model]["input"] / 1_000_000)
    return tokens, daily_tokens, daily_cost, daily_cost * days

system_prompt = (
    "You are a support assistant for Acme Cloud. Always answer in a friendly, "
    "professional tone. Never reveal internal pricing formulas. If the user asks "
    "about billing, refer them to the billing dashboard at app.acme.com/billing. "
    "If the user reports an outage, check the status page before responding, and "
    "always include the current incident ID if one is open. Keep responses under "
    "150 words unless the user explicitly asks for more detail."
)

for model in ["claude-sonnet", "claude-haiku"]:
    t, daily_tok, daily_cost, yearly_cost = estimate_cost(system_prompt, 1000, model)
    print(f"{model}: {t} tokens/call, {daily_tok:,} tokens/day, ${daily_cost:.2f}/day, ${yearly_cost:,.0f}/year")

Real output:

claude-sonnet: 87 tokens/call, 87,000 tokens/day, $0.26/day, $95/year
claude-haiku: 87 tokens/call, 87,000 tokens/day, $0.07/day, $25/year

One system prompt, sent unchanged on every one of 1,000 daily queries. $95/year on Sonnet doesn't sound alarming — until this is one of six prompts in your app, or volume is 100,000/day instead of 1,000. The formula doesn't change; the multiplier does.

Step 3: Cut the Bill Without Cutting the Feature

trimmed_prompt = (
    "You are Acme Cloud's support assistant. Friendly, professional tone. "
    "Never reveal pricing formulas. Billing questions -> app.acme.com/billing. "
    "Outage reports -> check status page, include open incident ID. Max 150 words."
)
t2, _, daily_cost2, yearly_cost2 = estimate_cost(trimmed_prompt, 1000, "claude-sonnet")
print(f"Trimmed: {t2} tokens/call ({(1 - t2/87)*100:.0f}% shorter), ${daily_cost2:.2f}/day, ${yearly_cost2:,.0f}/year")

Trimmed: 47 tokens/call (46% shorter), $0.14/day, $51/year

Denser prose cut the prompt nearly in half and the yearly cost from $95 to $51 — same rules enforced, zero functionality lost.

Step 4: What Temperature Actually Does

Temperature doesn't make a model smarter or dumber — it reshapes how sharply it commits to its top guess. Simulate it with a toy 5-word distribution over "The weather today is ___":

import math, random
random.seed(7)

candidates = ["sunny", "cloudy", "cold", "unpredictable", "magnificent"]
logits =     [2.2,      2.0,      1.5,    1.0,              0.3]

def softmax_with_temperature(logits, temperature):
    scaled = [l / max(temperature, 1e-6) for l in logits]
    m = max(scaled)
    exps = [math.exp(s - m) for s in scaled]
    total = sum(exps)
    return [e / total for e in exps]

def sample(candidates, probs):
    r = random.random()
    cum = 0.0
    for c, p in zip(candidates, probs):
        cum += p
        if r <= cum:
            return c
    return candidates[-1]

for temp in [0.2, 1.0]:
    probs = softmax_with_temperature(logits, temp)
    picks = [sample(candidates, probs) for _ in range(8)]
    print(f"temp={temp}: probs={dict(zip(candidates, [round(p,2) for p in probs]))}")
    print(f"  8 samples: {picks}")

temp=0.2: probs={'sunny': 0.71, 'cloudy': 0.26, 'cold': 0.02, 'unpredictable': 0.0, 'magnificent': 0.0}
  8 samples: ['sunny', 'sunny', 'sunny', 'sunny', 'sunny', 'sunny', 'sunny', 'sunny']

temp=1.0: probs={'sunny': 0.36, 'cloudy': 0.3, 'cold': 0.18, 'unpredictable': 0.11, 'magnificent': 0.05}
  8 samples: ['sunny', 'cloudy', 'sunny', 'sunny', 'cloudy', 'cold', 'sunny', 'sunny']

Same logits both times. At temp=0.2 the distribution collapses onto "sunny" — 8 for 8. At temp=1.0 the same model "opinion" produces four different words across 8 draws. Nothing about the model's knowledge changed — only how willing it is to pick outside its top guess.

Make It Reusable

def tokenize_and_cost(text, queries_per_day, model="claude-sonnet"):
    tokens, daily_tokens, daily_cost, yearly_cost = estimate_cost(text, queries_per_day, model)
    return {
        "tokens_per_call": tokens,
        "daily_cost_usd": round(daily_cost, 2),
        "yearly_cost_usd": round(yearly_cost, 0),
    }

One function, three inputs (text, volume, model), a real dollar figure out. Swap in your own system prompt before your next design review.

Colab Experiments to Try

Paste your own longest system prompt into Step 1 — is its tokens/word ratio closer to plain English or technical jargon?
Run estimate_cost at 10,000 queries/day. Watch the yearly figure move from "line item" to "budget conversation."
Add gpt-4o-mini ($0.15 input) to PRICING and compare its yearly cost against claude-haiku.
Change logits so all 5 candidates are nearly equal, then re-run both temperatures — does low temperature still look deterministic?

Interview Questions Hidden Inside This Notebook

Why does a technical or code-heavy prompt cost more than plain English at the same word count?

Sub-word tokenization splits compound technical terms and punctuation-heavy syntax (UUIDs, type hints) into more pieces than common English words. Word count and token count are only loosely correlated — measure tokens directly, never estimate from word count.

What lever actually reduces LLM cost on a fixed feature set?

Prompt density. The Step 3 rewrite cut cost 46% by removing redundant phrasing, not functionality. Token count is a writing-quality problem before it's an infrastructure problem.

Does temperature affect what the model "knows"?

No. It reshapes sampling over an unchanged probability distribution — it doesn't touch the model's underlying weights. Low temperature for tasks needing the same answer every time; higher temperature only where variety is the actual goal.

The Calculator Is the Easy Part

tokenize_and_cost() is maybe six lines. A real production cost model also has to account for output tokens (billed 3-5x the input rate, harder to bound), prompt caching (steep discounts on repeated system prompts this notebook doesn't model), and retries or tool-call round-trips that resend and re-bill context.

Interviewers ask about token cost because a candidate who can name a real number, on the spot, has clearly built something — not just read about it.

Want to Go Deeper?

This notebook covers the mechanics. The full session covers the "how does an LLM generate a response?" 5-beat answer end to end, including the hallucination root cause this notebook didn't touch:

https://confidentprep.com/courses/ai-ml-for-interview/1-how-llms-actually-work/

There's also a companion piece, "How Does an LLM Actually Generate a Response?" — The Interview Question Everyone Gets 80% Right, with 8 interview questions pulled straight from this same session. (Link it to that post's Dev.to URL once both are published.)

Measure your own prompt before your next standup. A real number beats a guess — in a cost review and in an interview.

"How Does an LLM Actually Generate a Response?" — The Interview Question Everyone Gets 80% Right

confident_prep — Thu, 01 Oct 2026 11:59:23 +0000

If you're prepping for an AI engineering interview, this question is coming: "How does an LLM generate a response?"

Most candidates get the first two beats right — tokens, embeddings — then start hand-waving. The ones who land the offer name all five beats without prompting, including the one most people skip.

There's a companion piece, "Your LLM Prompt Costs More Than You Think", that builds a token/cost calculator in Colab you can run in 10 minutes with zero API key. (Link it here once both are published.) This one is the pure Q&A version — no code, just the questions and the answers that separate a senior response from a junior one.

The core question, and the 5 beats interviewers listen for

"How does an LLM generate a response?"

A strong 90-second answer hits all five:

Tokens — sub-word units; the model reads these, not words
Embeddings — each token becomes a vector of numbers capturing meaning
Attention — scores relevance across every token in context, for each new token generated
Generation / sampling — next-token prediction, one token at a time, until done
Hallucination root cause — plausibility ≠ truth; there's no truth-check step in the loop

Miss beat 5 and expect a direct follow-up. It's the most commonly skipped beat, and interviewers know it.

8 questions from this session, answered

1. What is a token, and why does it matter for your job?

A sub-word chunk — the model's basic reading unit. Not a word, not a character. "unbelievable" is 3 tokens. "ChatGPT" is 3 tokens. It matters because you pay per token, not per word, and the model has a hard limit on how many it can process per request.

2. What's the difference between "context" and the "context window"?

Context is everything the model can see in one conversation — your messages, documents, its own replies. The context window is the size limit on that everything: the maximum token count (input plus output) per request. It resets completely on every new call — not persistent memory.

3. What is an embedding, in plain terms?

A list of numbers representing a token's meaning and usage, not its spelling. Similar words land at nearby coordinates; unrelated words land far apart. The same word can carry different embeddings depending on the sentence it's in ("bank" the institution vs. "bank" the river).

4. What does attention actually do?

For every token about to be generated, attention scores every other token in context for relevance. High-relevance tokens get more influence over what comes next. This is also why a cluttered, noisy prompt produces worse output — irrelevant tokens compete for attention against the instruction that actually matters.

5. Why does an LLM hallucinate — what's the real root cause?

The model predicts statistically plausible text. There is no truth-check step anywhere in generation. Confident, authoritative writing was abundant in training data, so the model learned confident tone as a default style — a style that shows up whether or not the underlying claim is true. Plausibility is not truth.

6. What does temperature control, and when do you use low vs. high?

How peaked or flat the next-token probability distribution is. Low (0.0–0.3): the model almost always picks its top-probability token — use for fact extraction, structured output, anything that needs to be the same every call. High (0.7–1.0): more willing to pick lower-probability tokens — use for brainstorming or creative variation. Temperature never changes what the model knows, only how consistently it says it.

7. How does a chatbot "remember" earlier messages if the model resets every call?

It doesn't, on its own. The application layer replays prior turns back into the context window on every new request. The continuity a user experiences is maintained by the app, not the model. Strip out that replay logic and the model has never heard of the user before.

8. Why can't an LLM answer questions about your company's internal documents?

Training data is frozen at a cutoff date, and internal documents were never in it to begin with. Anything private or created after the cutoff has to be explicitly supplied in context — which is the entire premise behind retrieval-augmented generation.

What separates a strong answer from a weak one

Say: "Tokens, not words." "Embeddings capture meaning as vectors." "Attention scores relevance across the full context." "Plausibility, not truth — there's no truth-check step." "Context window is a fixed buffer that resets."

Avoid: "It's just autocomplete" (technically true, reads as surface-level). "It thinks like a human" (signals a real misconception). "It searches the internet" (only true with retrieval tools wired in). "It hallucinates because it lacks data" (names the wrong root cause — it hallucinates even with the data, because nothing checks the output against it).

Want the full 5-beat model answer, word for word?

The course session covers this question with a complete scored model answer, the do's/don'ts list interviewers are actually listening for, and the hands-on Colab notebook that makes tokens and cost tangible before you ever sit down for the interview:

https://confidentprep.com/courses/ai-ml-for-interview/1-how-llms-actually-work/

Say your answer out loud before your next interview. Reading it silently and being able to say it under pressure are two different skills.

Walk Me Through How You'd Design a Production Prompt — 9 Interview Questions, Answered

confident_prep — Thu, 01 Oct 2026 11:57:35 +0000

There's one question that opens almost every AI engineering interview's prompt-design section:

"Walk me through how you'd design a production prompt for a structured extraction task."

It sounds open-ended. It isn't. Interviewers are listening for five specific beats, and the follow-up questions below are how they check for the ones you skip. This is the companion piece to the hands-on Colab notebook where we measured an 80%-to-0% format-drift swing from adding few-shot examples — if you want the receipts behind question 3 below, that's where they are.

1. What's the difference between a system prompt and a user message?

The system prompt is developer-controlled — role, rules, output format — and runs invisibly on every call. The user message is the runtime task or input, changing with each call. Miss this distinction and you'll bake per-request data into a prompt that should be static, or vice versa.

2. What does RCTF stand for, and what happens if you skip one letter?

Role, Context, Task, Format. Skip Format and the model chooses its own output shape — usually inconsistently. Skip Context and it fabricates facts it doesn't have (product names, plan tiers). Skip Role and tone drifts call to call. Skip Task and you get a summary when you wanted a classification. Interviewers who ask "what happens if you skip X" are testing whether you understand why each field exists, not just that it exists.

3. What temperature would you use for a support ticket classifier, and why?

Zero, or close to it. Classification feeds a parser, and parsers need identical output for identical input. Temperature above zero introduces sampling randomness — and that randomness doesn't just reword the answer, it can surface as a genuinely different output shape. Say the number if you have it: a naive zero-shot classifier can drift to a different JSON schema over 80% of the time at moderate temperature. Anchoring the same call with a few worked examples took that to zero.

4. When do you reach for few-shot instead of zero-shot?

Always start zero-shot — it's cheaper and easier to maintain. Move to few-shot only when zero-shot's content is right but the format is wrong or inconsistent. Few-shot examples teach output shape, not new facts. If the content itself is wrong, more examples won't fix it — that's a Context problem, not a technique problem.

5. What's the difference between few-shot prompting and fine-tuning?

Few-shot: examples live in the prompt, inference-time only, weights unchanged, cheap to update. Fine-tuning: the model is retrained on your dataset, weights change permanently, requires thousands of examples and real infrastructure. Try few-shot first. Fine-tuning is for stable, high-volume, narrowly-defined tasks where prompting has genuinely been exhausted — not a first instinct.

6. When does chain-of-thought help, and what does it cost you?

CoT helps on multi-condition or multi-step tasks where jumping straight to an answer causes errors — escalation decisions, trade-off analysis. The instruction "think step by step" forces intermediate reasoning tokens that improve the final answer. The cost: more tokens, more latency. Don't reach for it on simple single-condition tasks — that's needless spend.

7. What is prompt injection, and how do you actually prevent it?

User-supplied text overwrites developer instructions because, to the model, both are just tokens — there's no privileged execution layer separating "your rules" from "their input." The fix that holds up: XML-delimit user input (<user_input>...</user_input>) and instruct the model to treat that block as data only. The fix that doesn't hold up: asking nicely in the system prompt and hoping. A system prompt is a request, not a boundary.

8. Why does a model confidently answer with stale stock prices or outdated facts?

Knowledge cutoff. The model has no internet access and no live retrieval — it generates the statistically most plausible continuation from training data, which may be months or years stale, with the same confident tone as a correct answer. There's no "I don't know current data" reflex unless you build one. The fix is injecting current data into context directly (RAG), never trusting model memory for anything time-sensitive.

9. What does max_tokens actually control, and what breaks if you set it wrong?

The length of the response, not the input — that's the context window's job. Set too low, structured output truncates mid-JSON and your parser throws. Set too high, you're paying for and waiting on output nobody asked for. Rule of thumb: roughly 2x your expected output length, tighter (80-150) for constrained JSON classifiers.

The beat interviewers probe hardest: failure modes. A candidate who nails RCTF and parameters but never mentions injection, knowledge cutoff, or format validation unprompted gets asked "what could go wrong?" — and that follow-up is where senior and junior answers actually separate.

Want to see the 80%-to-0% drift number measured live, not just quoted? The hands-on notebook is here — zero API key required, about 12 minutes.

Are You Ready for Your Job? Try to Answer These Questions.

confident_prep — Thu, 01 Oct 2026 11:55:44 +0000

Not a quiz for fun. A gut check.

These are three questions that show up in real AI engineering interviews — not "define RAG" trivia, but the scenario-style questions interviewers actually use to tell a candidate who's built things apart from one who's only read about them. Read each one, answer it out loud before you read the framework underneath, and be honest about how close you got.

🎯 TL;DR: Three real interview questions, no trivia. Answer each one out loud before reading the framework. If you reach for a definition instead of a process, that's the exact gap other candidates are closing right now — not by reading more, but by building.

🩹 1. "Your RAG chatbot just gave a customer a confident, completely wrong answer. Walk me through how you'd find out why — and stop it from happening again."

This is a debugging question disguised as a RAG question. Interviewers ask it because "explain how RAG works" tells them you read an article, but this tells them whether you've actually operated one.

How to answer it:

Split the failure in two before you diagnose anything: was it a retrieval problem (the right chunk never made it into context) or a generation problem (the right chunk was there and the model ignored it anyway)? Say this split out loud first — it's the single biggest signal you know what you're doing.
Describe how you'd check retrieval: log the retrieved chunks alongside the query, and eyeball whether the answer's source text was even in there.
Describe how you'd check generation: if the source was there but the model still got it wrong, that's a prompt/grounding problem — talk about instructing the model to cite or refuse when the context doesn't support an answer.
Close with prevention, not just the one-time fix: a small eval set of known-answer questions you can re-run after every prompt or retrieval change, so this doesn't quietly regress next sprint.

⚖️ 2. "We could fix this with a better prompt, or by fine-tuning the model. How do you decide which one?"

This question is a trap for candidates who only know one hammer. It's testing judgment, not a definition.

How to answer it:

Lead with prompt engineering as the default: it's fast, cheap, needs no training data, and you can iterate on it in minutes.
Name the specific conditions that push you toward fine-tuning instead: you have a large, stable, labeled dataset; you need consistent output formatting or tone at a scale prompting can't reliably hold; or your prompt is already maxed out on context budget and instructions are still being ignored.
Mention the cost you're trading away: fine-tuning is slower to iterate and harder to reverse than a prompt change. Say that out loud — it shows you're weighing the decision, not just picking a technique.

🚨 3. "The feature demos perfectly, but real users are breaking it in ways you never saw in testing. What's your process?"

This is the question that separates "I built a demo" from "I've operated something in production."

How to answer it:

Start with the gap between demo inputs and real inputs: real users send empty strings, five-paragraph pastes, other languages, and adversarial prompts your test set never covered.
Talk about what you need logged before this happens, not after: full request/response pairs, retrieved context, any tool calls — without that, you're debugging blind.
Mention a containment step: a feature flag or circuit breaker so you can turn the feature off or roll back the version for affected users while you dig in, instead of leaving it broken in front of customers while you investigate.

🪞 How did you do?

If you answered all three with that level of specificity, unprompted — that's rare, and it means you've actually operated systems like this before.

If you found yourself reaching for a textbook definition instead of a process, that's useful information. It means you understand the concepts but haven't yet built the judgment that only comes from shipping and breaking a few of these yourself. Other candidates in your interview pool are closing that exact gap right now — not by reading more, but by building.

That judgment isn't something an article can hand you — including this one. It comes from building the actual systems these questions are about.

The takeaway: knowing the definition doesn't get the offer. Having broken the thing once does.

See the paths that build that judgment →

DEV Community: confident_prep

How training and inference work: build and test a model by hand

You need a pen, a sheet of paper and nothing else

Step 1: Write down labeled training data

Step 2: Run the algorithm and get a model

Step 3: Uncover the test rows and score the model

Step 4: Use the model on a new delivery

Step 5: Break it on purpose by memorizing

Step 6: Decide how the model would answer people

Why the paper version transfers

Key takeaways

Links

How would you make sure an AI system is responsible?

The strong answer names properties, not good intentions

Follow-up: bias is the measurement, fairness is the standard

Follow-up: safety, veracity and robustness fail in different ways

Follow-up: guardrails check two points and show evidence for only two features

Follow-up: sustainability counts when you choose a model

Follow-up: the legal risks are business exposures with three routes in

Follow-up: features and risks are the same list read from two ends

Key takeaways

Links

What is an agentic AI system actually made of?

A strong answer names the loop before any technology

Follow-up 1: the model requests, the runtime executes

Follow-up 2: MCP connects an agent to external systems

Follow-up 3: memory is a decision about what survives

Follow-up 4: arrangement and message flow are separate questions

Follow-up 5: orchestration owns the plan, including when to stop

Follow-up 6: one agent with good tools is the default

Key takeaways

Links

🌡️ What Do Temperature and Top-p Really Do? Test Them on Amazon Bedrock From the CLI (Hands-on)

🎤 The interview question

🧰 Before you start

🧊 Step 1: Low temperature, three runs

🔥 Step 2: High temperature, same prompt

🎯 Step 3: Top-p, low vs high

🧨 Break it on purpose

🧹 Cost and cleanup

💬 Say it in the interview

🎯 What you can now say (and the exam)

📚 Go deeper

💸 What Does One Amazon Bedrock Prompt Cost? Find Out From the CLI (Hands-on)

🎤 The interview question

🔧 Step 1: Set up

💬 Step 2: Send one prompt with the Converse API

🧮 Step 3: Turn tokens into money

🧨 Break it on purpose

🧹 Cost and cleanup

🎯 What you can now say in the interview (and the exam)

📚 Go deeper

When is machine learning the wrong answer to a business problem?

The strong answer: machine learning earns its place only when the rule is unknown and the data can teach it

Follow-up 1: a known rule beats a learned guess when output must be predictable

Follow-up 2: without enough clean, representative data there is no model worth building

Follow-up 3: a project with no measurable business goal cannot be justified

Follow-up 4: when a decision must be explained, an opaque model carries a real cost

Follow-up 5: choosing the wrong type of AI is also a wrong answer

How AIF-C01 turns this judgment into scored questions

Key takeaways

Links

Fine-tuning versus continued pre-training for Amazon Nova 2

The strongest answer separates knowledge acquisition from behavior shaping

The right choice begins with the gap you need to close

Raw domain corpora point toward continued pre-training

Continued pre-training may be only the first adaptation stage

Data quality and data shape are different requirements

Nova 2 Lite supports both paths, but model selection remains a tradeoff

Key takeaways

Links

AWS AIF-C01: Bias, Variance & the AWS Tools That Detect Them

Bias has two meanings, and the scenario tells you which one applies

Variance appears as a gap between training and unseen performance

Dataset characteristics explain where disparities begin

Aggregate accuracy cannot establish fairness

Timing selects Clarify, Model Monitor, or A2I

Common distractors fail because they answer the wrong question

Key takeaways

Your LLM Prompt Costs More Than You Think — Build a Token & Cost Calculator in Colab