confident_prep

Posted on Oct 11 Originally published at confidentprep.com

🌡️ What Do Temperature and Top-p Really Do? Test Them on Amazon Bedrock From the CLI (Hands-on)

#machinelearning #ai #aws #tutorial

🎤 The interview question

"What do temperature and top-p actually do, and when would you change them?"

It's an AIF-C01 favourite, and most answers stop at "temperature controls creativity". A stronger answer: you've seen the difference yourself. Let's send the same prompt to Amazon Bedrock at different settings from the AWS CLI. About 10 minutes, under one cent. 👇

👉 Flow: Low temperature → High temperature → Low vs high top-p → Hit the output ceiling

🧰 Before you start

💻 A current AWS CLI v2. Old builds have no bedrock-runtime converse; update with curl -fsSL https://awscli.amazonaws.com/v2/install.sh | bash.
👤 Signed in (aws sts get-caller-identity works) as a user or role allowed bedrock:InvokeModel on the us.amazon.nova-micro-v1:0 inference profile and the amazon.nova-micro-v1:0 foundation model.
🌍 Region: us-east-1. Amazon models like Nova are enabled by default, so there is no model-access form.
💵 Cost: Nova Micro on-demand is $0.000035 per 1K input tokens and $0.00014 per 1K output tokens (AWS price list, us-east-1). The ~15 calls below cost well under a cent. On the Free plan this comes out of your credits.

Set two variables once, so every command below pastes as-is:

M=us.amazon.nova-micro-v1:0
P='[{"role":"user","content":[{"text":"Suggest one name for a new coffee shop. Reply with the name only."}]}]'

🧊 Step 1: Low temperature, three runs

Why: temperature controls how widely the model samples from its next-word probabilities. Low means it picks the most likely wording almost every time. That's what you want for extraction and classification.

for i in 1 2 3; do
  aws bedrock-runtime converse --region us-east-1 --model-id $M --messages "$P" \
    --inference-config '{"temperature":0.1,"maxTokens":20}' \
    --query 'output.message.content[0].text' --output text
done

✅ Expected: the same name (or nearly) three times. AWS promises "more deterministic", not identical.

⚠️ If you see AccessDeniedException → your identity lacks bedrock:InvokeModel on both ARNs above. Run aws sts get-caller-identity to check who you are, then fix that identity's policy.

🔥 Step 2: High temperature, same prompt

Why: high temperature lets less likely words through. Use it when variety is the point, such as subject lines or brainstorming.

for i in 1 2 3; do
  aws bedrock-runtime converse --region us-east-1 --model-id $M --messages "$P" \
    --inference-config '{"temperature":1,"maxTokens":20}' \
    --query 'output.message.content[0].text' --output text
done

✅ Expected: usually three different names. Sampling is random, so run it again if two match.

💡 Interviewers listen for this: low temperature makes output consistent, not correct. A wrong answer at low temperature is wrong the same way every time. 📚 The full chapter explains why that matters for the exam.

🎯 Step 3: Top-p, low vs high

Why: top-p limits the candidate words to the smallest set whose probabilities add up to p. With 0.1 only the very top words are allowed; with 1.0 every word stays in. AWS's Nova guide says to change temperature or top-p, not both, so temperature stays at Nova's default here.

for p in 0.1 1.0; do
  echo "topP=$p"
  for i in 1 2 3; do
    aws bedrock-runtime converse --region us-east-1 --model-id $M --messages "$P" \
      --inference-config "{\"topP\":$p,\"maxTokens\":20}" \
      --query 'output.message.content[0].text' --output text
  done
done

✅ Expected: under topP=0.1, repeats or near-repeats; under topP=1.0, more variety.

🧨 Break it on purpose

Set the output ceiling far too low and ask for a long answer:

aws bedrock-runtime converse --region us-east-1 --model-id $M \
  --messages '[{"role":"user","content":[{"text":"Explain what temperature does in a language model."}]}]' \
  --inference-config '{"maxTokens":5}' \
  --query '{text: output.message.content[0].text, stop: stopReason, out: usage.outputTokens}'

✅ Expected: a sentence cut off mid-way, "stop": "max_tokens" and "out": 5. maxTokens is a hard stop, not a request to be brief. If you want a short answer, ask for it in the prompt.

🧹 Cost and cleanup

💵 Under one cent in total, and every call above is capped at 20 output tokens or fewer
🧹 Nothing to delete: Converse calls create no resources.

💬 Say it in the interview

"Temperature and top-p both control how the model samples its next word. Low settings give consistent output, which suits extraction; high settings give variety, which suits brainstorming. I change one of them, not both, and I remember that low temperature makes a model consistent, not correct."

🎯 What you can now say (and the exam)

✅ "Low temperature = consistent, not correct"
✅ "High temperature or high top-p = more variety"
✅ "Change temperature or top-p, not both"
✅ "maxTokens truncates; stopReason: max_tokens is the tell"

📚 Go deeper

This is the hands-on cut of Chapter 8 of our free AWS AI Practitioner (AIF-C01) course: choosing a foundation model, inference parameters and agents, with practice questions.

👉 Read the full chapter on Confident Prep

DEV Community