If you've worked with large language models (LLMs), you have likely come across a parameter called temperature.
Despite its name, temperature has nothing to do with hardware or system performance. It controls how predictable or creative an LLM's responses are.
What Is Temperature?
Temperature influences how the model chooses the next word from its list of possible predictions.
Think of it as a creativity slider:
- Low temperature (0-0.3) -> More predictable and focused responses.
- Medium temperature (0.5-0.7) -> Balanced creativity and accuracy.
- High temperature (0.8-1.5+) -> More diverse and creative outputs.
The higher the temperature, the more willing the model is to choose less likely words.
Example
Prompt:
Explain what Kubernetes is.
Temperature = 0
"Kubernetes is an open-source container orchestration platform that automates deployment, scaling, and management of containerized applications."
The answer is consistent and factual.
Temperature = 1
"Kubernetes is like an operating system for your containers, helping applications scale, recover from failures, and run efficiently across clusters."
Still correct, but phrased differently.
Temperature = 2
"Kubernetes acts as the conductor of a container orchestra, ensuring every application plays its part in harmony across a distributed environment."
More creative, but less precise.
When to Use Low Temperature
Low temperature is preferred when accuracy matters:
- RAG applications
- Technical support chatbots
- Code generation
- Documentation assistants
- Question-answering systems
The goal is consistency and reliability.
When to Use High Temperature
Higher temperatures work better for:
- Brainstorming
- Story writing
- Marketing content
- Social media posts
- Creative ideation
The goal is diversity and originality.
Temperature in RAG
For RAG systems, temperature is usually kept low (around 0-0.3).
Why?
The retrieved documents already provide the knowledge. The model's job is to use that information, not invent new details.
Higher temperatures can increase the likelihood of hallucinations and inconsistent answers.
Common Misconception
Many people assume:
Higher temperature = smarter AI
Not true.
Temperature only affects randomness. It does not increase the model's knowledge or intelligence.
A higher temperature simply makes the model explore less likely responses.
If you need accuracy, keep it low.
If you need creativity, increase it.
Top comments (0)