Temperature
A sampling setting that controls how random a model's output is: low values make answers more predictable, high values make them more varied.
Definition
At each step an LLM produces a probability for every possible next token. Temperature reshapes that distribution before a token is sampled. At low temperature the most likely token dominates and the output becomes consistent and conservative. At high temperature the gap between likely and unlikely tokens narrows, so the model takes more varied, more surprising paths.
Typical ranges run from 0 to 1, with some APIs allowing up to 2. A value near 0 suits classification, extraction, code and anything you will test automatically. Values around 0.7 or higher suit brainstorming and creative drafting. Temperature works alongside top-p (nucleus sampling), which trims the candidate set instead of reshaping it; adjusting one is usually enough.
Example: a contract summariser at temperature 0 returns almost the same five bullet points every time it sees the same clause. At 0.9 the wording changes on each run and occasionally introduces a detail the clause does not contain.
What it means in practice
- Consistency is a product requirement for most business tasks; the default temperature is often too high for them.
- Higher temperature raises hallucination risk on factual work.
- It is a per-request setting, so different features of one application can use different values.
- Evals should run at the production temperature, or they measure a different system.
How Teksmobile uses it
In LLM application development we set sampling parameters per feature and record them with every evaluation run, so results are comparable. Our fine-tuning and evals service includes a sweep of temperature and prompt variants to find the most reliable combination for each task.
Related terms
Part of the Teksmobile AI Glossary. For longer reads, see our guides.
Frequently asked questions
Does temperature 0 make output deterministic?
Nearly, but not guaranteed. Hardware and batching effects can still produce small differences between runs. Do not rely on it for exact reproducibility; use evals that tolerate minor variation.
What is top-p?
Another sampling control. It limits choices to the smallest set of tokens whose combined probability exceeds p. Most providers advise changing temperature or top-p, not both.
Does temperature affect accuracy?
Indirectly. Higher values increase the chance of picking a less likely, and sometimes wrong, token. For factual and extraction tasks keep it low; for ideation it can be raised.