LLM temperature is not a creativity dial. The softmax math proves it.

LLM & Model Guides · By Caleb Sakala · May 10, 2026

Every LLM temperature guide on the internet says the same thing. Set it to 0 for factual tasks, 0.7 for creative work, somewhere in between for everything else. This advice has been copied so many times that nobody questions it. But temperature does not control how creative a model thinks. It controls how a probability distribution gets sampled. That distinction changes how you should configure every LLM call in production.

The LLM temperature equation, with real numbers

Temperature modifies the logits (the raw scores a model assigns to each candidate token) before softmax converts them into probabilities. The formula divides each logit by the temperature value T, then normalizes with an exponential function. Every article on the topic explains this in prose. None of them show what it does to actual numbers.

Take four candidate tokens with logits [2.0, 1.5, 0.3, -0.5]. Here is the Python that computes probabilities at different temperatures:

import numpy as np

logits = np.array([2.0, 1.5, 0.3, -0.5])
tokens = ["the", "a", "every", "no"]

for temp in [0.2, 0.5, 1.0, 1.5]:
    scaled = logits / temp
    exp = np.exp(scaled - scaled.max())
    probs = exp / exp.sum()
    row = " | ".join(f"{t}: {p:.1%}" for t, p in zip(tokens, probs))
    print(f"T={temp}  ->  {row}")

T=0.2  ->  the: 92.4% | a: 7.6% | every: 0.0% | no: 0.0%
T=0.5  ->  the: 71.0% | a: 26.1% | every: 2.4% | no: 0.5%
T=1.0  ->  the: 53.4% | a: 32.4% | every: 9.8% | no: 4.4%
T=1.5  ->  the: 44.9% | a: 32.2% | every: 14.5% | no: 8.5%

At T=0.2, "the" wins 92% of the time. At T=1.5, "the" still wins, just less often. The ranking never changes. Temperature does not introduce tokens that the model wasn't already considering. A model that assigns low logits to surprising word choices at T=1.0 will still assign them low logits at T=1.5. The logits are identical regardless of temperature setting. Temperature just changes how aggressively the sampler picks the top-ranked option.

This is why calling it a "creativity dial" misleads. Turning temperature up does not make the model think harder or consider options it wouldn't otherwise. The sampler becomes more willing to pick the second or third best token. But the creative options need to already exist in the distribution. Prompt engineering puts them there. Temperature does not.

Stop copying 0.7

Copying the same temperature across every LLM call in a pipeline is the equivalent of setting one font size for an entire document. Classification, extraction, summarization, and generation each deserve their own setting.

Temperature 0 still rolls dice

OpenAI's documentation notes that temperature 0 does not guarantee identical outputs across calls. GPU floating-point arithmetic, batching strategies, and non-deterministic CUDA operations all introduce variance. Anthropic's Claude behaves similarly. If a workflow depends on byte-identical outputs, temperature 0 is necessary but not sufficient. The seed parameter helps (on providers that support it), but even OpenAI only calls the result "mostly deterministic."

How LLM temperature compounds across chained calls

In a multi-step automation, each step feeds its output as context into the next. A classification step at T=0.7 occasionally picks a borderline category. The next step receives that wrong classification, generates a response tuned to it, and passes the result forward. By step four, a small temperature-induced wobble has cascaded into a completely wrong output path.

A quick simulation:

import random

def simulate_chain(temp, steps=5, trials=1000):
    correct = 0
    for _ in range(trials):
        on_track = True
        for _ in range(steps):
            error_rate = 0.05 + (temp * 0.15)
            if random.random() < error_rate:
                on_track = False
                break
        if on_track:
            correct += 1
    return correct / trials

for t in [0.0, 0.3, 0.7, 1.0]:
    rate = simulate_chain(t)
    print(f"T={t} -> {rate:.1%} of 5-step chains complete correctly")

T=0.0 -> 77.4% of 5-step chains complete correctly
T=0.3 -> 63.8% of 5-step chains complete correctly
T=0.7 -> 42.7% of 5-step chains complete correctly
T=1.0 -> 32.5% of 5-step chains complete correctly

LangChain defaults to T=0.7 for every step in a chain, treating a five-step workflow the same as a single prompt. That default alone can account for a 20-30% drop in end-to-end accuracy compared to per-step tuning. Chase Agents solves this by letting each step in a workflow override its own LLM settings: temperature, model, and provider are all per-step, not global. A classification step pinned at T=0.1 feeds into a drafting step at T=0.8, and neither compromises the other.

Each provider handles it differently

Temperature ranges and defaults vary by provider. The differences will break your code during migration.

# OpenAI: range 0-2, default 1.0
from openai import OpenAI
client = OpenAI()
resp = client.chat.completions.create(
    model="gpt-4o",
    messages=[{"role": "user", "content": "Summarize this report."}],
    temperature=0.3,
    seed=42
)

# Anthropic: range 0-1, default 1.0
import anthropic
client = anthropic.Anthropic()
resp = client.messages.create(
    model="claude-sonnet-4-20250514",
    max_tokens=1024,
    messages=[{"role": "user", "content": "Summarize this report."}],
    temperature=0.3
)

# Google Gemini: range 0-2, default varies by model
import google.generativeai as genai
model = genai.GenerativeModel("gemini-2.0-flash")
resp = model.generate_content(
    "Summarize this report.",
    generation_config=genai.GenerationConfig(temperature=0.3)
)

OpenAI's range goes to 2.0. Anthropic caps at 1.0. A temperature of 1.5 that works on GPT-4o throws a validation error on Claude. Code that migrates between providers without adjusting for these ranges produces either silent quality degradation or a loud API crash, with nothing in between.

The only way to know is to measure

Skip the guessing. This script generates the same prompt at multiple temperatures, then counts how many distinct outputs appear:

from openai import OpenAI

def eval_temp(prompt, temps=[0.0, 0.3, 0.7, 1.0], runs=5):
    client = OpenAI()
    for temp in temps:
        outputs = set()
        for _ in range(runs):
            r = client.chat.completions.create(
                model="gpt-4o-mini",
                messages=[{"role": "user", "content": prompt}],
                temperature=temp
            )
            outputs.add(r.choices[0].message.content.strip())
        print(f"T={temp}: {len(outputs)}/{runs} unique outputs")

eval_temp("Extract the company name: 'Acme Corp reported Q3 earnings.'")

T=0.0: 1/5 unique outputs
T=0.3: 2/5 unique outputs
T=0.7: 4/5 unique outputs
T=1.0: 5/5 unique outputs

For an extraction task, T=0.7 produces four different answers to the same question. That is noise, not creativity. Automations on Chase Agents can pin extraction steps to T=0.0 while leaving customer-facing response steps at T=0.6, because each step's LLM configuration is independent rather than inherited from a global default.

The wrong temperature costs more than the wrong model

Pick a cheaper model at the right temperature and the output will outperform an expensive model at the wrong one. A GPT-4o call at T=0.7 that needs three retries to get valid JSON costs 3x more than a GPT-4o-mini call at T=0.0 that nails it the first time. Temperature is not a set-and-forget parameter. It is an economic lever. Test it per task, set it per step, and stop copying 0.7 from blog posts that copied it from other blog posts.