Temperature in the OpenAI API

Recently, I built an AI harness for an agent. Much of what I learned along the way was about different aspects of the OpenAI API. Today, I’m going to write about the temperature parameter.

Temperature controls how random the model’s token choices are. Low values make the model pick the most likely next token almost every time. High values make its choices more random. Different use cases call for different temperatures. Values usually range from 0 to 1, and in some cases go up to 2.

Use cases for different temperatures

Low: deterministic, precise

• Code generation and refactoring. You want the most likely, correct syntax, not creative variable names.
• Structured output. JSON, SQL, regex, or function/tool calls, where one stray token breaks parsing.
• RAG. Answering from retrieved context, where you want less invention.
• Math and logic.
• Evals and testing. More reproducible outputs make prompt changes easier to compare.

But there’s one catch: a model can’t be 100% deterministic. So if you need truly deterministic results, you should probably find another way to solve the problem.

High: diverse

• Brainstorming. Product names, taglines, feature ideas, where variety matters more than a single “best” answer.
• Creative content.
• Synthetic data generation. Varied test inputs, fake user personas, diverse training examples.