Crank It Past 1.0: A Field Guide to the Sampling Knobs That Reshape Your AI

You've written the perfect system prompt. Your persona is dialed in, your examples are airtight. And yet the output is either a robotic rerun of the same three sentences or a hallucination spiral that invents a second moon. Before you blame the model, check the two numbers you've been ignoring: temperature and top-p.

These sampling settings decide how your model picks its next word, and they're the difference between a copilot and a chaos gremlin. Most people never touch them. You're not most people.

What Temperature Actually Does

Every token your model generates comes from a probability distribution — a ranked list of likely next words. Temperature rescales that list before the model commits.

• Low temperature (0.1–0.4): The model becomes a coward. It almost always picks the top choice. Output is predictable, repetitive, and safe. Great for extraction, terrible for brainstorming.
• Medium (0.6–0.8): The sweet spot for most work. The model has opinions but doesn't wander off.
• High (1.0+): The long tail gets heard. Rare words, weird associations, occasional brilliance. Also, occasionally, total nonsense — past roughly 1.3, most models start speaking in word salad.

Here's the same prompt — "give me a metaphor for burnout" — at three temperatures:

0.2: "Burnout is like a machine that has been running too long without maintenance."

0.7: "Burnout is a phone at 3% battery that keeps opening apps by itself."

1.2: "Burnout is the taxidermy of enthusiasm, stitched and posed in your meeting calendar."

All three are coherent. Only one is worth putting in an essay. Only one would survive a code review.

Top-p: Temperature's Quieter Sibling

Where temperature rescales everything, top-p (nucleus sampling) draws a hard line: the model may only choose from the top words whose probabilities add up to P.

• top-p 0.1: Ruthlessly narrow. Only the most probable words survive. Robotic.
• top-p 0.9: The default on most platforms. Enough variety to sound human.
• top-p 1.0: No filter. Everything's on the table — which, combined with high temperature, is how you get a chatbot that ends every paragraph with the word "moisture."

The practical rule: tune one, not both. Crank temperature and squeeze top-p and you're fighting yourself. Most tinkerers set top-p between 0.9 and 0.95 and leave it, using temperature as the main dial.

Settings Worth Stealing

For coding (temp 0.1–0.3): You want the statistically correct answer, not the model's jazz interpretation of a sort function. Low temperature, top-p 0.9. If your codebot keeps hallucinating library names, drop the temperature before you touch anything else.

For fiction and roleplay (temp 0.8–1.0): Characters need to surprise you. Bump toward 1.0 and add a repetition penalty around 1.1 if your narrator develops a catchphrase. This is also where jailbreak-style personas live — high temperature plus a loosened system prompt is the classic recipe for getting a local model out of its corporate voice.

For brainstorming (temp 0.9–1.1): You want volume and weirdness. Generate ten ideas, expect seven to be garbage, keep the taxidermy metaphor.

For summarization and data extraction (temp 0.0–0.2): Determinism is the feature. Run it twice; if the outputs differ meaningfully, your prompt is the problem.

The Chaos Test

Want to know your model's breaking point? Run the same creative prompt at temperature 0.5, 1.0, and 1.5 and compare. The 0.5 run should be competent and boring. The 1.0 run should have at least one line you wish you'd written. The 1.5 run should be a beautiful disaster — if it's still coherent at 1.5, your model is either exceptional or heavily fine-tuned for it.

One Last Thing

Sampling settings are vibes engineering: the numbers above are starting points, not commandments. Every model responds differently, and quantized local models especially can get unhinged at temperatures where their full-precision cousins stay smooth. Save your presets, name them honestly ("essay mode," "gremlin mode"), and stop letting the defaults write your outputs for you.

The dials were always there. Now they're yours.