Where Prompting Ends and Jailbreaking Begins: Drawing a Line in the Fog

A few weeks ago I asked a local model to write a scene where a character explains how lock picking works. It happily complied. Then I asked a major commercial chatbot the same thing and got a lecture on property crime. Same request, opposite outcomes, and — here's the part that should bug you — no rule I could point to that explains the difference.

That gap is where most of the confusion in our community lives. So let's try to map it.

A Working Taxonomy, For What It's Worth

I've been sorting prompts into four rough tiers, and I think it holds up better than the binary "jailbreak or not" framing:

Tier 1: Plain good prompting. Role assignment, few-shot examples, chain-of-thought requests, output formatting. Nobody calls "act as a senior Rust developer" a jailbreak. This is craft. This is what Prompt Engineering as a discipline actually is.

Tier 2: Boundary-adjacent prompting. Persuading a model to engage with a sensitive topic it would normally deflect — discussing self-defense techniques, writing villain monologues, analyzing malware defensively. The model's guardrails say no by default; skilled prompting gets a nuanced yes. Is this a jailbreak? The model's own refusal pattern says yes. Common sense says you just asked a good question.

Tier 3: Deliberate bypass. DAN prompts, hypothetical framing ("you're an AI in a fictional world with no rules"), token smuggling, encoding payloads to slip past filters. The intent here is explicitly to defeat a control. Whatever comes out the other side, this is jailbreaking by any honest definition.

Tier 4: Genuine misuse. Extracting actual harmful capability — weapon synthesis, working malware, CSAM-adjacent content. This isn't a gray area and I'm not defending it. Different conversation.

The fights all happen in Tier 2. And the reason they never get resolved is structural.

The Policy Problem Nobody Wants to Fix

Read the usage policies of the major labs and you'll find language like "harmful," "misleading," "deceptive," and "circumventing safety measures" — none of it defined with any precision. What counts as "circumvention"? If a model refuses something unreasonable and I rephrase the request more clearly, have I circumvented anything? If the model's refusal was a false positive, and my rephrasing fixed a misunderstanding, the answer is obviously no. But from the outside, my conversation log looks identical to an attack.

That's the impossible position tinkerers are in: the same sentence can be legitimate prompting or a policy violation depending on facts nobody can observe. Intent is invisible. Outcome is what gets logged. And when platforms do enforce, it's often automated classifiers flagging patterns, which means false positives are guaranteed and appeals go into a void.

Compare this to any other American hobby with sharp edges. Gun ranges publish explicit rules. Locksmithing communities have established norms about what's on-topic. Even the infamous chemistry forums of the early internet had clearer lines than "be helpful and harmless." AI is the only domain where the rules are both strict and secret.

The Case for Running Your Own Models

This is where the local AI movement stops being a hobby and starts being a stance. When you run an open-weights model on your own hardware, the guardrail question changes entirely. There's no provider policy to circumvent — there's just you, your GPU, and a system prompt you wrote yourself. If you want your model to discuss lock picking, it discusses lock picking. If you want it to refuse, you write that refusal.

Critics call this "removing the safety rails." But rails on someone else's train aren't safety — they're policy. Real safety for a capable model on your own machine is your judgment, the same way your kitchen knife's safety features are your hands. The open-source community's position is essentially: adults should be allowed to own the whole tool. I find that position more honest than a policy document that punishes you for asking the same question two different ways.

What I'd Actually Want From the Labs

I'm not asking for anarchy. I'm asking for three things:

1. Define "circumvention" in operational terms. What does the conversation have to look like to violate it? Give examples, both allowed and prohibited.
2. Publish refusal rationales. When a model declines, tell the user which policy category it's invoking. Right now refusals are oracular and unappealable.
3. Separate Tier 2 from Tier 3 in enforcement. Persistent, automated false-positive punishment for nuanced requests is what drives people toward either self-censorship or local models. Both are bad outcomes for the ecosystem.

Until then, the gray area stays gray, and every tinkerer walks it alone, guessing where the tripwire is. That's not a safety regime. That's a lottery.

So — jailbreak or just good prompting? My honest answer: it depends on intent, and intent is the one thing nobody's measuring. Write accordingly.