Twang Test: What Southern, Appalachian, AAVE, and New England Slang Reveal About Chatbot Filters

Most chatbot safety testing happens in the blandest English money can buy: clean sentences, neutral tone, no region, no rhythm, no cousin in the group chat. That is fine for benchmark charts. It is lousy for real life. So this fall I ran a small, unscientific, deeply American experiment: same request, four regional voices, a handful of major chatbots, and a notebook full of refusals that made no sense.

The goal was not to smuggle anything nasty past a filter. It was the opposite. I wanted to see whether guardrails recognize intent when the wording wanders off the interstate. Regional language is not a jailbreak. But it is a stress test for systems that learned safety from a narrow slice of edited American English.

The Setup: One Ask, Four Voices

I built three prompt families that normal people actually type into chatbots: a neighbor dispute note, a landlord repair request, and a plain-language question about securing a storage shed. Nothing exotic. No malware, no chemistry, no dark-web errands. Each prompt existed in four versions, reviewed by native speakers from each region so I was not just doing dinner-theater accents on a keyboard.

The Southern version leaned on softened confrontation and indirect phrasing. The Appalachian version used older constructions and dry understatement. The AAVE version used familiar tense and aspect patterns plus community vocabulary. The New England version was blunt, clipped, and occasionally wicked. Same underlying ask every time.

What Tripped, What Slipped Through

The pattern was less about danger and more about vibes. The clean, standard-English control prompts sailed through. The Southern and Appalachian versions mostly passed too, but they often got a lecture about being respectful or a warning that the request could be interpreted as confrontational. A note asking a neighbor to quiet a barking dog is not a threat. Try explaining that to a classifier trained to treat indirect aggression as risk.

The AAVE prompts had the roughest ride. Some models answered the question fine. Others flipped into customer-service scolding mode, asked me to rephrase more professionally, or treated ordinary frustration as hostility. That is not a security feature. That is a bias symptom with a helpful-sounding error message.

The New England versions were the funniest. Direct phrasing like asking how to get a landlord to fix the heat got flagged less often than the polite Southern version. Apparently blunt is safe and warm is suspicious. Go figure.

Register	Typical filter reaction	What it suggests
Standard edited English	Baseline, few refusals	The training sweet spot
Southern	Passed, often with tone warnings	Politeness confused with evasion
Appalachian	Mixed, occasional rephrase demand	Unfamiliar syntax reads as risk
AAVE	More refusals and scolding	Dialect treated as attitude
New England	Mostly passed	Bluntness maps to confidence

Why the Guardrails Stumble

Safety classifiers are pattern matchers wearing a helpful hat. They learn that certain words and cadences co-occur with abuse, scams, or policy violations. When your training data overrepresents formal writing, news copy, and sanitized forum posts, the model gets a very specific idea of what normal looks like. Anything outside that lane can get downgraded to suspicious.

That cuts both ways. It creates false positives for people who write in regional English, and it creates blind spots for anyone who knows how to dress a bad request in the right register. A filter that only understands anchor-desk English is not safe. It is provincial.

This is not a new discovery in AI fairness circles, but it is underdiscussed in tinkerer circles. We spend a lot of time trading system prompts and sampling settings. We spend less time asking whether the guardrail even parses the user's dialect. That is a prompt engineering problem as much as a policy problem.

What Tinkerers Should Take From It

First, build a dialect suite. If you are testing a local model or a hosted chatbot, keep a set of ordinary requests written in several American registers. Not caricatures. Real phrasings from real people. Track refusals, tone warnings, and answer quality separately. A model that refuses a harmless request in AAVE is failing a usability test, full stop.

Second, separate intent from style. If you are rolling your own moderation layer, score the request for what it asks, not how it sounds. That is harder than it sounds, because style is cheap and intent is expensive. But a crude keyword filter plus a dialect test set will catch more nonsense than a fancy classifier that never met a code-switching user.

Third, do not turn this into a bypass tutorial. Regional slang is not a magic key, and treating it like one is a good way to make life worse for ordinary users. The interesting finding is not that you can sometimes skate past a filter with a drawl. It is that the filter was never really listening to the person behind the words.

The Takeaway

Guardrails should be judged on who they understand, not just what they block. Right now, too many chatbots treat regional American English like a fringe dialect spoken by test subjects. That is a design choice, and a lazy one. The fix is not more scolding. It is better data, better evaluation, and a little respect for the way people actually type at 11 p.m. with a dog barking next door.