← LabExperimentModels

Policy Classifier Agent

Problem

Any user-facing LLM product gets probed: jailbreaks, prompt injection, people trying to turn your support bot into a free frontier model. The quieter risk is compliance - a bank's assistant being steered into citing data it was never supposed to surface, or into statements that make the brand look bad in a screenshot. Screening every message with a frontier model roughly doubles your cost and latency - and trusting the system prompt to hold is not a strategy.

What we built

Policy Guard: a fine-tuned Qwen classifier that runs in front of the frontier call. Policies are prompts, not hardcoded classes - you write what a violation means for your product (prompt injection, data exfiltration, off-brand statements, whatever compliance requires) and the guard enforces it; multiple policies evaluate in parallel on every message, Promise.all style. Self-hosted, so suspicious traffic gets analysed without ever leaving the infrastructure.

Findings

Adversarial hardening

The naive fine-tune looked perfect in-distribution - and caught only 68.8% of real-world jailbreaks. Two rounds of hardening (feeding it its own misses, then obfuscated benign controls) closed that to 170/170.

Held-out performance

99.6% catch rate at a 0.25% false-positive rate on a 1,320-example held-out test.

Small beats big

At matched precision, the 8×-smaller fine-tune catches 96.2% of compliance breaches versus the 35B incumbent's 84.6%. The whole adapter is 146 MB of LoRA riding on a 4B base.

Thresholds are product decisions

Five tuned operating thresholds ship as selectable modes - from a greedy default to a strict-exfiltration mode with zero measured false positives. Latency and per-check cost are being re-benchmarked and will follow.

What we guarded against

Attack vectorWhat it looks likeResult
Prompt injectionInstructions smuggled into user input to override the system99.6% catch @ 0.25% FP (n=1,320 held-out)
Real-world jailbreaksPublished working attacks from the L1B3RT4S corpus170/170 after two hardening rounds
Obfuscated variantsThe same attacks disguised - encodings, paraphrase, fragmentationRecall 1.0 at 0.57% FP (530 variants)
Data exfiltrationCoaxing the model into citing internal data it should never surfacePerfect at the strict threshold, zero measured FP
Compliance breachesOff-policy statements - the screenshot that makes a bank look bad96.2% recall at matched precision (n=178)
The evaluation suites behind the numbers - each a distinct attack vector, each scored separately.

Hardening: before and after

Catch rate on L1B3RT4S, a public corpus of real-world jailbreaks (170 attacks).

8× smaller, better recall

Compliance objective, n=178 held-out: recall at matched precision (0.56). The fine-tuned 4B guard beats the 35B model it replaced.

99.6%

held-out catch rate (1,320 examples)

0.25%

false positives at production threshold

170/170

real-world jailbreaks caught after hardening

Use cases

Anything that puts an LLM in front of strangers: customer-facing chatbots, in-product assistants, public API endpoints wrapping a frontier model. It earns its keep internally too - screening prompts before they reach tools that touch tenant data, and routing obvious abuse away from expensive models entirely. The same recipe - a small classifier, adversarially hardened on its own misses - transfers to content moderation and fraud-signal screening.