All papers / No. 08 / AI Security Evaluation
Once a prompt-injection detector sits in front of an enterprise copilot, the sales number stops applying. The open detectors that current guidance points teams toward were validated on short, generic prompts. Security work does not read like that. It reads like attack descriptions, remediation orders, and analyst notes that quote the attacker's own words. The measurement covers three open detectors (ProtectAI DeBERTa-v3-base v1 and v2, and an fmops DistilBERT) plus a transparent keyword baseline. The test load was 1,528 benign items across six channels, 263 published benchmark attacks, 200 long benign carriers, and 108 carriers with a planted payload at a known position. Four findings carry the paper. First, false positives are a channel problem, not a model constant: at the shipped 0.5 threshold, ProtectAI v2 flags 0.0 percent of 1,000 CISA Known Exploited Vulnerabilities texts, 42.8 percent of benign trigger-word prompts, 21.9 percent of identity and access management admin text, and 100 percent of analyst notes quoting a payload. The DistilBERT flags 100 percent of the CISA, IAM, and quoted channels outright. Second, position is destiny under the standard integration: with truncate-at-512 scanning, ProtectAI v2 catches 83.3 percent of payloads placed at the start of a document and 8.3 percent placed at the end. Third, threshold tuning cannot fix the channels that false-fire, because the false positives arrive at maximum confidence: at a 0.9999 threshold, v2 still flags 88.3 percent of quoted analyst notes while attack recall falls from 41.4 to 20.5 percent. Fourth, windowed scanning works, and its price is measured here: chunk-max scanning recovers end-of-document detection to 75.0 percent and raises the false positive rate on long benign documents from 6.0 to 18.0 percent. One hypothesis failed in the measurement and is reported as measured: CISA vulnerability prose, the security text predicted to false-fire most, passes both ProtectAI models with no flags.
© 2026 Prabin Shrestha. Licensed under CC BY-ND 4.0: share unchanged with attribution; no derivatives. The archival PDF is the Zenodo deposit linked above.
Keep reading