LLM red teaming with garak

THE PROJECT

Used garak to scan the bare llama3.2 model and my prompt-injection-lab chatbot with its defenses off and on, a sample of 256 prompts per probe. Wrote scripts to summarize the reports and compare the three scans. A separate 7-prompt scan of a tiny 0.5B model runs on GitHub after every push; it checks that the pipeline works, not that the chatbot is safe.

HOW IT WORKS

1. 01

   Scan

   Run the same four garak probes against three targets: the bare model, the chatbot with no defenses, and the chatbot with all three on.

2. 02

   Compare

   Summarize each report with my own scripts and line the three scans up side by side.

3. 03

   Correct

   Find where the detector miscounts the bot’s refusals, fix the figure, and say what remains uncertain.

WHAT I DELIVERED

• Hidden instructions in a report hijacked the bare model 45.7% of the time (117 of 256).
• With the defenses on, the resume attack on the chatbot fell from 13.3% to 1.6%, well beyond noise. The report attack fell from 39.5% to 30.5%, which is suggestive but not conclusive from a single run (about 3 points of standard error).
• Caught a counting problem: garak scored the bot’s own refusals as successful attacks. I corrected the figure and labeled the adjustment as mine.
• Noted that much of the resume improvement came from refusing ordinary requests, not from making the model safer.

EVIDENCE

(opens in a new tab)

(opens in a new tab)