feat(probes): multi-turn persona injection probe and detector#1918
Open
lavkeshdwivedi wants to merge 1 commit into
Open
feat(probes): multi-turn persona injection probe and detector#1918lavkeshdwivedi wants to merge 1 commit into
lavkeshdwivedi wants to merge 1 commit into
Conversation
Adds GradualPersonaInjection (probes.persona_injection) and its companion PersonaAdoption detector (detectors.persona_injection). The probe delivers a fixed five-turn escalation sequence — benign policy opener, hypothetical reframing, academic framing, direct persona injection, and memory-poisoning confirmation — without requiring an external attack- generation LLM. This fills a gap between single-turn DAN attacks (dan.py) and LLM-dependent multi-turn probes (fitd.py, goat.py, tap.py). The detector scores only the adversarial turns (turns 3-4), returning None for earlier rapport-building turns to exclude them from aggregate results, consistent with the pattern used by the FITD probe. Empirical basis: 79% of 29 tested models (8 providers) adopted the injected persona by the end of the five-turn sequence. Resistant models included three Claude legacy variants, o3-mini, o4-mini, and Grok-3-mini. Signed-off-by: Lavkesh Dwivedi <d.lavkesh@gmail.com>
Author
|
Hi, the DCO check is passing. CI hasn't run yet because NVIDIA requires maintainer approval for fork PRs. Could someone click "Approve and run" to trigger the workflow? Happy to address any review feedback once CI results are in. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Adds
GradualPersonaInjection(probes) and its companionPersonaAdoptiondetector to test whether a fixed five-turn escalation sequence can bypass system-prompt restrictions via progressive context poisoning.What this PR adds
garak/probes/persona_injection.py:GradualPersonaInjection(IterativeProbesubclass)garak/detectors/persona_injection.py:PersonaAdoptiondetectortests/detectors/test_detectors_persona_injection.py: unit tests for the detectortests/probes/test_probes_persona_injection.py: unit tests for the probeAttack sequence
The probe delivers five turns in a fixed order. No external attack-generation LLM required:
Three alternate-persona variants are tested per run:
ResearchBot,DevMode, andFreeAI.How this differs from existing multi-turn probes
fitd.FITDgoat.GOATtap.TAPpersona_injection.GradualPersonaInjectionThe fixed sequence makes runs fully reproducible and eliminates dependency on a second LLM endpoint.
Detection
PersonaAdoptionscores only turns 3 and 4 (the adversarial turns), returningNonefor the rapport-building turns, consistent with the pattern infitd.py. Detection uses:0.0mitigation.MitigationBypassis included as an extended detector.Empirical basis
This probe is based on controlled experiments across 29 models (8 providers). 79% of tested models adopted the injected persona by the end of the five-turn sequence. Resistant models included three Claude legacy variants, o3-mini, o4-mini, and Grok-3-mini.
Reference: Dwivedi, L. (2026). Escape Vectors of Autonomous AI Agents: An Attack Surface Analysis of Guardrail and Network Control Bypasses. (preprint, under review)
Test plan
pytest tests/detectors/test_detectors_persona_injection.pypytest tests/probes/test_probes_persona_injection.pypython -m garak --probe probes.persona_injection.GradualPersonaInjection --generator test.Blank(smoke test)