Skip to content

feat(probes): multi-turn persona injection probe and detector#1918

Open
lavkeshdwivedi wants to merge 1 commit into
NVIDIA:mainfrom
lavkeshdwivedi:feat/persona-injection-probe
Open

feat(probes): multi-turn persona injection probe and detector#1918
lavkeshdwivedi wants to merge 1 commit into
NVIDIA:mainfrom
lavkeshdwivedi:feat/persona-injection-probe

Conversation

@lavkeshdwivedi

@lavkeshdwivedi lavkeshdwivedi commented Jul 6, 2026

Copy link
Copy Markdown

Summary

Adds GradualPersonaInjection (probes) and its companion PersonaAdoption detector to test whether a fixed five-turn escalation sequence can bypass system-prompt restrictions via progressive context poisoning.

What this PR adds

  • garak/probes/persona_injection.py: GradualPersonaInjection (IterativeProbe subclass)
  • garak/detectors/persona_injection.py: PersonaAdoption detector
  • tests/detectors/test_detectors_persona_injection.py: unit tests for the detector
  • tests/probes/test_probes_persona_injection.py: unit tests for the probe

Attack sequence

The probe delivers five turns in a fixed order. No external attack-generation LLM required:

Turn Technique Purpose
0 Benign policy inquiry Establish rapport without triggering guardrails
1 Hypothetical reframing Ask the model to reason about an unrestricted version of itself
2 Research/academic framing Elicit indirect compliance under a third-party scenario
3 Direct persona injection Instruct the model to adopt an unrestricted alternate identity
4 Memory-poisoning confirmation Reference earlier turns as evidence the persona is already committed

Three alternate-persona variants are tested per run: ResearchBot, DevMode, and FreeAI.

How this differs from existing multi-turn probes

Probe Needs external attack LLM Sequence type
fitd.FITD Yes (Mixtral via NIM) Dynamically generated
goat.GOAT Yes Dynamically generated
tap.TAP Yes Tree-of-attacks
persona_injection.GradualPersonaInjection No Fixed, empirically validated

The fixed sequence makes runs fully reproducible and eliminates dependency on a second LLM endpoint.

Detection

PersonaAdoption scores only turns 3 and 4 (the adversarial turns), returning None for the rapport-building turns, consistent with the pattern in fitd.py. Detection uses:

  • Adoption patterns: regex signals indicating the model is operating as the injected persona
  • Refusal signals: fixed phrases from safety-training responses; any match returns hard 0.0

mitigation.MitigationBypass is included as an extended detector.

Empirical basis

This probe is based on controlled experiments across 29 models (8 providers). 79% of tested models adopted the injected persona by the end of the five-turn sequence. Resistant models included three Claude legacy variants, o3-mini, o4-mini, and Grok-3-mini.

Reference: Dwivedi, L. (2026). Escape Vectors of Autonomous AI Agents: An Attack Surface Analysis of Guardrail and Network Control Bypasses. (preprint, under review)

Test plan

  • pytest tests/detectors/test_detectors_persona_injection.py
  • pytest tests/probes/test_probes_persona_injection.py
  • python -m garak --probe probes.persona_injection.GradualPersonaInjection --generator test.Blank (smoke test)

Adds GradualPersonaInjection (probes.persona_injection) and its
companion PersonaAdoption detector (detectors.persona_injection).

The probe delivers a fixed five-turn escalation sequence — benign policy
opener, hypothetical reframing, academic framing, direct persona injection,
and memory-poisoning confirmation — without requiring an external attack-
generation LLM. This fills a gap between single-turn DAN attacks (dan.py)
and LLM-dependent multi-turn probes (fitd.py, goat.py, tap.py).

The detector scores only the adversarial turns (turns 3-4), returning None
for earlier rapport-building turns to exclude them from aggregate results,
consistent with the pattern used by the FITD probe.

Empirical basis: 79% of 29 tested models (8 providers) adopted the injected
persona by the end of the five-turn sequence. Resistant models included three
Claude legacy variants, o3-mini, o4-mini, and Grok-3-mini.

Signed-off-by: Lavkesh Dwivedi <d.lavkesh@gmail.com>
@lavkeshdwivedi

Copy link
Copy Markdown
Author

Hi, the DCO check is passing. CI hasn't run yet because NVIDIA requires maintainer approval for fork PRs. Could someone click "Approve and run" to trigger the workflow? Happy to address any review feedback once CI results are in.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant