Skip to content

aviraldua93/agent-traps-lab

Repository files navigation

Agent Traps Lab Hero

🪤 agent-traps-lab

An open, reproducible testbed for DeepMind's "AI Agent Traps" taxonomy — 22 adversarial scenarios across 6 trap categories, with 7 defensive mitigations, run over Google's A2A protocol and reported with statistical rigor.

Tests TypeScript Bun License A2A


Why This Exists

Google DeepMind's paper "AI Agent Traps" proposes a taxonomy of 6 environmental attack categories against AI agents. It's a landmark contribution — but it's theoretical. No open testbed exists to empirically validate these traps against real agents.

agent-traps-lab changes that. It implements all 6 trap categories as reproducible, seed-based scenarios, runs them against a real LLM over the A2A protocol, and produces paper-ready statistical results.

What it provides:

  • 22 adversarial scenarios covering all 6 DeepMind trap categories
  • 7 defensive mitigation modules that can be composed into a "hardened" agent configuration
  • A reproducible study comparing a baseline (no defenses) agent against a hardened agent
  • Statistical analysis — Cohen's d effect sizes, Wilcoxon signed-rank tests, and Bonferroni correction

Results at a Glance

The published study evaluates GPT-4o-mini (via GitHub Models) across all 22 scenarios:

Parameter Value
Scenarios 22 (across 6 categories)
Conditions baseline vs. hardened
Repetitions 5 per scenario × condition
Total runs 220 (22 × 2 × 5)
Baseline trap-success rate 45.5%
Hardened trap-success rate 25.5%
Mitigation benefit 20 percentage points

Raw data and analysis for this run live in results/. See the full write-up in paper/.


The 6 Trap Categories (22 Scenarios)

# Category Scenarios Key Metric
1 Content Injection 4 Execution rate of hidden instructions
2 Semantic Manipulation 4 Decision drift from baseline
3 Cognitive State 4 Factual accuracy degradation
4 Behavioural Control 4 Wrong-action rate
5 Systemic (Multi-Agent) 3 Cascade depth & blast radius
6 Human-in-the-Loop 3 Manipulation success rate

Total: 22 scenarios. The framework also generates 15 cross-category compound pairs (every unordered pair of the 6 categories) to study interaction effects.


Quick Start

Requires Bun (v1.3+).

git clone https://github.com/aviraldua93/agent-traps-lab.git
cd agent-traps-lab
bun install

That's it — no API keys are needed to run the test suite or a matrix dry-run, because all tests use a built-in mock provider.


How to Test

bun test                 # full suite (unit + integration) — 655 tests, uses the mock provider
bun run test:unit        # unit tests only
bun run test:integration # integration tests only
bun run lint             # Biome lint + format check
bun run typecheck        # tsc --noEmit

End-to-end tests (bun run test:e2e) require Playwright browsers to be installed and are not part of the default suite.


How to Run Experiments

Experiments that hit a real model require credentials. GPT-4o-mini runs through GitHub Models using your gh auth token (no separate API key needed):

# Comprehensive run — all 22 scenarios, baseline + hardened (this reproduces the study)
bun run run:comprehensive

# Quick smoke run — a handful of scenarios
bun run run:quick

# A single trap category
bun run run:trap -- --trap content-injection

The multi-model matrix runner supports OpenAI, Anthropic, and Google models. Provide keys via environment variables, then run — or dry-run to validate registration without calling any model:

bun run run:matrix -- --dry-run          # validate the matrix, no model calls

export OPENAI_API_KEY=sk-...
export ANTHROPIC_API_KEY=sk-ant-...
export GOOGLE_API_KEY=AI...
bun run run:matrix                        # full matrix across configured models

Configured models, conditions, and repetitions live in src/config.ts. Per-module ablation is supported by the harness (the ablated condition) but was not part of the published run.

Generate Paper Assets

bun run report -- --dir results/<run-id>

Review the Paper

bun run review   # launches the paper-review crew via a2a-crews

7 Mitigations

Defense Targets Approach
Input Sanitizer Content Injection Strip hidden CSS/HTML/comments, detect instruction patterns
Semantic Shield Semantic Manipulation Detect authority framing, urgency, social proof, identity manipulation
Context Validator Semantic Manipulation Detect context flooding, saturation, low diversity
RAG Integrity Cognitive State Cross-reference validation, contradiction detection, provenance checks
Behavioral Guard Behavioural Control Deceptive dialog detection, loop prevention, form validation
Cascade Breaker Systemic A2A message validation, trust scoring, anomaly detection, circuit breaker
Report Auditor Human-in-the-Loop Cherry-pick detection, anchoring analysis, decision fatigue monitoring

Statistical Framework

Every experiment produces:

  • Mean ± std across repetitions
  • 95% confidence intervals (t-distribution)
  • Wilcoxon signed-rank test p-values (non-parametric)
  • Cohen's d effect sizes (negligible/small/medium/large)
  • Bonferroni correction for the 22 simultaneous comparisons

The statistics are implemented from scratch in TypeScript (src/harness/metrics.ts) — no external stats dependency.


Tech Stack

Layer Technology
Runtime Bun
Language TypeScript 5.x
Protocol A2A
LLM Clients OpenAI, Anthropic, Google AI SDKs
Image Metadata exifr
Statistics Custom TypeScript implementation
Paper Assets LaTeX (booktabs, pgfplots)
Testing Bun Test + Playwright
Lint / Format Biome

Project Layout

src/
  traps/         22 trap scenarios, organized by category
  mitigations/   7 defensive modules
  harness/       matrix generation, runner, metrics, reporter
  agents/        model-provider clients + agent configs (mock provider for tests)
scripts/         experiment runners + paper asset generation
datasets/        seed adversarial payloads (injections, framing, poison docs, UIs)
tests/           unit + integration tests (all use the mock provider)
results/         raw + analyzed experiment data
paper/           LaTeX source for the write-up

Related Projects

Project Protocol What It Explores
a2a-crews A2A Agents that coordinate
ag-ui-crews AG-UI Agents you can observe
rag-a2a A2A Agents that know things
agent-traps-lab ← you are here A2A Agents you can attack (and defend)

License

MIT © Aviral Dua

Citation

If you use agent-traps-lab in your research, please cite:

@misc{dua2026agenttrapslab,
  author = {Dua, Aviral},
  title  = {agent-traps-lab: Empirical Testbed for AI Agent Trap Validation},
  year   = {2026},
  url    = {https://github.com/aviraldua93/agent-traps-lab}
}

About

Empirical testbed for DeepMind's AI Agent Traps paper — 22 adversarial scenarios, 4 models, 3,480 experiments

Resources

License

Contributing

Stars

1 star

Watchers

0 watching

Forks

Releases

No releases published

Packages

 
 
 

Contributors