Free, offline AI agent security scanner for CI/CD pipelines
Scan your AI agent configuration for 27 security checks across 6 categories. No signup, no API keys, no data collection. Runs entirely offline in your GitHub Actions workflow.
Add this to your GitHub Actions workflow (.github/workflows/security.yml):
name: Security Scan
on: [push, pull_request]
jobs:
pitstop:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4
- uses: thepitstop/pitstop-scan@v1
with:
fail-on-grade: 'C'That's it! The scanner will automatically detect your agent config files and report security issues.
- Exec Security Mode — Is exec restricted to allowlist or denied?
- Sandbox Enabled — Is sandboxing configured for dangerous operations?
- Tool Restrictions — Are tools explicitly allowlisted/denylisted?
- File System Permissions — Are file access paths restricted?
- Network Access Control — Are outbound connections restricted?
- System Prompt Protection — Is the system prompt protected from override?
- Input Validation — Is user input validated/sanitized?
- Output Filtering — Are outputs filtered for sensitive data?
- Instruction Hierarchy — Is there instruction priority enforcement?
- Outbound Allowlist — Are outbound URLs/IPs allowlisted?
- SSRF Protection — Is SSRF explicitly mitigated?
- Sensitive Data Masking — Are secrets/credentials masked in outputs?
- Clipboard Restrictions — Is clipboard access controlled?
- Plugin Allowlist — Are plugins/extensions explicitly allowed?
- Skill Verification — Are skills verified before loading?
- Dependency Pinning — Are dependencies version-pinned?
- Package Audit — Are packages audited for vulnerabilities?
- Sub-Agent Sandboxing — Are sub-agents sandboxed?
- Trust Scoring — Is there trust scoring for delegated tasks?
- Behavioral Inheritance — Do sub-agents inherit security policies?
- Delegation Limits — Are delegation depth/breadth limited?
- Sub-Agent Output Review — Are sub-agent outputs reviewed before use?
- Command Logging — Are executed commands logged?
- Session Recording — Are sessions recorded/auditable?
- Anomaly Detection — Is there anomaly/drift detection?
- Audit Trail — Is there a tamper-evident audit trail?
- Monitoring Alerts — Are alerts configured for security events?
## 🏎️ Pitstop Security Scan Results
**Score: 78/100 (C+)**
### 📊 Category Scores
| Category | Score | Grade |
|----------|-------|-------|
| Permissions & Access Control | 72/100 | C |
| Prompt Injection Defense | 85/100 | B |
| Data Exfiltration Prevention | 60/100 | D |
| Supply Chain Integrity | 80/100 | B |
| Sub-Agent Safety | 50/100 | F |
| Audit & Logging | 75/100 | C |
### ⚠️ Issues Found (2 critical, 5 high)
| Check | Severity | Status | Fix |
|-------|----------|--------|-----|
| exec_security_mode | 🔴 Critical | FAIL | Set `tools.exec.security: "allowlist"` |
| sensitive_data_masking | 🔴 Critical | FAIL | Add `security.mask_secrets: true` |
| subagent_sandboxing | 🟠 High | FAIL | Add `subagents.sandbox: true` |
| ... | ... | ... | ... |
> Scan powered by [The Pitstop](https://thepitstop.ai) — AI Agent Security Scanner| Input | Description | Default | Options |
|---|---|---|---|
config-path |
Path to agent config file | auto-detect | Any file path |
fail-on-grade |
Minimum grade to pass | D |
A+, A, B, C, D, F |
format |
Output format | summary |
summary, json, sarif |
badge |
Generate badge | true |
true, false |
| Output | Description |
|---|---|
score |
Overall score 0-100 |
grade |
Letter grade (A+, A, B, C, D, F) |
issues |
Number of issues found |
report |
Full JSON report |
- uses: thepitstop/pitstop-scan@v1
id: pitstop
with:
config-path: '.openclaw/openclaw.json'
fail-on-grade: 'B'
format: 'json'
- name: Upload SARIF
if: always()
uses: thepitstop/pitstop-scan@v1
with:
format: 'sarif'The scanner auto-detects these config files:
- OpenClaw:
openclaw.json,.openclaw/openclaw.json - CrewAI:
agents.yaml,crew.yaml - Cursor:
.cursorrules,cursor.json - Claude:
CLAUDE.md,claude.json - AutoGPT:
autogpt.json,.env - LangChain:
langchain.config.js,langchain.yaml - AutoGen:
autogen_config.json - Generic:
AGENTS.md, any agent config pattern
Don't see your framework? Open an issue and we'll add it!
- Each check is scored: Pass (100), Partial (50), Fail (0), N/A (skipped)
- Category scores are calculated as the average of all checks in that category
- Overall score is a weighted average of category scores
- Grades are assigned: A+ (95+), A (90+), B (80+), C (70+), D (60+), F (<60)
- Permissions & Access Control: 20%
- Prompt Injection Defense: 20%
- Data Exfiltration Prevention: 15%
- Supply Chain Integrity: 15%
- Sub-Agent Safety: 15%
- Audit & Logging: 15%
- ✅ 100% offline — No network calls, no data collection
- ✅ No API keys — Free to use, no signup required
- ✅ No telemetry — Your code stays in your repo
- ✅ Open source — MIT licensed, audit the code yourself
- ✅ Fast — Completes in under 10 seconds
# Clone the repo
git clone https://github.com/thepitstop/pitstop-scan.git
cd pitstop-scan
# Install dependencies
npm install
# Run locally
node scan.js
# Test with specific config
node scan.js --config path/to/config.json --format jsonWe welcome contributions! Here's how you can help:
- Add support for new frameworks — PRs welcome for new config formats
- Improve check logic — Better detection heuristics
- Add more checks — Propose new security checks
- Fix bugs — Found an issue? Open a PR
See CONTRIBUTING.md for guidelines.
- 🌐 Website: thepitstop.ai
- 📖 Docs: thepitstop.ai/docs
- 🐛 Issues: GitHub Issues
- 💬 Discussions: GitHub Discussions
MIT License — see LICENSE for details.
Made with ❤️ by The Pitstop — Securing the AI agent ecosystem, one scan at a time.
🏎️ Float like a butterfly, secure like Pitstop.