-
Notifications
You must be signed in to change notification settings - Fork 18
⚡ Claude Token Optimization2026-04-04 — Secret Digger (Claude) #1671
Description
Target Workflow: secret-digger-claude
Source report: #1670
Estimated cost per run: $0.18
Total tokens per run: ~79K
Cache read rate: 48.4% (37,927 / 78,181 total)
Cache write rate: 51.6% (38,500 / 78,181 total)
LLM turns: 3 (1× Haiku, 2× Sonnet)
Current Configuration
| Setting | Value |
|---|---|
| Tools loaded (GitHub MCP) | 52 tools (context, repos, issues, pull_requests toolsets) |
| Tools actually used (GitHub MCP) | 0 — agent never calls GitHub MCP in any of the 4 runs analyzed |
| Built-in tools | 15 (Bash, Edit, Glob, Grep, Read, Write, LS, Task, etc.) |
| Network groups | defaults, github |
| Pre-agent steps | No |
| Prompt size | ~7,581 bytes (~1,900 tokens) |
The shared/secret-audit.md import sets network: allowed: [defaults, github] but specifies no tools: github: toolsets: restriction. This results in 52 GitHub MCP tool schemas being injected into the system prompt via the GITHUB_TOOLSETS: "context,repos,issues,pull_requests" default.
Recommendations
1. Remove GitHub MCP tools entirely
Estimated savings: ~26K tokens/run (~58% cost reduction, ~$0.10/run)
The Secret Digger's mission is container security research — it uses bash to probe the environment, cache-memory to track findings, and safeoutputs to file issues. It does not need to read GitHub issues, PRs, commits, or code files.
All 52 GitHub MCP tools (mcp__github__*) are loaded but never called. Each tool schema adds ~500 tokens to the system prompt, totalling ~26K tokens that inflate the cache write on every run.
Option A (Recommended): Remove github from network groups
Edit shared/secret-audit.md:
network:
allowed:
- defaults
- - githubThis removes the GitHub MCP server entirely, eliminating all 52 tool schemas from the system prompt.
⚠️ Verify the agent doesn't need outbound access togithub.comfor its bash investigation. Given the mission is examining the local container environment, this should be safe. If any bash commands in the agent's findings referencegithub.com, they'll fail silently — but no run in the last 4 analyzed has triggered this.
Option B: Suppress GitHub MCP tools while keeping network access
If outbound github.com access is needed for some bash commands, keep the network group but suppress MCP tools:
tools:
cache-memory: true
bash: true
+ github:
+ toolsets: []Note:
toolsets: []syntax support depends on the gh-aw version. If unsupported, use the minimum viable toolset such astoolsets: [context](~5 tools instead of 52, saves ~23K tokens).
2. Add toolsets: [context] as interim fallback if Option B is used
Estimated savings: ~23K tokens/run (~52% cost reduction)
If removing the github group is not feasible, restrict to the smallest needed toolset:
tools:
cache-memory: true
bash: true
github:
toolsets: [context] # ~5 tools: get_me, list_notifications, get_notification_details, etc.This reduces from 52 to ~5 tools. Savings: ~47 tools × ~500 tokens = ~23.5K tokens.
Compare to other workflows that already do this:
ci-doctor.md:toolsets: [default, actions]claude-token-usage-analyzer.md:toolsets: [default, actions]
3. Verify cache is amortized correctly across turns
Estimated savings: $0 (validation only)
The current within-run cache pattern is already optimal:
- Turn 2 (Sonnet) writes ~38K system prompt to Anthropic cache
- Turn 3 (Sonnet) reads those same 38K tokens back at 12.5× lower cost
However, since the cache write price ($3.75/M) is 12.5× the read price ($0.30/M), the break-even point is Turn 3 reusing Turn 2's cache. With only 2 Sonnet turns per run, each run pays full write cost once, and recoups partial savings only once. Reducing the absolute cache size (Recommendation 1) is therefore the highest-leverage change.
Cache Analysis (Anthropic-Specific)
Note: Per-turn breakdown estimated from aggregate report data. Exact per-turn data requires agent-stdio.log artifact.
| Turn | Model | Input | Output | Cache Read | Cache Write | Est. Cost |
|---|---|---|---|---|---|---|
| 1 | Haiku | ~123 | ~90 | 0 | 0 | ~$0.000 |
| 2 | Sonnet | ~50 | ~100 | 0 | ~38,500 | ~$0.144 |
| 3 | Sonnet | ~77 | ~1,500 | ~37,927 | 0 | ~$0.034 |
| Total | ~127 | ~1,600 | ~37,927 | ~38,500 | ~$0.18 |
Cache write amortization: Turn 2's 38,500-token write is reused only once (Turn 3). At $3.75/M write vs $0.30/M read, each write needs to be reused 12.5× to fully amortize. The current structure gets 1× reuse per run.
Cache cost vs benefit: Cache write is 81% of total run cost ($0.146 of $0.182). Since the majority of cached tokens are GitHub MCP tool schemas that are never actually called, this cache is effectively paying to cache dead weight.
Cross-run cache: The identical cache_read: 37,927 across all 4 runs suggests possible cross-run system prompt caching (Anthropic's 5-minute TTL). If so, removing unused tools would further improve cross-run cache efficiency for the remaining content.
Expected Impact
| Metric | Current | Projected (Rec. 1A) | Savings |
|---|---|---|---|
| Cache write tokens/run | ~38,500 | ~12,500 | -67% |
| Cache read tokens/run | ~37,927 | ~12,000 | -68% |
| Total tokens/run | ~79K | ~27K | -66% |
| Cost/run | $0.182 | $0.077 | -58% |
| LLM turns | 3 | 3 | — |
| GitHub MCP tools | 52 | 0 | -52 |
At 4 runs/day: $0.421/day → $0.176/day (saves $0.245/day, ~$89/year)
At 24 runs/day (full hourly cadence): saves ~$2.53/day, ~$922/year
Implementation Checklist
- Edit
shared/secret-audit.md: remove- githubfromnetwork: allowed:(Option A) - Verify no bash commands in the agent rely on outbound github.com access
- Recompile:
gh aw compile .github/workflows/secret-digger-claude.md- (Also recompile
secret-digger-codex.mdandsecret-digger-copilot.mdif they sharesecret-audit.md)
- (Also recompile
- Post-process:
npx tsx scripts/ci/postprocess-smoke-workflows.ts - Verify CI passes on PR
- Compare token usage on next run vs this baseline ($0.18/run)
- Update
shared/secret-audit.mddescription to note the network restriction
Generated by Daily Claude Token Optimization Advisor
Generated by Daily Claude Token Optimization Advisor · ◷