Pre-commit content scanning: block literal-bearing env assignments regardless of filename (defense-in-depth for op:// migration)

[npstorey]

Problem

Workspace-wide content-layer remediation to op:// references is tracked in npstorey/civic-ai-tools-website#101. That control aims to eliminate intentional literal-value secret storage. It does not prevent accidental literal-bearing files surviving on disk in places .gitignore doesn't cover, then potentially being committed by a wide-net git add or via filenames that evade name-pattern matching.

Concrete instance (2026-05-21). A phantom file named (single space character) was discovered untracked in civic-ai-tools-website, 606 bytes, mtime 2026-05-05 (two days after the 2026-05-03 audit). Contents: a copy of a template-shaped file, with one variable carrying a literal value while others used op:// references.

• git check-ignore -v " " returned exit code 1 — the filename did not match any .gitignore pattern.
• git log --all --full-history -- " " returned empty — the file was never committed to any branch, so no remote exposure occurred.
• BUT: the file lived in the working tree for 16 days and would have been committed by any wide-net git add that didn't audit the staged set first.

.gitignore is name-anchored. It cannot catch shape-of-secrets in non-secret-named files (notes.txt, tmp, scratch.md, files with whitespace in their names, etc.). Defense-in-depth requires content scanning at the pre-commit stage.

Proposed approach

Install a pre-commit hook in each repo that scans staged content (not the full working tree) for env-assignment-with-literal-value patterns, blocking the commit if any are found.

Detection rule (starting point — tunable):

• Lines that match the shape of an environment-variable assignment (uppercase identifier, then equals sign, then value).
• AND the RHS does not start with a known-safe reference shape — op://, double-quoted op://, or shell variable interpolation ${...}.
• AND the file is not in an explicit allow-list (test fixtures, mocks).

Three implementation options, in order of recommendation:

1. Adopt gitleaks — mature Go-based scanner, pre-commit integration, allow-list via .gitleaks.toml, baseline support for existing tracked content. Most maintained. Recommended default.
2. Adopt detect-secrets (Yelp) — Python, baseline-driven, pre-commit integration via pre-commit.
3. Custom hook — small shell or Python script per repo, or shared via the pre-commit framework. Lowest dep cost, highest maintenance cost; pick only if (1) and (2) are too heavy.

Cross-repo coordination. Each repo is independent. Hook config should be checked-in (so it travels with the repo) but pre-commit install (or equivalent) is required per fresh clone. Document the install step in workspace CLAUDE.md and per-repo CLAUDE.md.

Self-demonstration note. The drafting of this issue itself produced a working demonstration of the value of the proposed control: the local Claude Code pre-tool hook blocked an early version of the gh issue create command because the body contained example shapes resembling a literal-bearing env assignment. That is the same class of pattern this issue proposes detecting at the git-commit boundary across the three repos. The gap is that an analogous control exists at the AI-agent tool-use boundary but not at the git-commit boundary.

Spec changes the work produces

• New config file in each of the three repos (e.g., .gitleaks.toml or .pre-commit-config.yaml).
• Updates to each repo's CLAUDE.md noting the install step for fresh clones.
• Workspace CLAUDE.md's secret-hygiene section updated to reference the machine-enforcement layer.
• Optional: a SECURITY.md per repo noting the scanner in use and how to suppress false positives.

Relationship to other work

• Complement to npstorey/civic-ai-tools-website#101. That issue migrates content (literal → op://). This issue prevents regressions (tooling enforces the rule). Both should ship for full coverage.
• 2026-05-03 audit produced Standards/competitive landscape refresh (last surveyed March–April 2026) #101 (after discovering a signing-key literal on disk). The 2026-05-21 phantom-space-file finding is a new occurrence of the same trapdoor class (different secret), 16 days post-audit, demonstrating that content remediation alone is insufficient — operator discipline and ignore patterns both have holes.
• Workspace CLAUDE.md secret-hygiene rules are operator-facing (rules for sessions / humans to follow). This issue adds machine enforcement of the same intent.
• Adjacent: Threat model documentation: enumerate what evidence-package signatures attest, by capture method #63 (threat model documentation by capture method) — different scope; threat model is about what signatures attest, this is about preventing secret leaks in the source tree.

Scope

In:

• Pick a tool (gitleaks recommended) and install it in all three repos: civic-ai-tools, civic-ai-tools-website, socrata-mcp-server.
• Configure detection rule for env-assignment-with-literal-value pattern.
• Establish an allow-list mechanism for legitimate exceptions (e.g., test fixtures with mock keys).
• Document the install step in each repo's CLAUDE.md.
• Backfill baseline scan of existing tracked content; should be clean post-Standards/competitive landscape refresh (last surveyed March–April 2026) #101, but verify and allow-list anything pre-existing with explicit rationale.

Out:

• Service-side scanning (GitHub Advanced Security, push-time scanning) — separate decision; service-side is a second layer, not a substitute for the commit-time gate.
• General entropy-based detection beyond env-assignment shapes — could be enabled via gitleaks defaults but the goal here is the specific trapdoor pattern; broader scanning is a possible follow-up.
• Migration of any newly-discovered literals — that's Standards/competitive landscape refresh (last surveyed March–April 2026) #101's scope. This issue only adds the guardrail.

Acceptance criteria

• A pre-commit hook is active in each of civic-ai-tools, civic-ai-tools-website, socrata-mcp-server.
• Attempting to git commit a file containing a literal-bearing env-shaped assignment is blocked with a clear error message naming the offending file/line and the suppression mechanism.
• A # gitleaks:allow (or equivalent) inline directive can authorize legitimate exceptions, with a written rationale in the line or in .gitleaks.toml.
• A fresh clone of each repo includes the hook config; CLAUDE.md documents the one-time install step.
• A baseline scan of each repo's tracked content passes cleanly, or any findings are explicitly allow-listed with rationale.
• Workspace CLAUDE.md secret-hygiene section references the tooling as the machine-enforcement layer.

Dependencies

• No hard dependencies. Can ship independently of Standards/competitive landscape refresh (last surveyed March–April 2026) #101.
• Soft coordination with npstorey/civic-ai-tools-website#101. Both should land before declaring the trapdoor class closed. Practically, this issue is easier and could ship first as a guardrail while Standards/competitive landscape refresh (last surveyed March–April 2026) #101's migration completes.

Risk

• False positives blocking legitimate commits. Mitigation: allow-list mechanism with inline # gitleaks:allow directives + clear error messages. Audit existing tracked content during install to bootstrap a clean baseline.
• Hook bypass via --no-verify. Mitigation: workspace CLAUDE.md already prohibits --no-verify; the commit guidance reinforces it. The hook is a check, not a hard gate; human discipline still enforces use. Server-side scanning is the eventual hardening if needed.
• Drift between the three repos' configs. Mitigation: maintain a canonical config in civic-ai-tools (the hub repo) with a documented sync step; or use identical config files reviewed periodically.
• Adoption friction on new clones. Mitigation: clear install instructions in each repo's README and CLAUDE.md. Consider a single bootstrap.sh per repo that wraps the install.
• The trapdoor pattern that produced the phantom file (a literal value inside template-shaped content saved to an oddly-named file) is itself a content shape that gitleaks-style tools may or may not flag by default. Mitigation: confirm during tool evaluation that the chosen tool's default ruleset catches uppercase-identifier-equals-non-reference shapes in arbitrary files, not just template-named files. If not, add a custom rule.

Reproducible at

• 2026-05-21 phantom-space-file incident (this issue's trigger): file named (single space), 606 bytes, mtime 2026-05-05, untracked in civic-ai-tools-website, never committed.
• 2026-05-03 audit — original incident, source of npstorey/civic-ai-tools-website#101.
• Workspace CLAUDE.md secret-hygiene section — operator-facing rules this tooling backstops.
• User memory feedback_migration_trapdoors.md — documents trapdoor mechanics for the operator side.
• gitleaks — recommended tool.
• detect-secrets — alternative.
• pre-commit — hook orchestrator (works with either tool).