Skip to content

DEPYLER-3005: Phase 4 — Static-first feedback (skill graduation pipeline) #260

Description

@noahgift

Umbrella: #255
Spec: docs/specifications/depyler-repair-mode.md (branch `feat/repair-mode-spec`)
Depends on: DEPYLER-3004 (Phase 3)
Blocks:

Scope

Close the loop. Mine repair-session telemetry for patterns that fire often enough to justify lifting into the static transpiler. Each promotion deletes the skill markdown so the same fix is never paid for twice.

This phase is the success signal for the entire effort. Without it, repair mode is a permanent band-aid that masks bugs in `depyler-core` and stagnates the single-shot compile rate. The whole architecture is justified by the graduation pipeline working.

Mechanics

  1. Quarterly telemetry rollup: for each skill, count `firings` and `distinct_files` across all corpora
  2. Skills meeting the threshold (≥50 firings / ≥10 distinct files per quarter, per C-SKILL-GRADUATION) become graduation candidates
  3. Engineer picks up the candidate, writes a deterministic rule in `depyler-core`, adds corpus regression tests
  4. PR includes BOTH the new rule AND the deletion of `skills/.md` — same commit, atomically
  5. Cache entries that referenced the deleted skill auto-invalidate (the skill set hash changes)

Deliverables

  • Telemetry rollup tool (CLI subcommand: `depyler repair-stats --quarter 2026Q3`)
  • Graduation candidate report (markdown table, one row per candidate skill)
  • PR template for skill graduations: requires the matching skill deletion in the same PR
  • Corpus regression test framework that ties tests to skill IDs (so deleting a skill without a test is caught)
  • Quarterly review checklist in `docs/design/repair-quarterly-review.md`

Contracts enforced

  • C-SKILL-GRADUATION (`contracts/skill-graduation-v1.yaml`) — the whole contract, especially `repair_invocation_rate_trends_down` and `graduated_skills_are_deleted`

Exit criterion

This phase doesn't "complete" in the usual sense — it's an ongoing process. But the milestone is:

  • ≥3 skills graduated to `depyler-core` static rules within 6 months of Phase 3 GA
  • Quarter-over-quarter repair-invocation rate is flat or decreasing in every tier for at least two consecutive quarters
  • Zero graduated-skill regressions caught in CI (i.e., promoted rules stay correct)

If repair-invocation rate trends up for two quarters in any tier, that triggers a graduation audit (per C-SKILL-GRADUATION).

Metadata

Metadata

Assignees

No one assigned

    Labels

    enhancementNew feature or request

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions