feat(plugin): DTPR authoring studio — five skills + research corpus#273
feat(plugin): DTPR authoring studio — five skills + research corpus#273
Conversation
…us scaffold Substrate for the authoring studio: a qualitative comprehension rubric with seven checklist items, a shared block template that schema-tier skills inline, and an empty research corpus with INDEX + README documenting the contract (slug convention, frontmatter, authority_tier enum, applicability_tags vocabulary, consumer-side degradation, _-prefix privacy convention).
Adds four new checks to the offline conformance validator:
* verifyCorpusEntry — enforces the YYYY-MM-DDThhmm-<slug>.md filename
convention, required frontmatter (source, date_accessed,
authority_tier, applicability_tags), the closed 8-value
authority_tier enum, ISO 8601 dates, and flags plausible typos of
known fields.
* verifyIndex — validates the INDEX.md header, checks every data row
references an existing entry file, and (when git history is
available) asserts the file is a byte-prefix of its merge-base
version with origin/main.
* verifyEvalSymmetry — every should_not_trigger id prefixed
cross-sibling:<skill>:<positive-id> must match a should_trigger id
on the named sibling skill.
* loadMcpToolNames — now scans api/src/mcp/tools/*.ts alongside
tools.ts, picking up render_datachain and future split-out tools.
Extends knownNonTools with corpus and template frontmatter terms so
SKILL.md bodies can reference them in backticks without tripping drift
detection. Scopes tool-drift checking to SKILL.md only — research/ and
references/ remain free prose. CI workflow sets fetch-depth: 0 so the
append-only check can resolve the merge base.
Ships the comprehension-tier skill that applies the shared rubric to an element, category, datachain-instance, pasted YAML, or arbitrary markdown. Phase 0 classifies the input; Phase 1 loads via MCP when the user gives an id; Phase 2 reads the rubric + block template; Phase 3 does a corpus lookup with graceful Task/Read/Write degradation; Phase 4 applies the rubric item-by-item; Phase 5 emits the block and a one-paragraph summary that routes to a sibling skill when a gap needs schema-level fix. Eval: 6 positives (element grading, category clarity, datachain-instance audit, pasted YAML, sign-copy readability, explicit comprehension-check phrasing) and 8 non-cross-sibling negatives. Cross-sibling negatives land in Unit 10 once all siblings exist.
Meta-structure schema-tier skill. Critiques or designs the datachain-type shape itself — which categories exist, which are required, whether a whole category should retire or split. Output is the ported Scenario/Gaps/Proposed changes template with a non-optional element-migration plan when a category retires, an inline Comprehension check, and the pnpm schema:new handoff. Eval: 6 positives (incl. generative-output ported verbatim) and 9 non-cross-sibling negatives.
Category-tier schema skill. Audits one category's element collection for coherence, overlap, and gaps. Phase 0 accepts a category id with fuzzy match; Phase 2 surfaces overlap pairs by querying list_elements with each element's title scoped to the same category; Phase 4 produces the audit (Coverage map / Overlap pairs / Gaps / Proposed changes) and an inline Comprehension check before the schema:new handoff. Eval: 6 positives (incl. accountable-deep-dive ported verbatim) and 8 non-cross-sibling negatives.
Element-tier schema skill. Drafts one proposed element (add, edit, or retire) as a YAML fragment skeleton matching the canonical shape of accept_deny.yaml — six-locale title/description placeholders, citation, symbol_id, optional variables. Phase 1 does a collision check; Phase 4 emits a symbol direction as prose (not SVG) suitable for a designer. Inlines the Comprehension check before the schema:new handoff. Eval: 6 positives (incl. llm-hallucination, third-party-processor, retire-element ported verbatim from the retiring skill) and 8 non-cross-sibling negatives.
Prepends Phase 0 — Inventory and classify. Trial-call-and-degrade probe for Read, WebFetch, Task, Write (no env-var introspection; cross-host safe). Size-band classification for attached artifacts (verbal / inline ≤2k / inline-full 2–10k / chunk-relevant 10–20k / reject >20k) with a 300-tokens/page PDF heuristic. Structured ask for budget overflow. Explicit degradation paths for password-protected PDFs, 401/403 URLs, and artifact-vs-verbal conflicts. Mid-flow artifact drop loops back to Phase 0 for just the new artifact. Inserts an optional Research context step between Phase 2 and Phase 3 for corpus lookup + Task-dispatched researcher on miss. Phase 2 captures content_hash into any corpus entry written during the session. Sibling routing replaces the retired dtpr-schema-brainstorm reference with the four-way split (datachain-structure, category-audit, element-design, comprehension-audit). Verbal-only input preserves behavior byte-for-byte. Eval expansion: 10 positives (5 preserved + 5 Phase-0-specific covering PDF, URL, mixed, over-budget, password-protected) and 7 non-cross-sibling negatives.
Rewrites plain-id negatives that are conceptually cross-sibling into the cross-sibling:<target-skill>:<positive-id> form so verify.mjs's symmetry check can validate they route to an existing should_trigger positive on the named sibling. Every in-studio skill now carries at least one cross-sibling negative for each of the four peers, giving the router an explicit counter-example per bordering skill. Retired-skill prompts (llm-hallucination, third-party-processor, retire-element, accountable-deep-dive, generative-output) already ported to their primary targets in Units 4–6; this pass adds them as cross-sibling negatives on the siblings they could plausibly straddle, matching Decision 15's mapping.
Rewrites the README as the studio's entry point: five-skill table keyed by judgment tier (instance, meta-structure, category, element, comprehension), a capability matrix per host (Claude Code / Claude Cowork / Claude.ai) showing which optional tools each skill can rely on at session start, install/upload instructions per host, and new sections for the shared References (comprehension rubric and block template) and the Research corpus (contract, privacy, consumer-side degradation). Troubleshoot table adds an empty-corpus row and a Task-unavailable row so first-time users on Claude.ai know what the warnings mean.
Hard-deletes the retiring skill and its eval file. The three schema-tier replacements (dtpr-datachain-structure, dtpr-category-audit, dtpr-element-design) cover its responsibility; eval prompts ported in Units 4–6 and the straddlers landed as cross-sibling negatives in Unit 10, so no retired prompt falls through. plugin.json bumps to 0.2.0 with an authoring-studio description; .mcp.json User-Agent tracks the bump for server-side telemetry; a new CHANGELOG.md captures the 0.1.0 retro-entry and the 0.2.0 studio expansion; README links to the CHANGELOG.
Greptile SummaryExpands the Confidence Score: 5/5Safe to merge — all remaining findings are P2 style/completeness suggestions that do not affect runtime correctness. Cross-sibling eval symmetry is fully verified programmatically and manually confirms IDs match across all five skills. The corpus verifier is well-structured with graceful degradation on missing git history. The two findings are an unused import and a missing reverse-index check that is a best-practice gap rather than a runtime defect. plugin/dtpr/evals/verify.mjs — unused
|
| Filename | Overview |
|---|---|
| plugin/dtpr/evals/verify.mjs | Major extension: adds corpus-entry frontmatter validation, INDEX.md append-only integrity, and cross-sibling eval-symmetry checks. Two minor issues: unused basename import and missing reverse check (every corpus file must appear in INDEX.md). |
| plugin/dtpr/skills/dtpr-describe-system/SKILL.md | Significant expansion with Phase 0 trial-call-and-degrade, multi-artifact budgeting (inline/chunk-relevant/reject bands), multi-host degradation matrix, and research corpus integration. Sibling routing table and security framing are well-specified. |
| plugin/dtpr/evals/describe-system.evals.json | Adds five new positive prompts (PDF/URL/verbal artifact scenarios) and four cross-sibling negatives; cross-sibling IDs match their counterpart should_trigger entries in the sibling eval files. |
| plugin/dtpr/evals/element-design.evals.json | New file; 6 should_trigger, 8 should_not_trigger (4 cross-sibling). Provides the target positive IDs referenced by all four sibling eval files' cross-sibling entries. |
| .github/workflows/plugin-test.yaml | Adds fetch-depth: 0 to actions/checkout so git merge-base origin/main HEAD in verify.mjs has the full history it needs for the append-only check. |
| plugin/dtpr/research/README.md | New file; fully specifies the corpus contract (retrieval, write path, concurrency, append-only invariant, privacy prefix convention, authority tier enum). |
| plugin/dtpr/evals/schema-brainstorm.evals.json | Deleted; its should_trigger prompts were redistributed as cross-sibling negatives across the four replacement skills. |
Flowchart
%%{init: {'theme': 'neutral'}}%%
flowchart TD
User([User prompt]) --> Router{Skill router}
Router -->|"describe / disclose / datachain"| DS[dtpr-describe-system]
Router -->|"datachain-type shape / category add-retire"| DC[dtpr-datachain-structure]
Router -->|"audit elements inside a category"| CA[dtpr-category-audit]
Router -->|"draft / retire element"| ED[dtpr-element-design]
Router -->|"grade for public comprehension"| CX[dtpr-comprehension-audit]
DS -->|Phase 0| Probe[Trial-call-and-degrade\nRead · WebFetch · Task · Write]
DS -->|Phase 2| MCP[(DTPR MCP\napi.dtpr.io)]
DS -->|Research ctx| Corpus[(research/\nINDEX.md)]
DS -->|Schema gap| ED
DS -->|Category off| CA
DS -->|Shape off| DC
DS -->|Comprehension| CX
subgraph verify.mjs
V1[Skill frontmatter]
V2[Eval JSON structure]
V3[MCP tool drift]
V4[Corpus frontmatter]
V5[INDEX append-only]
V6[Cross-sibling symmetry]
end
Prompt To Fix All With AI
This is a comment left during a code review.
Path: plugin/dtpr/evals/verify.mjs
Line: 38
Comment:
**Unused `basename` import**
`basename` is imported from `node:path` but is never referenced anywhere in the file. This was likely added in anticipation of `entry.name` vs `basename(entry.path)` usage that never landed.
```suggestion
import { dirname, join } from 'node:path'
```
How can I resolve this? If you propose a fix, please make it concise.
---
This is a comment left during a code review.
Path: plugin/dtpr/evals/verify.mjs
Line: 508-511
Comment:
**No reverse-index check: orphaned corpus files go undetected**
`verifyIndex` validates that every slug row in `INDEX.md` points to an existing file, but the inverse is not enforced — a corpus entry file created in `research/` without a corresponding `INDEX.md` row will pass the verifier silently. Because retrieval is routed entirely through `INDEX.md`, such files are invisible to the skills at runtime and effectively wasted. Consider adding a check that every file in `corpusEntries` is referenced in `INDEX.md`.
```js
// After verifyIndex(), add:
for (const entry of corpusEntries) {
if (!indexedSlugs.has(entry.name)) {
fail(`${entry.path}: corpus file has no corresponding row in INDEX.md.`)
}
}
```
(Requires extracting the validated slug set out of `verifyIndex`.)
How can I resolve this? If you propose a fix, please make it concise.Reviews (1): Last reviewed commit: "feat(plugin): retire dtpr-schema-brainst..." | Re-trigger Greptile
| import { execSync } from 'node:child_process' | ||
| import { readFileSync, readdirSync } from 'node:fs' | ||
| import { dirname, join } from 'node:path' | ||
| import { basename, dirname, join } from 'node:path' |
There was a problem hiding this comment.
basename is imported from node:path but is never referenced anywhere in the file. This was likely added in anticipation of entry.name vs basename(entry.path) usage that never landed.
| import { basename, dirname, join } from 'node:path' | |
| import { dirname, join } from 'node:path' |
Prompt To Fix With AI
This is a comment left during a code review.
Path: plugin/dtpr/evals/verify.mjs
Line: 38
Comment:
**Unused `basename` import**
`basename` is imported from `node:path` but is never referenced anywhere in the file. This was likely added in anticipation of `entry.name` vs `basename(entry.path)` usage that never landed.
```suggestion
import { dirname, join } from 'node:path'
```
How can I resolve this? If you propose a fix, please make it concise.| const corpusEntries = listCorpusEntries() | ||
| const corpusFileNames = new Set(corpusEntries.map((e) => e.name)) | ||
| for (const entry of corpusEntries) verifyCorpusEntry(entry.path, entry.name) | ||
| verifyIndex(INDEX_PATH, corpusFileNames) |
There was a problem hiding this comment.
No reverse-index check: orphaned corpus files go undetected
verifyIndex validates that every slug row in INDEX.md points to an existing file, but the inverse is not enforced — a corpus entry file created in research/ without a corresponding INDEX.md row will pass the verifier silently. Because retrieval is routed entirely through INDEX.md, such files are invisible to the skills at runtime and effectively wasted. Consider adding a check that every file in corpusEntries is referenced in INDEX.md.
// After verifyIndex(), add:
for (const entry of corpusEntries) {
if (!indexedSlugs.has(entry.name)) {
fail(`${entry.path}: corpus file has no corresponding row in INDEX.md.`)
}
}(Requires extracting the validated slug set out of verifyIndex.)
Prompt To Fix With AI
This is a comment left during a code review.
Path: plugin/dtpr/evals/verify.mjs
Line: 508-511
Comment:
**No reverse-index check: orphaned corpus files go undetected**
`verifyIndex` validates that every slug row in `INDEX.md` points to an existing file, but the inverse is not enforced — a corpus entry file created in `research/` without a corresponding `INDEX.md` row will pass the verifier silently. Because retrieval is routed entirely through `INDEX.md`, such files are invisible to the skills at runtime and effectively wasted. Consider adding a check that every file in `corpusEntries` is referenced in `INDEX.md`.
```js
// After verifyIndex(), add:
for (const entry of corpusEntries) {
if (!indexedSlugs.has(entry.name)) {
fail(`${entry.path}: corpus file has no corresponding row in INDEX.md.`)
}
}
```
(Requires extracting the validated slug set out of `verifyIndex`.)
How can I resolve this? If you propose a fix, please make it concise.Adds /plugin to the docs-site covering the five-skill Claude plugin. One page per skill with triggers, workflow, and non-goals pointing at the SKILL.md in the repo as source of truth. Dedicated pages for install (per-host instructions + capability matrix + troubleshoot), the research corpus contract (slug convention, frontmatter schema, authority_tier enum, applicability_tags vocabulary), and the comprehension rubric (seven items, block template, rendering rules). Top-level index.md gains a fifth card; getting-started adds the authoring-team audience and a plugin link. 7.changelog.md renumbers to 8.changelog.md to slot /plugin between /concepts and /changelog.
Deploying with
|
| Status | Name | Latest Commit | Updated (UTC) |
|---|---|---|---|
| 🔵 In progress View logs |
dtpr-ai | 6c39b69 | Apr 20 2026, 02:13 PM |
Expands
plugin/dtpr/from a two-skill pair into a five-skill authoring studio (dtpr-describe-system,dtpr-datachain-structure,dtpr-category-audit,dtpr-element-design,dtpr-comprehension-audit) backed by a shared comprehension rubric, a file-based research corpus with append-onlyINDEX.md, and an extendedverify.mjsthat validates corpus frontmatter, INDEX append-only integrity against theorigin/mainmerge base, andcross-sibling:*eval symmetry.dtpr-describe-systemnow accepts 0+ artifacts (PDFs, URLs, verbal) via a new Phase 0 trial-call-and-degrade tool probe with multi-host degradation for Claude Code / Cowork / Claude.ai.dtpr-schema-brainstormis hard-deleted; its eval prompts port to the three schema-tier replacements and straddlers land ascross-sibling:*negatives so the router has explicit counter-examples for every peer. Plugin version bumps to 0.2.0 with a newCHANGELOG.mdand a README rewrite that adds the per-host capability matrix, the References section (rubric + block template), and the Research corpus contract. Implementsdocs/plans/2026-04-20-001-feat-dtpr-authoring-studio-plan.mdend-to-end; 77 eval prompts across 5 files,pnpm test:pluginpasses.