diff --git a/skills/visual-diff/SKILL.md b/skills/visual-diff/SKILL.md index 3b3f353..dad57b2 100644 --- a/skills/visual-diff/SKILL.md +++ b/skills/visual-diff/SKILL.md @@ -1,9 +1,9 @@ --- name: visual-diff -description: Compare two renderings of the same set of pages - a "local" base against a "live" base - using odiff, and report which pages match. Use to verify a port/migration, check parity after edits, or QA before shipping, instead of eyeballing full-page screenshots. Triggered by /visual-diff. +description: Compare two renderings of the same set of pages - a "local" base against a "live" base - using odiff, and report which pages match. Use to verify a port/migration, check parity after edits, or QA before shipping, instead of eyeballing full-page screenshots. Drop to --sections to localize where a page differs. Triggered by /visual-diff. trigger: /visual-diff user-invocable: true -argument-hint: "[page-slug] [--mobile] [--full]" +argument-hint: "[page-slug] [--sections] [--mobile] [--full]" disable-model-invocation: true --- @@ -56,10 +56,11 @@ you only look at images when something is actually off. 2. **Run the comparison.** Pass `$ARGUMENTS` through: - no arg → all pages, desktop viewport - a slug (e.g. `pricing`) → `--page ` + - `--sections` → diff each `
` of the page(s) separately (see below) - `--mobile` → also capture 390×844 - `--full` → full-page captures (padded diff; height differences raise the %) ```bash - node path/to/skills/visual-diff/scripts/visual-diff.mjs [--page ] [--mobile] [--full] + node path/to/skills/visual-diff/scripts/visual-diff.mjs [--page ] [--sections] [--mobile] [--full] ``` Bases can also be overridden ad-hoc: `--local-base --live-base `. The command prints the run directory, e.g. `tmp/visual-diff//`. @@ -67,9 +68,12 @@ you only look at images when something is actually off. 3. **Read ONLY `tmp/visual-diff//report.md`.** It is a small table: `Page | Viewport | Diff % | Status | Diff image`. -4. **For rows marked `❌ fail` (and only those), open the listed `diff*.png`** with the - Read tool to see the changed regions (highlighted in magenta `#ff00ff`). Do **not** - read `local.png` / `live.png` unless the diff image alone is ambiguous. Skip +4. **For every row marked `❌ fail` you MUST open its `diff*.png`** with the Read tool to + see the changed regions (highlighted in magenta `#ff00ff`). This is required, not + optional: the diff % tells you *that* a row failed, never *what* changed. Do not + propose a fix from the number or section name alone. The report ends with a + "Required next step" checklist of the exact images to read. Do **not** read + `local.png` / `live.png` unless the diff image alone is ambiguous, and skip `✅ pass` rows entirely - they are within the gate. 5. **Report** the pass/fail table back to the user. For each failure, name the likely @@ -77,6 +81,32 @@ you only look at images when something is actually off. line-height, exact brand-color hex, border-radius, SVG/icon size, section padding, or a missing/resized section (large diff % with a "size mismatch" note). +## Section by section (`--sections`) + +A full-page fail tells you a page differs but not *where*, and one tall section can +drag the whole number. `--sections` localizes it: it auto-detects each `
` on +the **local** page by its first semantic class (`hero`, `pricing`, `faq`, …), clips the +**same selector** on both sides, and diffs each region independently. + +```bash +node path/to/scripts/visual-diff.mjs --page --sections # one page, all its sections +node path/to/scripts/visual-diff.mjs --sections # every page, section by section +``` + +- The `report.md` gains a **Section** column; read it the same way and open a + `
-diff.png` only for `❌ fail` rows. Per-section PNGs are + `
-local.png` / `
-live.png` / `
-diff.png`. +- If the live side is missing a section's class, that row reports `⚠️ error` + ("Selector not found") - a signal the ported section didn't get its semantic class. +- **Override the auto-detected list** per page in your config/manifest by adding a + `sections` array (strings, or `{ "selector": ".cards .grid", "label": "grid" }` to + target a child): + ```json + { "slug": "pricing", "path": "/pricing/", "sections": [".hero", ".tiers", "#faq"] } + ``` +- Capture both sides at the **same viewport width** (`--mobile` / default) - a width + mismatch reflows the section and inflates the diff. + ## Notes - The acceptance gate defaults to `< 0.1%` diff. Override with `thresholdPct` in config, diff --git a/skills/visual-diff/config.example.json b/skills/visual-diff/config.example.json index 150b31c..2d4adb9 100644 --- a/skills/visual-diff/config.example.json +++ b/skills/visual-diff/config.example.json @@ -1,5 +1,5 @@ { - "$comment": "Copy to your project root as visual-diff.config.json and edit. localBase/liveBase are the two renderings to compare; pages share `path` across both. thresholdPct is the pass gate (default 0.1). You can instead keep pages in a sibling visual-diff.pages.json.", + "$comment": "Copy to your project root as visual-diff.config.json and edit. localBase/liveBase are the two renderings to compare; pages share `path` across both. thresholdPct is the pass gate (default 0.1). You can instead keep pages in a sibling visual-diff.pages.json. A page may add an optional `sections` array (e.g. [\".hero\", \".pricing\"]) to override --sections auto-detection.", "localBase": "http://127.0.0.1:8765", "liveBase": "https://staging.example.com", "thresholdPct": 0.1, diff --git a/skills/visual-diff/scripts/lib/capture.mjs b/skills/visual-diff/scripts/lib/capture.mjs index fdce58f..4d26b99 100644 --- a/skills/visual-diff/scripts/lib/capture.mjs +++ b/skills/visual-diff/scripts/lib/capture.mjs @@ -32,6 +32,8 @@ export async function launchBrowser() { * @param {number} [o.width] Viewport width (default 1280). * @param {number} [o.height] Viewport height (default 800). * @param {number} [o.timeout] Navigation timeout ms (default 30000). + * @param {string} [o.selector] CSS selector to clip to (captures just that element, + * scrolled into view - used for section-by-section diffs). * @param {import('playwright').Browser} [o.browser] Reuse an existing browser. * @returns {Promise<{ width:number, height:number }>} viewport used. */ @@ -42,6 +44,7 @@ export async function captureScreenshot({ width = 1280, height = 800, timeout = 30_000, + selector = null, browser = null, }) { const ownBrowser = !browser; @@ -70,7 +73,7 @@ export async function captureScreenshot({ await page.evaluate(() => document.fonts && document.fonts.ready).catch(() => {}); await page.waitForTimeout(300); - if (full) { + if (full || selector) { // Scroll through the page so IntersectionObserver reveal triggers for every section. await page.evaluate(async () => { await new Promise((resolve) => { @@ -87,7 +90,20 @@ export async function captureScreenshot({ await page.waitForTimeout(300); } - await page.screenshot({ path: outPath, fullPage: full }); + if (selector) { + // Section-by-section: clip to a single element. scrollIntoView re-triggers its + // reveal observer; the element-clip keeps both sides the same region/size. + const el = page.locator(selector).first(); + if (!(await el.count())) { + await page.close(); + throw new Error(`Selector not found: ${selector} on ${url}`); + } + await el.scrollIntoViewIfNeeded(); + await page.waitForTimeout(150); + await el.screenshot({ path: outPath }); + } else { + await page.screenshot({ path: outPath, fullPage: full }); + } await page.close(); return { width, height }; } finally { diff --git a/skills/visual-diff/scripts/visual-diff.mjs b/skills/visual-diff/scripts/visual-diff.mjs index a98d724..2640306 100644 --- a/skills/visual-diff/scripts/visual-diff.mjs +++ b/skills/visual-diff/scripts/visual-diff.mjs @@ -19,12 +19,17 @@ * node visual-diff.mjs # all pages, desktop (1280×800) * node visual-diff.mjs --mobile # desktop + mobile (390×844) * node visual-diff.mjs --page # a single page from the manifest + * node visual-diff.mjs --sections # diff each
separately (localize a fail) * node visual-diff.mjs --full # full-page captures (padded diff) * node visual-diff.mjs --local-base http://127.0.0.1:8765 * node visual-diff.mjs --live-base https://staging.example.com * node visual-diff.mjs --config path/to/visual-diff.config.json * - * Exit codes: 0 = every page passes (< threshold), 2 = one or more fail/error. + * --sections auto-detects each page's
elements on the LOCAL page and + * diffs each by its semantic class (hero, pricing, faq, …). A page in the manifest + * may override detection with a `sections` array (selectors or {selector,label}). + * + * Exit codes: 0 = every row passes (< threshold), 2 = one or more fail/error. * Output: /tmp/visual-diff// (report.md, report.json, PNGs). */ @@ -63,6 +68,7 @@ const LIVE_BASE = String(pick('--live-base', 'VDIFF_LIVE_BASE', 'liveBase', const gatePct = Number(pick('--threshold', 'VDIFF_THRESHOLD', 'thresholdPct', THRESHOLD_PCT)); const fullPage = flag('--full'); const onlyPage = opt('--page', null); +const sectionsMode = flag('--sections'); const VIEWPORTS = [{ name: 'desktop', width: 1280, height: 800 }]; if (flag('--mobile')) VIEWPORTS.push({ name: 'mobile', width: 390, height: 844 }); @@ -123,37 +129,42 @@ for (const page of pages) { for (const vp of VIEWPORTS) { const suffix = vp.name === 'desktop' ? '' : `-${vp.name}`; - const localPath = join(pageDir, `local${suffix}.png`); - const livePath = join(pageDir, `live${suffix}.png`); - const diffPath = join(pageDir, `diff${suffix}.png`); - const row = { slug: page.slug, viewport: vp.name, path: page.path }; - try { - await captureScreenshot({ - url: LOCAL_BASE + page.path, outPath: localPath, - full: fullPage, width: vp.width, height: vp.height, browser, - }); - await captureScreenshot({ - url: LIVE_BASE + page.path, outPath: livePath, - full: fullPage, width: vp.width, height: vp.height, browser, - timeout: 45_000, // remote/live side is usually slower than localhost - }); + if (!sectionsMode) { + const row = await diffPair({ page, vp, suffix, pageDir, section: null }); + results.push(row); + logRow(row); + continue; + } - const cmp = await odiffCompare(localPath, livePath, diffPath, { gatePct }); - row.diffPercentage = cmp.diffPercentage; - row.diffCount = cmp.diffCount; - row.status = cmp.pass ? 'pass' : 'fail'; - row.message = cmp.message; - row.diffImage = cmp.pass ? null : relize(cmp.diffPath, ROOT); - console.log(` ${row.status === 'pass' ? '✓' : '✗'} ${page.slug} [${vp.name}] - ${cmp.message}`); + // ── section-by-section ────────────────────────────────────────────────── + let sections; + try { + sections = await resolveSections(page, vp); } catch (err) { - row.status = 'error'; - row.message = err.message; - row.diffPercentage = null; - console.log(` ! ${page.slug} [${vp.name}] - ERROR: ${err.message}`); + const row = { slug: page.slug, section: '(detect)', viewport: vp.name, path: page.path, + status: 'error', message: `section detect failed: ${err.message}`, diffPercentage: null }; + results.push(row); + logRow(row); + continue; } - - results.push(row); + if (!sections.length) { + const row = { slug: page.slug, section: '(none)', viewport: vp.name, path: page.path, + status: 'error', message: 'no
with a semantic class found on local', diffPercentage: null }; + results.push(row); + logRow(row); + continue; + } + for (const section of sections) { + const row = await diffPair({ page, vp, suffix, pageDir, section }); + results.push(row); + logRow(row); + } + // Whole-page guard: catches sections that exist on live but not on local. + // Per-section rows only confirm matched sections pass; this ensures structural parity. + const guardRow = await diffPair({ page, vp, suffix, pageDir, section: { selector: null, label: '(whole-page)' } }); + results.push(guardRow); + logRow(guardRow); } } @@ -172,37 +183,165 @@ const failed = results.filter((r) => r.status !== 'pass'); console.log(`\nReport: ${relize(join(runDir, 'report.md'), ROOT)}`); console.log(`${results.length - failed.length}/${results.length} passed.`); if (failed.length) { - console.log(`Open the diff image for these rows only:`); - for (const r of failed) console.log(` - ${r.slug} [${r.viewport}]${r.diffImage ? ` → ${r.diffImage}` : ''}`); + console.log(`\nREQUIRED NEXT STEP - open each diff image below with the Read tool and`); + console.log(`look at the magenta (#ff00ff) regions. The diff % alone does NOT tell you`); + console.log(`what changed; do not propose a fix without viewing the image first.`); + for (const r of failed) { + const note = r.diffImage ? '' : ` (no diff image - ${r.message})`; + console.log(` - ${rowTag(r)} [${r.viewport}]${r.diffImage ? ` → ${r.diffImage}` : note}`); + } process.exit(2); } +// ── capture/diff one pair (whole page, or one section when `section` is set) ── +async function diffPair({ page, vp, suffix, pageDir, section }) { + const stem = section ? `${slugify(section.label)}${suffix}` : null; + const localPath = join(pageDir, section ? `${stem}-local.png` : `local${suffix}.png`); + const livePath = join(pageDir, section ? `${stem}-live.png` : `live${suffix}.png`); + const diffPath = join(pageDir, section ? `${stem}-diff.png` : `diff${suffix}.png`); + const row = { slug: page.slug, viewport: vp.name, path: page.path }; + if (section) row.section = section.label; + + try { + await captureScreenshot({ + url: LOCAL_BASE + page.path, outPath: localPath, + full: fullPage, width: vp.width, height: vp.height, browser, + selector: section ? section.selector : null, + }); + await captureScreenshot({ + url: LIVE_BASE + page.path, outPath: livePath, + full: fullPage, width: vp.width, height: vp.height, browser, + timeout: 45_000, // remote/live side is usually slower than localhost + selector: section ? section.selector : null, + }); + + const cmp = await odiffCompare(localPath, livePath, diffPath, { gatePct }); + row.diffPercentage = cmp.diffPercentage; + row.diffCount = cmp.diffCount; + row.status = cmp.pass ? 'pass' : 'fail'; + row.message = cmp.message; + row.diffImage = cmp.pass ? null : relize(cmp.diffPath, ROOT); + } catch (err) { + row.status = 'error'; + row.message = err.message; + row.diffPercentage = null; + } + return row; +} + +/** Section list for a page: manifest `sections` override, else auto-detect on local. */ +async function resolveSections(page, vp) { + if (Array.isArray(page.sections) && page.sections.length) { + return page.sections.map((s) => + typeof s === 'string' ? { selector: s, label: s.replace(/^[.#]/, '') } : s); + } + return detectSections(LOCAL_BASE + page.path, vp); +} + +/** Open the local page and read each
's first semantic class. */ +async function detectSections(url, vp) { + const p = await browser.newPage(); + try { + await p.setViewportSize({ width: vp.width, height: vp.height }); + const resp = await p.goto(url, { waitUntil: 'load', timeout: 30_000 }); + if (resp && !resp.ok()) throw new Error(`HTTP ${resp.status()} for ${url}`); + const found = await p.evaluate(() => { + // Skip generic layout/utility tokens and Tailwind-style responsive prefixes (e.g. md:py-24). + const GENERIC = new Set(['section', 'container', 'wrapper', 'inner', 'content', 'row', 'col']); + const isUtility = (c) => GENERIC.has(c) || /^[a-z]{2,3}:/.test(c); + const list = []; + document.querySelectorAll('section').forEach((el) => { + // Skip sections hidden by CSS or feature toggles — screenshotting them would timeout. + const style = window.getComputedStyle(el); + if (style.display === 'none' || style.visibility === 'hidden') return; + const cls = [...el.classList].find((c) => !isUtility(c)); + if (!cls) return; + // Qualify with `section` tag so we target the element itself, not any unrelated `.cls` node. + // CSS.escape handles class names with special chars (colons, slashes, etc.). + list.push({ selector: 'section.' + CSS.escape(cls), label: cls }); + }); + return list; + }); + const seen = new Set(); // de-dup by selector, keep first occurrence + return found.filter((s) => (seen.has(s.selector) ? false : (seen.add(s.selector), true))); + } finally { + await p.close(); + } +} + // ── helpers ────────────────────────────────────────────────────────────────────── function relize(p, root) { return p && p.startsWith(root) ? p.slice(root.length + 1) : p; } +function slugify(s) { + return String(s).replace(/[^a-z0-9]+/gi, '-').replace(/^-+|-+$/g, '').toLowerCase(); +} + +function rowTag(r) { + return r.section ? `${r.slug} › ${r.section}` : r.slug; +} + +function logRow(r) { + const mark = r.status === 'pass' ? '✓' : r.status === 'fail' ? '✗' : '!'; + console.log(` ${mark} ${rowTag(r)} [${r.viewport}] - ${r.message}`); +} + function buildMarkdown(rows) { const passed = rows.filter((r) => r.status === 'pass').length; + const hasSections = rows.some((r) => r.section); const lines = [ `# Visual diff report`, ``, `- Generated: ${new Date().toISOString()}`, `- Local: ${LOCAL_BASE} · Live: ${LIVE_BASE}`, - `- Capture: ${fullPage ? 'full-page' : 'viewport'} · Gate: < ${gatePct}% diff`, + `- Capture: ${hasSections ? 'section-by-section' : fullPage ? 'full-page' : 'viewport'} · Gate: < ${gatePct}% diff`, `- Result: **${passed}/${rows.length} passed**`, ``, - `> Read this table first. Open a diff image ONLY for rows marked \`fail\`.`, + `> Read this table first. Diff % tells you *whether* a row failed, not *what* changed -`, + `> for that you MUST open the diff image (see the required step below).`, ``, - `| Page | Viewport | Diff % | Status | Diff image |`, - `| --- | --- | ---: | --- | --- |`, + hasSections + ? `| Page | Section | Viewport | Diff % | Status | Diff image |` + : `| Page | Viewport | Diff % | Status | Diff image |`, + hasSections + ? `| --- | --- | --- | ---: | --- | --- |` + : `| --- | --- | ---: | --- | --- |`, ]; for (const r of rows) { const pct = r.diffPercentage == null ? '-' : `${r.diffPercentage.toFixed(2)}%`; const badge = r.status === 'pass' ? '✅ pass' : r.status === 'fail' ? '❌ fail' : '⚠️ error'; const img = r.status === 'pass' ? '-' : (r.diffImage ? `\`${r.diffImage}\`` : `(${r.message})`); - lines.push(`| ${r.slug} | ${r.viewport} | ${pct} | ${badge} | ${img} |`); + lines.push(hasSections + ? `| ${r.slug} | ${r.section || '-'} | ${r.viewport} | ${pct} | ${badge} | ${img} |` + : `| ${r.slug} | ${r.viewport} | ${pct} | ${badge} | ${img} |`); } + + // Explicit, imperative next-step block so the agent actually inspects the + // diffs instead of reporting percentages. Listed only for failing rows. + const failed = rows.filter((r) => r.status !== 'pass'); + if (failed.length) { + lines.push( + ``, + `## ⛔ Required next step - inspect every diff below`, + ``, + `${failed.length} row(s) failed. **Before proposing any fix, open each diff image`, + `with the Read tool** and look at the magenta (\`#ff00ff\`) regions - that is the only`, + `way to know *what* differs. Do **not** infer the cause from the diff % or section`, + `name. Work the list top to bottom; for each, name the changed element and the likely`, + `cause (font/letter-spacing, line-height, color hex, radius, icon size, padding, or a`, + `size mismatch = a section that grew/shrank).`, + ``, + ); + for (const r of failed) { + const where = `${rowTag(r)} [${r.viewport}]`; + const pct = r.diffPercentage == null ? '' : ` - ${r.diffPercentage.toFixed(2)}%`; + lines.push(r.diffImage + ? `- [ ] \`${r.diffImage}\` - ${where}${pct}` + : `- [ ] ${where}${pct} - no diff image (${r.message})`); + } + } + lines.push(''); return lines.join('\n'); }