otel-genai-diff

Diff two otel-genai-rollup outputs and flag breaking changes for a CI gate — e.g., a new model appeared, per-row cost jumped > 10%, span count exploded, or currency changed.

Makes the diff family a quintet alongside the per-protocol diffs:

• agent-card-diff — A2A AgentCards
• mcp-tool-card-diff — MCP Tool Cards
• prompt-provenance-diff — prompt provenance
• evidence-bundle-diff — evidence bundles
• otel-genai-diff — OTel GenAI rollups (today)

Part of the Kinetic Gain Suite.

---

Why

otel-genai-rollup produces daily aggregates of GenAI usage and cost. This diff tool lets you compare two such rollups — today vs yesterday, this PR vs main, this region vs that — and fail a CI gate when something noteworthy changed.

Change reasons

dimension	reasons
fleet	model-added, model-removed, provider-added, provider-removed
volume	spans-jumped, spans-dropped, input-tokens-jumped, output-tokens-jumped
economics	cost-increased, cost-decreased, currency-changed, pricing-status-changed
meta	window-shifted

Breaking reasons (set diff.breaking = true, CLI exits 1):

• model-added / model-removed (new model = needs re-evaluation)
• cost-increased, input-tokens-jumped, output-tokens-jumped, spans-jumped (cost / scale regressions)
• currency-changed

cost-decreased, spans-dropped, and pricing-status-changed are informational.

CLI

otel-genai-diff previous-rollup.json next-rollup.json
    [--format json|markdown|summary]
    [--threshold 0.10]
    [--strict] [--out FILE]

--threshold (default 0.10) — minimum relative delta to flag a token / span / cost change. Pass 0.25 for tail-of-week reports where you only care about > 25% jumps.

Exit codes:

• 0 — no changes (or only non-breaking)
• 1 — breaking diff (or --strict and any change)
• 2 — usage / I/O error

Library

import { diffRollups, toMarkdown } from "otel-genai-diff";
import type { RollupReport } from "otel-genai-diff";

const prev: RollupReport = JSON.parse(readFileSync("yesterday.json", "utf8"));
const next: RollupReport = JSON.parse(readFileSync("today.json", "utf8"));

const diff = diffRollups(prev, next, { threshold: 0.1 });
console.log(diff.breaking, diff.changes.length, diff.added.models);
console.log(toMarkdown(diff));

Composes with

• otel-genai-rollup — produces the inputs.
• otel-genai-validator — validates the raw OTLP spans before they get rolled up.
• otel-genai-test-vectors — conformance corpus.
• llm-cost-rollup-action — GitHub Action that runs the rollup; pair this diff after it to gate PRs.

License

AGPL-3.0-or-later