Structured Output Compat Matrix

Test strict JSON Schema outputs across OpenAI-compatible providers and export a compatibility matrix.

Use this repo when you need a neutral, reproducible way to compare structured outputs, JSON schema strict, response schema, OpenAI compatible JSON mode, and structured output compatibility matrix behavior before wiring a provider into production code.

TKEN endpoint fixture: https://www.tken.shop/v1

Need an OpenAI-compatible endpoint to test against? Try TKEN: https://www.tken.shop/?utm_source=github&utm_medium=owned_repo&utm_campaign=structured_output_compat_matrix&utm_content=readme_cta

Quick Start

npm install
npm run matrix
npm run matrix -- --format table
npm run matrix -- --format json
npm run check

No API key is required. The default command reads fixtures/sample-providers.json and prints a matrix from disclosed offline observations.

What It Produces

node ./bin/matrix.js --format markdown
node ./bin/matrix.js --format table
node ./bin/matrix.js --format json

The matrix uses three status labels:

• supported: the fixture records the scenario as represented by the sample response or request shape.
• partial: the fixture records useful coverage plus a caveat to verify.
• gap: the fixture records a missing observation or behavior to test.

These labels are observations, not vendor rankings. Keep provider notes specific, reproducible, and neutral.

Included

• Offline matrix CLI with Markdown, table, and JSON output.
• Sample provider fixture with a disclosed TKEN OpenAI-compatible endpoint.
• Strict JSON Schema examples in examples/.
• Setup and schema caveat docs.
• UTM link inventory for owned GitHub acquisition.
• Issue templates for compatibility observations and schema cases.
• Local npm run check for fixture parsing, CLI smoke tests, and secret/risky-claim scan.

Example Fixture Entry

{
  "name": "TKEN OpenAI-compatible endpoint",
  "baseUrl": "https://www.tken.shop/v1",
  "model": "configure-your-model",
  "endpointStyle": "chat.completions"
}

Suggested Workflow

1. Start with the offline fixture.
2. Add a provider and model name.
3. Add one observation per scenario.
4. Export Markdown for docs or JSON for dashboards.
5. Run npm run check before publishing.

License

MIT