@
What
Add a TPC-DI generator alongside the existing TPC-H / TPC-DS crates — same shape: a tpcdigen crate plus a tpcdi subcommand on the CLI, zero-dependency, single static binary.
How it differs from H/DS (the actual work)
TPC-DI doesnt emit a set of relational tables. It generates a heterogeneous source staging area that mimics an OLTP/CRM/HR landscape:
- pipe- and comma-delimited flat files,
- one XML file (
CustomerMgmt.xml),
- fixed-width FINWIRE records (CMP/SEC/FIN),
- split into an initial historical load (Batch1) plus incremental batches (Batch2/Batch3).
Scale factor drives row counts. So the interesting part is the multi-format writers and the batch model, not new query answer sets.
The reason — not speed, not scale
The only official generator is DIGen (DIGen.jar), which wraps PDGF and needs a JDK, the license-gated TPC-DI kit, and a working Java classpath. Functionally its fine; this isnt a performance complaint. The problem is friction — I lost a weekend getting Java/PDGF configured just to produce a handful of files. A cargo install (or pip install) binary that just writes the files out would have saved all of it. You already removed this exact pain for dbgen and dsdgen; TPC-DI is the last major TPC data benchmark still gated behind a JVM-only toolchain.
Thats the whole ask — user empathy, not throughput.
@
@
What
Add a TPC-DI generator alongside the existing TPC-H / TPC-DS crates — same shape: a
tpcdigencrate plus atpcdisubcommand on the CLI, zero-dependency, single static binary.How it differs from H/DS (the actual work)
TPC-DI doesnt emit a set of relational tables. It generates a heterogeneous source staging area that mimics an OLTP/CRM/HR landscape:
CustomerMgmt.xml),Scale factor drives row counts. So the interesting part is the multi-format writers and the batch model, not new query answer sets.
The reason — not speed, not scale
The only official generator is DIGen (
DIGen.jar), which wraps PDGF and needs a JDK, the license-gated TPC-DI kit, and a working Java classpath. Functionally its fine; this isnt a performance complaint. The problem is friction — I lost a weekend getting Java/PDGF configured just to produce a handful of files. Acargo install(orpip install) binary that just writes the files out would have saved all of it. You already removed this exact pain for dbgen and dsdgen; TPC-DI is the last major TPC data benchmark still gated behind a JVM-only toolchain.Thats the whole ask — user empathy, not throughput.
@