This affects only TPC-DS because TPC-H has no sales/returns pair concept.
Follow-up to #354 and #391; part of #218.
Requirement
For each sales/returns pair:
- Sales only → generate and output only sales.
- Returns only → generate and output only returns.
- Both → generate both in one shared pass and output both.
Generated work and output files must exactly match the selected tables.
Issues
1. Selecting one table produces both
Affected formats: DAT and CSV.
Selecting only sales or only returns generates and writes both tables.
2. Selecting both traverses the shared source twice
Affected format: Parquet.
Selecting both tables writes the correct outputs but generates each independently instead of sharing one source pass.
Behavior matrix
Using store_sales and store_returns as the representative pair:
| Format |
Sales only |
Returns only |
Both |
| DAT¹ |
❌ Outputs both |
❌ Outputs both |
✅ One pass, both outputs |
| CSV |
❌ Outputs both |
❌ Outputs both |
✅ One pass, both outputs |
| Parquet |
✅ Sales only |
✅ Returns only |
❌ Two source passes |
¹ DAT is the TPC-DS equivalent of TBL.
Reproduction
Issue 1: DAT and CSV output both tables
# DAT: each command incorrectly produces both paired files
cargo run -p tpcgen-cli --bin tpcgen-cli tpcds dat --scale-factor=1 --tables=store_sales -o "$(mktemp -d)"
cargo run -p tpcgen-cli --bin tpcgen-cli tpcds dat --scale-factor=1 --tables=store_returns -o "$(mktemp -d)"
# CSV: each command incorrectly produces both paired files
cargo run -p tpcgen-cli --bin tpcgen-cli tpcds csv --scale-factor=1 --tables=store_sales -o "$(mktemp -d)"
cargo run -p tpcgen-cli --bin tpcgen-cli tpcds csv --scale-factor=1 --tables=store_returns -o "$(mktemp -d)"
Selecting both already works:
cargo run -p tpcgen-cli --bin tpcgen-cli tpcds dat --scale-factor=1 --tables=store_sales,store_returns -o "$(mktemp -d)"
cargo run -p tpcgen-cli --bin tpcgen-cli tpcds csv --scale-factor=1 --tables=store_sales,store_returns -o "$(mktemp -d)"
Issue 2: Parquet traverses the shared source twice
Single-table selection works:
cargo run -p tpcgen-cli --bin tpcgen-cli tpcds parquet --scale-factor=1 --tables=store_sales -o "$(mktemp -d)"
cargo run -p tpcgen-cli --bin tpcgen-cli tpcds parquet --scale-factor=1 --tables=store_returns -o "$(mktemp -d)"
Selecting both produces the correct files but traverses the shared source twice:
cargo run -p tpcgen-cli --bin tpcgen-cli tpcds parquet --scale-factor=1 --tables=store_sales,store_returns -o "$(mktemp -d)"
Acceptance criteria
- DAT/CSV single-table selection generates and writes only the selected table.
- DAT/CSV paired selection continues to generate both in one pass.
- Parquet paired selection generates both in one shared pass.
- Cover store, catalog, and web pairs across every affected format.
Repeated selection is already handled by #391 (TPC-DS) and #392 (TPC-H) and is outside this issue.
This affects only TPC-DS because TPC-H has no sales/returns pair concept.
Follow-up to #354 and #391; part of #218.
Requirement
For each sales/returns pair:
Generated work and output files must exactly match the selected tables.
Issues
1. Selecting one table produces both
Affected formats: DAT and CSV.
Selecting only sales or only returns generates and writes both tables.
2. Selecting both traverses the shared source twice
Affected format: Parquet.
Selecting both tables writes the correct outputs but generates each independently instead of sharing one source pass.
Behavior matrix
Using
store_salesandstore_returnsas the representative pair:¹ DAT is the TPC-DS equivalent of TBL.
Reproduction
Issue 1: DAT and CSV output both tables
Selecting both already works:
Issue 2: Parquet traverses the shared source twice
Single-table selection works:
Selecting both produces the correct files but traverses the shared source twice:
cargo run -p tpcgen-cli --bin tpcgen-cli tpcds parquet --scale-factor=1 --tables=store_sales,store_returns -o "$(mktemp -d)"Acceptance criteria
Repeated selection is already handled by #391 (TPC-DS) and #392 (TPC-H) and is outside this issue.