Name the system table ctk cfr sys-export is reading

[florinutz]

Summary of the changes / Why this is an improvement

ctk cfr sys-export shows a percentage and nothing else, so when it slows down there is no telling which system table it is on. Working it out by hand means knowing how many tables the sys schema has, that they are walked alphabetically, and that tqdm counts the tables already finished rather than the one being read. #429 has waited sixteen months on that question, and #416 was closed without an answer to it.

The progress bar names the table it is reading:

Exporting sys:  24%|##3       | 5/21 [00:00<00:01, 13.21it/s, jobs_log]
Exporting sys:  24%|##3       | 5/21 [00:02<00:01, 13.21it/s, jobs_metrics]

Between the two frames the elapsed counter jumps from 00:00 to 00:02 with jobs_log on screen. The Capturing definitions bar names the relation it is on the same way.

Every read is also logged at debug level with what it cost. Against a sys.jobs_log seeded with 50,000 statements, roughly 410 MB for a single SELECT *:

Read sys.jobs_log: 50000 rows, ~388.1M in memory, 1.467s

A system table's row count and width depend on the cluster, so a support case about an export that took hours gets per-table row counts, sizes and durations to reason from. The size is polars' estimate of the frame in memory: an upper bound for struct columns, and above the bytes that crossed the wire.

Stacked on #895 for its test fixtures; GitHub retargets this to main when #895 merges.

Tests

Two tests in tests/cfr/test_systable.py, each failing with the change removed:

• The progress bar has to name allocations, jobs_log and shards as it reads them.
• The row count logged for information_schema.tables has to move by exactly one when one relation is created, so a count that is hardcoded or always zero fails.

pytest tests/cfr against crate/crate:nightly: 76 passed. poe lint clean.

Checklist

• Link to issue this PR refers to (if applicable): Refs CFR: ctk cfr sys-export stalls after 24% ... #429, CFR: sys-export raises "overflows the data-type's capacity" #416
• Add changes in CHANGES.md

[coderabbitai]

Important

Review skipped

Auto reviews are disabled on this repository. Please check the settings in the CodeRabbit UI or the .coderabbit.yaml file in this repository. To trigger a single review, invoke the @coderabbitai review command.

⚙️ Run configuration

Configuration used: Organization UI

Review profile: CHILL

Plan: Advanced

Run ID: b760becf-e86d-4193-87b2-6411aa422480

You can disable this status message by setting the reviews.review_status to false in the CodeRabbit configuration file.

Use the checkbox below for a quick retry:

• 🔍 Trigger review

---

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

• X
• Mastodon
• Reddit
• LinkedIn

_{Comment @coderabbitai help to get the list of available commands.}