feat(from-eval): model-agnostic reasoning-preserving automerge into self-describing SFT rows

[signalrush]

What

Makes rllm dataset from-eval able to SFT-distill a reasoning model — which it currently can't — model-agnostically, and fixes SFT checkpoints so the trained LoRA survives. No new flags.

1. Model-agnostic reasoning-preserving automerge → self-describing SFT rows

from-eval read only the final step's conversation and dropped reasoning_content (_clean_message), so distilling a reasoning teacher yielded reasoning-free data.

It now walks each attempt's steps and emits rows where every message carries a trainable flag, deterministically — no flag or config decides it:

• Reasoning is preserved model-agnostically as a structured ThinkingPart, not a hardcoded <think> string. The model's renderer picks the reasoning format at training time (deepseek <think>, qwen, harmony, …). Content is a uniform list of parts so the parquet column stays one type. Verified: the same curated dataset renders correctly under both the deepseek and qwen renderers, each in its own format.
• Automerge walk (message-level analogue of the RL path's verl.transform._process_trajectory): merges steps that appear identically as history into one row (all turns trained) and splits where they don't. A non-thinking trajectory is one prefix chain → single row; a reasoning turn seals its segment (its thinking is stripped when it becomes history), so the next turn starts a new row → per-turn rows, the inference-faithful form (at inference the harness strips prior-turn thinking, matching each row's stripped history). A mid-episode context reset (summarization) also splits.
• Every turn is trained; only a step with no assistant turn is skipped.

2. Always-CUSTOMIZED masking (removes resolve_train_on_what)

SFT masking is now a single path driven by each message's trainable flag. Self-describing rows (from automerge) use their flags; flag-less rows (external --train-file, other datasets) derive a default in the loader — all assistant turns train, or just the last when tokenize_and_mask_method=stepwise — reproducing the old ALL_ASSISTANT/LAST_ASSISTANT behavior. This removes resolve_train_on_what and the train_on_what enum threading. Backward compatible.

Removes the obsolete last-step-only extraction (_episode_to_messages/_clean_message/_load_messages).

Behavior note: from-eval output now carries trainable and multi-turn traces automerge. Training outcome is equivalent for non-thinking data and strictly better for reasoning data (reasoning preserved). Existing curation tests pass.

3. Promote the final Fireworks LoRA so it survives teardown

The SFT loop saved every checkpoint promotable=False and deletes the trainer job on close, so the trained LoRA was a resumable-only DCP blob GC'd after ~30 days — never servable. The final save is now promotable=True and promote_latest()'d to a permanent account model before infra.close(), mirroring the RL path and the SDK's own sft recipe. Non-fatal on failure.

Validation

• Tests (tests/eval/test_curation_automerge.py): per-turn split, non-thinking merge, every-step-trained, reasoning-seals-segment, context-reset split, prefix primitive, _ensure_trainable derive.
• End-to-end on a real 1,329-episode deepseek-v4-pro terminus2 run: curate() → 32,824 rows (24.7/episode), 0 skipped, 473 merged. All content is structured list-of-parts, no hardcoded <think> strings. Every row renders through the actual CUSTOMIZED datum path with non-zero weights — 0 failures over 300 sampled rows. The same rows render correctly under deepseek (<think>…) and qwen3 (its format), proving model-agnosticism.

🤖 Generated with Claude Code