feat(slime): multi-turn trace merging, validation at an input dataset and gateway config fix in custom rollout function#81
Merged
Conversation
… and gateway config fix in custom rollout function
luyuzhe111
approved these changes
Jul 6, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
This PR adds drift-free multi-turn trajectory merging to the slime custom rollout
function and fixes several issues that surfaced when running on full training /
validation datasets (eval episodes timing out and crashing the engine, connection-pool
churn, and unreadably noisy logs). It also brings
SlimeRunnerand the example scriptsin line with these changes.
Custom rollout function (
integration/)traces.py). Addedmerge_traces_to_samples: an episode's turns are folded into one trainingSample per contiguous prefix-extending segment (bridge tokens masked out,
completions trained), instead of one Sample per turn. Pairs with the gateway's
cumulative token mode; falls back to one-Sample-per-turn when a turn breaks the
prefix.
rollout.py,gateway.py). Newcumulative_token_mode/renderer_familyconfig knobs, forwarded to thegateway;
merge_traces_to_samplesis wired in as the rollout's sample builder.rollout.py). Supported doing validation on an input validationset periodically with an interval in training. Bounded in-flight agent sessions in validation
by
max_concurrentin yaml config.rollout.py). Addedmax_pool_connectionsso user can set
>= max_concurrentto stop boto3 client logging "Connection poolis full, discarding connection" under concurrent ACR sessions.
rollout.py,gateway.py). Replaced per-session loggingwith a single per-batch summary (
episodes / succeeded / failed / sequences);failed episodes log once at INFO with the full ACR
session_idfor CloudWatchlookup. Added
gateway_log_level(defaultwarning) to silence the gateway'suvicorn/httpx access logs.
/generate(gateway.py). The slimebackend now enables
use_sglangunconditionally and always passes the served--model; removed the deaduse_sglangconfig field. (The/generatemechanics live in the rllm-model-gateway PR.)
Training scripts & runner
runner.py— exposed the new knobs onSlimeRunner(
sglang_tool_call_parser,sglang_reasoning_parser,cumulative_token_mode,renderer_family,max_pool_connections,gateway_log_level,sglang_context_length) and addedcuda_hometo pin CUDA_HOME/LD_LIBRARY_PATH(incl.
--train-env-varsfor the Megatron actors), fixing TransformerEngine's"Multiple libcudart" abort. The tool-call parser is now a field instead of a
hardcoded
qwen25.examples/math_agent/train.sh— ported the CUDA-toolchain pinning,torch_memory_saverfixup, eval flags, log suppression, and checkpointing fromthe working run; all paths/credentials stay env-var placeholders.
config.yaml.example— documented the newmax_pool_connections,cumulative_token_mode, andrenderer_familysettings.This PR must work together with rllm-org/rllm#715.
A follow-up PR with slime environment setup guide in docs changes will come soon.