feat(gateway): add use_sglang mode in model gateway that translates upstream agent's /chat/completions requests to sglang server or sgl-router endpoint's /generate requests, required for RL training with sglang (e.g. slime)

[lyzustc]

What this adds

A new use_sglang mode for rllm-model-gateway that routes every agent
/v1/chat/completions turn through SGLang's native /generate API
(token-in input_ids, token-out via meta_info.output_token_logprobs) instead
of the OpenAI /v1/{chat,}/completions endpoints.

Why: RL training needs the exact prompt/completion token ids + logprobs
of every turn. SGLang's OpenAI-compatible endpoints don't expose token ids, and
sgl-router rejects token-id prompts on /v1/completions. The only path that
yields token ids and works behind sgl-router is the native /generate
endpoint. use_sglang makes the gateway:

1. Tokenize each turn's prompt client-side with the model's HF chat
   template — byte-identical to how SGLang's own /v1/chat/completions
   tokenizes (flattens block content, parses tool-call args to dicts, render +
   encode split to avoid double-BOS). A rollout turn therefore matches the
   agent-harness + SGLang-server deployment path exactly.
2. POST input_ids to <worker>/generate with return_logprob=True,
   capturing token ids + logprobs natively — no SGLang server patches, and
   it works through sgl-router.
3. Parse the completion back into structured
   {content, tool_calls, reasoning_content} using SGLang's own
   FunctionCallParser / ReasoningParser, then return a normal OpenAI chat
   response so the agent is unchanged. Content is surfaced losslessly (never
   dropped when a tool call is present), so the agent can replay the full
   message list next turn and it re-tokenizes identically.

Composes with the existing cumulative_token_mode: turn-0 (or after a
prefix break) full-renders via the chat template; subsequent turns use the
renderer bridge to prefix-extend, keeping sampled tokens verbatim for drift-free
multi-turn merging.

sglang remains an optional dependency — all sglang imports are
function-local, so vLLM-only users are unaffected.

Code changes

File	Change
sglang_helper.py (new)	All stateless SGLang logic as free functions: chat-template prep (flatten_*, prepare_template_message, apply_chat_template), parsers (get_fc_parser, parse_completion — lazy sglang imports here), and /generate request/response shaping (sglang_sampling_params, parse_sglang_generate, build_generate_trace, to_openai_tool_calls, generate_to_chat_response).
proxy.py	Adds the use_sglang interception in handle() plus the orchestration skeleton — _render_sglang_prompt, _handle_sglang_generate, _sglang_generate_streaming — which own routing / HTTP / trace-persistence / accumulator state and delegate all SGLang specifics to sglang_helper. New __init__ fields: use_sglang, tokenizer, sglang_tool_call_parser, sglang_reasoning_parser, plus a one-time tokenizer.encode("") probe for special-token handling.
server.py	New CLI flags (--use-sglang, --sglang-tool-call-parser, --sglang-reasoning-parser); builds the tokenizer when use_sglang / cumulative_token_mode is set and wires the new fields into ReverseProxy.
models.py	GatewayConfig gains use_sglang, sglang_tool_call_parser, sglang_reasoning_parser.
tests/	New tests/unit/test_sglang.py (client-side tokenization, block-content regression, structured tool-call parsing, native token + logprob trace capture, cumulative prefix-extension vs non-cumulative, streaming) and tests/helpers/mock_sglang.py (mock /generate server).

Usage

Launch the gateway in use_sglang mode pointed at an SGLang server (or
sgl-router in front of several), then register the worker and point any
OpenAI-compatible agent at a gateway session — no agent code changes.

python -m rllm_model_gateway \
  --port 9090 \
  --use-sglang \
  --model Qwen/Qwen3-4B-Instruct-2507 \
  --sglang-tool-call-parser qwen
# add  --cumulative-token-mode --renderer-family qwen3   for drift-free multi-turn merging
# add  --sglang-reasoning-parser qwen3                    for <think>…</think> models

Each turn's TraceRecord then carries the native prompt_token_ids,
completion_token_ids, and aligned logprobs — ready to feed a training engine
(e.g. slime GRPO).