-
Notifications
You must be signed in to change notification settings - Fork 6
Expand file tree
/
Copy pathupstream_watchlist.yaml
More file actions
1548 lines (1415 loc) · 88.3 KB
/
Copy pathupstream_watchlist.yaml
File metadata and controls
1548 lines (1415 loc) · 88.3 KB
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
296
297
298
299
300
301
302
303
304
305
306
307
308
309
310
311
312
313
314
315
316
317
318
319
320
321
322
323
324
325
326
327
328
329
330
331
332
333
334
335
336
337
338
339
340
341
342
343
344
345
346
347
348
349
350
351
352
353
354
355
356
357
358
359
360
361
362
363
364
365
366
367
368
369
370
371
372
373
374
375
376
377
378
379
380
381
382
383
384
385
386
387
388
389
390
391
392
393
394
395
396
397
398
399
400
401
402
403
404
405
406
407
408
409
410
411
412
413
414
415
416
417
418
419
420
421
422
423
424
425
426
427
428
429
430
431
432
433
434
435
436
437
438
439
440
441
442
443
444
445
446
447
448
449
450
451
452
453
454
455
456
457
458
459
460
461
462
463
464
465
466
467
468
469
470
471
472
473
474
475
476
477
478
479
480
481
482
483
484
485
486
487
488
489
490
491
492
493
494
495
496
497
498
499
500
501
502
503
504
505
506
507
508
509
510
511
512
513
514
515
516
517
518
519
520
521
522
523
524
525
526
527
528
529
530
531
532
533
534
535
536
537
538
539
540
541
542
543
544
545
546
547
548
549
550
551
552
553
554
555
556
557
558
559
560
561
562
563
564
565
566
567
568
569
570
571
572
573
574
575
576
577
578
579
580
581
582
583
584
585
586
587
588
589
590
591
592
593
594
595
596
597
598
599
600
601
602
603
604
605
606
607
608
609
610
611
612
613
614
615
616
617
618
619
620
621
622
623
624
625
626
627
628
629
630
631
632
633
634
635
636
637
638
639
640
641
642
643
644
645
646
647
648
649
650
651
652
653
654
655
656
657
658
659
660
661
662
663
664
665
666
667
668
669
670
671
672
673
674
675
676
677
678
679
680
681
682
683
684
685
686
687
688
689
690
691
692
693
694
695
696
697
698
699
700
701
702
703
704
705
706
707
708
709
710
711
712
713
714
715
716
717
718
719
720
721
722
723
724
725
726
727
728
729
730
731
732
733
734
735
736
737
738
739
740
741
742
743
744
745
746
747
748
749
750
751
752
753
754
755
756
757
758
759
760
761
762
763
764
765
766
767
768
769
770
771
772
773
774
775
776
777
778
779
780
781
782
783
784
785
786
787
788
789
790
791
792
793
794
795
796
797
798
799
800
801
802
803
804
805
806
807
808
809
810
811
812
813
814
815
816
817
818
819
820
821
822
823
824
825
826
827
828
829
830
831
832
833
834
835
836
837
838
839
840
841
842
843
844
845
846
847
848
849
850
851
852
853
854
855
856
857
858
859
860
861
862
863
864
865
866
867
868
869
870
871
872
873
874
875
876
877
878
879
880
881
882
883
884
885
886
887
888
889
890
891
892
893
894
895
896
897
898
899
900
901
902
903
904
905
906
907
908
909
910
911
912
913
914
915
916
917
918
919
920
921
922
923
924
925
926
927
928
929
930
931
932
933
934
935
936
937
938
939
940
941
942
943
944
945
946
947
948
949
950
951
952
953
954
955
956
957
958
959
960
961
962
963
964
965
966
967
968
969
970
971
972
973
974
975
976
977
978
979
980
981
982
983
984
985
986
987
988
989
990
991
992
993
994
995
996
997
998
999
1000
watch:
- upstream: vllm#39598
status: closed # network sweep 2026-07-05: PR CLOSED unmerged (was 'open'); the residual stays unmitigated upstream — keep the probe gate
severity: P1-correctness
local_patches:
- P64
action: a-b-test # probe-on-our-stack (stream_toolcall_probe.py); enum-mapped
since: 2026-07-02
notes: |
club-3090 #145 close-out (2026-06-13, read from issue comments): v0.22.0
fixed tool_choice=auto + streaming natively, BUT a RESIDUAL remains on stock
vLLM: tool_choice=REQUIRED + thinking + streaming DROPS 13/20 tool calls
with MTP n=3 (clean 0/20 without MTP; parser-independent) - vllm#39598
resurfacing un-mitigated. Genesis P64 was the mitigation and is version-
capped off on >=dev491 pins (engine-native parser owns streaming). Our 35B
runs MTP K=5 + qwen3_xml -> DIRECT exposure for tool agents that use
tool_choice=required with streaming. Their 8-packs never caught it because
packs run non-streaming; they built scripts/stream-toolcall-probe.py for it.
PROBED 2026-07-02 on the live 35B (dev672, MTP K=5, qwen3_xml, 190W caps):
scripts/stream_toolcall_probe.py --n 20 -> 0/20 dropped, 0 errors — the
residual does NOT reproduce on our pin. Two turns showed a cosmetic
content-channel XML fragment ALONGSIDE a correctly delivered tool_call
delta (no loss; watch if agent frontends render stray XML). KEEP the probe
as a pin-bump gate: the drop reproduced on club's stack/pins, and the
fleet_boot_smoke tool-call check is auto/non-streaming so it cannot see it.
- upstream: vllm#40886
status: open # network sweep 2026-07-05 (audit 2026-07-04 #27): gh pr view says OPEN, NOT merged — the 'merged / absorbed-in-v0.24.0' claim was a club-3090 CHANGELOG assertion taken on faith (iron-rule-#11 class). Do NOT retire G4_04 until a verified merge (gh pr view) or a verified native equivalent in the target pin.
severity: P2-cleanup
local_patches:
- G4_04
action: watch # was retire; retire is gated on VERIFIED merge (audit #27) — restore action=retire only with gh-pr-view proof
since: 2026-07-02
notes: |
CORRECTION 2026-07-05: vllm#40886 is OPEN unmerged (gh pr view). The
v0.24.0-absorption claim below is club-3090's, unverified — if their
v0.24.0 ships the remap it came from another commit/PR; identify it
before any G4_04 retire-audit.
club-3090 shipped vllm-stable v0.24.0 "overlay-free" (their #533, 2026-06-30):
upstream absorbed the Gemma-4 AWQ MoE keys remap (#40886) that G4_04 vendors.
We are NOT on v0.24.0 yet (current pin dev672 = 0.23.1). ON A v0.24.0 BUMP,
deep-diff G4_04 remap vs native #40886 (iron-rule #11): RETIRE if byte-equivalent,
else KEEP the extra keys. Source: club-3090 CHANGELOG (v0.22.0 -> v0.24.0).
ALSO ON THE v0.24.0 BUMP — further overlay-free retire-audit candidates club
dropped (was a separate `v0.24.0-overlay-free` entry; folded here because none
of these track a single mergeable PR — G4_25 is genesis_original/upstream_pr=None
— so they cannot be schema-modeled as standalone PR entries). iron-rule #11: do
NOT retire on a title match.
(a) Gemma p-RoPE long-context: a v0.24.0 config fix may supersede G4_25 (Gemma 4
dual-RoPE base-freq divergence guard) — but G4_25 is a genesis_original
DIAGNOSTIC (warns on rope_theta == global_rope_theta), not a PR backport;
keep unless the upstream config validation makes the warning unreachable.
(b) truncate_prompt_tokens (#41800): native in v0.24.0; we vendor no equivalent.
(c) marlin-pad native: audit our marlin-path patches (P37/P81 family).
(d) club found MTP SLOWER on Gemma-4 MoE with a FIXED K -> they repointed their
dual to MTP-off (#537). We do NOT ship a fixed K: SNDR_MTP_DYNAMIC_K_001
(per-seq adaptive-K MTP proposer, port of vllm#26504 — rolling accept-rate
window len=10 + hysteresis; K++ when acceptance high, K-- down to 1 when low)
+ PN40-C (adaptive K) + per-task K presets (gemma4-*-mtp-chat-k3 /
-structured-k4). Our adaptive-K lowers K exactly on the low-acceptance tasks
where club's fixed-K lost -> their finding VALIDATES our design. Action:
confirm adaptive-K is enabled/tuned on the gemma-4 MTP presets (per-task,
acceptance-driven) — NOT disable MTP wholesale.
(e) W8A8 int8 (weights+activations) ~zero quality cost, +17-51% prefill,
TTFT 122 vs 158 ms (their experimental v0.24.0 row) -> a prefill-optimised
variant worth an A/B if we add one.
Source: club-3090 CHANGELOG + BENCHMARKS (2026-06-27..07-01).
- upstream: vllm#43409
status: merged
fix_upstream: vllm#45656
fix_status: merged-in-pin # network sweep 2026-07-05: #45656 (merged 06-18) is an ancestor of BOTH live pin bases (dev714/dev748); pristine dev748 auto_gptq.py:761-769 carries the is_sym use_zp guard natively
severity: P0-correctness
local_patches:
- PN400
action: drift-check # reconciled 2026-06-23: PN400 IS the backport of fix_upstream vllm#45656 (port complete); drift-check until #45656 lands in our pin, then retire PN400. DONE: PN400 retired 2026-06-24 (dev301 bump); row kept for provenance.
since: 2026-06-20
notes: |
REGRESSION IN dev148 (found by the /loop upstream sweep 2026-06-20).
#43409 (merged 06-12, IN dev148) removed the `if not is_sym` qzeros
guard in AutoGPTQMoEMethod.get_fused_moe_quant_config AND
CompressedTensorsWNA16MarlinMoEMethod.process_weights_after_loading.
Symmetric (is_sym=True) AutoRound/GPTQ Marlin MoE on NVIDIA then passes
meaningless qzeros to the Marlin kernel -> INCORRECT MoE expert outputs.
Our 27B = Qwen3.6-27B-int4-AutoRound is CONFIRMED on the broken path:
checkpoint config.json quantization_config = {quant_method: auto-round,
sym: True, bits: 4, group_size: 128} on NVIDIA A5000 (verified live
2026-06-20). Fix #45656 ("Restore is_sym guard for zp in GPTQ/CT MoE",
merged 06-18 16:20Z) landed ~12h AFTER the dev148 base commit
(b4c80ec0f @ 06-18 04:18Z) -> NOT in pin.
ACTION: backport #45656 as a Genesis TextPatch (2-file additive: restore
`use_zp = not is_sym` gating of w1_zp/w2_zp in auto_gptq.py + the
`if w13/w2_qzeros is not None` guards in
compressed_tensors_moe_wna16_marlin.py), version-capped to pins lacking
#45656; OR wait for the next pin bump including #45656 and treat dev148
as not-27B-correctness-safe until then. VALIDATE via 27B greedy-output
A/B (dev148 vs dev148+patch) on a deterministic prompt. The 27B is not
currently running (35B PROD is up) so there is no live mis-serving now.
- upstream: vllm#43559
status: merged
fix_upstream: vllm#43650
fix_status: open-NOT-in-pin
severity: P0-correctness
local_patches: []
affects_models:
- Qwen3.6-27B-int4-AutoRound
- Qwen3.6-35B-A3B-FP8
action: watch
since: 2026-06-21
notes: |
REGRESSION in dev148 (found by the /loop upstream sweep 2026-06-21).
MTP + --enable-prefix-caching on hybrid GDN silently drops accuracy
~20% (tool-eval 90%->50%), confirmed upstream on BOTH Qwen3.6-27B and
35B-A3B. Fix #43650 is OPEN, NOT in dev148. RIG STATUS: 35B PROD is
SAFE — start_qwen3.6-35b-balanced.sh runs MTP K=5 but has NO
--enable-prefix-caching (verified live 2026-06-21). 27B INT4 is exposed
only if launched with MTP + prefix caching. ACTION: never combine MTP +
--enable-prefix-caching on hybrid GDN models; if prefix caching is
required, force --kv-cache-dtype fp8 (issue-confirmed workaround). When
#43650 merges, take the final-mamba-block-drop fix. Gemma-4-26B-A4B is
NOT hybrid -> unaffected.
- upstream: vllm#46384
status: open
severity: P0-correctness
local_patches:
- PN346
- PN346B
affects_models:
- Qwen3.6-35B-A3B-FP8
- Qwen3.6-27B-int4-AutoRound
action: drift-check
since: 2026-06-26
notes: |
INCOMING ANCHOR-DRIFT (deep re-study 2026-06-26, pin dev424
0.23.1rc1.dev424+g3f5a1e173). #46384 ("[2/N] partial prefix-cache hits in
hybrid coordinator") is an UNCONDITIONAL structural rewrite of
find_longest_cache_hit in BOTH v1/core/kv_cache_coordinator.py AND
v1/core/single_type_kv_cache_manager.py — the exact files+regions PN346
(manager half, vendor #43650) and PN346B (coordinator curr_hit_length
clamp #45614 + Part-B FA/Mamba post-trim belt #46281) anchor into with
required=True sub-patches. NOT a runtime bug for us today: the partial-hash
DATA path is gated on enable_partial_hash_hits (needs
mamba_cache_mode="align"; PROD runs "none" -> inert). BUT the rewrite is
mode-independent: it deletes the _get_block_hashes closure, reshapes the
clamp region (_new_hit_length -> get_cache_hit_length, cdiv truncation,
KVCacheBlockListWithHitLength) and edits the MambaManager loop region. On
the pin that carries #46384 the PN346/PN346B required anchors VANISH -> the
transactional patcher SKIPs -> SILENT LOSS of the #43559 Mamba+EAGLE/MTP+APC
poison guard (the exact 35B-A3B MTP+APC path #43559 targets). The anchor-SOT
bump_preflight catches this mechanically (PN346/PN346B land in
genuine_anchor_drift on the carrying pin); this entry records the PLAN.
ACTION on merge: re-anchor PN346 onto the new MambaManager partial branch;
for PN346B verify whether the merged clamp (get_cache_hit_length + cdiv)
already encodes the monotonic-min — if so RETIRE PN346B Part-A, else
RE-ANCHOR onto the new clamp shape; re-derive the Part-B belt against the
cdiv truncation. Do NOT vendor #46384 itself (half-merged [2/N] partial-hash
machine, inert on mamba_cache_mode="none"). Cross-ref sweep: pr 46384.
- upstream: vllm#46446
status: open
severity: P3-perf
local_patches:
- PN129
affects_models:
- Qwen3.6-35B-A3B-FP8
- Qwen3.6-27B-int4-AutoRound
action: retire
since: 2026-06-26
notes: |
RETIRE-ON-MERGE (deep re-study 2026-06-26). #46446 ("Warm up slot-mapping
kernel through BlockTable") adds BlockTable.warmup_compute_slot_mapping()
(+ MultiGroupBlockTable fan-out), called from
gpu_worker.compile_or_warm_up_model before cudagraph capture — the CLEAN
upstream of PN129 (both vendor #42165). PN129 monkeypatches
Worker.compile_or_warm_up_model and pokes the private Triton
do_not_specialize attribute (self-documented hacky); #46446 does it the
supported way (real compute_slot_mapping over multiple shapes via
BlockTable, no private-API poke) and runs for MultiGroupBlockTable which our
hybrid (FA+GDN+Mamba) uses. PN129 is default-OFF and NOT enabled on prod
YAMLs -> NO correctness exposure (boot-JIT latency only). ACTION on merge:
retire PN129 entirely. OPTIONAL before merge: re-base PN129 onto #46446's
warmup_compute_slot_mapping (drop the private-Triton hack). Not urgent.
Cross-ref sweep: pr 46446.
- upstream: vllm#38999
status: open
severity: P1-crash
local_patches: []
affects_models:
- Gemma-4-26B-A4B
action: watch
since: 2026-06-21
notes: |
Gemma-4-26B-A4B CRASHES with --data-parallel-size>1 (AssertionError in
cuda_communicator all_gather on the FIRST request — loads fine, graphs
build, /health returns 200, then dies in serving). Found by the /loop
sweep 2026-06-21. TRAP: setting DP=2 for "parallelism" = a silent
in-flight incident after a healthy boot. ACTION: never set
--data-parallel-size>1 or --enable-eplb on Gemma4. The validated speedup
path is --enable-expert-parallel (EP-over-TP) — rig-validated -20% TPOT
(journal §78), now canonically emitted by compose() for gemma4_moe
multi-GPU. Fresh EP fixes #43219/#43110/#45002 do not regress that path.
- upstream: vllm#44740
status: open
fix_upstream: vllm#44209
fix_status: open
severity: P1-crash
local_patches: []
affects_models:
- Qwen3.6-27B-int4-AutoRound
action: watch
since: 2026-06-21
notes: |
MTP cudagraph negative memory estimate (-35 GiB) -> KV reallocation ->
OOM. On hybrid GDN the KV pool size is non-deterministic -> a silent
crash-loop under restart:unless-stopped AFTER /health already returned
200. Found by the /loop sweep 2026-06-21. Affects our 27B INT4 (MTP +
hybrid GDN + cudagraph). WORKAROUND: pass an explicit
--kv-cache-memory-bytes instead of pure --gpu-memory-utilization; do NOT
treat the healthcheck as a success indicator when MTP is on.
- upstream: vllm#38479
status: merged
local_patches:
- P3
- P4
- P5
- P6
- P22
- P26
- P28
- P36
- P38
- P44
- P46
action: a-b-test
since: 2026-04-15
notes: 'Upstream merged TurboQuant KV cache compression. Genesis stack
(P3..P46) overlaps significantly. Need A/B vs upstream''s
`--kv-cache-dtype turboquant_k8v4` to confirm Genesis advantage
on hybrid (Mamba/DeltaNet) models that upstream excludes per
§10.10 audit note.
'
- upstream: vllm#40914
status: open
local_patches:
- P67
- P67b
action: a-b-test
since: 2026-05-08
notes: 'Routes uniform K+1 spec-verify batches through the multi-query
kernel. P67/P67b are Genesis''s own implementation. After upstream
merge: A/B comparison required — keep Genesis path if it
outperforms (likely so on Ampere; uncertain on Hopper+).
'
- upstream: vllm#40069
status: open
local_patches:
- P4
- P5
- P67
- P67b
- P78
- P98
- P99
- P101
- PN57
action: drift-check
since: 2026-05-08
notes: 'TurboQuant/HIGGS attention follow-up tracking issue. Genesis stack
should auto-retire patches as their corresponding bullet points
land in upstream PRs.
'
- upstream: vllm#43432
status: closed # network sweep 2026-07-05: PR CLOSED unmerged (was 'open'); the value-MSE quality idea remains un-landed upstream — do-not-auto-adopt guidance below stands
local_patches:
- P44
- P67
- PN119
- P22
- P26
- P32
- PN261
action: a-b-test
since: 2026-06-19
notes: '[TurboQuant] Use MSE (Lloyd-Max) quantization for VALUES — a QUALITY
candidate; the only non-applicable-condemned delta of the delta-scan.
NOT condemned by the FIX2 dead-end: FIX2 was a decode-THROUGHPUT
MSE-KEY tensor-core experiment that died at Gemma GQA group=2 (needs
M>=16); THIS is scalar-Triton VALUE-quant accuracy, independent of the
GQA ratio. PR benches our exact 27B-int4-AutoRound 36/36 NIAH; Gemma
V-MSE mean NLL 0.297 vs 0.607 uniform + 2B/value-vector smaller slot.
DO NOT auto-adopt: OPEN/bot-review-only, 1124-line rewrite of value
store/decode/dequant on the live k8v4 hot path BOTH PROD models run,
changes value_packed_size 68->66 / slot_size 196->194 (must co-version
PN261/PN119/P22/P26/P32), extra per-decode inverse-Hadamard@Pi GEMM
unquantified for k8v4 TPOT. If pursued: backport ONLY the value-MSE
half (drop bundled mm_prefix_range — already vendored in
overlays/pr42637), default-OFF PNxxx extending overlays/pr42637
store+decode, with a mandatory dev148 rig A/B (TPOT + tool-call
clean-rate + NIAH on 27B & 35B) BEFORE enabling.
'
- upstream: vllm#45748
status: open
local_patches: []
action: watch
since: 2026-06-19
notes: 'Turboquant native fp8 v4 (CUDA) store. NOT APPLICABLE — hardware-gated
SM>=8.9 (Hopper/Ada E4M3). Our 2x RTX A5000 = SM (8,6) deterministically
take the fp8e4b15 branch (live store, Genesis P3 workaround), which is
byte-incompatible with the native kernel''s E4M3; the native op is not
even compiled into dev148 (torch.ops._C.turboquant_store_fp8_v4 = False).
Zero perf/correctness delta. Revisit ONLY on a Hopper/L40 migration; if
ever ported, preserve the fp8_e4b15==0 / device_capability>=89 guards
VERBATIM or it silently corrupts the A5000 KV cache. Do NOT re-flag as
a missed backport.
'
- upstream: vllm#41803
status: open
local_patches: []
action: watch
since: 2026-06-19
notes: '[Attention][MLA] Triton-fused TurboQuant decode backend. NOT APPLICABLE
— MLA-only. Our models are GQA/hybrid, not DeepSeek-MLA: 35B =
qwen3_5_moe (n_kv_heads=2, head_dim=256, no kv_lora_rank), 27B = DFlash
GDN+Mamba hybrid. The backend enters only via the MLA priority list +
the kv_lora_rank-gated spec hook, never touches our GQA decode kernel
(P67/P40/PN119). Do NOT re-flag as a missed backport despite the
"TurboQuant decode" title match. Revisit only if a DeepSeek-MLA model
joins the fleet (then with vllm#38479).
'
- upstream: vllm#43887
status: closed # network sweep 2026-07-05: PR CLOSED unmerged (was 'open'); the consciously-declined assessment below stands, drift risk gone with the closure
local_patches:
- P67b
- G4_67
- G4_81
action: drift-check
since: 2026-06-19
notes: '[TurboQuant] Add MTP spec-decode routing. ALREADY COVERED live by P67b
(+32% TPS) with G4_67 (our #40914 backport) and G4_81 as backups. The
PR flips supports_spec_as_decode=True — the OPPOSITE of our deliberate
design (live=False); G4_81 checklist item 5 documents that flip as a
silent-corruption + PN243-empty-max hazard. Its only NEW half
(value-MSE) comes from base #43432, tracked there. Assessed and
consciously declined. IF it merges: verify G4_81 wrap-outer
interception still matches the new upstream forward() drift markers
(no double-route).
'
- upstream: vllm#43878
status: closed # network sweep 2026-07-05: PR CLOSED unmerged (was 'open'); P101 mitigation remains our path
local_patches:
- P101
- PN116
action: drift-check
since: 2026-06-19
notes: '[TurboQuant] streaming fallback for long prefill. NOT a miss — the
residual 0.5GB FP16 continuation-alloc spike that could affect us is
already mitigated differently by P101 (cached_len>=32768 -> sliced
decode-replay). The PR streaming core is gated behind
`not _can_use_flash_prefill`, unreachable on our flash-enabled
head_dim=256 / sliding_window=None / text-only PROD path -> would be
600+ dormant LOC. PR self-flags "could not validate k8v4 serving" =
our exact cache layout.
'
- upstream: vllm#40269
status: merged # network sweep 2026-07-05 (audit 2026-07-04 #28): MERGED 2026-05-14 — sat hidden as 'open' for 7 weeks because the watchlist audit is offline-only
local_patches:
- P77
- PN40
action: a-b-test # was port; nothing left to port — the merge is an ancestor of the live pin bases, so the code is IN dev714/dev748. Remaining task is the enable/verify probe described in the notes
since: 2026-05-08
notes: 'Probabilistic draft rejection — passes draft_probs from drafter
to verifier. Genesis P77 (adaptive ngram K) is orthogonal but
complementary. Port as PN90 when upstream merges.
UPDATE 2026-07-05: MERGED 2026-05-14 and an ancestor of the live pin
bases — the draft_probs wiring is IN dev714/dev748 natively. The
"port" here is now an ENABLE/VERIFY task, not a backport: check
whether our MTP drafters populate draft_probs on the live pin and
whether the probabilistic rejection path engages (or needs a config
knob) before writing any patch. Do NOT vendor a copy of merged code.
'
- upstream: vllm#26504
status: closed # network sweep 2026-07-05: PR CLOSED unmerged (was 'open'); the MTP-adapt design sketch below remains the reference if we ever revive it (SNDR_MTP_DYNAMIC_K_001, our port, is itself retired)
local_patches: []
action: port
since: 2026-05-31
notes: "DynamicProposer — per-sequence adaptive K based on rolling\nacceptance-rate window. Claims +5-12%\
\ TPS on mixed workload\nvs static K. **Limitation for Genesis**: PR extends\n`EagleProposer` directly,\
\ NOT MTP-compatible as-written. Our\n4 PROD models use MTP K=3/K=4 (not Eagle). Backport requires\n\
MTP adaptation: either inherit from MTP-proposer instead of\nEagle, or extract a mixin/composition\
\ pattern. STUDY done\n2026-05-31 (see commit a074e900 follow-up notes). PR has\n`needs-rebase` label\
\ — wait for author to rebase before\nattempting the MTP-adapt backport. Acceptance threshold +\n\
hysteresis algorithm is generic enough to lift.\n\nMTP-adapt design sketch (2026-05-31 study, ready\
\ for vendor\nsession when upstream rebases):\n\nArchitecture target — extend DraftModelProposer instead\
\ of\nEagleProposer. The DraftModelProposer (at\nvllm/v1/spec_decode/draft_model.py in pin 0.21.1rc1.dev354+g626fa9bba)\n\
is the assistant-model-based MTP proposer all 4 of our PROD\ngemma4/qwen3.6 PROD models go through.\
\ It inherits from\nSpecDecodeBaseProposer which already owns\n`self.num_speculative_tokens` — the\
\ same attribute the\nDynamicProposer mutates per-batch.\n\nAlgorithm port (1:1 with #26504 file):\n\
\ 1. Per-seq SequenceState (acceptance_rate_history deque\n of len 10, num_spec_tokens int).\n\
\ 2. ACCEPTANCE_HISTORY_LEN=10, ACCEPTANCE_RATE_HYSTERESIS=0.05,\n MIN_HISTORY_FOR_ADJUSTMENT=3,\
\ MIN_SPEC_TOKENS=1 — copy\n the constants.\n 3. In propose() override:\n a. Update sequence\
\ states from prev step's\n `self.runner.input_batch.num_accepted_tokens_cpu`.\n b. Cleanup\
\ states for finished req_ids.\n c. Compute per_sequence_k via\n _adjust_and_get_spec_tokens_for_batch().\n\
\ d. Save self.num_speculative_tokens, set to\n max(per_seq_k), call super().propose(),\
\ restore.\n e. Pad the returned tensor to self.max_spec_tokens\n columns so the runner\
\ expects a fixed shape.\n\nConfig plumbing:\n - Add SpecConfig field `acceptance_rate_threshold:\
\ float =\n 0.7` (PR's default).\n - Add method-name route: `--speculative-config\n '{\"method\"\
: \"mtp_dynamic\", \"num_speculative_tokens\": 4,\n \"acceptance_rate_threshold\": 0.7, ...}'`\n\
\ - SpecConfig.proposer_class resolution must dispatch\n mtp_dynamic → Genesis MTPDynamicProposer.\n\
\nImplementation surface (estimated 1-2 days):\n - vllm/sndr_core/integrations/spec_decode/mtp_dynamic_proposer.py\n\
\ (NEW, ~250 LOC mirroring PR #26504's structure with\n DraftModelProposer base).\n - vllm/sndr_core/integrations/spec_decode/spec_decode_config_routing.py\n\
\ patch — add mtp_dynamic to method enum + dispatch.\n - registry.py entry — PN_NEW with method=mtp_dynamic\
\ gate.\n - Test on gemma4-31B (4-class workload) + qwen3.6-27B (chat\n + structured). Expected\
\ +5-12% per the PR's claim; the\n actual gain on our hybrid GDN+Mamba (27B) and MoE A3B\n (35B)\
\ stacks would be the first empirical check.\n\nOperator value: 31B chat-K=3 + structured-K=4 split\
\ currently\nrequires the spec_decode gateway to route per request. A\nmtp_dynamic proposer would\
\ converge a SINGLE launcher to the\nright K per-sequence automatically — no gateway needed for\n\
the K-axis. The chat/structured workload-class semantic split\nstays valid (different compression_plan\
\ / drafter behavior),\nbut the K choice within each profile becomes self-adapting.\n"
- upstream: vllm#42006
status: closed # network sweep 2026-07-05: PR CLOSED unmerged (was 'open'); the retire-on-merge trigger can never fire from THIS PR — keep the overlays, superseded locally by the #42237 v2 vendor anyway
local_patches:
- G4_T1
action: retire
since: 2026-05-30
notes: 'Gemma4 MTP streaming multi-tool-calls fix — segment-replay
refactor of `_extract_streaming`. Vendored as G4_T1 v1 overlay
`g4_t1_gemma4_tool_parser_pr42006_overlay.py` (matches
noonghunna/club-3090 stack 2026-05-08). On merge: delete both
v1 and v2 overlay files + remove `-v` mount in launcher
`start_gemma4-31b-tq-mtp-structured-k4.sh`. SUPERSEDED locally by
the v2 vendor of PR #42237 (see next entry) but kept here for
retire-on-merge symmetry.
'
- upstream: vllm#42237
status: closed # network sweep 2026-07-05: PR CLOSED unmerged (was 'open'); retire-on-merge cannot fire — the G4_T1 v2 overlay stays REQUIRED for gemma4 streaming tool calls until an equivalent lands upstream
local_patches:
- G4_T1
action: retire
since: 2026-05-31
notes: 'Gemma4 streaming tool parser REWRITE in Hermes/Kimi style
(accumulated text + rescan + diff). Net -103 LOC vs PR #42006
and drops the third-party `regex` import. Currently vendored
as G4_T1 v2 overlay
`g4_t1_v2_gemma4_tool_parser_pr42237_overlay.py` — this is
the file the launcher bind-mounts as of 2026-05-31. Empirical
result on gemma4-31B AWQ-4bit + TQ k8v4 + MTP K=4: 35/35 tool
calls (7 cases × 5 runs, both streaming and non-streaming,
with `Connection: close` header — see Bug Class 13 notes in
docs/TROUBLESHOOTING.md). On merge: delete both overlay files
+ remove launcher `-v` mount. The HTTP keep-alive parser
state-leak (case 5 nested-object missing closing `"}` two
chars on runs 2..N when client uses keep-alive) is a residual
vLLM-side bug worth flagging to the PR author when the merge
is being prepared.
'
- upstream: vllm#43074
status: closed # network sweep 2026-07-05: PR CLOSED unmerged (was 'open'); retire-on-merge cannot fire — Q3_T1 v1 stays live; watch the related PR list below for a successor fix
local_patches:
- Q3_T1
action: retire
since: 2026-05-31
notes: "[Bugfix] Qwen3Coder streaming: emit args when whole tool body\nlands in one delta. PR #43074\
\ fixes the EXACT bug class our\nQ3_T1 v1 overlay addresses — the upstream state machine drops\ntool_calls\
\ when MTP K=3 packs the whole `<function=NAME>...\n</function></tool_call>` body into a single delta\
\ after\nis_tool_call_started flipped on the prior delta. Reproduces\nwith `--tool-call-parser qwen3_coder\
\ --stream-interval 20\n--speculative-config '{\"method\":\"mtp\",\"num_speculative_tokens\":3}'`\n\
on Qwen3.6-27B — exactly our 27B PROD launcher config.\n\nAuthor's fix is a surgical +81/-41 LOC patch\
\ to the state\nmachine (\"collapsed body\" detection + emit args inline).\nOur Q3_T1 v1 takes a different\
\ approach — full Hermes/Kimi\nrescan + diff rewrite (~250 LOC of new method) — which is\nmore general\
\ (covers ALL packing patterns, not just \"whole\nbody in one delta\") but heavier. After upstream\
\ merges,\nverify the fix covers our 7-case bench at 100% on both 27B\nand 35B, then retire Q3_T1\
\ v1.\n\nRelated (track but not primary retire triggers):\n - vllm#40861 (Qwen3 XML and Coder streaming\
\ regression\n fix, needs-rebase)\n - vllm#39598 (streaming tool call empty fields with MTP\n\
\ + qwen3coder early return, needs-rebase)\n - vllm#41970 (Enable tool parser tests for Qwen3\
\ Coder)\n - vllm#37829 (stop string streaming at next <parameter=>\n boundary, needs-rebase)\n\
\ - vllm#43363 (refactor Use shared type coercion utilities\n in Qwen3 XML — touches type-coercion\
\ path our Q3_T1 v1\n also depends on via _convert_param_value)\n\nEmpirical state of Q3_T1 v1\
\ on pin 0.21.1rc1.dev354+g626fa9bba:\n qwen3.6-27b stream: 35/35 ✓ (was 1/7 = 14%)\n qwen3.6-35b\
\ stream: 35/35 ✓ (was 2/14 = 14%)\n non-streaming on both: unchanged 7/7\nSee Bug Class 15 in docs/TROUBLESHOOTING.md\
\ for full bench.\n"
- upstream: vllm#43909
status: merged
local_patches: []
action: a-b-test
since: 2026-05-31
notes: "[Bug] Fix gemma4 MTP IMA issue when TP>1 (CUDA error: illegal\nmemory access). Merged 2026-05-30\
\ (after our pin 626fa9bb of\n2026-05-28). Minimal +9/-1 LOC fix in\n`vllm/model_executor/models/gemma4_mtp.py`:\
\ caches\n`_stable_full_lm_head_weight` to prevent CUDA graph from\ncapturing a dangling pointer to\
\ a temp tensor (post all_gather\n+ slice). Author bench on Gemma-4-E2B-it shows 0.64s avg\nlatency\
\ for batch=16 input-len=512 output-len=128.\n\nVendored 2026-05-31 as Genesis G4_77 overlay (4-file\n\
bind-mount to `/usr/local/lib/python3.12/dist-packages/\nvllm/model_executor/models/gemma4_mtp.py`).\
\ A/B bench on\ngemma4-31b AWQ + TQ k8v4 + MTP K=4 + TP=2 + 2× A5000 +\ngpu-mem-util=0.80 (Bug Class\
\ 14 fix), 3 samples × 4\nworkloads, greedy decode, `Connection: close`:\n\n | Workload | Baseline\
\ | With G4_77 | Δ |\n |---|---:|---:|---:|\n | chat | 23.54 | 20.27 | -13.9% |\n | code | 41.73\
\ | 42.95 | +2.9% |\n | count | 52.37 | 50.05 | -4.4% |\n | json | 53.47 | 47.63 | -10.9% |\n\n\
NET NEGATIVE. Reverted. Likely cause: the cached tensor\n(post all_gather across TP=2) consumes additional\
\ VRAM at\na config that is already running gpu-mem-util=0.80 to give\nGenesis P38 TQ workspace its\
\ 1 GiB alloc. Reduced KV cache\n→ more KV churn on longer prompts → measurable regression\non chat\
\ (longest output) and JSON (structured but long).\nThe PR's author bench was on Gemma-4-E2B-it (much\
\ smaller\nmodel, presumably more memory headroom). The IMA workaround\nGenesis already ships (G4_71..G4_76\
\ stack in launcher) may\nalso be incompatible with the upstream cache in subtle ways.\nRe-evaluate\
\ after next pin bump if the upstream lands in\nproduction builds.\n"
- upstream: vllm#43803
status: merged
local_patches: []
action: a-b-test
since: 2026-05-31
notes: "[Perf] remove seqlen from Mamba SSD chunk kernels. Merged\n2026-05-28. +0/-10 LOC — removes\
\ a dead `seqlen` parameter\nfrom 5 Mamba prefill kernels (`_chunk_cumsum_fwd_kernel`,\n`_chunk_state_fwd_kernel`,\
\ `_chunk_scan_fwd_kernel`,\n`_bmm_chunk_fwd_kernel`, `_causal_conv1d_fwd_kernel`).\nTriton specializes\
\ plain `int` args by default, so every\nprefill with a new total-token-count was triggering a\nfresh\
\ JIT compile. Author claim on Nemotron-H-8B / A10G /\nShareGPT: Mean TTFT -17%, P99 TTFT -46%. Throughput\n\
unchanged.\n\nVendored 2026-05-31 to qwen3.6-27b-int4-autoround\nlauncher as a 4-file bind-mount (`/tmp/mamba_fixed/\n\
{causal_conv1d,ssd_bmm,ssd_chunk_scan,ssd_chunk_state}.py`).\nCleared ~/.triton/cache before measurement\
\ to force fresh\nJIT path. A/B vs baseline 27B, 8-sample TTFT × 3 prompt\nlengths, 3-sample TPS ×\
\ 4 workloads, greedy, `Connection:\nclose`:\n\n | Metric | Baseline | With #43803 | Δ |\n |---|---:|---:|---:|\n\
\ | TTFT short (5 tok prompt) | 151.3ms | 139.4ms | -7.9% |\n | TTFT medium (80 tok prompt) | 169.7ms\
\ | 176.0ms | +3.7% (noise) |\n | TTFT long (400 tok prompt) | 233.2ms | 232.5ms | -0.3% (noise)\
\ |\n | TPS chat | 91.68 | 86.95 | -5.2% |\n | TPS code | 113.53 | 113.05 | -0.4% (noise) |\n |\
\ TPS count | 117.79 | 116.43 | -1.2% (noise) |\n | TPS json | 105.61 | 107.77 | +2.0% |\n\nNET marginal-to-negative.\
\ Reverted. Does NOT reproduce the\nPR's claimed -17%/-46% on our config. Likely cause: our\nqwen3.6-27B\
\ is a hybrid GDN + Mamba architecture, NOT pure\nMamba like Nemotron-H. The GDN layers dominate the\
\ prefill\nhot path and the Mamba SSD kernels see fewer invocations\nwith varying seqlen — fewer JIT\
\ recompiles to save. The\nTPS chat regression (-5.2%) is just outside the run-to-run\nnoise band\
\ (~3%) and may reflect a different code path\ntaken without the dead `seqlen` arg in our specific\n\
configuration. Re-evaluate if Genesis later vendors a pure\nMamba model (e.g. nemotron-h-* family)\
\ where this PR's\narchitecture would actually apply.\n"
- upstream: vllm#36138
status: open # network sweep 2026-07-05: gh pr view says OPEN, NOT merged (was 'merged' since 2026-04-25 — same claimed-merged-on-faith class as #40886, audit #27/#28)
local_patches:
- P59
- P62
- P64
action: drift-check
since: 2026-04-25
notes: 'Grammar ignored when reasoning ended in speculated tokens.
Already integrated as P62 + structured-output fixes. Verify
anchor md5 not drifted after upstream pin bump.
CORRECTION 2026-07-05: the PR is OPEN unmerged (gh pr view) — the
original "merged" status was never verified. P62 remains OUR fix,
not a backport-of-merged; do not treat it as a retire candidate.
'
- upstream: vllm#40270
status: open
local_patches: []
action: port
since: 2026-05-08
notes: 'Pluggable GPU KV cache eviction (LRU/2Q/ARC). Critical for
agent/RAG workloads. Port as PN91 when merged.
'
- upstream: vllm#37160
status: merged # network sweep 2026-07-05 (audit 2026-07-04 #28): MERGED 2026-04-01 — sat hidden as 'open' for 3 months; the second merged action=port row the offline-only audit missed
local_patches: []
action: watch # was port; nothing left to port — the live pins ship the native kv-offload stack. Remaining task is an evaluation on non-hybrid lanes (see notes)
since: 2026-05-08
notes: 'CPU KV offload connector. Compatibility: NOT compatible with
hybrid Mamba (Qwen3.5/3.6 GDN). Port as PN92 with `applies_to.is_hybrid: [False]`
hard guard.
UPDATE 2026-07-05: MERGED 2026-04-01; the live pins ship a native
kv-offload stack (vllm/v1/kv_offload/ package + the offloading
connector PN383 was written against). Nothing to port — the task is
now an EVALUATION of native CPU KV offload on the non-hybrid lanes
(Gemma-4), still with the is_hybrid guard rationale. Note the PN92
name mentioned below was later used by an unrelated patch.
'
- upstream: vllm#37190
status: open
local_patches: []
action: watch
since: 2026-05-08
notes: 'MoE expert CPU offloading. Allow fitting 35B-A3B (256 experts)
on single 24GB. High throughput cost (30-60% TPS hit). Port as
experimental-only PN93.
'
- upstream: vllm#38330
status: open
local_patches: []
action: port
since: 2026-05-08
notes: 'Multimodal encoder cache. Cache vision embeddings across
requests. Port as PN94 with preprocessor-version key.
'
- upstream: vllm#37951
status: open
local_patches: []
action: watch
since: 2026-05-08
notes: 'KV cache memory error at 262K context. Padding waste in hybrid
layers. Genesis memory explain (DA-018) should surface this.
'
- upstream: vllm#39998
status: open
local_patches:
- P14
action: watch
since: 2026-05-08
notes: 'Qwen3 + TurboQuant 4-bit KV + high concurrency + 32K context
index out of bounds. P14 (block_table tail zero-fill) related;
verify it covers the index path upstream is fixing.
'
- upstream: vllm#40281
status: closed
local_patches: []
action: watch
since: 2026-05-08
notes: 'Gemma4 support — closed unmerged. Wait for replacement PR.
When Gemma4 lands upstream, create patch family + preset.
'
- upstream: vllm#39133
status: open
local_patches: []
action: watch
since: 2026-05-08
notes: 'Gemma 4 31B INT4 KV/cache limits on 2x24GB. SWA underutilized.
Will need Gemma-specific config profile (memory estimator).
'
- upstream: vllm#40265
status: closed
local_patches:
- P7
- P7b
action: drift-check
since: 2026-05-08
notes: 'FLA TP device index overflow — closed unmerged. Pattern matters.
Add regression guard in P7/P7b ensuring int64 device-index dtype.
'
- upstream: Sandermage/sndr_core_engine#1
status: open
local_patches: []
action: cookbook
since: 2026-05-08
notes: 'OOM with long context. See docs/COOKBOOK.md §1. Memory explain
(DA-018) should preflight detect and warn.
'
- upstream: vllm#40124
status: open
local_patches:
- P3
- P4
- P5
- P6
- P14
- P15
- P22
- P26
- P28
- P31
- P36
- P38
- P44
- P46
- P67
- P67b
- P95
action: drift-check
since: 2026-05-08
notes: 'Sander''s own master tracking issue: Ampere FP8 + TurboQuant +
Qwen3.6-35B-A3B + hybrid MoE. Every Genesis patch in this list
should map to a sub-bullet of the issue.
'
- upstream: vllm#42102
status: closed # network sweep 2026-07-05: PR CLOSED unmerged (was 'open'); the independent-KV-groups backport plan below is not obsoleted by any merge — revisit only if DFlash lanes leave the archive
local_patches:
- PN21
- PN23
- PN24
- PN38
- PN40
action: port
since: 2026-05-12
notes: "DFlash drafter coexistence with quantized target KV. Upstream\nproposes three mechanisms:\n\
\ 1. partition DFlash draft KV specs before page-size unify\n (draft and target live in separate\
\ KV groups).\n 2. drafter cache_dtype=\"auto\" when target global KV dtype is\n quantized (drafter\
\ inherits fp16/bf16 instead).\n 3. per-spec dtype in FlashAttention metadata scheduler.\n\nGenesis\
\ already has PN21..PN40 for DFlash on Ampere. The relevant\nbackport target is the hybrid path where\
\ target KV =\nturboquant_k8v4 while the drafter keeps fp16 (head_size=256\nconstraint).\n\nBackport\
\ plan: see docs/_internal/research/upstream_42102_dflash_\nindependent_kv_groups_plan_2026-05-12.md\
\ (S4.1 closure).\n\nCritical compatibility check: refined COMPAT-001 (qwen-next only)\nfrees Qwen3.6-27B\
\ Lorbus + DFlash for further development.\n"
- upstream: vllm#38864
status: open
local_patches:
- PN66
- P12
- P27
- P59
- PN71
- PN58
action: a-b-test
since: '2026-06-19'
notes: 'Cluster of OPEN qwen3 reasoning/tool-parser edge-case fixes found 2026-06-19 (loop scan), NOT
vendored, candidates for the next pin bump: #38864 (</think> leak in streaming WITH stop sequences
— token/text desync; distinct from our PN66 multiturn-leak), #45164 (Qwen3 parsing when reasoning
end-tag present), #45787 (count reasoning tokens when <think> present — our P89 area), #44812 (stop
thinking budget at implicit tool calls — our PN16 area), #45299 (reasoning->content short resp), #40861
(Qwen3 XML/Coder streaming tool call). #39055 (tool calls embedded in think) + #45389 (braces in required-tool
streaming) ALREADY referenced in registry. All OPEN, all edge cases. Live probe of #38864 on the 35B
was INCONCLUSIVE (default traffic did not trigger reasoning_content + an SSE-capture issue) — no confirmed
current exposure on the validated config (tool-call 7/7). Evaluate + vendor the applicable ones when
a pin carrying them is pulled.'
- upstream: vllm#43730
status: open
local_patches:
- P23
- P87
- G4_08
action: watch
since: '2026-06-19'
notes: '[Bugfix] Marlin MoE c_tmp FP32-reduce IMA (drops a min() clamp in csrc/moe/marlin_moe_wna16/ops.cu;
re-file of closed #36889). NOT overlay-patchable — the fix is in a COMPILED .cu kernel, not Python,
so it can only arrive via a pin bump that carries it. Reproducer is RTX PRO 6000 Blackwell SM 12.0;
our fleet is SM 8.6 A5000 and the live 35B runs c_tmp/marlin-IMA-clean. It is the WNA16 (INT4) MoE
path (more the 27B than the 35B FP8). No confirmed current exposure; pull it when a pin includes it.
Our P23 (marlin_fp32_reduce_wire) + P87 (sub-tile pad) + G4_08 (kdim pad) touch adjacent Marlin config
but not this .cu clamp. 2026-06-19 loop MoE/quant scan.'
- upstream: vllm#45656
status: merged
local_patches:
- P87
- P91
action: drift-check
since: '2026-06-19'
notes: '[Bugfix] Restore is_sym guard for zp in GPTQ/CT MoE — fixes a symmetric-quant regression
introduced by #43409 (CPU W4A16 INT4 MoE, merged 2026-06-12, IN dev148). #45656 merged
2026-06-18T16:20Z, AFTER our dev148 cutoff (b4c80ec0f, 04:18Z) -> next-pin candidate. The
regression only mis-feeds qzeros to Marlin when is_sym=True (symmetric GPTQ MoE). Our 27B/35B
are AutoRound INT4 ASYMMETRIC (real per-group zero-points present; P87 pads qzeros, P91 tags
scales/qzeros — AutoRound emits these only for sym=False), so is_sym=False and the #45656 use_zp
path is byte-identical pre/post fix -> NOT exposed. Drift-check only: when a pin carries #45656,
re-confirm our checkpoints still load is_sym=False (no behavior change expected). 2026-06-19 loop
MoE/quant scan.'
- upstream: vllm#45466
status: merged
local_patches:
- P67
action: watch
since: '2026-06-19'
notes: '[Bugfix][Kernel] Check output alignment in vectorize_with_alignment — fixes a misaligned
store for non-16B-multiple head rows in reshape_and_cache_flash (fuzz model head_size=46). Merged
2026-06-18T16:58Z, AFTER our dev148 cutoff -> next-pin. PR states "no behavior change for any
currently-working caller". Our head dims (128/256/512) are all 16B multiples -> NOT exposed; P67
next_power_of_2(hpk) handles the 27B GQA=6 grouping at the block level, separate from this per-store
alignment. Track only so a future odd/non-pow2 head-dim model would not silently hit the misaligned
store landmine. Free on bump. 2026-06-19 loop kernel scan.'
sweep:
- pr: 46384
genesis_patch: PN346, PN346B
trigger: reanchor-on-merge
note: 'INCOMING ANCHOR-DRIFT (deep re-study 2026-06-26, dev424 3f5a1e173). #46384 ("[2/N]
partial prefix-cache hits in hybrid coordinator") is an UNCONDITIONAL structural rewrite of
find_longest_cache_hit in BOTH v1/core/kv_cache_coordinator.py AND
v1/core/single_type_kv_cache_manager.py — the exact files+regions PN346 (manager half, vendor
#43650) and PN346B (coordinator curr_hit_length clamp #45614 + Part-B FA/Mamba post-trim belt
#46281) anchor into with required=True sub-patches. NOT a runtime bug for us today: its partial-hash
DATA path is gated on enable_partial_hash_hits (needs mamba_cache_mode="align"; PROD runs "none" so
the feature is inert). BUT the rewrite is mode-independent — it DELETES the _get_block_hashes closure,
reshapes the clamp region (_new_hit_length -> get_cache_hit_length, cdiv truncation,
KVCacheBlockListWithHitLength), and edits the MambaManager loop region. On the pin that carries #46384
PN346/PN346B required anchors will be ABSENT -> the transactional patcher SKIPs -> silent LOSS of the
#43559 Mamba+EAGLE/MTP+APC poison guard on the exact 35B-A3B MTP+APC path #43559 targets. On merge:
re-anchor PN346 onto the new MambaManager partial branch; for PN346B check whether the merged clamp
(get_cache_hit_length + cdiv) already encodes the monotonic-min — if so PN346B Part-A is a RETIRE
candidate, else RE-ANCHOR onto the new clamp shape; re-derive the Part-B belt against the cdiv
truncation. The anchor-SOT build_manifest already surfaces this mechanically (PN346/PN346B land in
genuine_anchor_drift on the carrying pin); this row records the re-anchor-vs-retire DECISION + the
#43559 guard rationale the mechanical detector cannot infer. Also tracked in the watch: section above.'
- pr: 46446
genesis_patch: PN129
trigger: retire-on-merge
note: 'RETIRE-ON-MERGE (deep re-study 2026-06-26). #46446 ("Warm up slot-mapping kernel through
BlockTable") adds BlockTable.warmup_compute_slot_mapping() (+ MultiGroupBlockTable fan-out) called
from gpu_worker.compile_or_warm_up_model before cudagraph capture — the CLEAN upstream of PN129 (both
vendor #42165). PN129 monkeypatches Worker.compile_or_warm_up_model and pokes the private Triton
do_not_specialize attribute (self-documented hacky); #46446 does it the supported way (real
compute_slot_mapping over multiple shapes via BlockTable, covering more specializations, no private-API
poke) and runs for MultiGroupBlockTable which our hybrid (FA+GDN+Mamba) uses. PN129 is default-OFF and
NOT enabled on prod YAMLs (no correctness exposure; boot-JIT latency only). On merge: retire PN129
entirely. OPTIONAL before merge: re-base PN129 onto #46446 warmup_compute_slot_mapping (drop the
private-Triton hack). Not urgent.'
- pr: 42006
genesis_patch: G4_T1
trigger: retire-on-merge
note: 'G4_T1 RACING CLUSTER (#42006 / #42237 / #42300 / #44741 / #45068 / #44844) — whichever merges
first decides retirement of the overlay stack (now THREE on-disk overlay generations: v1 rollback
+ v2 live mount + v3 prep). This PR = v1 overlay g4_t1_gemma4_tool_parser_pr42006_overlay.py (rollback
copy). Also tracked in the legacy watch: section above.'
- pr: 42237
genesis_patch: G4_T1
trigger: retire-on-merge
note: 'G4_T1 RACING CLUSTER member — v2 overlay (live bind-mount; 35/35 tool calls with Connection:close).
Also tracked in the legacy watch: section above. On merge delete overlay files + launcher -v mount.'
- pr: 42300
genesis_patch: G4_T1
trigger: retire-on-merge
note: G4_T1 RACING CLUSTER member — third competing upstream fix for the Gemma-4 streaming tool parser;
not vendored; its merge also triggers overlay retirement review.
- pr: 44741
genesis_patch: PN375
trigger: retire-on-merge
note: 'G4_T1 RACING CLUSTER member. DONE in the 2026-06-11 sweep: vendored as PN375 (pn375_gemma4_multiboundary_streaming.py)
— _extract_streaming_delta_segments runtime hook rebinding _extract_streaming on the PRISTINE pin
parser; the G4_14 pad-token set IS stripped from current_text AND delta_text before the consistency
check (mutation-verified: without the strip the fix silently degrades to the single-pass path). Self-skips
on the G4_T1 v2 overlay variant (accumulated-rescan, structurally immune). Combined regression test
(multi-boundary + <pad> + G4_14 active) in tests/unit/integrations/tool_parsing/ test_pn375_gemma4_multiboundary_streaming.py;
the pristine-bug reproduction test flips to FAILED when an upstream fix lands — then deep-diff and
retire PN375 (iron-rule-#11).'
- pr: 45068
genesis_patch: G4_T1
trigger: retire-on-merge
note: 'G4_T1 RACING CLUSTER member — Gemma4 parallel tool calls + MTP. DONE in the 2026-06-11 W2 sweep:
(1) MTP-split regression corpus ported into test_g4_t1_streaming_mtp_corpus.py (first streaming parser
tests for the family; runs against v2 AND the v3 prep), (2) token-id start-gate vendored into the
v2 overlay (gate on tool_call_start_token_id in current_token_ids, Genesis adaptations documented
inline), (3) keep-alive state leak (31/35 = remaining 4/35 failures on Gemma-4 + MTP) reproduced as
a strict-xfail on v2 with the `if not previous_text` reset-guard analysis; the FIX ships only in the
v3 prep overlay (Genesis reset-guard hardening) pending the server-stage A/B.'
- pr: 45207
genesis_patch: G4_60E
trigger: reanchor-on-merge
detection: manual
note: 'Mamba page padding — patches the SAME function (unify_kv_cache_spec_page_size) that G4_60E owns,
with the same page_size_padded technique. W2 plan: fold the MambaSpec elif into G4_60E (standalone
vendor would double-monkey-patch) + add Genesis-unique 3-way test (TQ + Mamba + bf16-drafter). On
merge re-derive G4_60E anchors and drop the duplicated branch. detection=manual: G4_60E sub-patches
are required=False (independent branches), so an incoming anchor drift soft-skips rather than failing
genuine_anchor_drift — caught by the planned operator reconciliation pass over the shared function,
not the anchor-SOT manifest.'
- pr: 45181
genesis_patch: G4_60E
trigger: reanchor-on-merge
detection: manual
note: 'Mixed KV page sizes (DFlash) — generic AttentionSpec padding fallback supersedes G4_60E Patch
2 (keep TQ-native Patches 1/3/4); same function as #45207, do ONE reconciliation pass over G4_60E
+ the pr42637 overlay. Unblocks the Gemma-4-31B TQ k8v4 profile on this pin (~2.7x KV memory vs fp16).
detection=manual: same shared-function reconciliation as #45207 (G4_60E sub-patches required=False).'
- pr: 45199
genesis_patch: 'planned: PN371'
trigger: retire-on-merge
note: 'Deferred ref-pinned encoder-cache eviction — W1 vendor, flag-gated ON for gemma4 (engine-fatal
"Encoder cache miss" on Gemma-4 vision + MTP K=3 + async-scheduling, our exact #38551 triple). Self-skip
probe on eager_eviction signature; EXTEND: demote fatal assert to warn+skip in drafter path only.'
- pr: 45202
genesis_patch: 'planned: P86'
trigger: retire-on-merge
note: Prefix-cache stats double-count on retries — W2 vendor as the safer variant (suppress recording
in get_computed_blocks, record on allocate_slots success); fallback-disable if a connector is configured.
Requires re-baselining p85/TQ hit-rate numbers collected under max_num_seqs=8.
- pr: 45182
genesis_patch: watch-only
trigger: review-on-merge
note: TRTLLM BF16 MoE modular kernel is SM100-only (dead on Ampere) — borrow ONLY the logger.info_once("Using
%s experts") one-liner into fp8/int_wna16 oracles + selection test asserting MarlinExperts (guards
PN96b/PN368 against silent selection drift across pin bumps).
- pr: 45197
genesis_patch: watch-only
trigger: review-on-merge
note: Disable cudagraph memory estimate default — upstream CHANGES_REQUESTED; do NOT flip configs blindly.
MEASURE =1 vs =0 on 35B/27B first; if >~200 MiB overestimate confirmed, write the root-cause capture-all-and-measure
patch (descs <= 16) and reconcile presets.py(=0) vs a5000 YAML(=1) inconsistency. Interacts with P66
capture-size filter.
- pr: 45196
genesis_patch: 'planned: PN-tbd (loader fail-fast validation)'
trigger: retire-on-merge
note: Loader/LoadConfig fail-fast validation — W2 near-clean 4-hunk vendor + tests; safety prerequisite
for the enable_multithread_load + num_threads=8 experiment on 35B/27B (~30-60s saved per restart);
mirror checks into audit_config_keys.py.
- pr: 45184
genesis_patch: watch-only
trigger: review-on-merge