11# PLIP-RS Roadmap: Model Expansion and Backend Refactoring
22
3- ** Current version:** v1.0.3
4- ** Target version:** v1.1.0 (trait refactor + new transformer models), v1. 2.0 (RWKV-6)
5- ** Status:** Planning
6- ** Last updated:** 2026-02-07
3+ ** Current version:** v1.1.0
4+ ** Target version:** v1.2.0 (RWKV-6)
5+ ** Status:** v1.1.0 released (Phases 0-2 complete); Phases 3-6 not started
6+ ** Last updated:** 2026-02-09
77
88---
99
@@ -238,16 +238,16 @@ fn apply_chat_template(&self, prompt: &str, system_prompt: Option<&str>) -> Stri
238238
239239### 4.3 Implementation Checklist
240240
241- - [ ] Download and inspect ` config.json ` from ` codellama/CodeLlama-7b-hf `
242- - [ ] Create ` src/forward_llama.rs ` with ` LlamaConfig ` struct
243- - [ ] Implement ` PlipBackend ` for ` PlipLlama `
244- - [ ] Handle sharded weights (reuse Qwen2's shard-detection logic)
245- - [ ] Add ` Llama ` variant to ` ModelArchitecture ` + detection for ` "llama" ` , ` "codellama" ` in model ID
246- - [ ] Add match arm in ` from_pretrained_with_arch `
247- - [ ] Test basic inference (greedy generation on a short prompt)
248- - [ ] Test attention extraction (verify ` [batch, heads, seq, seq] ` output shape)
249- - [ ] Test knockout (reuse existing knockout test patterns)
250- - [ ] Run existing PLIP experiments on Code-LLaMA and compare with transformer baselines
241+ - [x ] Download and inspect ` config.json ` from ` codellama/CodeLlama-7b-hf `
242+ - [x ] Create ` src/forward_llama.rs ` with ` LlamaConfig ` struct
243+ - [x ] Implement ` PlipBackend ` for ` PlipLlama `
244+ - [x ] Handle sharded weights (reuse Qwen2's shard-detection logic)
245+ - [x ] Add ` Llama ` variant to ` ModelArchitecture ` + detection for ` "llama" ` , ` "codellama" ` in model ID
246+ - [x ] Add match arm in ` from_pretrained_with_arch `
247+ - [x ] Test basic inference (greedy generation on a short prompt)
248+ - [x ] Test attention extraction (verify ` [batch, heads, seq, seq] ` output shape)
249+ - [x ] Test knockout (reuse existing knockout test patterns)
250+ - [x ] Run existing PLIP experiments on Code-LLaMA and compare with transformer baselines
251251
252252### 4.4 Validation
253253
@@ -292,19 +292,19 @@ Similarly, the MLP uses a fused `gate_up_proj` instead of separate `gate_proj` a
292292
293293### 5.3 Implementation Checklist
294294
295- - [ ] Download and inspect ` config.json ` from ` microsoft/Phi-3-mini-4k-instruct `
296- - [ ] Verify RoPE variant (standard vs. SuRoPE/longrope) — check for ` rope_scaling ` in config
297- - [ ] Create ` src/forward_phi3.rs ` with ` Phi3Config ` struct
298- - [ ] Implement fused QKV split logic in attention
299- - [ ] Implement fused gate+up split logic in MLP
300- - [ ] Implement ` PlipBackend ` for ` PlipPhi3 `
301- - [ ] Add ` Phi3 ` variant to ` ModelArchitecture ` + detection for ` "phi-3" ` , ` "phi3" ` in model ID
302- - [ ] Add match arm in ` from_pretrained_with_arch `
303- - [ ] Implement Phi-3 chat template (` <|user|>\n{prompt}<|end|>\n<|assistant|>\n ` )
304- - [ ] Test basic inference
305- - [ ] Test attention extraction
306- - [ ] Test knockout
307- - [ ] Run PLIP experiments
295+ - [x ] Download and inspect ` config.json ` from ` microsoft/Phi-3-mini-4k-instruct `
296+ - [x ] Verify RoPE variant (standard vs. SuRoPE/longrope) — check for ` rope_scaling ` in config
297+ - [x ] Create ` src/forward_phi3.rs ` with ` Phi3Config ` struct
298+ - [x ] Implement fused QKV split logic in attention
299+ - [x ] Implement fused gate+up split logic in MLP
300+ - [x ] Implement ` PlipBackend ` for ` PlipPhi3 `
301+ - [x ] Add ` Phi3 ` variant to ` ModelArchitecture ` + detection for ` "phi-3" ` , ` "phi3" ` in model ID
302+ - [x ] Add match arm in ` from_pretrained_with_arch `
303+ - [x ] Implement Phi-3 chat template (` <|user|>\n{prompt}<|end|>\n<|assistant|>\n ` )
304+ - [x ] Test basic inference
305+ - [x ] Test attention extraction
306+ - [x ] Test knockout
307+ - [x ] Run PLIP experiments
308308
309309### 5.4 Validation
310310
@@ -317,13 +317,13 @@ Same as Code-LLaMA: compare greedy output against HuggingFace Python, verify att
317317** Content:** Trait refactor + Code-LLaMA 7B + Phi-3-mini
318318
319319** Checklist before release:**
320- - [ ] All existing tests pass
321- - [ ] All 6 models produce correct inference output
322- - [ ] Attention extraction works for all 6 models
323- - [ ] Knockout experiments produce non-trivial results for all 6 models
324- - [ ] ` cargo clippy ` clean
325- - [ ] Update ` Cargo.toml ` version to ` 1.1.0 `
326- - [ ] Update README model support table
320+ - [x ] All existing tests pass
321+ - [x ] All 6 models produce correct inference output
322+ - [x ] Attention extraction works for all 6 models
323+ - [x ] Knockout experiments produce non-trivial results for all 6 models
324+ - [x ] ` cargo clippy ` clean
325+ - [x ] Update ` Cargo.toml ` version to ` 1.1.0 `
326+ - [x ] Update README model support table
327327
328328** Architecture coverage after v1.1.0:**
329329
0 commit comments