fix: raise a clear error on stale KV cache, avoid double allocation in chat (#2190) [1/3]

[OnePunchMonk]

Part of #2190.

Part 1 of 3, split out of #2300 so each piece is reviewable on its own. Addresses the KV cache half of #2190.

Two things here:

Stale KV cache now raises a clear error. If you call forward with input_pos after changing max_seq_length without re-running set_kv_cache, the old code either printed a warning and carried on with a stale cache, or died with a cryptic IndexError from the cache index. Either way you could silently get wrong attention results. It now raises a RuntimeError that says what happened and what to do about it.

chat no longer double-allocates the cache. When a chat turn needs a longer context, process_prompt grew the cache by calling set_kv_cache again. The old cache was still alive at that point, so both sat in memory at once and peak usage roughly doubled for the rest of the session. Added an explicit clear_kv_cache() before the reallocate.

Also documented the init/destroy lifecycle on set_kv_cache/clear_kv_cache, since the ordering rules were not written down anywhere.

Tests

• test_stale_kv_cache_raises_clear_error covers the new error.
• test_process_prompt_clears_kv_cache_before_growing covers the chat path.
• test_kv_cache_full_context_length exercises set_kv_cache/forward at a real model's full block_size with a realistic batch size, which is the "test at full context lengths" ask from Memory Management Issues #2190. The existing tests only used the tiny block_size=25 configs.

Related: #2311 (profiling) and #2312 (tunable chunking).

AI Usage Disclaimer

• AI assistance (Claude Code) was used for this change.