Fix DeepScaleR dynamic 4bit mapping to an unrelated 16bit model#7069
Fix DeepScaleR dynamic 4bit mapping to an unrelated 16bit model#7069vineethsaivs wants to merge 2 commits into
Conversation
In __INT_TO_FLOAT_MAPPER the entry for "unsloth/DeepScaleR-1.5B-Preview-unsloth-bnb-4bit" listed its 16bit target (values[0]) as "unsloth/DeepHermes-3-Llama-3-8B-Preview", an unrelated 8B Llama-3 model, instead of the DeepScaleR mirror. Every other "-unsloth-bnb-4bit" entry sets values[0] to the unsloth 16bit mirror of the same base model. values[0] is fanned out (mapper.py:1409-1463) into INT_TO_FLOAT_MAPPER, MAP_TO_UNSLOTH_16bit, and FLOAT_TO_INT_MAPPER, so this single wrong string mis-routes DeepScaleR loads in both directions: loading the dynamic 4bit repo with load_in_4bit=False, or loading agentica-org/DeepScaleR-1.5B-Preview or the bnb-4bit repo in 16bit, all resolve to DeepHermes-8B. Same class of bug as the gemma duplicate-key fix in unslothai#6891. Point values[0] at the correct mirror, "unsloth/DeepScaleR-1.5B-Preview". Add a GPU-free regression test that ast-extracts the mapper and asserts the DeepScaleR target, plus the general invariant that every "-unsloth-bnb-4bit" entry whose values[0] is an unsloth mirror shares the key's base model name.
There was a problem hiding this comment.
Code Review
This pull request corrects a mapping error in unsloth/models/mapper.py where the 4-bit key for DeepScaleR-1.5B-Preview was incorrectly mapped to DeepHermes-3-Llama-3-8B-Preview instead of its own 16-bit model. It also introduces a test suite to verify this mapping and enforce a general invariant that ensures all -unsloth-bnb-4bit entries point to their matching 16-bit models. The reviewer suggested improving the invariant test to also handle nested dictionary structures (used for FP8 and 16-bit targets) rather than only tuples, ensuring complete test coverage.
Important
The consumer version of Gemini Code Assist on GitHub is being sunset. Starting June 18, 2026, new organization installations will be blocked, and all code review activity will officially cease on July 17, 2026.
For more details on the timeline and next steps, please review the Help Documentation.
| mismatched = [ | ||
| (key, values[0]) | ||
| for key, values in mapper.items() | ||
| if key.endswith("-unsloth-bnb-4bit") | ||
| and isinstance(values, tuple) | ||
| and values | ||
| and values[0].startswith("unsloth/") | ||
| and _base_name(values[0]) != _base_name(key) | ||
| ] |
There was a problem hiding this comment.
The current test only checks entries where values is a tuple. However, several models in __INT_TO_FLOAT_MAPPER (such as Llama-3.1, Llama-3.2, Llama-3.3, and Qwen3) use a nested dict structure to support both FP8 and 16-bit targets. This means the test currently skips validating those models.
We should update the test to extract the "16" key from the dictionary when a nested structure is encountered, ensuring complete test coverage for all dynamic quantization entries.
| mismatched = [ | |
| (key, values[0]) | |
| for key, values in mapper.items() | |
| if key.endswith("-unsloth-bnb-4bit") | |
| and isinstance(values, tuple) | |
| and values | |
| and values[0].startswith("unsloth/") | |
| and _base_name(values[0]) != _base_name(key) | |
| ] | |
| mismatched = [] | |
| for key, values in mapper.items(): | |
| if not key.endswith("-unsloth-bnb-4bit"): | |
| continue | |
| actual_values = values["16"] if isinstance(values, dict) else values | |
| if isinstance(actual_values, tuple) and actual_values: | |
| target = actual_values[0] | |
| if target.startswith("unsloth/") and _base_name(target) != _base_name(key): | |
| mismatched.append((key, target)) |
for more information, see https://pre-commit.ci
Summary
In
unsloth/models/mapper.py, the__INT_TO_FLOAT_MAPPERentry for the DeepScaleR dynamic quant pointed its 16bit target at an unrelated model:Every other
-unsloth-bnb-4bitentry setsvalues[0]to the unsloth 16bit mirror of the same base model (e.g.unsloth/OpenThinker-7B-unsloth-bnb-4bit -> unsloth/OpenThinker-7B). This entry is the only one where the base names differ.values[0]is fanned out atmapper.py:1409-1463intoINT_TO_FLOAT_MAPPER,MAP_TO_UNSLOTH_16bit, andFLOAT_TO_INT_MAPPER, so the wrong string silently mis-routes DeepScaleR in both directions: loading the dynamic 4bit repo withload_in_4bit=False, or loadingagentica-org/DeepScaleR-1.5B-Preview/ the-bnb-4bitrepo in 16bit, all resolve to DeepHermes-8B. This is the same class of bug as the gemma duplicate-key fix in #6891.Fix
Point
values[0]at the correct mirror,unsloth/DeepScaleR-1.5B-Preview.Test
tests/python/test_mapper_deepscaler_16bit_target.pyast-extracts__INT_TO_FLOAT_MAPPER(no GPU / noimport unsloth) and asserts the DeepScaleR target, plus the general invariant that every-unsloth-bnb-4bitentry whosevalues[0]is an unsloth mirror shares the key's base model name. Fails before, passes after.