Skip to content

Fix DeepScaleR dynamic 4bit mapping to an unrelated 16bit model#7069

Open
vineethsaivs wants to merge 2 commits into
unslothai:mainfrom
vineethsaivs:fix/deepscaler-16bit-mapper-target
Open

Fix DeepScaleR dynamic 4bit mapping to an unrelated 16bit model#7069
vineethsaivs wants to merge 2 commits into
unslothai:mainfrom
vineethsaivs:fix/deepscaler-16bit-mapper-target

Conversation

@vineethsaivs

Copy link
Copy Markdown
Contributor

Summary

In unsloth/models/mapper.py, the __INT_TO_FLOAT_MAPPER entry for the DeepScaleR dynamic quant pointed its 16bit target at an unrelated model:

"unsloth/DeepScaleR-1.5B-Preview-unsloth-bnb-4bit" : (
    "unsloth/DeepHermes-3-Llama-3-8B-Preview",   # unrelated 8B Llama-3
    "agentica-org/DeepScaleR-1.5B-Preview",
    "unsloth/DeepScaleR-1.5B-Preview-bnb-4bit",
),

Every other -unsloth-bnb-4bit entry sets values[0] to the unsloth 16bit mirror of the same base model (e.g. unsloth/OpenThinker-7B-unsloth-bnb-4bit -> unsloth/OpenThinker-7B). This entry is the only one where the base names differ.

values[0] is fanned out at mapper.py:1409-1463 into INT_TO_FLOAT_MAPPER, MAP_TO_UNSLOTH_16bit, and FLOAT_TO_INT_MAPPER, so the wrong string silently mis-routes DeepScaleR in both directions: loading the dynamic 4bit repo with load_in_4bit=False, or loading agentica-org/DeepScaleR-1.5B-Preview / the -bnb-4bit repo in 16bit, all resolve to DeepHermes-8B. This is the same class of bug as the gemma duplicate-key fix in #6891.

Fix

Point values[0] at the correct mirror, unsloth/DeepScaleR-1.5B-Preview.

Test

tests/python/test_mapper_deepscaler_16bit_target.py ast-extracts __INT_TO_FLOAT_MAPPER (no GPU / no import unsloth) and asserts the DeepScaleR target, plus the general invariant that every -unsloth-bnb-4bit entry whose values[0] is an unsloth mirror shares the key's base model name. Fails before, passes after.

In __INT_TO_FLOAT_MAPPER the entry for
"unsloth/DeepScaleR-1.5B-Preview-unsloth-bnb-4bit" listed its 16bit target
(values[0]) as "unsloth/DeepHermes-3-Llama-3-8B-Preview", an unrelated 8B
Llama-3 model, instead of the DeepScaleR mirror. Every other "-unsloth-bnb-4bit"
entry sets values[0] to the unsloth 16bit mirror of the same base model.

values[0] is fanned out (mapper.py:1409-1463) into INT_TO_FLOAT_MAPPER,
MAP_TO_UNSLOTH_16bit, and FLOAT_TO_INT_MAPPER, so this single wrong string
mis-routes DeepScaleR loads in both directions: loading the dynamic 4bit repo
with load_in_4bit=False, or loading agentica-org/DeepScaleR-1.5B-Preview or the
bnb-4bit repo in 16bit, all resolve to DeepHermes-8B. Same class of bug as the
gemma duplicate-key fix in unslothai#6891.

Point values[0] at the correct mirror, "unsloth/DeepScaleR-1.5B-Preview". Add a
GPU-free regression test that ast-extracts the mapper and asserts the DeepScaleR
target, plus the general invariant that every "-unsloth-bnb-4bit" entry whose
values[0] is an unsloth mirror shares the key's base model name.

@gemini-code-assist gemini-code-assist Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review

This pull request corrects a mapping error in unsloth/models/mapper.py where the 4-bit key for DeepScaleR-1.5B-Preview was incorrectly mapped to DeepHermes-3-Llama-3-8B-Preview instead of its own 16-bit model. It also introduces a test suite to verify this mapping and enforce a general invariant that ensures all -unsloth-bnb-4bit entries point to their matching 16-bit models. The reviewer suggested improving the invariant test to also handle nested dictionary structures (used for FP8 and 16-bit targets) rather than only tuples, ensuring complete test coverage.

Important

The consumer version of Gemini Code Assist on GitHub is being sunset. Starting June 18, 2026, new organization installations will be blocked, and all code review activity will officially cease on July 17, 2026.
For more details on the timeline and next steps, please review the Help Documentation.

Comment on lines +42 to +50
mismatched = [
(key, values[0])
for key, values in mapper.items()
if key.endswith("-unsloth-bnb-4bit")
and isinstance(values, tuple)
and values
and values[0].startswith("unsloth/")
and _base_name(values[0]) != _base_name(key)
]

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

medium

The current test only checks entries where values is a tuple. However, several models in __INT_TO_FLOAT_MAPPER (such as Llama-3.1, Llama-3.2, Llama-3.3, and Qwen3) use a nested dict structure to support both FP8 and 16-bit targets. This means the test currently skips validating those models.

We should update the test to extract the "16" key from the dictionary when a nested structure is encountered, ensuring complete test coverage for all dynamic quantization entries.

Suggested change
mismatched = [
(key, values[0])
for key, values in mapper.items()
if key.endswith("-unsloth-bnb-4bit")
and isinstance(values, tuple)
and values
and values[0].startswith("unsloth/")
and _base_name(values[0]) != _base_name(key)
]
mismatched = []
for key, values in mapper.items():
if not key.endswith("-unsloth-bnb-4bit"):
continue
actual_values = values["16"] if isinstance(values, dict) else values
if isinstance(actual_values, tuple) and actual_values:
target = actual_values[0]
if target.startswith("unsloth/") and _base_name(target) != _base_name(key):
mismatched.append((key, target))

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant