Studio: route models by CONFIG_MAPPING_NAMES instead of hardcoded tables#7043
Conversation
A model whose model_type is absent from an overlay's transformers cannot load there, so a new MoE arch not yet in the tier tables gets routed to default and fails (e.g. lfm2_moe, deepseek_v4). Add a static resolver that parses each overlay's CONFIG_MAPPING_NAMES straight from source (AST only, no import, no network, no trust_remote_code) and picks the lowest tier that ships the model_type. Runs after the existing checks and only ever upgrades default, so no existing routing changes and new archs no longer need a table edit.
for more information, see https://pre-commit.ci
![gemini-code-assist[bot]](https://avatars.githubusercontent.com/in/956858?s=60&v=4){kind=small}
There was a problem hiding this comment.
Code Review
This pull request introduces a static tier resolution mechanism for the transformers library by parsing configuration files using the AST module. This allows the system to determine the lowest tier that supports a given model type without importing the library or making network requests. The review feedback highlights a potential robustness issue where AST traversal and key extraction are performed outside of a try-except block, which could lead to unhandled exceptions and crash the model loading process if unexpected structures are encountered in third-party files. It is recommended to wrap the entire parsing and traversal logic in the try-except block to ensure graceful fallback.
Important
The consumer version of Gemini Code Assist on GitHub is being sunset. Starting June 18, 2026, new organization installations will be blocked, and all code review activity will officially cease on July 17, 2026.
For more details on the timeline and next steps, please review the Help Documentation.
| if tdir: | ||
| for rel in ("models/auto/configuration_auto.py", "models/auto/auto_mappings.py"): | ||
| path = Path(tdir) / rel | ||
| if not _safe_is_file(path): | ||
| continue | ||
| try: | ||
| tree = ast.parse(path.read_text(encoding = "utf-8")) | ||
| except Exception: | ||
| continue |
There was a problem hiding this comment.
The AST traversal (ast.walk(tree)) and key extraction (_mapping_first_keys) are currently executed outside of the try...except block. If any unexpected structure or malformed AST is encountered in the third-party transformers source files, an unhandled exception could propagate and crash the entire get_transformers_tier call, preventing any model from loading.
Wrapping the entire parsing and traversal logic inside the try...except block ensures that any unexpected failures are caught gracefully, allowing the system to fall back safely.
| if tdir: | |
| for rel in ("models/auto/configuration_auto.py", "models/auto/auto_mappings.py"): | |
| path = Path(tdir) / rel | |
| if not _safe_is_file(path): | |
| continue | |
| try: | |
| tree = ast.parse(path.read_text(encoding = "utf-8")) | |
| except Exception: | |
| continue | |
| try: | |
| tree = ast.parse(path.read_text(encoding="utf-8")) | |
| for node in ast.walk(tree): | |
| if isinstance(node, ast.Assign) and any( | |
| isinstance(t, ast.Name) and t.id == "CONFIG_MAPPING_NAMES" for t in node.targets | |
| ): | |
| keys |= _mapping_first_keys(node.value) | |
| except Exception: | |
| continue |
References
- For purely informational or advisory checks that are non-critical to the main execution flow, use a broad exception handler (e.g.,
except Exception:) with a silent return to ensure that any unexpected failures in the check do not propagate and disrupt the main execution path.
There was a problem hiding this comment.
Fixed in 287631e: the ast.walk and key extraction now run inside the try/except, so a malformed source file can never propagate out of tier resolution.
![chatgpt-codex-connector[bot]](https://avatars.githubusercontent.com/in/1144995?s=60&v=4){kind=small}
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 9dc34ac0b3
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
| spec = importlib.util.find_spec("transformers") | ||
| except Exception: | ||
| return None | ||
| return os.path.dirname(spec.origin) if spec and spec.origin else None |
There was a problem hiding this comment.
When in-process export has previously activated a 5.x sidecar, find_spec("transformers") resolves to that prepended sidecar, so the cached "default" CONFIG_MAPPING_NAMES can actually be 5.x. In that state, a mapping-only model whose model_type exists in the active sidecar is classified as default, and ensure_transformers_version may deactivate the sidecar before loading a config that only 5.x can parse. The default mapping needs to be read with the .venv_t5_* entries excluded or from a fixed 4.57.x path.
Useful? React with 👍 / 👎.
There was a problem hiding this comment.
Fixed in 287631e: the default tier map is now resolved from the base install, skipping any .venv_t5_* or llmcompressor overlay on sys.path. Verified that with a 5.10 sidecar prepended, the default map stays 4.57.x (421 model types, not 643).
| result = frozenset(keys) | ||
| _config_mapping_cache[tier] = result |
There was a problem hiding this comment.
_config_model_types runs before any _ensure_venv_t5_*_exists() call, so on a fresh or lazy-provisioned install where a .venv_t5_* directory is not present yet this stores frozenset() for that tier. A mapping-only architecture then falls through to the default tier instead of selecting and installing the sidecar, and the empty cache persists even if another model provisions the sidecar later in the same process. Please avoid caching absent/incomplete overlays, or ensure the sidecar before parsing it.
Useful? React with 👍 / 👎.
There was a problem hiding this comment.
Fixed in 287631e: _config_model_types no longer caches a result when the overlay dir is absent, so a later call re-reads it once the sidecar is provisioned instead of serving a stale empty map.
| remote_cfg = _cached_config_json(model_name, hf_token) | ||
| if remote_cfg is not None: | ||
| static = _tier_from_config_mapping(remote_cfg) |
There was a problem hiding this comment.
When _load_config_json serves a cached Hub config after a transient network failure, it intentionally does not populate _config_json_cache, so this added block sees remote_cfg is None even though the preceding slow config checks just read the model's config. In that temporary-outage/cached-model scenario, mapping-only architectures such as lfm2_moe or deepseek_v4 still fall through to the default tier and fail to load; the mapping resolver should use the config returned by the transient read or explicitly reload it here rather than relying only on the definitive cache.
Useful? React with 👍 / 👎.
There was a problem hiding this comment.
Fixed in 287631e: the mapping fallback now reads via _load_config_json instead of the cache-only reader, so a config served from the hub cache during a transient outage still routes lfm2_moe / deepseek_v4 correctly.
| if isinstance(node, ast.Assign) and any( | ||
| isinstance(t, ast.Name) and t.id == "CONFIG_MAPPING_NAMES" for t in node.targets | ||
| ): | ||
| keys |= _mapping_first_keys(node.value) |
There was a problem hiding this comment.
Some transformers releases build CONFIG_MAPPING_NAMES by importing the generated map and then applying CONFIG_MAPPING_NAMES.update(...) or OrderedDict(**{...}, **CONFIG_MAPPING_NAMES) additions in configuration_auto.py. Because this walk only collects direct assignments, those sidecar-only model types are invisible to the resolver and still fall through to the default tier even though the sidecar can load them; the parser needs to account for these mutations as well as the literal assignment.
Useful? React with 👍 / 👎.
There was a problem hiding this comment.
Fixed in 287631e: the parser now also collects model types from CONFIG_MAPPING_NAMES.update({...}) and OrderedDict(**{...}) unpacking, not just the literal assignment. Verified 5.10's mlcd / vibevoice / evolla / gpt-sw3 additions are now included.
- Resolve the default tier map from the base install, skipping any .venv_t5_*
sidecar on sys.path, so an in-process 5.x activation cannot make a 5.x-only
model look loadable by 4.x.
- Do not cache an overlay whose sidecar dir is absent, so a later call re-reads
it once provisioned instead of serving a stale empty map.
- Also collect model types added via CONFIG_MAPPING_NAMES.update({...}) and
**{...} unpacking, not just the literal assignment (5.10 uses both).
- Wrap the AST walk in the try/except so a malformed source can never crash tier
resolution.
- Feed the mapping fallback from _load_config_json so a config served from the
hub cache during a transient outage still routes new architectures.
for more information, see https://pre-commit.ci
|
@codex review |
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 9c13dd5503
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
| tdir = _overlay_transformers_dir(tier) | ||
| if tdir is None: | ||
| return frozenset() # overlay not provisioned yet; do not cache so a later call re-reads |
There was a problem hiding this comment.
When a .venv_t5_* sidecar is absent (or stale enough that its transformers source is not present), this returns an empty mapping and _tier_from_config_mapping falls through to default for models that now rely only on the new map resolver, such as lfm2_moe/deepseek_v4. Because no 5.x tier is selected, activation never calls the corresponding _ensure_venv_t5_*_exists() path that would install or refresh the sidecar, so these models still try to load under 4.57.x in fresh/runtime-repair environments. The hardcoded routes selected the tier before ensuring, so the resolver should validate/provision sidecars or treat missing maps as inconclusive rather than unsupported.
Useful? React with 👍 / 👎.
There was a problem hiding this comment.
The .venv_t5_* sidecars are pre-installed by setup.sh (see the module docstring), and the existing AutoConfig subprocess probe already requires them present, so this is not a regression introduced here. When a sidecar is genuinely absent the resolver returns None and falls through to the existing tokenizer / saved-by-5.x / probe checks rather than misrouting, and provisioning inside this static resolver would break its no-import / no-network / no-exec contract and could install sidecars a model does not need. Sidecar provisioning stays with setup.sh and the activation path.
|
@codex review |
|
Codex Review: Didn't find any major issues. Chef's kiss. Reviewed commit: ℹ️ About Codex in GitHubYour team has set up Codex to review pull requests in this repo. Reviews are triggered when you
If Codex has suggestions, it will comment; otherwise it will react with 👍. Codex can also answer questions or update the PR. Try commenting "@codex address that feedback". |
|
@codex review |
|
Codex Review: Didn't find any major issues. Hooray! Reviewed commit: ℹ️ About Codex in GitHubYour team has set up Codex to review pull requests in this repo. Reviews are triggered when you
If Codex has suggestions, it will comment; otherwise it will react with 👍. Codex can also answer questions or update the PR. Try commenting "@codex address that feedback". |
1 participant
Summary
Studio picks a transformers version (default 4.57.x, or a 5.3.0 / 5.5.0 / 5.10.x sidecar) per model. Today that choice leans on hardcoded architecture and model_type tables, so a new MoE architecture that is not yet listed falls through to the default tier and fails to load. This is what happened with
lfm2_moe(LFM2-8B-A1B), and the same gap applies todeepseek_v4.The underlying fact is simple: a model whose
model_typeis absent from an overlay'sCONFIG_MAPPING_NAMEScannot be loaded by that overlay. So instead of maintaining a table per model, resolve the tier directly from that mapping.Changes
studio/backend/utils/transformers_version.py:CONFIG_MAPPING_NAMESstraight from the installed source (models/auto/configuration_auto.py, orauto_mappings.pyin 5.10) usingastonly. No import, no network, notrust_remote_code, no subprocess.model_type.Why this is safe
model_typeis absent from the default transformers. If amodel_typeis absent from default, default cannot load it anyway, so upgrading is correct by construction.qwen3_moeandqwen3_next, whose default implementation needs 5.3.0), are still handled by the existing checks first, so their routing is unchanged.Testing
Resolved tier for a local
config.jsonpermodel_type, on a tree without any hardcodedlfm2_moeentry:Parsed model_type counts per overlay: default 421, 530 475, 550 499, 510 643.
This supersedes #7040, which added
lfm2_moeto the 5.3.0 tables as a stopgap. With this resolver that entry is no longer needed, so #7040 can be closed once this lands.