fix(ui): size model fit against the cluster and move node labels into the selector - #11765
Merged
Conversation
The scheduling page kept a node-label browser open above the rules whether or not anyone was writing one, while the field that actually needs labels, the rule's node selector, was two bare text inputs with no hint of what the cluster reports. The browser is gone. The selector's key input now completes against the label keys the cluster uses, and the value input offers only the values that key takes. The roster already loads for the page, so the suggestions cost no request, and a roster that fails to load costs the admin the hints and nothing else. Suggestions stay suggestions: a key no node reports yet still commits as typed, which is how an admin writes a rule before labelling the nodes for it. Assisted-by: Claude:claude-opus-5 golangci-lint eslint playwright Signed-off-by: Ettore Di Giacinto <mudler@localai.io>
The models page asked the frontend how much memory a model may occupy. In distributed mode the frontend is usually a GPU-less pod while every model runs on a worker, so a fleet of GPU nodes was told it could only run the smallest CPU build. The variant picker's fits flag and its auto-selection came from the same place, as did the hardware recommendations. The registry now reports the largest single healthy backend node. The largest node, not the fleet total: a model loads into one node, so four 16GB workers are not a home for a 40GB model. An operator-set VRAM budget caps a node's contribution, because the scheduler refuses a load above that ceiling anyway, and a GPU node beats a CPU node holding more system RAM. GET /api/resources and GET /api/models carry this as an additional cluster object. Their aggregate and ram fields keep reporting the frontend's own hardware, which is what the resource monitor shows. Variant selection judges backends against the union of the capabilities present in the cluster, the way backend discovery already did. Every path degrades to the local host: no cluster object in single-node mode, and none when the registry cannot be read, so a hiccup narrows the answer back to single-node behaviour rather than marking the whole catalog too large. The verdicts now name the node they belong to, since a model fits somewhere or nowhere. Assisted-by: Claude:claude-opus-5 golangci-lint eslint playwright Signed-off-by: Ettore Di Giacinto <mudler@localai.io>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Description
Two UX defects reported from a live distributed deployment.
1. The models page sized every model against the frontend, not the cluster
In distributed mode the frontend is usually a GPU-less pod while models run on the workers, so a fleet of GPU nodes was told it could only run the smallest CPU build. The same wrong number drove three surfaces, two of them server-side:
Models.jsxreadresources.aggregate.total_memory, the frontend's own host.useRecommendedModels.jstiered its "recommended for your hardware" picks off it.GET /api/models/variants/:iddecidedfitsandauto_selectedin Go against the controller'sSystemState.VRAM, and judged backend compatibility against a host with no GPU.NodeRegistry.HealthyNodeMemorynow reports the largest single healthy backend node. The largest node, not the fleet total: a model loads into one node, so four 16GB workers are not a home for a 40GB model. An operator-set VRAM budget caps a node's contribution, since the scheduler refuses a load above that ceiling anyway, and a GPU node beats a CPU node holding more system RAM.ClusterMemoryProviderFormirrors the existingClusterCapabilityProviderFor, andgallery.ClusterResolveEnvhands variant selection the cluster's memory plus the union of capabilities present in the cluster.GET /api/resourcesandGET /api/modelscarry this as an additionalclusterobject. Theiraggregateandram*fields keep reporting the frontend's own hardware, which is what the resource monitor legitimately shows. Every degradation path lands back on single-node behaviour: no cluster object in single-node mode, and none when the registry cannot be read, so a hiccup narrows the answer rather than marking the whole catalog too large.Verdicts now name the node they belong to ("Headroom on dgx-01", "80 GB on dgx-01, the largest of 4 nodes"), because a model fits somewhere or nowhere.
2. The scheduling page showed node labels whether or not you needed them
The node-label browser stood open above the rules at all times, while the field that actually needs labels, the rule's node selector, was two bare text inputs with no hint of what the cluster reports.
The browser is gone. The selector's key input completes against the label keys the cluster uses, and the value input offers only the values that key takes. The roster already loads for the page, so suggestions cost no request, and a roster that fails to load costs the admin the hints and nothing else. Suggestions stay suggestions: a key no node reports yet still commits as typed, which is how you write a rule before labelling the nodes for it.
Notes for Reviewers
Both commits are TDD: each behaviour has a test that failed first. One caveat worth naming, since it shaped the final diff. An early e2e assertion ("the pane names the node") was a false green —
toContainTextpollstextContentand matched the zero-state hero still mounted right after the click. Chasing it down revealed that the fit badge inModelDetailis unreachable dead code (the stat grid deliberately withholdsvramDisplay). That edit was reverted, the node name moved to the headroom stat that actually renders, and the assertion re-verified red then green.Verification:
go test ./core/services/nodes/ --ginkgo.focus=HealthyNodeMemorygo test ./core/gallery/ --ginkgo.focus=ClusterResolveEnvgo test ./core/http/routes/node --test src/utils/modelBudget.test.js src/utils/nodeLabelSuggestions.test.jsnpx playwright testmake lintnpm run lint:inline-stylesThe single e2e failure (
console-narrow.spec.js) passes 6/6 in isolation and touches nothing in this change; it is the parallelism flakinessplaywright.config.jsalready documents. An earlier full run flaked on a different set of four specs, all likewise green in isolation.Signed commits