Skip to content

fix(ui): size model fit against the cluster and move node labels into the selector - #11765

Merged
mudler merged 2 commits into
masterfrom
fix/cluster-aware-model-fit
Aug 28, 2026
Merged

fix(ui): size model fit against the cluster and move node labels into the selector#11765
mudler merged 2 commits into
masterfrom
fix/cluster-aware-model-fit

Conversation

@localai-bot

Copy link
Copy Markdown
Collaborator

Description

Two UX defects reported from a live distributed deployment.

1. The models page sized every model against the frontend, not the cluster

In distributed mode the frontend is usually a GPU-less pod while models run on the workers, so a fleet of GPU nodes was told it could only run the smallest CPU build. The same wrong number drove three surfaces, two of them server-side:

  • Models.jsx read resources.aggregate.total_memory, the frontend's own host.
  • useRecommendedModels.js tiered its "recommended for your hardware" picks off it.
  • GET /api/models/variants/:id decided fits and auto_selected in Go against the controller's SystemState.VRAM, and judged backend compatibility against a host with no GPU.

NodeRegistry.HealthyNodeMemory now reports the largest single healthy backend node. The largest node, not the fleet total: a model loads into one node, so four 16GB workers are not a home for a 40GB model. An operator-set VRAM budget caps a node's contribution, since the scheduler refuses a load above that ceiling anyway, and a GPU node beats a CPU node holding more system RAM.

ClusterMemoryProviderFor mirrors the existing ClusterCapabilityProviderFor, and gallery.ClusterResolveEnv hands variant selection the cluster's memory plus the union of capabilities present in the cluster.

GET /api/resources and GET /api/models carry this as an additional cluster object. Their aggregate and ram* fields keep reporting the frontend's own hardware, which is what the resource monitor legitimately shows. Every degradation path lands back on single-node behaviour: no cluster object in single-node mode, and none when the registry cannot be read, so a hiccup narrows the answer rather than marking the whole catalog too large.

Verdicts now name the node they belong to ("Headroom on dgx-01", "80 GB on dgx-01, the largest of 4 nodes"), because a model fits somewhere or nowhere.

2. The scheduling page showed node labels whether or not you needed them

The node-label browser stood open above the rules at all times, while the field that actually needs labels, the rule's node selector, was two bare text inputs with no hint of what the cluster reports.

The browser is gone. The selector's key input completes against the label keys the cluster uses, and the value input offers only the values that key takes. The roster already loads for the page, so suggestions cost no request, and a roster that fails to load costs the admin the hints and nothing else. Suggestions stay suggestions: a key no node reports yet still commits as typed, which is how you write a rule before labelling the nodes for it.

Notes for Reviewers

Both commits are TDD: each behaviour has a test that failed first. One caveat worth naming, since it shaped the final diff. An early e2e assertion ("the pane names the node") was a false greentoContainText polls textContent and matched the zero-state hero still mounted right after the click. Chasing it down revealed that the fit badge in ModelDetail is unreachable dead code (the stat grid deliberately withholds vramDisplay). That edit was reverted, the node name moved to the headroom stat that actually renders, and the assertion re-verified red then green.

Verification:

Suite Command Result
Registry go test ./core/services/nodes/ --ginkgo.focus=HealthyNodeMemory 8/8
Gallery go test ./core/gallery/ --ginkgo.focus=ClusterResolveEnv 8/8
Routes go test ./core/http/routes/ 77/77
JS units node --test src/utils/modelBudget.test.js src/utils/nodeLabelSuggestions.test.js 16/16
UI e2e npx playwright test 469 passed, 1 known flake
Lint make lint 0 issues
Inline styles npm run lint:inline-styles at baseline

The single e2e failure (console-narrow.spec.js) passes 6/6 in isolation and touches nothing in this change; it is the parallelism flakiness playwright.config.js already documents. An earlier full run flaked on a different set of four specs, all likewise green in isolation.

Signed commits

  • Yes, I signed my commits.
  • Documentation updated (docs/content/) for user-facing changes, or not applicable

mudler added 2 commits August 28, 2026 17:13
The scheduling page kept a node-label browser open above the rules
whether or not anyone was writing one, while the field that actually
needs labels, the rule's node selector, was two bare text inputs with no
hint of what the cluster reports.

The browser is gone. The selector's key input now completes against the
label keys the cluster uses, and the value input offers only the values
that key takes. The roster already loads for the page, so the
suggestions cost no request, and a roster that fails to load costs the
admin the hints and nothing else.

Suggestions stay suggestions: a key no node reports yet still commits as
typed, which is how an admin writes a rule before labelling the nodes
for it.

Assisted-by: Claude:claude-opus-5 golangci-lint eslint playwright
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>
The models page asked the frontend how much memory a model may occupy.
In distributed mode the frontend is usually a GPU-less pod while every
model runs on a worker, so a fleet of GPU nodes was told it could only
run the smallest CPU build. The variant picker's fits flag and its
auto-selection came from the same place, as did the hardware
recommendations.

The registry now reports the largest single healthy backend node. The
largest node, not the fleet total: a model loads into one node, so four
16GB workers are not a home for a 40GB model. An operator-set VRAM
budget caps a node's contribution, because the scheduler refuses a load
above that ceiling anyway, and a GPU node beats a CPU node holding more
system RAM.

GET /api/resources and GET /api/models carry this as an additional
cluster object. Their aggregate and ram fields keep reporting the
frontend's own hardware, which is what the resource monitor shows.
Variant selection judges backends against the union of the capabilities
present in the cluster, the way backend discovery already did.

Every path degrades to the local host: no cluster object in single-node
mode, and none when the registry cannot be read, so a hiccup narrows the
answer back to single-node behaviour rather than marking the whole
catalog too large.

The verdicts now name the node they belong to, since a model fits
somewhere or nowhere.

Assisted-by: Claude:claude-opus-5 golangci-lint eslint playwright
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>
@mudler
mudler merged commit 29899cd into master Aug 28, 2026
70 of 71 checks passed
@mudler
mudler deleted the fix/cluster-aware-model-fit branch August 28, 2026 20:57
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants