Skip to content

feat: local-model-fit domain — catalog, hardware profiles, scoring engine, and UI#2141

Draft
Copilot wants to merge 130 commits intofeat/ts-backend-migration-origin-rebasefrom
copilot/add-local-model-fit-feature
Draft

feat: local-model-fit domain — catalog, hardware profiles, scoring engine, and UI#2141
Copilot wants to merge 130 commits intofeat/ts-backend-migration-origin-rebasefrom
copilot/add-local-model-fit-feature

Conversation

Copy link
Contributor

Copilot AI commented Mar 15, 2026

Self-contained local-model-fit domain that ranks models by how well they fit a user's hardware. Four layers: catalog ingestion → hardware profile → deterministic scoring → UI consumers. One canonical ranked result set powers all views.

Core domain (web/src/local_model_fit/)

  • types.tsHardwareProfile, ModelCatalogEntry, ModelVariant, RankedModelFit (canonical result shape with score 0–100, tier S/A/B/C/D/F, fit label, memory %, reasons[])
  • modelCatalog.ts — 30+ normalized models from Ollama/llama.cpp/LM Studio ecosystem (chat, code, reasoning, vision). Static v1 seed with documented ingestion strategy for future automation
  • hardwareProfiles.ts — 21 presets (NVIDIA GPUs, Apple Silicon, CPU-only) + buildCustomProfile() for manual VRAM/RAM overrides
  • hardwareProfileDetection.ts — WebGPU adapter probe → platform/UA heuristics → navigator.deviceMemory fallback
  • scoreModelFit.ts — Deterministic heuristic: effective memory (VRAM for discrete, 75% unified for macOS, 60% RAM for CPU-only) → memory ratio → score with quant bonus. No ML, fully explainable
  • rankModelFits.ts / filterModelFits.ts / summaries.ts — Composable ranking, filtering (search/tags/families/tiers/fits), tier counts

Hooks & store

  • useHardwareProfile() — detect/set profile
  • useRankedModelFits() — two-layer memo: score on profile change, filter on UI interaction
  • useModelFitSummary() — tier counts + one-liner
  • localModelFitStore — Zustand with persist for hardware profile + view mode

UI (web/src/components/local_model_fit/)

  • LocalModelFitPage — hardware selector → tier summary → search/filter bar → card/list toggle
  • HardwareProfileSelector — preset dropdown, auto-detect, custom VRAM/RAM inputs
  • TierSummary — clickable tier chips that toggle as filters
  • LocalModelFitCardGrid / LocalModelFitList — two renderers, same RankedModelFit[]
  • FitBadge — reusable tier chip for later integration into model menus

Usage

import { useRankedModelFits, useHardwareProfile } from "../local_model_fit";

const { profile } = useHardwareProfile();
const { results } = useRankedModelFits();
// results: RankedModelFit[] — sorted, filtered, ready to render

Route: /local-model-fit (lazy-loaded, protected).

Tests

43 unit tests across scoring, ranking, filtering, and summaries — all pure functions, no mocking needed.


🔒 GitHub Advanced Security automatically protects Copilot coding agent pull requests. You can protect all pull requests by enabling Advanced Security for your repositories. Learn more about Advanced Security.

… scoring, ranking, filtering, summaries, store, hooks, and UI components

Co-authored-by: heavy-d <3121000+heavy-d@users.noreply.github.com>
@cloudflare-workers-and-pages
Copy link

cloudflare-workers-and-pages bot commented Mar 15, 2026

Deploying nodetool-staging with  Cloudflare Pages  Cloudflare Pages

Latest commit: a735fa6
Status:🚫  Build failed.

View logs

@cloudflare-workers-and-pages
Copy link

cloudflare-workers-and-pages bot commented Mar 15, 2026

Deploying nodetool with  Cloudflare Pages  Cloudflare Pages

Latest commit: a735fa6
Status:🚫  Build failed.

View logs

Copilot AI and others added 2 commits March 15, 2026 02:19
…Fits, summaries

Co-authored-by: heavy-d <3121000+heavy-d@users.noreply.github.com>
Co-authored-by: heavy-d <3121000+heavy-d@users.noreply.github.com>
Copilot AI changed the title [WIP] Add nodetool-native local model fit feature feat: local-model-fit domain — catalog, hardware profiles, scoring engine, and UI Mar 15, 2026
Copilot AI requested a review from heavy-d March 15, 2026 02:27
georgi and others added 18 commits March 15, 2026 07:46
Incorporates 74 new commits from the rebased branch via patch apply:
- Systematic TypeScript any type removal across frontend
- Memory leak fixes (setTimeout/setInterval cleanup)
- Error normalization and sanitization utilities
- Accessibility improvements (ARIA labels, roles)
- Performance optimizations (data-attribute pattern, memoization)
- Python bridge resilience and interpreter discovery
- Output type normalization between TS and Python backends
- New component and store tests
- Security hardening (noopener/noreferrer, CORS)
- Cross-platform build script compatibility

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
…nodes

Three compounding issues prevented controlled nodes from completing:
1. _runControlled() used iterAny() (all handles) instead of iterInput("__control__")
   — blocked on data handles even after controllers finished
2. _sendEOS() conditionally skipped closing __control__ via _controlEdgesRouted guard
   — controlled nodes never saw EOS when events used sendControlEvent() API
3. sendControlEvent() inflated upstream count via addUpstream() without ever closing
   — phantom source kept __control__ handle permanently open

Also add static requiredSettings to search (SERPAPI_API_KEY), apify (APIFY_API_KEY),
and messaging nodes (DISCORD_BOT_TOKEN, TELEGRAM_BOT_TOKEN) so _injectSecrets()
loads API keys from the encrypted secrets store.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
ControlAgent fixes:
- Use configurable system prompt (inputs.system) instead of hardcoded 2-sentence constant
- Match Python user prompt format with markdown code fences and instructions
- Add empty model validation (throw on empty provider)
- Add early return with warning when no _control_context is provided
- Add metadata extraction for LLM responses with node_id/node_type wrapper keys
- Add image and audio props for multimodal control decisions
- Route __control_output__ from ControlAgent to controlled nodes via control edges
  (the core bug: runner only looked for __control__ sourceHandle, missing __control_output__)
- Suppress __control_output__ from output_update emissions

Node metadata: TS is now source of truth
- Registry.register() no longer merges Python metadata for TS nodes
- Python package_metadata JSON only serves Python-only node packs (huggingface, mlx, etc.)
- mergeMetadata keeps TS authoritative with Python backfill for layout/model_packs only
- Updated metadata-parity test to verify TS superset of Python properties

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
…edges

- Graph._detectControlledNodes(): auto-set is_controlled on nodes that have
  incoming control edges, matching Python's BaseNode.is_controlled() runtime check
- ProcessingContext: add setSendControlEvent/sendControlEvent/hasControlEventSupport
  so agent nodes can dispatch control events to controlled nodes and await results

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
- replicate-base.ts: API key extraction, submit/poll with Prefer:wait,
  output converters (image/video/audio/string), asset URL resolution
- 40 tests for all utility functions
- index.ts stub with empty REPLICATE_NODES array

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
- types.ts: NodeSpec/FieldDef/EnumDef/NodeConfig/ModuleConfig with returnType
- schema-fetcher.ts: fetch Replicate OpenAPI schemas with caching
- schema-parser.ts: parse Replicate Input schema to NodeSpec
- node-generator.ts: generate TS node classes with Replicate API pattern

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
14 config files organized by namespace (image-generate, video-generate,
audio-generate, text-generate, etc.) covering all active Replicate models.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Supports --all, --module, --no-cache, --output-dir flags.
Follows the same pattern as fal-codegen generate.ts.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
- metadata-parser.ts: parse Python package metadata JSON → NodeSpec
- generate.ts: --from-metadata flag for offline generation without API token
- 14 generated files with 156 node classes across image/video/audio/text
- Tests: 44 passing (40 base + 4 generated nodes)
- Registered in websocket server alongside fal-nodes

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
- ReplicateProvider extends OpenAIProvider using Replicate's OpenAI-compatible
  endpoint (api.replicate.com/v1/openai/v1) for chat completions with streaming
- textToImage via Replicate predictions REST API with polling
- 8 language models (Llama 3 variants, Llama Guard, Snowflake Arctic)
- 3 image models (FLUX Schnell, SDXL, FLUX Dev)
- Registered as "replicate" provider in provider registry
- metadata-parser.ts: generate nodes from Python metadata JSON without API token
- 12 provider tests passing

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
- replicateSubmit now accepts both "owner/name" (latest version) and
  "owner/name:version" (pinned) formats
- Generator no longer wraps model ID in extractVersion()
- Deduplicate nodes by className in generateFromMetadata
- 155 nodes, 44 tests passing

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
5 models had owner changes on Replicate:
- fofr/consistent-character → sdxl-based/consistent-character
- batouresearch/magic-image-refiner → fermatresearch/magic-image-refiner
- batouresearch/high-resolution-controlnet-tile → fermatresearch/high-resolution-controlnet-tile
- batouresearch/magic-style-transfer → fermatresearch/magic-style-transfer
- lucataco/nsfw_image_detection → falcons-ai/nsfw_image_detection

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Image generation (20): Seedream 4.5/5-lite/3, Recraft V4/V4-SVG/V4-Pro/V4-Pro-SVG,
  Ideogram V3-Balanced/V3-Quality/V2A-Turbo, Imagen 4/4-Ultra/3-Fast,
  Nano Banana Pro, Grok Imagine Image, Fibo, Bria Image 3.2,
  Flux 2 Klein 4B/Kontext Max, Hunyuan Image 3

Video generation (27): Gen4.5, Kling V3/V3-Omni/V2.5-Turbo-Pro/V2.6,
  Veo 3/3-Fast/2, Hailuo 2.3/2.3-Fast, Pixverse V5.6/V4/V4.5,
  Wan 2.5-T2V/T2V-Fast/I2V/I2V-Fast, Seedance 1-Pro/1-Lite/1-Pro-Fast,
  Ray 2-540p/2-720p/Flash-2, Sora 2/2-Pro, Video 01 Director

Text/LLM (16): Gemini 3.1-Pro/2.5-Flash/3-Pro, Claude Opus 4.6/4.5-Sonnet/
  4.5-Haiku/4-Sonnet, GPT 5.2/O4-Mini/O1/4o/4o-Mini, Grok 4,
  Deepseek V3, Qwen3 235B, Kimi K2.5

Audio (1): Speech 2.8 HD

Total configs: 219 (155 existing + 64 new)

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
New config files:
- image-background.ts (12): background removal, eraser, genfill
- audio-speech.ts (39): ElevenLabs, Chatterbox, Kokoro, XTTS, Minimax TTS
- video-face.ts (16): lipsync, face swap, talking head, avatar
- video-process.ts (12): video upscale, interpolation, captioning
- image-3d.ts (7): Trellis, ShapE, Hunyuan3D, SeedVR2

Updated existing configs:
- audio-transcribe.ts (+9): Whisper, WhisperX, diarization
- image-ocr.ts (+4): Surya, Deepseek OCR, Datalab
- image-upscale.ts (+18): ESRGAN variants, Aura SR, Topaz, Crystal
- image-enhance.ts (+10): Deoldify, VQFR, NAFNet, AnimeSR
- image-generate.ts (+19): PuLID, Kontext apps, Gen4 Image, OmniGen2
- image-analyze.ts (+7): Apollo, VideoLlama3, LLaVA v1.6, Gemini Flash

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Full code generation with REPLICATE_API_TOKEN. 19 categories:
- image-generate (99), video-generate (49), audio-speech (38),
  text-generate (34), image-upscale (26), image-enhance (16),
  image-analyze (16), video-face (13), image-background (12),
  audio-transcribe (11), video-process (10), image-generate (99),
  audio-generate (8), image-face (8), image-3d (7), image-process (7),
  image-ocr (6), audio-enhance (1), audio-separate (1)

9 models skipped (404 on Replicate API - removed/renamed)

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
…_models, required_settings

Replicate nodes were missing layout (undefined instead of "default"),
which could cause React rendering errors when the UI expected a string.
Also default the_model_info to {}, recommended_models to [],
and required_settings to [] to match the OpenAPI schema contract.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Legacy HuggingFace model info field that was always {} except for
one cover_image_url usage in NodeInfo.tsx. Removed from:
- NodeMetadata interface (node-sdk, protocol, api.ts, mobile)
- BaseNode.theModelInfo static property
- getNodeMetadata generation
- All test fixtures (21 web test files)
- Default metadata objects, comfy schema converter, placeholder nodes

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
georgi and others added 30 commits March 18, 2026 08:41
…ty graph

POST /api/workflows without a graph field now returns 200 with an
empty graph ({nodes:[], edges:[]}) instead of 400, aligning tests
with the actual API behaviour.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Four example workflows demonstrating the DSL API:
add_numbers, concat_text, list_operations, and flux_3_dogs (image gen).

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
…Properties bug (#2199)

* Initial plan

* fix(kernel): add @nodetool/config alias to vitest.config.ts

Co-authored-by: georgi <19498+georgi@users.noreply.github.com>

* feat(kernel): implement 8 todo parity tests and fix allowUndefinedProperties bug

Co-authored-by: georgi <19498+georgi@users.noreply.github.com>

---------

Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com>
Co-authored-by: georgi <19498+georgi@users.noreply.github.com>
…ve stub comments from RealtimeAgent/Transcription

Co-authored-by: georgi <19498+georgi@users.noreply.github.com>
…etry fix in GriffinLim, clamp min values in NoiseGate

Co-authored-by: georgi <19498+georgi@users.noreply.github.com>
…rowser automation

BrowserUse was a workflow node that embedded an LLM agent loop directly. This
functionality belongs in the agent system using existing browser tools (browser,
dom_examine, dom_search, dom_extract, take_screenshot). Removed the node, its
tests, generated DSL bindings, and docs.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* Initial plan

* Implement stub nodes: NoiseGate, Phaser, GriffinLim, BrowserUse; remove stub comments from RealtimeAgent/Transcription

Co-authored-by: georgi <19498+georgi@users.noreply.github.com>

* Fix code review issues: SSRF protection in BrowserUse, conjugate symmetry fix in GriffinLim, clamp min values in NoiseGate

Co-authored-by: georgi <19498+georgi@users.noreply.github.com>

* refactor(browser): remove BrowserUseLibNode in favor of agent-based browser automation

BrowserUse was a workflow node that embedded an LLM agent loop directly. This
functionality belongs in the agent system using existing browser tools (browser,
dom_examine, dom_search, dom_extract, take_screenshot). Removed the node, its
tests, generated DSL bindings, and docs.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

---------

Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com>
Co-authored-by: georgi <19498+georgi@users.noreply.github.com>
Co-authored-by: georgi <matti.georgi@gmail.com>
Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
…2208)

* Initial plan

* Changes before error encountered

Co-authored-by: georgi <19498+georgi@users.noreply.github.com>

* feat: add comprehensive test suite for replicate-codegen (151 tests)

Co-authored-by: georgi <19498+georgi@users.noreply.github.com>

---------

Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com>
Co-authored-by: georgi <19498+georgi@users.noreply.github.com>
* Initial plan

* Add tests/index.test.ts to cover the untested barrel file in packages/deploy

Co-authored-by: georgi <19498+georgi@users.noreply.github.com>

---------

Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com>
Co-authored-by: georgi <19498+georgi@users.noreply.github.com>
* Initial plan

* Initial plan for finishing vectorstore tasks

Co-authored-by: georgi <19498+georgi@users.noreply.github.com>

* Fix bugs, add comprehensive tests for vectorstore package

Co-authored-by: georgi <19498+georgi@users.noreply.github.com>

---------

Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com>
Co-authored-by: georgi <19498+georgi@users.noreply.github.com>
* Initial plan

* Changes before error encountered

Co-authored-by: georgi <19498+georgi@users.noreply.github.com>

* packages/agents: add tests for all untested source files, 34/34 pass

Co-authored-by: georgi <19498+georgi@users.noreply.github.com>

---------

Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com>
Co-authored-by: georgi <19498+georgi@users.noreply.github.com>
* Initial plan

* Initial plan for elevenlabs-nodes fixes

Co-authored-by: georgi <19498+georgi@users.noreply.github.com>

* Fix elevenlabs-nodes: tests, source bugs, exports, workspace registration

Co-authored-by: georgi <19498+georgi@users.noreply.github.com>

---------

Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com>
Co-authored-by: georgi <19498+georgi@users.noreply.github.com>
* Initial plan

* Changes before error encountered

Co-authored-by: georgi <19498+georgi@users.noreply.github.com>

* Fix test failures, improve detectFromJson with exact pipeline class table

Co-authored-by: georgi <19498+georgi@users.noreply.github.com>

---------

Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com>
Co-authored-by: georgi <19498+georgi@users.noreply.github.com>
…coverage for 22 files (#2209)

* Initial plan

* Changes before error encountered

Co-authored-by: georgi <19498+georgi@users.noreply.github.com>

* packages/replicate-nodes: finish tasks — add tests for submit/poll, upload paths, registry

Co-authored-by: georgi <19498+georgi@users.noreply.github.com>

---------

Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com>
Co-authored-by: georgi <19498+georgi@users.noreply.github.com>
* Initial plan

* Changes before error encountered

Co-authored-by: georgi <19498+georgi@users.noreply.github.com>

* Fix and implement HTTP API stubs in packages/websocket

- Implement workflow generate-name endpoint (derives name from node type categories)
- Fix workflow DSL export: silent 200 stub → proper 501 with 404-awareness
- Fix workflow Gradio export: silent 200 stub → proper 501 with 404-awareness
- Implement thread summarize endpoint (derives title from first message text)
- Document Ollama streaming 501 with clear explanation for clients
- Fix workflow create/update to require graph field (was silently defaulting to empty)
- Fix getAllModels() to always include recommended models as baseline
- Fix code-node.ts: lazy-load isolated-vm to avoid native addon at import time
- Export StreamingInputs/StreamingOutputs from packages/runtime and packages/node-sdk
- Fix resolveProvider tests to handle missing API keys gracefully
- Add 17 new tests for all stub endpoints (generate-name, DSL, Gradio, summarize, triggers)

Co-authored-by: georgi <19498+georgi@users.noreply.github.com>

---------

Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com>
Co-authored-by: georgi <19498+georgi@users.noreply.github.com>
…urce files (#2215)

* Initial plan

* Add comprehensive CLI test suite: 101 tests across 6 test files

Co-authored-by: georgi <19498+georgi@users.noreply.github.com>

---------

Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com>
Co-authored-by: georgi <19498+georgi@users.noreply.github.com>
…orter, and CLI run command

- core.ts: replace stubs with full run()/runGraph() implementation using WorkflowRunner;
  surface node-level actor errors via result.messages scan
- export.ts: new workflowToDsl() converts workflow JSON to importable TypeScript DSL source;
  fixes identifier collision suffixing, workflowName case check, O(n²) find replaced with Map
- index.ts: re-export export.ts
- tests/core.test.ts: complex graph tests — diamond, long chain, string template,
  parallel branches, conditional error, runGraph chain
- tests/export.test.ts: comprehensive workflowToDsl tests — diamond, identifier collision,
  reserved words, workflowName normalisation, complex props, cycle detection, missing fields,
  topological order, multi-namespace imports, multiple source nodes
- packages/cli: add `nodetool run <dsl-file>` command; new run-dsl.ts uses tsx/esm/api
  tsImport to load TypeScript DSL files, find Workflow exports, and run them; CLI registers
  base nodes automatically before execution; unit + integration tests with fixtures
- packages/websocket: null-guard on workflow.graph in DSL export endpoint

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
- Updated OutputContextMenu to handle null values for saveNodeMetadata.
- Enhanced OutputRenderer to ensure proper BlobPart typing when creating object URLs.
- Adjusted ReactFlowWrapper to correctly type props.
- Modified TaskPlanView to provide default values for optional properties.
- Improved useVideoSrc hook to ensure BlobPart typing.
- Cleaned up NodeInfo by removing unnecessary loading state.
- Refactored PanelRight to use correct functional component typing.
…ration

# Conflicts:
#	packages/cli/package.json
#	packages/dsl/package.json
#	packages/dsl/src/core.ts
#	packages/dsl/src/generated/lib.browser.ts
#	packages/websocket/package.json
#	packages/websocket/src/http-api.ts
Replace sandboxed V8 isolate with `new AsyncFunction()` for simpler,
dependency-free code execution. Timeout uses Promise.race instead of
isolate-level CPU limits.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
- Introduced a new Local Model Fit page with a dedicated route and UI components.
- Enhanced ModelListHeader to include a link to the Local Model Fit page.
- Refactored LocalModelFitList and LocalModelFitRow to utilize shared column metrics for layout consistency.
- Implemented search filtering logic to support multi-token queries and tag prefix matching.
- Added navigation and back button functionality in the Local Model Fit page.
- Updated PanelLeft to include a button for quick access to the Local Model Fit feature.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants