feat: local-model-fit domain — catalog, hardware profiles, scoring engine, and UI#2141
Draft
Copilot wants to merge 130 commits intofeat/ts-backend-migration-origin-rebasefrom
Draft
feat: local-model-fit domain — catalog, hardware profiles, scoring engine, and UI#2141Copilot wants to merge 130 commits intofeat/ts-backend-migration-origin-rebasefrom
Copilot wants to merge 130 commits intofeat/ts-backend-migration-origin-rebasefrom
Conversation
… scoring, ranking, filtering, summaries, store, hooks, and UI components Co-authored-by: heavy-d <3121000+heavy-d@users.noreply.github.com>
…Fits, summaries Co-authored-by: heavy-d <3121000+heavy-d@users.noreply.github.com>
Co-authored-by: heavy-d <3121000+heavy-d@users.noreply.github.com>
Copilot
AI
changed the title
[WIP] Add nodetool-native local model fit feature
feat: local-model-fit domain — catalog, hardware profiles, scoring engine, and UI
Mar 15, 2026
Incorporates 74 new commits from the rebased branch via patch apply: - Systematic TypeScript any type removal across frontend - Memory leak fixes (setTimeout/setInterval cleanup) - Error normalization and sanitization utilities - Accessibility improvements (ARIA labels, roles) - Performance optimizations (data-attribute pattern, memoization) - Python bridge resilience and interpreter discovery - Output type normalization between TS and Python backends - New component and store tests - Security hardening (noopener/noreferrer, CORS) - Cross-platform build script compatibility Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
…nodes
Three compounding issues prevented controlled nodes from completing:
1. _runControlled() used iterAny() (all handles) instead of iterInput("__control__")
— blocked on data handles even after controllers finished
2. _sendEOS() conditionally skipped closing __control__ via _controlEdgesRouted guard
— controlled nodes never saw EOS when events used sendControlEvent() API
3. sendControlEvent() inflated upstream count via addUpstream() without ever closing
— phantom source kept __control__ handle permanently open
Also add static requiredSettings to search (SERPAPI_API_KEY), apify (APIFY_API_KEY),
and messaging nodes (DISCORD_BOT_TOKEN, TELEGRAM_BOT_TOKEN) so _injectSecrets()
loads API keys from the encrypted secrets store.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
ControlAgent fixes: - Use configurable system prompt (inputs.system) instead of hardcoded 2-sentence constant - Match Python user prompt format with markdown code fences and instructions - Add empty model validation (throw on empty provider) - Add early return with warning when no _control_context is provided - Add metadata extraction for LLM responses with node_id/node_type wrapper keys - Add image and audio props for multimodal control decisions - Route __control_output__ from ControlAgent to controlled nodes via control edges (the core bug: runner only looked for __control__ sourceHandle, missing __control_output__) - Suppress __control_output__ from output_update emissions Node metadata: TS is now source of truth - Registry.register() no longer merges Python metadata for TS nodes - Python package_metadata JSON only serves Python-only node packs (huggingface, mlx, etc.) - mergeMetadata keeps TS authoritative with Python backfill for layout/model_packs only - Updated metadata-parity test to verify TS superset of Python properties Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
…edges - Graph._detectControlledNodes(): auto-set is_controlled on nodes that have incoming control edges, matching Python's BaseNode.is_controlled() runtime check - ProcessingContext: add setSendControlEvent/sendControlEvent/hasControlEventSupport so agent nodes can dispatch control events to controlled nodes and await results Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
- replicate-base.ts: API key extraction, submit/poll with Prefer:wait, output converters (image/video/audio/string), asset URL resolution - 40 tests for all utility functions - index.ts stub with empty REPLICATE_NODES array Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
- types.ts: NodeSpec/FieldDef/EnumDef/NodeConfig/ModuleConfig with returnType - schema-fetcher.ts: fetch Replicate OpenAPI schemas with caching - schema-parser.ts: parse Replicate Input schema to NodeSpec - node-generator.ts: generate TS node classes with Replicate API pattern Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
14 config files organized by namespace (image-generate, video-generate, audio-generate, text-generate, etc.) covering all active Replicate models. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Supports --all, --module, --no-cache, --output-dir flags. Follows the same pattern as fal-codegen generate.ts. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
- metadata-parser.ts: parse Python package metadata JSON → NodeSpec - generate.ts: --from-metadata flag for offline generation without API token - 14 generated files with 156 node classes across image/video/audio/text - Tests: 44 passing (40 base + 4 generated nodes) - Registered in websocket server alongside fal-nodes Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
- ReplicateProvider extends OpenAIProvider using Replicate's OpenAI-compatible endpoint (api.replicate.com/v1/openai/v1) for chat completions with streaming - textToImage via Replicate predictions REST API with polling - 8 language models (Llama 3 variants, Llama Guard, Snowflake Arctic) - 3 image models (FLUX Schnell, SDXL, FLUX Dev) - Registered as "replicate" provider in provider registry - metadata-parser.ts: generate nodes from Python metadata JSON without API token - 12 provider tests passing Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
- replicateSubmit now accepts both "owner/name" (latest version) and "owner/name:version" (pinned) formats - Generator no longer wraps model ID in extractVersion() - Deduplicate nodes by className in generateFromMetadata - 155 nodes, 44 tests passing Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
5 models had owner changes on Replicate: - fofr/consistent-character → sdxl-based/consistent-character - batouresearch/magic-image-refiner → fermatresearch/magic-image-refiner - batouresearch/high-resolution-controlnet-tile → fermatresearch/high-resolution-controlnet-tile - batouresearch/magic-style-transfer → fermatresearch/magic-style-transfer - lucataco/nsfw_image_detection → falcons-ai/nsfw_image_detection Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Image generation (20): Seedream 4.5/5-lite/3, Recraft V4/V4-SVG/V4-Pro/V4-Pro-SVG, Ideogram V3-Balanced/V3-Quality/V2A-Turbo, Imagen 4/4-Ultra/3-Fast, Nano Banana Pro, Grok Imagine Image, Fibo, Bria Image 3.2, Flux 2 Klein 4B/Kontext Max, Hunyuan Image 3 Video generation (27): Gen4.5, Kling V3/V3-Omni/V2.5-Turbo-Pro/V2.6, Veo 3/3-Fast/2, Hailuo 2.3/2.3-Fast, Pixverse V5.6/V4/V4.5, Wan 2.5-T2V/T2V-Fast/I2V/I2V-Fast, Seedance 1-Pro/1-Lite/1-Pro-Fast, Ray 2-540p/2-720p/Flash-2, Sora 2/2-Pro, Video 01 Director Text/LLM (16): Gemini 3.1-Pro/2.5-Flash/3-Pro, Claude Opus 4.6/4.5-Sonnet/ 4.5-Haiku/4-Sonnet, GPT 5.2/O4-Mini/O1/4o/4o-Mini, Grok 4, Deepseek V3, Qwen3 235B, Kimi K2.5 Audio (1): Speech 2.8 HD Total configs: 219 (155 existing + 64 new) Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
New config files: - image-background.ts (12): background removal, eraser, genfill - audio-speech.ts (39): ElevenLabs, Chatterbox, Kokoro, XTTS, Minimax TTS - video-face.ts (16): lipsync, face swap, talking head, avatar - video-process.ts (12): video upscale, interpolation, captioning - image-3d.ts (7): Trellis, ShapE, Hunyuan3D, SeedVR2 Updated existing configs: - audio-transcribe.ts (+9): Whisper, WhisperX, diarization - image-ocr.ts (+4): Surya, Deepseek OCR, Datalab - image-upscale.ts (+18): ESRGAN variants, Aura SR, Topaz, Crystal - image-enhance.ts (+10): Deoldify, VQFR, NAFNet, AnimeSR - image-generate.ts (+19): PuLID, Kontext apps, Gen4 Image, OmniGen2 - image-analyze.ts (+7): Apollo, VideoLlama3, LLaVA v1.6, Gemini Flash Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Full code generation with REPLICATE_API_TOKEN. 19 categories: - image-generate (99), video-generate (49), audio-speech (38), text-generate (34), image-upscale (26), image-enhance (16), image-analyze (16), video-face (13), image-background (12), audio-transcribe (11), video-process (10), image-generate (99), audio-generate (8), image-face (8), image-3d (7), image-process (7), image-ocr (6), audio-enhance (1), audio-separate (1) 9 models skipped (404 on Replicate API - removed/renamed) Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
…_models, required_settings
Replicate nodes were missing layout (undefined instead of "default"),
which could cause React rendering errors when the UI expected a string.
Also default the_model_info to {}, recommended_models to [],
and required_settings to [] to match the OpenAPI schema contract.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Legacy HuggingFace model info field that was always {} except for
one cover_image_url usage in NodeInfo.tsx. Removed from:
- NodeMetadata interface (node-sdk, protocol, api.ts, mobile)
- BaseNode.theModelInfo static property
- getNodeMetadata generation
- All test fixtures (21 web test files)
- Default metadata objects, comfy schema converter, placeholder nodes
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
…ty graph
POST /api/workflows without a graph field now returns 200 with an
empty graph ({nodes:[], edges:[]}) instead of 400, aligning tests
with the actual API behaviour.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Four example workflows demonstrating the DSL API: add_numbers, concat_text, list_operations, and flux_3_dogs (image gen). Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
…Properties bug (#2199) * Initial plan * fix(kernel): add @nodetool/config alias to vitest.config.ts Co-authored-by: georgi <19498+georgi@users.noreply.github.com> * feat(kernel): implement 8 todo parity tests and fix allowUndefinedProperties bug Co-authored-by: georgi <19498+georgi@users.noreply.github.com> --------- Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com> Co-authored-by: georgi <19498+georgi@users.noreply.github.com>
…ve stub comments from RealtimeAgent/Transcription Co-authored-by: georgi <19498+georgi@users.noreply.github.com>
…etry fix in GriffinLim, clamp min values in NoiseGate Co-authored-by: georgi <19498+georgi@users.noreply.github.com>
…rowser automation BrowserUse was a workflow node that embedded an LLM agent loop directly. This functionality belongs in the agent system using existing browser tools (browser, dom_examine, dom_search, dom_extract, take_screenshot). Removed the node, its tests, generated DSL bindings, and docs. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* Initial plan * Implement stub nodes: NoiseGate, Phaser, GriffinLim, BrowserUse; remove stub comments from RealtimeAgent/Transcription Co-authored-by: georgi <19498+georgi@users.noreply.github.com> * Fix code review issues: SSRF protection in BrowserUse, conjugate symmetry fix in GriffinLim, clamp min values in NoiseGate Co-authored-by: georgi <19498+georgi@users.noreply.github.com> * refactor(browser): remove BrowserUseLibNode in favor of agent-based browser automation BrowserUse was a workflow node that embedded an LLM agent loop directly. This functionality belongs in the agent system using existing browser tools (browser, dom_examine, dom_search, dom_extract, take_screenshot). Removed the node, its tests, generated DSL bindings, and docs. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> --------- Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com> Co-authored-by: georgi <19498+georgi@users.noreply.github.com> Co-authored-by: georgi <matti.georgi@gmail.com> Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
…2208) * Initial plan * Changes before error encountered Co-authored-by: georgi <19498+georgi@users.noreply.github.com> * feat: add comprehensive test suite for replicate-codegen (151 tests) Co-authored-by: georgi <19498+georgi@users.noreply.github.com> --------- Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com> Co-authored-by: georgi <19498+georgi@users.noreply.github.com>
* Initial plan * Add tests/index.test.ts to cover the untested barrel file in packages/deploy Co-authored-by: georgi <19498+georgi@users.noreply.github.com> --------- Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com> Co-authored-by: georgi <19498+georgi@users.noreply.github.com>
* Initial plan * Initial plan for finishing vectorstore tasks Co-authored-by: georgi <19498+georgi@users.noreply.github.com> * Fix bugs, add comprehensive tests for vectorstore package Co-authored-by: georgi <19498+georgi@users.noreply.github.com> --------- Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com> Co-authored-by: georgi <19498+georgi@users.noreply.github.com>
* Initial plan * Changes before error encountered Co-authored-by: georgi <19498+georgi@users.noreply.github.com> * packages/agents: add tests for all untested source files, 34/34 pass Co-authored-by: georgi <19498+georgi@users.noreply.github.com> --------- Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com> Co-authored-by: georgi <19498+georgi@users.noreply.github.com>
* Initial plan * Initial plan for elevenlabs-nodes fixes Co-authored-by: georgi <19498+georgi@users.noreply.github.com> * Fix elevenlabs-nodes: tests, source bugs, exports, workspace registration Co-authored-by: georgi <19498+georgi@users.noreply.github.com> --------- Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com> Co-authored-by: georgi <19498+georgi@users.noreply.github.com>
* Initial plan * Changes before error encountered Co-authored-by: georgi <19498+georgi@users.noreply.github.com> * Fix test failures, improve detectFromJson with exact pipeline class table Co-authored-by: georgi <19498+georgi@users.noreply.github.com> --------- Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com> Co-authored-by: georgi <19498+georgi@users.noreply.github.com>
…coverage for 22 files (#2209) * Initial plan * Changes before error encountered Co-authored-by: georgi <19498+georgi@users.noreply.github.com> * packages/replicate-nodes: finish tasks — add tests for submit/poll, upload paths, registry Co-authored-by: georgi <19498+georgi@users.noreply.github.com> --------- Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com> Co-authored-by: georgi <19498+georgi@users.noreply.github.com>
* Initial plan * Changes before error encountered Co-authored-by: georgi <19498+georgi@users.noreply.github.com> * Fix and implement HTTP API stubs in packages/websocket - Implement workflow generate-name endpoint (derives name from node type categories) - Fix workflow DSL export: silent 200 stub → proper 501 with 404-awareness - Fix workflow Gradio export: silent 200 stub → proper 501 with 404-awareness - Implement thread summarize endpoint (derives title from first message text) - Document Ollama streaming 501 with clear explanation for clients - Fix workflow create/update to require graph field (was silently defaulting to empty) - Fix getAllModels() to always include recommended models as baseline - Fix code-node.ts: lazy-load isolated-vm to avoid native addon at import time - Export StreamingInputs/StreamingOutputs from packages/runtime and packages/node-sdk - Fix resolveProvider tests to handle missing API keys gracefully - Add 17 new tests for all stub endpoints (generate-name, DSL, Gradio, summarize, triggers) Co-authored-by: georgi <19498+georgi@users.noreply.github.com> --------- Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com> Co-authored-by: georgi <19498+georgi@users.noreply.github.com>
…urce files (#2215) * Initial plan * Add comprehensive CLI test suite: 101 tests across 6 test files Co-authored-by: georgi <19498+georgi@users.noreply.github.com> --------- Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com> Co-authored-by: georgi <19498+georgi@users.noreply.github.com>
…orter, and CLI run command - core.ts: replace stubs with full run()/runGraph() implementation using WorkflowRunner; surface node-level actor errors via result.messages scan - export.ts: new workflowToDsl() converts workflow JSON to importable TypeScript DSL source; fixes identifier collision suffixing, workflowName case check, O(n²) find replaced with Map - index.ts: re-export export.ts - tests/core.test.ts: complex graph tests — diamond, long chain, string template, parallel branches, conditional error, runGraph chain - tests/export.test.ts: comprehensive workflowToDsl tests — diamond, identifier collision, reserved words, workflowName normalisation, complex props, cycle detection, missing fields, topological order, multi-namespace imports, multiple source nodes - packages/cli: add `nodetool run <dsl-file>` command; new run-dsl.ts uses tsx/esm/api tsImport to load TypeScript DSL files, find Workflow exports, and run them; CLI registers base nodes automatically before execution; unit + integration tests with fixtures - packages/websocket: null-guard on workflow.graph in DSL export endpoint Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
- Updated OutputContextMenu to handle null values for saveNodeMetadata. - Enhanced OutputRenderer to ensure proper BlobPart typing when creating object URLs. - Adjusted ReactFlowWrapper to correctly type props. - Modified TaskPlanView to provide default values for optional properties. - Improved useVideoSrc hook to ensure BlobPart typing. - Cleaned up NodeInfo by removing unnecessary loading state. - Refactored PanelRight to use correct functional component typing.
…ration # Conflicts: # packages/cli/package.json # packages/dsl/package.json # packages/dsl/src/core.ts # packages/dsl/src/generated/lib.browser.ts # packages/websocket/package.json # packages/websocket/src/http-api.ts
Replace sandboxed V8 isolate with `new AsyncFunction()` for simpler, dependency-free code execution. Timeout uses Promise.race instead of isolate-level CPU limits. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
- Introduced a new Local Model Fit page with a dedicated route and UI components. - Enhanced ModelListHeader to include a link to the Local Model Fit page. - Refactored LocalModelFitList and LocalModelFitRow to utilize shared column metrics for layout consistency. - Implemented search filtering logic to support multi-token queries and tag prefix matching. - Added navigation and back button functionality in the Local Model Fit page. - Updated PanelLeft to include a button for quick access to the Local Model Fit feature.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Self-contained local-model-fit domain that ranks models by how well they fit a user's hardware. Four layers: catalog ingestion → hardware profile → deterministic scoring → UI consumers. One canonical ranked result set powers all views.
Core domain (
web/src/local_model_fit/)HardwareProfile,ModelCatalogEntry,ModelVariant,RankedModelFit(canonical result shape with score 0–100, tier S/A/B/C/D/F, fit label, memory %, reasons[])buildCustomProfile()for manual VRAM/RAM overridesnavigator.deviceMemoryfallbackHooks & store
persistfor hardware profile + view modeUI (
web/src/components/local_model_fit/)LocalModelFitPage— hardware selector → tier summary → search/filter bar → card/list toggleHardwareProfileSelector— preset dropdown, auto-detect, custom VRAM/RAM inputsTierSummary— clickable tier chips that toggle as filtersLocalModelFitCardGrid/LocalModelFitList— two renderers, sameRankedModelFit[]FitBadge— reusable tier chip for later integration into model menusUsage
Route:
/local-model-fit(lazy-loaded, protected).Tests
43 unit tests across scoring, ranking, filtering, and summaries — all pure functions, no mocking needed.
🔒 GitHub Advanced Security automatically protects Copilot coding agent pull requests. You can protect all pull requests by enabling Advanced Security for your repositories. Learn more about Advanced Security.