Skip to content

feat(sam3): enable SDK-based remote execution for SAM3 workflow blocks#2042

Merged
hansent merged 3 commits intomainfrom
feat/sam3-workflow-remote-exec-via-sdk
Feb 26, 2026
Merged

feat(sam3): enable SDK-based remote execution for SAM3 workflow blocks#2042
hansent merged 3 commits intomainfrom
feat/sam3-workflow-remote-exec-via-sdk

Conversation

@hansent
Copy link
Collaborator

@hansent hansent commented Feb 26, 2026

Summary

  • Decouple SAM3 workflow block remote execution from the API inference proxy. SAM3 blocks (v1, v2, v3) previously always routed through the inference proxy when running remotely, unlike every other model block which uses the SDK with configurable HOSTED_*_URL endpoints. Now when WORKFLOWS_STEP_EXECUTION_MODE=remote and SAM3_EXEC_MODE=local, SAM3 blocks use the standard SDK-based remote execution pattern via HOSTED_CORE_MODEL_URL, matching the behavior of object detection, classification, instance segmentation, CLIP, OCR, and SAM3 3D blocks.
  • Add SAM3 SDK client methods: sam3_concept_segment, sam3_visual_segment, sam3_embed_image (sync + async) to InferenceHTTPClient, following the established _post_images pattern used by sam2_segment_image, infer_lmm, etc. This ensures proper execution_id propagation and X-Processing-Time header collection via the standard RemoteProcessingTimeCollector mechanism.
  • Add SAM3_FINE_TUNED_MODELS_ENABLED env var to control whether fine-tuned SAM3 models are allowed, independent of SAM3_EXEC_MODE. Default is derived from SAM3_EXEC_MODE for full backward compatibility (False when remote, True when local) — existing deployments require no config changes.

Execution mode matrix

SAM3_EXEC_MODE WORKFLOWS_STEP_EXECUTION_MODE Behavior
remote any Inference proxy (unchanged)
local local Local model execution (unchanged)
local remote NEW: SDK-based remote via HOSTED_CORE_MODEL_URL

Files changed

File Change
inference/core/env.py Add SAM3_FINE_TUNED_MODELS_ENABLED
inference/core/interfaces/http/http_api.py Decouple fine-tuned check from SAM3_EXEC_MODE
inference_sdk/http/client.py Add 6 SAM3 SDK methods (3 sync + 3 async)
segment_anything3/v1.py Three-way run() branching + run_remotely()
segment_anything3/v2.py Three-way run() branching + run_remotely()
segment_anything3/v3.py Three-way run() branching + run_remotely()

Test plan

  • Verify SAM3_EXEC_MODE=remote still routes through inference proxy (backward compat)
  • Verify SAM3_EXEC_MODE=local + WORKFLOWS_STEP_EXECUTION_MODE=local runs locally (unchanged)
  • Verify SAM3_EXEC_MODE=local + WORKFLOWS_STEP_EXECUTION_MODE=remote uses SDK client with HOSTED_CORE_MODEL_URL
  • Verify SAM3_FINE_TUNED_MODELS_ENABLED=false blocks non-sam3/ model IDs on /sam3/concept_segment
  • Verify SAM3_FINE_TUNED_MODELS_ENABLED defaults to false when SAM3_EXEC_MODE=remote (no config change needed for existing deployments)
  • Verify remote processing time is collected via X-Processing-Time header when using SDK path

🤖 Generated with Claude Code

SAM3 workflow blocks previously always used the API inference proxy for
remote execution, regardless of configuration. This change decouples
the workflow step execution mode from SAM3_EXEC_MODE, enabling SAM3
blocks to use the standard SDK-based remote execution pattern used by
all other model blocks when WORKFLOWS_STEP_EXECUTION_MODE=remote.

Also adds SAM3 SDK client methods (concept_segment, visual_segment,
embed_image) and a new SAM3_FINE_TUNED_MODELS_ENABLED env var to
control fine-tuned model access independently of execution mode.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
@hansent hansent merged commit 2387be5 into main Feb 26, 2026
57 checks passed
@hansent hansent deleted the feat/sam3-workflow-remote-exec-via-sdk branch February 26, 2026 19:22
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants