Skip to content

Latest commit

 

History

History
223 lines (162 loc) · 10 KB

CLAUDE.md

File metadata and controls

223 lines (162 loc) · 10 KB

Flash Worker (worker-flash)

Auto-generated by /analyze-repos on 2026-02-22. Manual edits will be overwritten on next analysis.

Project Overview

worker-flash (v1.0.1), a RunPod Serverless worker that executes @remote functions and classes inside GPU/CPU containers. Receives serialized FunctionRequest (cloudpickle + base64), installs dependencies on-the-fly, executes user code, returns FunctionResponse. Two modes: Live Serverless (dynamic code per-request) and Flash Deployed Apps (pre-deployed artifacts). Dual entry points: RunPod serverless handler (QB) and FastAPI LB handler. Python 3.10-3.14, base images: pytorch 2.9.1 (GPU), python:3.12-slim (CPU).

Architecture

Key Abstractions

  1. RemoteExecutor (src/remote_executor.py:30) -- Central orchestrator using composition pattern. Coordinates dependency installation, execution routing (function vs class), log streaming.
  2. DependencyInstaller (src/dependency_installer.py:12) -- Python (uv/pip) and system (apt/nala) package installation with env-aware config (Docker vs local).
  3. FunctionExecutor (src/function_executor.py:12) -- Sync/async function execution via exec() with stdout/stderr/log capture.
  4. ClassExecutor (src/class_executor.py:14) -- Class instantiation, method dispatch, instance persistence in unbounded registry.
  5. CacheSyncManager (src/cache_sync_manager.py:12) -- Bidirectional cache sync between local /root/.cache and network volume tarballs.

Entry Points

  • QB Handler (src/handler.py) -- RunPod serverless entry via runpod.serverless.start(). Receives jobs from queue, delegates to RemoteExecutor.
  • LB Handler (src/lb_handler.py) -- FastAPI app served by uvicorn. HTTP endpoints for load-balanced requests.

Module Structure

src/
  handler.py                # RunPod serverless entry point (QB mode)
  lb_handler.py             # FastAPI Load Balancer entry point (LB mode)
  remote_executor.py        # Central orchestrator (composition: DependencyInstaller + Executors)
  function_executor.py      # Function execution with stdout/stderr/log capture
  class_executor.py         # Class instantiation, method dispatch, instance persistence
  dependency_installer.py   # Python (uv/pip) + system (apt/nala) package installation
  serialization_utils.py    # CloudPickle + base64 encode/decode utilities
  subprocess_utils.py       # Centralized subprocess with logging via run_logged_subprocess
  log_streamer.py           # Thread-safe log buffering for captured output
  logger.py                 # Logging configuration
  cache_sync_manager.py     # Network volume <-> local cache bidirectional sync
  manifest_reconciliation.py # TTL-based flash_manifest.json refresh
  unpack_volume.py          # Build artifact extraction from network volume
  constants.py              # Named constants (NAMESPACE, LARGE_SYSTEM_PACKAGES)

Public API Surface

No public Python API. The worker exposes its interface through:

  • QB protocol: FunctionRequest in, FunctionResponse out (via runpod.serverless.start())
  • LB protocol: HTTP endpoints mapped from @remote(method=..., path=...) decorators
  • Protocol definitions: imported from runpod_flash.protos.remote_execution

Environment Variables

Variable Required Purpose
RUNPOD_API_KEY Yes RunPod API authentication
RUNPOD_ENDPOINT_ID Auto Workspace isolation (set by RunPod)
FLASH_ENDPOINT_TYPE Auto QB vs LB mode selection
FLASH_RESOURCE_NAME Auto Resource identification
FLASH_MAIN_FILE Deploy Entry file for deployed apps
FLASH_APP_VARIABLE Deploy App variable name for deployed apps
FLASH_BUILD_ARTIFACT_PATH Deploy Path to build artifacts
FLASH_DISABLE_UNPACK Deploy Skip artifact extraction
LOG_LEVEL No Logging verbosity (default: INFO)
HF_HUB_ENABLE_HF_TRANSFER No Accelerated HuggingFace downloads
HF_HOME No HuggingFace cache location (default: /hf-cache)
HF_TOKEN No Auth for private/gated HF models

Cross-Repo Dependencies

Depends On

  • flash (runpod_flash package) -- imports FunctionRequest, FunctionResponse, RemoteExecutorStub, ServiceRegistry, StateManagerClient from runpod_flash.protos and runpod_flash.runtime.
  • runpod-python (runpod package) -- runpod.serverless.start() for QB mode handler registration.

Depended On By

  • flash -- builds Docker images that run this worker. Docker image names hardcoded in flash's core/resources/constants.py.
  • flash-examples -- indirectly; user code runs inside this worker.

Interface Contracts

  • FunctionRequest/FunctionResponse protocol -- the primary contract between flash and flash-worker. Any field changes require coordinated releases across both repos.
  • FunctionResponse as generic envelope -- subprocess_utils.py reuses FunctionResponse for subprocess results (coupling to protocol schema).
  • Docker image tags -- flash deploys specific image tags; worker image names/tags must match flash's constants.py.
  • Manifest schema -- manifest_reconciliation.py parses flash_manifest.json generated by flash's build step.

Dependency Chain

flash-examples --> flash (runpod_flash) --> runpod-python (runpod)
flash-worker   --> flash (protocols)    --> runpod-python (serverless.start)

Known Drift

  • Python version: runpod-python supports 3.8+, flash-worker requires 3.10+
  • Coverage thresholds: runpod-python 90%, flash 35%, flash-worker 35%

Development Commands

Setup

make setup                    # Initialize project and sync dependencies
make dev                      # Install all development dependencies
uv sync --all-groups          # Alternative: sync all dependency groups

Testing

make test                     # Run all tests
make test-unit                # Unit tests only
make test-integration         # Integration tests only
make test-coverage            # Tests with coverage report
make test-fast                # Tests with fail-fast mode
make test-handler             # Test handler locally with all test_*.json files (matches CI)

Quality

make quality-check            # REQUIRED BEFORE ALL COMMITS (format + lint + tests + coverage)
make lint                     # Ruff linter
make lint-fix                 # Auto-fix lint issues
make format                   # Ruff formatter
make format-check             # Check formatting
make typecheck                # mypy type checking

Build and Deploy

make build                    # Build GPU Docker image (single-platform, loads locally)
make build-cpu                # Build CPU-only Docker image
make build-lb                 # Build Load Balancer image
make build-wip                # Multi-platform build, pushes to Docker Hub (NOT visible in docker images)
make smoketest                # Test built images locally
make smoketest-lb             # Test LB images locally

Code Intelligence

make index                    # Rebuild MCP code intelligence index

Code Health

High Severity

  • None critical. Architecture is clean with composition pattern.

Medium Severity

  • ExecuteFunction is 150 lines (remote_executor.py:58) -- main execution path, consider extracting sub-methods
  • sync_to_volume is 170 lines (cache_sync_manager.py:103) -- complex tarball sync logic
  • RemoteExecutor only tested indirectly -- direct unit tests needed

Low Severity

  • _UNPACKED flag set before extraction completes (unpack_volume.py:130) -- race condition if extraction fails
  • Off-by-one retry sleep in unpack_volume.py:147 -- first retry has no backoff
  • Unbounded instance registry in class_executor.py:19 -- memory leak for long-running workers with many unique classes
  • No mutable defaults, no bare except, no print(), no TODOs, no commented-out code -- clean

Testing

Structure

  • tests/unit/ -- component-level testing with mocking
  • tests/integration/ -- end-to-end workflow testing
  • tests/conftest.py -- shared fixtures and mock objects
  • src/tests/ -- handler test JSON files for make test-handler
  • Coverage threshold: 35% minimum

Coverage Gaps

File Coverage Risk
log_streamer.py None MEDIUM -- thread-safe buffering untested
subprocess_utils.py None HIGH -- all subprocess calls flow through here
logger.py None LOW
remote_executor.py Indirect only HIGH -- central orchestrator needs direct tests

Patterns

  • Arrange-Act-Assert in all tests
  • Mock external services (RunPod API, file system for volumes)
  • Use make test-handler for handler validation (matches CI behavior)
  • Do NOT run individual test files manually with RUNPOD_TEST_INPUT -- use make test-handler

Docker Testing

  • Docker containers should never reference src/ paths directly
  • Use make build for local testing (visible in docker images)
  • Use make build-wip only for pushing to Docker Hub (NOT visible locally)

Code Intelligence (MCP)

Server: worker-flash-code-intel

Always prefer MCP tools over Grep/Glob for semantic code searches.

Tool Use Case Example
find_symbol(symbol) Find classes, functions, methods by name find_symbol("RemoteExecutor")
list_classes() Get all classes in codebase Exploring class hierarchy
get_class_interface(class_name) Inspect class methods/properties get_class_interface("DependencyInstaller")
list_file_symbols(file_path) View file structure list_file_symbols("src/handler.py")
find_by_decorator(decorator) Find decorated items find_by_decorator("remote")

Indexed codebase includes:

  • Project source (src/) -- all 83 worker-flash symbols
  • runpod_flash dependency -- all 552 protocol definitions, resources, and core components

When to use Grep instead: Content searches (error messages, string literals, log statements, env var usage).

Rebuild index when dependencies change: make index

Last analyzed: 2026-02-22