Skip to content

feat: add wasm web support - #138

Merged
jhen0409 merged 6 commits into
mainfrom
codex/add-wasm-web-support
Jun 3, 2026
Merged

feat: add wasm web support#138
jhen0409 merged 6 commits into
mainfrom
codex/add-wasm-web-support

Conversation

@jhen0409

@jhen0409 jhen0409 commented Jun 2, 2026

Copy link
Copy Markdown
Member

Summary

Adds browser WebAssembly support for llama.node through a new @fugood/node-llama-wasm package. The package keeps the high-level llama.node API shape for browser use, loads GGUF models by URL by default, supports session save/load with buffers and URLs, and includes CPU plus WebGPU WASM artifacts.

The CPU path now auto-selects the pthread artifact on cross-origin isolated pages and defaults n_threads to min(4, navigator.hardwareConcurrency). The WASM runtime runs in a dedicated worker by default so model loading, tokenize/detokenize, completion, state, embedding/rerank, multimodal staging, and bench calls do not block the UI thread.

Validation

  • node --check packages/node-llama-wasm/index.js
  • node --check packages/node-llama-wasm/worker.js
  • node --check scripts/build-wasm-package.js
  • node --check scripts/build-wasm-docker.js
  • npm run typecheck
  • npm pack --dry-run ./packages/node-llama-wasm
  • Browser HTML harness with Gemma 3 270M CPU threaded: load/tokenize/detokenize/state/completion/tool formatting passed
  • Browser HTML harness with Gemma 3 270M WebGPU: load/tokenize/detokenize/state/completion/tool formatting passed
  • Apple Silicon Docker smoke: npm run build-wasm-docker selected emscripten/emsdk:4.0.14-arm64 (linux/arm64) and copied the CPU WASM artifact
  • Browser smoke with the arm64-built CPU artifact: tiny model load, tokenize/detokenize, worker runtime, and bench passed

Mac Docker Benchmark

Clean CPU WASM build on Apple Silicon Docker Desktop with JOBS=4 and isolated Emscripten caches:

  • emscripten/emsdk:4.0.13 with --platform linux/amd64: 1268.44s
  • emscripten/emsdk:4.0.14-arm64 with --platform linux/arm64: 122.51s

The native arm64 image is about 10.4x faster for this build on Mac, so npm run build-wasm-docker now selects 4.0.14-arm64 on arm64 hosts.

Workflow Dispatches

Dispatched .github/workflows/build-release.yml on branch codex/add-wasm-web-support with upload-artifacts=YES and publish=NO:

@jhen0409
jhen0409 marked this pull request as ready for review June 3, 2026 05:14
@jhen0409
jhen0409 merged commit 425b4a9 into main Jun 3, 2026
2 checks passed
@jhen0409
jhen0409 deleted the codex/add-wasm-web-support branch June 3, 2026 05:20
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant