Skip to content

v1.12.0 — Reliability and compatibility release#8106

Merged
skyfallsin merged 91 commits into
masterfrom
sync/jo-browser-jul17
Jul 19, 2026
Merged

v1.12.0 — Reliability and compatibility release#8106
skyfallsin merged 91 commits into
masterfrom
sync/jo-browser-jul17

Conversation

@skyfallsin

@skyfallsin skyfallsin commented Jul 18, 2026

Copy link
Copy Markdown
Contributor

v1.12.0 — Reliability and compatibility release

This release upstreams reliability work from running Camofox in production at Jo, alongside community fixes.

What changed

Linux and Docker

  • Await VirtualDisplay.get() so launches no longer receive DISPLAY=[object Promise].
  • Remove the fixed context viewport that triggered viewport.isMobile Juggler errors.
  • Default Xvfb to 1280×720×24.
  • Pin camoufox-js to 0.10.2 for reproducible installs.
  • Improve Docker build and dependency installation.

Fixes #6807 and related reports #8043, #8042, #5916, #5812, #5643, #5408, and #5166.

Browser reliability

  • Scope process cleanup to browsers owned by the current server.
  • Prevent close/relaunch races and clean up orphaned browser processes.
  • Improve idle-shutdown health checks and memory-pressure recovery.
  • Harden popup, stale-tab, and concurrent tab lifecycle handling.
  • Improve multi-instance tab routing, proxy rotation, and Google prewarming.
  • Bound click fallback lookups and return an actionable 422 instead of a 30-second 500.
  • Skip unstable captcha and bot-detection iframes during ref construction.
  • Normalize browser failures into stable status codes and recovery hints.

Persistence and startup

OpenClaw and dependencies

  • Fix factory tool registration so newer OpenClaw versions expose all camofox_* tools. Fixes Toolset not available to agent #5437.
  • Restore the generated plugin.js runtime artifact.
  • Fix the E2E test command and test teardown.
  • Update qs to 6.15.2 and ua-parser-js to 2.0.10, including its Client Hints ReDoS fix.

Community PRs

Validation

  • 707 unit tests
  • 28 plugin tests
  • 67 real-browser E2E tests
  • Fresh packed-plugin install on OpenClaw 2026.4.25, with all 11 tools loaded and a successful live tab open

- createTabState gets navigateAbort AbortController field
- navigateCurrentPage races page.goto() with abort signal
- DELETE /tabs/:tabId calls abort() before safePageClose, cancelling
  in-flight navigation immediately instead of waiting 30s timeout
- Add camofox_snapshot_bytes histogram for snapshot size tracking
- 6 new abort tests passing
…tion, fixes

Backported from camofox-browser v1.5.2:

- Lazy Prometheus metrics: off by default, enable with PROMETHEUS_ENABLED=1.
  Noop stubs when disabled — no prom-client import overhead. Fixes OpenClaw
  install blocking on optional dep.
- Extract extractPageImages to lib/images.js (OpenClaw scanner isolation)
- Extract actionFromReq + classifyError to lib/request-utils.js
- buildRefs timeout leak: clearTimeout in both success and error paths
- YT transcript session cleanup: close phantom __yt_transcript__ context
  when all tabs are gone
- Session cleanup on empty tab groups: tab reaper now closes sessions with
  zero remaining tabs + triggers browser idle shutdown
- Horizontal scroll: mouse.wheel now supports left/right directions
- Delete tab/group: accept userId from query string or body
- plugin.ts: proc.kill() on startup timeout, screenshot Content-Type guard
  (returns text error instead of base64-encoded JSON on non-image response)
- 2 new test files: sessionCleanup.test.js, updated screenshotToolResult.test.js
Race 1 (YT transcript): browserTranscript cleanup counted tabGroups which
was always 0 (YT pages aren't registered in tabGroups). First concurrent
request to finish would close the context, killing other requests' pages.
Fix: use context.pages() to check actual live pages before closing.

Race 2 (tab reaper / session expiry): context.close() fired without await,
then session deleted from map. A request that already got the session ref
(between getSession return and newPage) would use a closing context.
Fix: set session._closing = true before teardown; getSession() treats
_closing sessions as dead and creates a new one.

Both fixes also applied to the session expiry timer (same pattern).
16 unit tests added covering _closing flag, getSession skip, YT concurrent
cleanup, and session expiry sentinel.
When a click/navigate/open_url times out (e.g., Cloudflare holding the
connection for 30s), the Decodo proxy session becomes poisoned — all
subsequent requests through the same BrowserContext hang. Previously,
timeouts only tracked per-tab consecutive counts without destroying the
session, so recovery tabs inherited the poisoned proxy.

Now navigation-related timeouts (click, navigate, open_url) destroy the
entire user session, so the next request gets a fresh BrowserContext
with a fresh proxy session. Non-navigation timeouts (type, scroll) still
use the per-tab consecutive timeout tracking.

Root cause: WWDC cron clicked a MacRumors Cloudflare-protected link,
the proxy session got poisoned, recovery tab's prewarm to google.com
also timed out on the same session, browser agent gave up entirely.
…act, crash reporter, CI improvements

New features from upstream:
- OpenAPI spec auto-generated from JSDoc + swagger-stripey docs at /docs (#78)
- Opt-in session tracing via Playwright (#68)
- Structured extract endpoint with JSON Schema + x-ref hints (#70)
- session:destroying event for persistence checkpoint (#75)
- Anonymized crash/frustration reporter with GitHub App auth
- GitHub Actions CI workflow for unit tests

Also: compressed fox.png, added README badges, GitHub Sponsors funding link
Upstream refactor moves crash reporter credentials to camofox.config.json.
Watchdog now includes active sessions/tabs/URLs in event-loop stall reports.

jo-browser config uses its own GitHub App (Jo Browser Crash Reporter,
app ID 3505356) targeting jo-inc/jo-browser.
Drifts >120s are OS sleep/hibernate, not real event-loop stalls.
Skip them and suppress the next 5 ticks to avoid post-wake jitter
false positives (like the 6s stall in #83).

Fixes #82, #83
# Conflicts:
#	lib/reporter.js
#	openapi.json
Merge artifact from v1.7.4 sync introduced duplicate 'let lastHeapUsed'
at lines 976 and 1004 in reporter.js. Node.js SyntaxError on startup
caused all 3 Fly machines to crash-loop and exhaust retries.

All machines have been stopped since ~05:11 UTC, breaking browser-dependent
cron jobs and use_browser tool calls.
deploy.yml had zero dependency on CI — deployed immediately on push
regardless of whether tests passed. This is how the duplicate let
declaration shipped to prod and took down all 3 machines for 14+ hours.

Now deploy requires: syntax check of all JS files + reporter unit tests.
- Install @sentry/node, init from SENTRY_DSN env var
- lib/sentry.js: isolated module (no process.env, scanner-safe)
- lib/config.js: add sentryDsn config key
- server.js: capture 500s in sendError, uncaughtException, unhandledRejection
- setupExpressErrorHandler after all routes, flush on graceful shutdown
- fly.toml: add SENTRY_DSN for jo-browser project
- StaleRefsError (422) filtered out via beforeSend
Root cause: .internal DNS bypassed Fly Proxy — all traffic pinned to one
machine. 11 concurrent BrowserContexts × ~300MB = 3.3GB on 4GB → OOM.

Changes:
- MAX_SESSIONS=10 (was 50 default — way too high for 4GB)
- concurrency type connections soft=25 (was requests soft=8 — wrong metric)
- Memory admission control: 503 + fly-replay at >90% RAM
- Memory pressure eviction: 30s loop, multi-evict until <80% RAM
- Session overflow redirect: fly-replay when >ceil(MAX/3) sessions
- tabNotFoundResponse: 410 Gone for local tabs after browser restart
- SSL error handling: 502 recoverable:false for SEC_ERROR
- Sentry: skip capture for intentional 503s
Dead context, timeouts, proxy errors, nav aborts, and tab lifecycle
errors are expected operational behavior — not bugs. They were flooding
Sentry with noise issues, hiding real problems.

Added classifyError patterns for NS_ERROR_ABORT, tab deleted navigation
abort, and page crash. Expanded beforeSend to drop operational errors.
…, timeout tuning

Fix 1: Drain tab locks BEFORE closing browser context in closeSession().
Queued operations now get clean 'Tab destroyed' (410) instead of cascading
'Target page closed' (500) errors. Dead-context errors in handleRouteError
now return 503 with session_expired code — the root cause (proxy/timeout)
is still reported, but cascade errors no longer flood Sentry.

Fix 2: Transparent retry on proxy/timeout errors during navigation.
When page.goto fails with NS_ERROR_PROXY_FORBIDDEN or timeout, destroy
the session, get a fresh proxy, and retry once before failing to caller.
Applied to POST /tabs (create with URL), POST /tabs/:tabId/navigate, and
POST /tabs/open. Uses existing context rotation pattern.

Fix 3: Increase goBack timeout from 10s to 20s.
Back navigation uses browser cache and shouldn't need aggressive timeouts.
10s was too short for complex SPA re-renders.
# Conflicts:
#	camofox.config.json
#	package-lock.json
#	server.js
@skyfallsin skyfallsin changed the title Reliability and compatibility release v1.11.2 — Reliability and compatibility release Jul 19, 2026
@skyfallsin skyfallsin changed the title v1.11.2 — Reliability and compatibility release v1.12.0 — Reliability and compatibility release Jul 19, 2026
@skyfallsin
skyfallsin merged commit b00cf21 into master Jul 19, 2026
2 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment