feat: add file watching capabilities with inotify support#324
feat: add file watching capabilities with inotify support#324whoiskatrin wants to merge 23 commits intomainfrom
Conversation
🦋 Changeset detectedLatest commit: a2e6bcc The changes in this PR will be included in the next version bump. This PR includes changesets to release 1 package
Not sure what this means? Click here to learn what changesets are. Click here if you're a maintainer who wants to add another changeset to this PR |
…ing or encoding Co-authored-by: Copilot Autofix powered by AI <62310815+github-advanced-security[bot]@users.noreply.github.com>
Documents the new sandbox.watch() method that enables real-time filesystem monitoring using Linux inotify. Includes: - API reference for watch() method in files.mdx - Comprehensive how-to guide with practical examples - Cross-references between related documentation This documentation covers: - Basic file watching with callbacks - Filtering by file type with glob patterns - Event type filtering (create, modify, delete, rename) - Cancellation with AbortController - Hot reload and log monitoring examples - Best practices and troubleshooting Synced from cloudflare/sandbox-sdk#324
commit: |
🐳 Docker Images PublishedDefault: FROM cloudflare/sandbox:0.0.0-pr-324-95f82e2With Python: FROM cloudflare/sandbox:0.0.0-pr-324-95f82e2-pythonWith OpenCode: FROM cloudflare/sandbox:0.0.0-pr-324-95f82e2-opencodeVersion: Use the 📦 Standalone BinaryFor arbitrary Dockerfiles: COPY --from=cloudflare/sandbox:0.0.0-pr-324-95f82e2 /container-server/sandbox /sandbox
ENTRYPOINT ["/sandbox"]Download via GitHub CLI: gh run download 21896848909 -n sandbox-binaryExtract from Docker: docker run --rm cloudflare/sandbox:0.0.0-pr-324-95f82e2 cat /container-server/sandbox > sandbox && chmod +x sandbox |
…ses. Update tests to validate new event parsing logic and ensure proper handling of inotifywait output.
…efault and custom excludes
Resolve conflicts in container.ts and sandbox.ts: - Keep WatchService import in container.ts - Keep watch methods and merge updated exposePort with token parameter
Streaming responses (like file watch events) were blocking the WebSocket message handler because handleStreamingResponse was awaited. This prevented other messages from being processed while a stream was active. Run streaming response handlers in the background with error logging, allowing the message handler to return immediately and process subsequent messages.
If inotifywait fails to output 'Watches established' within 10 seconds, the function will return and allow the stream to proceed. This prevents indefinite hangs if inotifywait behaves unexpectedly.
For WebSocket streaming, errors were deferred until stream consumption. This caused issues where watchStream() would return successfully even when the server returned an error response. Now requestStream() waits for the first message before returning: - If it's a stream chunk, return the stream (success case) - If it's an error response, throw immediately (error case) This makes WebSocket streaming behavior match HTTP streaming, where errors are thrown immediately rather than deferred.
|
hey! I noticed there are docs for this but doesn't seem like the changes are live yet would love to use this API, any timelines on when this'll get merged? |
|
/bonk resolve conflict and review this PR |
There was a problem hiding this comment.
Merge Conflict Resolved + Code Review
Conflict Resolution
Resolved the merge conflict in packages/sandbox-container/src/core/container.ts. The conflict was between the feature branch's watchHandler and main's ptyWsHandler — both are needed, so I included both in the Dependencies interface, the initialize() method, and the dependency registration block.
Build/typecheck/lint/unit tests all pass after the merge.
Code Review
Overall this is a well-architected feature that follows the three-layer SDK pattern correctly. The SSE streaming approach, DI integration, client pattern, and type safety are all solid. There are however several issues worth addressing, roughly in order of priority.
Issues to Address Before Merge
1. Empty catch blocks in cleanup paths (multiple locations)
Several catch blocks silently swallow errors that could indicate real problems:
-
watch-service.ts:390-393—proc.kill()in thefinallyblock catches everything with just a comment "Process may already be dead". This hides permission errors (EPERM), system failures, etc. At minimum log at debug level and distinguish ESRCH (expected) from other errors. -
watch-service.ts:477— Controller enqueue failure caught silently. Could lose error messages from inotifywait. -
watch-service.ts:563— Same pattern incontinueStderrMonitoring. -
file-watch.ts:168—this.reader.cancel().catch(() => {})in thefinallyblock.
Per AGENTS.md: "silent failures are unacceptable." These should at minimum log at debug level.
2. FileWatch.established() can hang forever with AbortSignal
file-watch.ts:115-129 — If the watch is cancelled via AbortSignal during establishment before receiving the "watching" event, the runLoop throws and calls establishedReject, which is correct. However, if abort fires after established() is called but before the runLoop picks it up, there's a window where the promise could hang. The runLoop does check signal.aborted in the while condition, but if reader.read() is blocked waiting for data, the abort won't interrupt it until data arrives or the stream ends.
Consider adding a timeout or connecting the abort signal to reject the establishment promise directly.
3. isFileWatchSSEEvent type guard is weak
file-watch.ts:175-180 — Only validates that type is a string in the allowed set, but doesn't validate that required fields exist for each event type. For event type events, eventType, path, and isDirectory could all be undefined, leading to runtime errors or silent data loss when passed to onEvent callback.
4. Changeset description is implementation-focused
Per AGENTS.md: changesets should be user-facing with clear benefit and usage example. Current:
feat: add file watching capabilities with inotify support
Should explain benefit and show usage:
Add real-time file watching for detecting filesystem changes as they happen.
Use `sandbox.watch()` to monitor directories for create, modify, delete, and rename events:
const watcher = await sandbox.watch('/app/src', {
recursive: true,
onEvent: (event) => console.log(event.type, event.path),
});
await watcher.stop();
5. Debug logging left in E2E tests
file-watch-workflow.test.ts:204,257,404-408,436,451-452,485-488 — Multiple console.log('[DEBUG]...') statements should be removed before merge. Other E2E tests in the codebase don't have debug logging.
6. stopWatch uses UNKNOWN_ERROR instead of specific code
watch-service.ts:114 — When process.kill() fails in stopWatch(), it returns ErrorCode.UNKNOWN_ERROR. The codebase convention is to use specific error codes (like PROCESS_NOT_FOUND, PORT_NOT_EXPOSED). Consider adding WATCH_STOP_ERROR or reusing WATCH_START_ERROR (since it's a watch lifecycle error).
7. WatchService not integrated into server shutdown
The WatchService spawns inotifywait processes but there's no cleanup hook in packages/sandbox-container/src/server.ts. When the container shuts down, these processes could become zombies. Add watchService.stopAllWatches() to the server cleanup sequence.
Minor Items
-
watch-service.ts:131-137— Comments reference historical context ("Expected case: process already exited"). Per AGENTS.md, describe current behavior instead of referencing what the code prevents. -
E2E tests use fixed delays (
1500ms,3000ms) that could cause flakiness in slower CI environments. ThewatchWithActionshelper's delays should be configurable or event-driven. -
file-watch.ts:189-191— Malformed JSON is silently ignored with no logging. This makes debugging "missing events" very difficult in production. Consider logging at debug level.
Strengths
- Secure glob matching: Character-by-character regex construction with comprehensive ReDoS prevention tests is exemplary.
- State machine design in FileWatch: Clean establishing → active → stopped transitions with proper error handling at each transition.
- WatchService follows established patterns: DI via constructor injection,
ServiceResult<T>return type, proper logger child creation. - Comprehensive unit tests: Good coverage of glob matching, inotify event parsing, and argument building. Uses
createNoOpLogger()as documented. - E2E tests cover real workflows: Creation, modification, deletion, filtering, stop/list APIs, and error cases.
- Proper error codes:
WATCH_NOT_FOUNDandWATCH_START_ERRORwith correct HTTP status mappings. - WebSocket transport improvements: Partial SSE event state tracking across chunks is a correct fix for chunked event parsing.
Co-authored-by: whoiskatrin <whoiskatrin@users.noreply.github.com>
|
Done. Here's a summary of what I did: Merge Conflict ResolutionResolved the single conflict in Verification
Review SummaryThe PR adds a well-architected file watching feature using inotify that follows the three-layer SDK pattern correctly. Key strengths include secure glob matching with ReDoS prevention, a clean state machine in FileWatch, and comprehensive test coverage. 7 issues identified ranging from silent error swallowing in cleanup paths, a potential hang in |
really keen to get this in this week or next week. sorry about the docs, someone got them merged ahead of time |
|
/bonk fix these code review issues that you listed |
Replace empty catch blocks with debug/warn logging throughout WatchService and FileWatch to make failures visible. Fix FileWatch.established() to reject on AbortSignal during establishment, preventing indefinite hangs. Strengthen the SSE event type guard to validate required fields per event type. Add WATCH_STOP_ERROR code, integrate watch cleanup into server shutdown, and rewrite changeset for end users.
|
All 7 review issues have been fixed. Here's a summary of what was changed: Changes Made1. Empty catch blocks in
2.
3. Weak
4. Changeset description (
5. Debug logging in E2E tests (
6.
7. Missing server shutdown cleanup (
|
|
/bonk investigate tests failures and fix if needed |
|
Bonk workflow was cancelled. |
The watchWithActions timeout started at stream creation but blocked for ~6.5s inside the event handler (pre-action delay + file ops + post-action delay), leaving insufficient time to read events in CI. Reset the timeout after actions complete so the full window is available for event collection.
The combined wait time (~18s) approached the 30s Vitest timeout. With excludes filtering most events, only 2-4 arrive, so the high stopAfterEvents threshold was never reached and the test always fell through to the full 12s reader timeout.
There was a problem hiding this comment.
OpenCode Review
This PR implements a comprehensive file watching feature using inotify, adding significant value for development tools and automation workflows. The implementation follows the project's three-layer architecture properly and includes extensive testing.
Strengths
Solid architecture: The three-layer design is well-executed:
- SDK layer (
FileWatchclass) provides high-level lifecycle management - Container layer (
WatchService) handles inotify process spawning and SSE streaming - Proper separation of concerns with modular client pattern
Robust implementation: Key quality aspects are well-handled:
- State machine pattern in
FileWatchprevents race conditions during establishment - Comprehensive error handling and cleanup (process killing, stream cancellation)
- Security-conscious glob matching prevents ReDoS attacks
- Extensive unit tests with private method testing via type assertions
Performance optimizations: Server-side filtering with inotify --exclude prevents unnecessary events from reaching the JavaScript layer, which is much more efficient than client-side filtering.
Technical Considerations
WebSocket streaming complexity: The deferred stream return logic in ws-transport.ts adds significant complexity to handle first-message waiting. While this may be necessary for the WebSocket transport, the promise-within-stream pattern makes error handling intricate. Worth reviewing if there's a simpler approach.
Stream lifecycle management: The activeStreams Map and reader acquisition timing changes in ws-adapter.ts address Bun GC issues, but this fix should be verified to ensure it actually prevents the original problem.
Container shutdown: Watch cleanup during shutdown could potentially hang if inotify processes are unresponsive. Consider adding timeouts to prevent blocking container shutdown.
Path resolution: The relative path handling in watch-handler.ts is functional but could be more robust for edge cases like ./ prefixes or .. components.
Documentation Alignment
The implementation aligns well with the Cloudflare documentation. The API surface matches the documented interfaces and the feature set is complete.
Testing Coverage
Excellent E2E test coverage covers the major workflows. The unit tests for WatchService are comprehensive and include security testing for glob patterns.
Verdict
This is a well-implemented feature that adds significant value. The technical considerations noted above are worth reviewing but don't block the core functionality. The architecture follows project patterns and the testing is thorough.
Looks good to merge - the file watching capability is solid and will be valuable for development tools and automation workflows.
|
@whoiskatrin any rough estimates as to when this'll be in the next release? 🙏🏻 |
|
@parssak this will get in this week |
|
That's great to hear! Tysm |
Adds real-time file watching using Linux's native inotify via inotifywait.
API
Implementation