ci: run SQL E2E over all three transports (subprocess + http + unix)#1
Merged
Conversation
The integration job previously ran only the subprocess/stdio transport. Add http and unix coverage so the full es_search suite is exercised over every transport the vgi extension supports. - main.go: wire a --unix <path> flag (RunUnix); --http already existed. - ci/run-integration.sh: parameterize by TRANSPORT (subprocess|http|unix). For http, start `--http`, parse PORT:<n>, set the bare http://host:port LOCATION, and inject `INSTALL httpfs FROM core; LOAD httpfs;` after each `LOAD vgi;` (the worker-RPC rides DuckDB's httpfs client). For unix, start `--unix`, wait for UNIX:<path> + the socket, set the unix:// LOCATION. Out-of-band workers are trap-killed with the exit code preserved, and a silent-skip guard fails any http leg the runner would otherwise skip into a fake green. - ci.yml: make the integration job a transport:[subprocess,http,unix] matrix, keeping the OpenSearch service container + healthcheck + seed for each leg. - ci/README.md: document the transport matrix, httpfs injection, and the silent-skip guard. No cursor fix needed: es_search already externalizes its scan position (PIT id + search_after sort values are plain gob-encodable state, one page per Process tick), so the streaming table function resumes correctly over HTTP's stateless continuation token. Verified GREEN on all three transports against a live OpenSearch 2.17 (subprocess/unix: 23 assertions, http: 25 with the injected httpfs loads). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
The integration job previously ran only the subprocess/stdio transport. This adds http and unix coverage so the full
es_searchsuite runs over every transport thevgiextension supports, against the live OpenSearch 2.17 service container on every leg.What changed
--unix <path>flag (RunUnix);--httpalready existed.TRANSPORT(subprocess|http|unix). http parsesPORT:<n>and uses a barehttp://host:portLOCATION + injectsINSTALL httpfs FROM core; LOAD httpfs;after eachLOAD vgi;; unix waits forUNIX:<path>+ the socket. Out-of-band workers are trap-killed (exit code preserved), plus a silent-skip guard fails any http leg the runner would otherwise skip into a fake green.transport:[subprocess,http,unix]matrix; the OpenSearch service container + healthcheck + seed run for each leg.Cursor: no fix needed
es_searchalready externalizes its scan position — the PIT id + the last hit'ssearch_aftersort values are plain gob-encodable scan-state fields, emitted one page perProcesstick. The framework snapshots them into HTTP's stateless continuation token, so the streaming table function resumes correctly over HTTP (noDone-only / restart-from-row-0 loop).Verified locally (live OpenSearch 2.17)
🤖 Generated with Claude Code