Skip to content

Commit 383ded9

Browse files
committed
feat: add portable CLI binaries for multi-platform Supabase CLI distribution
This commit implements portable, self-contained PostgreSQL binaries for the Supabase CLI across macOS (ARM), Linux (x64), and Linux (ARM64), along with automated CI/CD workflows for building and releasing these artifacts. The Supabase CLI needs to ship PostgreSQL binaries that work on user machines without requiring Nix or other system dependencies. This means extracting the actual binaries from Nix's wrapper scripts, bundling all necessary shared libraries, and patching them to use relative paths instead of hardcoded Nix store paths. A `variant` parameter was added to the postgres build pipeline to distinguish between "full" (all extensions) and "cli" (minimal extensions for Supabase CLI). The `cliExtensions` list contains 6 extensions required for running Supabase migrations: supautils, pg_graphql, pgsodium, supabase_vault, pg_net, and pg_cron. Built-in extensions (uuid-ossp, pgcrypto, pg_stat_statements) are included automatically with PostgreSQL. `makeOurPostgresPkgs`/`makePostgresBin` were modified to accept this parameter. A new `psql_17_cli` package is created using `variant = "cli"`, while the full extension set is preserved for base packages (`psql_15`, `psql_17`, `psql_orioledb-17`). The portable CLI variant (`psql_17_cli_portable`) includes 6 extensions for migration support while maintaining a significantly smaller size than the full build. The implementation in `nix/packages/postgres-portable.nix` extracts binaries from `psql_17_cli` using a `resolve_binary()` function that follows wrapper layers to find the actual ELF/Mach-O binaries behind Nix's environment setup scripts. All Nix-provided libraries (ICU, readline, zlib, etc.) are bundled while excluding system libraries (`libc`, `libpthread`, `libm`, `glibc`, `libdl`) that must come from the host. This distinction is critical: Linux bundles must exclude glibc due to kernel ABI dependencies, while macOS can include more libs due to its different linking model. Dependency resolution runs multiple passes to catch transitive deps (e.g., ICU → charset → etc.). Platform-specific patching is applied: Linux binaries use the system interpreter (`/lib64/ld-linux-*.so.2`) and `$ORIGIN`-based RPATHs, while macOS binaries use `@rpath` with `@executable_path`. Wrapper scripts set `LD_LIBRARY_PATH` (Linux) or `DYLD_LIBRARY_PATH` (macOS) to find bundled libraries. The bundle includes PostgreSQL config templates (`postgresql.conf`, `pg_hba.conf`, `pg_ident.conf`) tailored for CLI usage with minimal local dev settings, plus the complete Supabase migration script (`migrate.sh`) with all init-scripts and migration SQL files (55 files, 236KB). A GitHub Actions workflow builds portable binaries across all three platforms using a matrix strategy. Each build runs automated portability checks that verify no `/nix/store` references remain, validate RPATH configuration, confirm transitive dependencies are bundled, ensure system libraries are NOT bundled, and check wrapper scripts contain proper library path setup. Post-build testing validates binaries work without Nix (`postgres --version`, `psql --version`). On tagged releases (`v*-cli`), the workflow creates GitHub releases with tarball artifacts and checksums. The test infrastructure needed significant changes to support variants with different extension sets. An `isCliVariant` parameter was added to `makeCheckHarness`, and the hardcoded `shared_preload_libraries` list in `postgresql.conf.in` was replaced with a `@PRELOAD_LIBRARIES@` placeholder. A `generatePreloadLibs` script now parses `receipt.json` at test time and dynamically builds the preload list based on available extensions, removing the previous timescaledb removal hack for OrioleDB. For CLI variant tests, `prime.sql` is skipped (supautils is preloaded via `shared_preload_libraries`), 35 extension-specific tests are filtered out, and migrations tests are skipped as they may depend on extensions.
1 parent 28fe8c1 commit 383ded9

File tree

10 files changed

+1121
-36
lines changed

10 files changed

+1121
-36
lines changed

.github/workflows/cli-release.yml

Lines changed: 243 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,243 @@
1+
name: CLI Release
2+
3+
on:
4+
push:
5+
tags:
6+
- "v*-cli"
7+
pull_request:
8+
workflow_dispatch:
9+
inputs:
10+
version:
11+
description: "Version tag (e.g., v1.0.0-cli)"
12+
required: true
13+
type: string
14+
15+
jobs:
16+
build-native:
17+
name: Build ${{ matrix.arch }}
18+
runs-on: ${{ matrix.runner }}
19+
strategy:
20+
fail-fast: false
21+
matrix:
22+
include:
23+
- system: aarch64-darwin
24+
runner: macos-14
25+
arch: darwin-arm64
26+
- system: x86_64-linux
27+
runner: ubuntu-latest
28+
arch: linux-x64
29+
- system: aarch64-linux
30+
runner: ubuntu-24.04-arm
31+
arch: linux-arm64
32+
steps:
33+
- name: Checkout repository
34+
uses: supabase/postgres/.github/actions/shared-checkout@HEAD
35+
36+
- name: Install Nix
37+
uses: ./.github/actions/nix-install-ephemeral
38+
39+
- name: Run portability check
40+
run: |
41+
nix build .#checks.${{ matrix.system }}.psql_17_cli_portable --system ${{ matrix.system }} -L --accept-flake-config
42+
43+
- name: Build portable CLI bundle
44+
run: |
45+
nix build .#psql_17_cli_portable --system ${{ matrix.system }} -L --accept-flake-config
46+
47+
- name: Determine version
48+
id: version
49+
run: |
50+
if [ "${{ github.event_name }}" = "workflow_dispatch" ]; then
51+
VERSION="${{ github.event.inputs.version }}"
52+
else
53+
VERSION="${{ github.ref_name }}"
54+
fi
55+
# Sanitize version by replacing slashes with dashes
56+
VERSION="${VERSION//\//-}"
57+
echo "version=${VERSION}" >> "$GITHUB_OUTPUT"
58+
59+
- name: Prepare build directory
60+
run: |
61+
VERSION="${{ steps.version.outputs.version }}"
62+
ARCH="${{ matrix.arch }}"
63+
BUILD_DIR="supabase-postgres-${VERSION}-${ARCH}"
64+
65+
mkdir -p "${BUILD_DIR}"
66+
shopt -s dotglob
67+
cp -rL result/* "${BUILD_DIR}/"
68+
shopt -u dotglob
69+
70+
- name: Upload artifacts
71+
uses: actions/upload-artifact@v4
72+
with:
73+
name: supabase-postgres-${{ matrix.arch }}
74+
path: supabase-postgres-${{ steps.version.outputs.version }}-${{ matrix.arch }}
75+
retention-days: 90
76+
include-hidden-files: true
77+
78+
test-portability:
79+
name: ${{ matrix.arch }} without Nix
80+
needs: [build-native]
81+
runs-on: ${{ matrix.runner }}
82+
strategy:
83+
fail-fast: false
84+
matrix:
85+
include:
86+
- runner: macos-14
87+
arch: darwin-arm64
88+
- runner: ubuntu-latest
89+
arch: linux-x64
90+
- runner: ubuntu-24.04-arm
91+
arch: linux-arm64
92+
steps:
93+
- name: Download artifact
94+
uses: actions/download-artifact@v4
95+
with:
96+
name: supabase-postgres-${{ matrix.arch }}
97+
path: .
98+
99+
- name: Test binaries work without Nix
100+
run: |
101+
cd bin
102+
find . -maxdepth 1 -type f -exec chmod +x {} +
103+
ls -la
104+
105+
echo "Testing postgres --version..."
106+
./postgres --version
107+
108+
echo "Testing pg_config --version..."
109+
./pg_config --version
110+
111+
echo "Testing psql --version..."
112+
./psql --version
113+
114+
echo "Testing initdb --version..."
115+
./initdb --version
116+
117+
- name: Test migrations with CLI variant
118+
run: |
119+
set -e
120+
121+
# Create getkey script for pgsodium and vault
122+
cat > pgsodium_getkey.sh << 'EOF'
123+
#!/bin/bash
124+
echo "0000000000000000000000000000000000000000000000000000000000000000"
125+
EOF
126+
chmod +x pgsodium_getkey.sh
127+
GETKEY_SCRIPT="$(pwd)/pgsodium_getkey.sh"
128+
129+
# Initialize test database
130+
PGDATA="$(pwd)/pgdata-test"
131+
./bin/initdb -D "$PGDATA" -U supabase_admin --no-instructions
132+
133+
# Copy CLI config and add pgsodium/vault getkey scripts
134+
cp share/supabase-cli/config/postgresql.conf.template "$PGDATA/postgresql.conf"
135+
cat >> "$PGDATA/postgresql.conf" << EOF
136+
137+
# pgsodium and vault configuration for testing
138+
pgsodium.getkey_script = '$GETKEY_SCRIPT'
139+
vault.getkey_script = '$GETKEY_SCRIPT'
140+
EOF
141+
142+
# Start PostgreSQL
143+
./bin/postgres -D "$PGDATA" -p 54322 > postgres.log 2>&1 &
144+
PG_PID=$!
145+
146+
# Wait for PostgreSQL to be ready
147+
for i in {1..30}; do
148+
if ./bin/pg_isready -p 54322 -h 127.0.0.1 -U supabase_admin > /dev/null 2>&1; then
149+
echo "PostgreSQL is ready"
150+
break
151+
fi
152+
if [ $i -eq 30 ]; then
153+
echo "PostgreSQL failed to start"
154+
cat postgres.log
155+
exit 1
156+
fi
157+
sleep 1
158+
done
159+
160+
# Run migrations
161+
cd share/supabase-cli/migrations
162+
chmod +x migrate.sh
163+
echo "=========================================="
164+
echo "Running migrations..."
165+
echo "=========================================="
166+
PATH="$(pwd)/../../../bin:$PATH" \
167+
POSTGRES_PASSWORD=postgres \
168+
POSTGRES_PORT=54322 \
169+
POSTGRES_DB=postgres \
170+
POSTGRES_USER=supabase_admin \
171+
./migrate.sh 2>&1 | tee migration.log
172+
MIGRATION_STATUS=${PIPESTATUS[0]}
173+
174+
echo ""
175+
echo "=========================================="
176+
echo "Migration output complete"
177+
echo "=========================================="
178+
179+
# Check migration results (allow pgbouncer migration to fail as it's not in CLI variant)
180+
if [ $MIGRATION_STATUS -ne 0 ]; then
181+
if grep -q "pgbouncer" migration.log; then
182+
echo "Migration failed on expected pgbouncer error (not in CLI variant) - continuing"
183+
else
184+
echo "Migrations failed with unexpected errors"
185+
exit 1
186+
fi
187+
fi
188+
189+
# Verify extensions are installed
190+
cd ../../..
191+
./bin/psql -p 54322 -h 127.0.0.1 -U supabase_admin -d postgres -c "\dx" | tee extensions.log
192+
193+
# Check for required extensions
194+
for ext in pg_graphql pgcrypto uuid-ossp supabase_vault; do
195+
if ! grep -q "$ext" extensions.log; then
196+
echo "Required extension $ext not found"
197+
exit 1
198+
fi
199+
done
200+
201+
# Stop PostgreSQL
202+
./bin/pg_ctl -D "$PGDATA" stop -m fast
203+
204+
echo "Migration test completed successfully"
205+
206+
release:
207+
name: Create GitHub Release
208+
needs: [build-native, test-portability]
209+
runs-on: ubuntu-latest
210+
if: startsWith(github.ref, 'refs/tags/')
211+
permissions:
212+
contents: write
213+
steps:
214+
- name: Download all artifacts
215+
uses: actions/download-artifact@v4
216+
with:
217+
path: artifacts
218+
219+
- name: Prepare release assets
220+
run: |
221+
mkdir -p release
222+
223+
# Create tarballs from each artifact directory
224+
for dir in artifacts/*/supabase-postgres-*; do
225+
if [ -d "$dir" ]; then
226+
basename=$(basename "$dir")
227+
tarball="release/${basename}.tar.gz"
228+
229+
echo "Creating tarball: ${tarball}"
230+
tar -czhf "${tarball}" -C "$(dirname "$dir")" "$(basename "$dir")"
231+
232+
# Generate checksum
233+
sha256sum "${tarball}" > "${tarball}.sha256"
234+
fi
235+
done
236+
237+
ls -lh release/
238+
239+
- name: Create Release
240+
uses: softprops/action-gh-release@v2
241+
with:
242+
files: release/*
243+
generate_release_notes: true

0 commit comments

Comments
 (0)