Conversation
a CLI to package an app directly into app-compose.json (no docker, no registry pull). the embedded binary is decoded at boot into a tmpfs runtime dir and run under a systemd service (Restart=always), so the exact bytes are measured into the compose-hash / RTMR3 and supervised like a container. - `sca new <dir>`: scaffold sca.toml + README + dist/ - `sca build`: emit app-compose.json, print compose-hash / app-id / size decoding uses openssl (the guest busybox userland has no base64); the unit is written to /run/systemd/system (tmpfs); payloads live under .sca_payloads. verified end-to-end on a live CVM incl. crash-restart.
- model: lay out a rootfs/ dir mirroring the CVM fs; build packs the whole tree (deterministic tar.gz + base64) into .sca_rootfs and the boot script extracts it onto / via 'openssl base64 -d | tar -xz -C /' (busybox has no base64 but has openssl/tar/gzip; verified on a CVM) - config switched from TOML to config.json (stdlib json, no version reqs) - new scaffolds rootfs/run/sca/bin/entrypoint.sh and rootfs/etc/systemd/system/sca.service as editable defaults - gzip compression raises the effective payload budget under the 50MiB cap - reproducible build (normalized tar metadata) -> stable compose-hash
- pin tarfile GNU_FORMAT so output bytes don't shift across Python
versions (default flipped GNU->PAX in 3.8), keeping compose-hash stable
- normalize archive modes (dirs 0755, files 0644/0755 by exec bit) so the
hash no longer depends on the builder's umask; verified exec bit and
symlinks still extract correctly on a live CVM
- validate service unit names against a strict regex and shell-quote them
with shlex.quote (they reach a root shell at boot); reject shell-metachar
names instead of emitting them
- require compose flags to be real JSON booleans (bool("false") was True)
- check the 50 MiB cap BEFORE writing so a failed build leaves no oversized
artifact; reject unsupported file types (sockets/fifos) with a clear error
- document the in-memory build footprint
- replace the legacy kms_enabled/local_key_provider_enabled booleans with the modern key_provider enum (none|kms|local|tpm), matching dstack-types KeyProviderKind; gateway_enabled stays an independent flag. verified the new 'key_provider: kms' form drives KMS + gateway end-to-end on a live CVM - expose all app-compose options as CLI flags on both 'new' (bake into config.json) and 'build' (override config): --key-provider, --gateway, --public-logs, --public-sysinfo, --secure-time, --(no-)instance-id, --allowed-env, --key-provider-id (tri-state so unset keeps config/default) - validate key_provider and warn when gateway is on without kms
- README.md: what/why, boot-flow diagram, requirements, quick start, subcommands, config.json schema, CLI options, security model, the three transport size limits (nginx/prpc/in-guest), determinism notes - examples/hello-c: ~30KB static C HTTP server exposed via gateway (key_provider kms); build.sh compiles with musl then runs sca build - examples/heartbeat: zero-toolchain app (the service is a shell script) plus an embedded config file to show multi-file rootfs packing - build artifacts (compiled binary, app-compose.json) are gitignored
Contributor
There was a problem hiding this comment.
Pull request overview
Adds a new sca (“self-contained app”) builder under tools/sca/ that embeds an app’s rootfs/ into app-compose.json (tar.gz + base64) and generates a boot-time bash_script that extracts onto / and starts specified systemd units—enabling “no Docker / no registry pull” deployments that are still measured by compose-hash/RTMR3.
Changes:
- Introduces
tools/sca/sca.pyCLI withnew(scaffold) andbuild(deterministic pack + emit compose + print compose-hash/app-id) subcommands. - Adds documentation for the tool (
tools/sca/README.md) including security model and size limits. - Adds two runnable examples (
hello-c,heartbeat) with rootfs layouts, systemd units, and build/deploy instructions.
Reviewed changes
Copilot reviewed 13 out of 15 changed files in this pull request and generated 4 comments.
Show a summary per file
| File | Description |
|---|---|
| tools/sca/sca.py | New Python CLI to scaffold projects and build self-contained app-compose.json with embedded rootfs + generated boot script. |
| tools/sca/README.md | User documentation for sca, including workflow, requirements, config schema, and security/size notes. |
| tools/sca/examples/hello-c/src/server.c | Example static single-threaded HTTP server source for “hello-c”. |
| tools/sca/examples/hello-c/rootfs/run/sca/bin/entrypoint.sh | Example entrypoint used by systemd to launch the hello-c server binary. |
| tools/sca/examples/hello-c/rootfs/etc/systemd/system/sca.service | Example systemd unit for supervising the hello-c app. |
| tools/sca/examples/hello-c/README.md | Example-specific build and deployment instructions for hello-c (gateway + KMS). |
| tools/sca/examples/hello-c/config.json | Example config enabling key_provider: kms and gateway_enabled: true. |
| tools/sca/examples/hello-c/build.sh | Convenience script to compile with musl-gcc and run sca build. |
| tools/sca/examples/hello-c/.gitignore | Ignores generated binary and compose output for hello-c example. |
| tools/sca/examples/heartbeat/rootfs/run/sca/bin/entrypoint.sh | Example “app is a shell script” heartbeat loop reading /etc/heartbeat/interval. |
| tools/sca/examples/heartbeat/rootfs/etc/systemd/system/sca.service | Example systemd unit for supervising the heartbeat app. |
| tools/sca/examples/heartbeat/rootfs/etc/heartbeat/interval | Example embedded config file for heartbeat interval. |
| tools/sca/examples/heartbeat/README.md | Example-specific build/deploy notes for heartbeat. |
| tools/sca/examples/heartbeat/config.json | Example config using key_provider: none and no gateway. |
| tools/sca/examples/heartbeat/.gitignore | Ignores generated compose output for heartbeat example. |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
- validate allowed_envs is a list of strings (a stray string would otherwise be spread into per-character entries) - validate key_provider_id is a hex string and swap_size_mb a non-negative int, instead of crashing later with a traceback - treat 'services': [] as an explicit error, not a silent fallback to the default unit (only default when the key is absent/null) - 'sca new' on a path that exists as a file now errors cleanly instead of raising NotADirectoryError
- add SPDX headers to example shell/service/C sources; register the config.json and interval data files in REUSE.toml (can't carry comments) - add docstrings to public functions in sca.py and apply ruff-format so the prek ruff (E,F,I,D) + ruff-format hooks pass
gettarinfo() only returns None for sockets; FIFOs, device nodes and hardlinks still produce a TarInfo and were silently packed via the addfile fallback. reject any entry that isn't a regular file, directory or symlink so they can't be embedded and extracted onto the guest /.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Adds
sca(self-contained app), a CLI undertools/sca/that packages an application directly into anapp-compose.json— no Docker, no registry pull. The app's files are decoded at boot into the CVM and run under a systemd service, so the exact bytes are measured into the compose-hash / RTMR3 and supervised like a container.The branch is purely additive (only
tools/sca/**).How it works
rootfs/tree mirroring the CVM filesystem (your binary/script, a systemd unit, an entrypoint).sca buildpacks the whole tree into a deterministictar.gz, base64-encodes it, and embeds it inapp-compose.json. At boot the guest extracts it onto/viaopenssl base64 -d | tar -xz -C /(the guest busybox userland has nobase64, but hasopenssl/tar/gzip).Restart=always) runs the entrypoint, so crashes are supervised.Subcommands
sca new <dir>— scaffoldconfig.json,README, and arootfs/with defaultentrypoint.sh+sca.servicesca build— emitapp-compose.jsonand print compose-hash / app-id / payload sizeNotable design points
tarfileGNU format, normalized archive modes (dirs0755, files0644/0755by exec bit) and metadata, so the compose-hash is stable across Python versions and builder umask.newandbuild:--key-provider(none|kms|local|tpm, matchingdstack-typesKeyProviderKind),--gateway,--public-logs,--public-sysinfo,--secure-time,--(no-)instance-id,--allowed-env,--key-provider-id(tri-state). Validateskey_providerand warns on gateway-without-kms.shlex.quoted (they reach a root shell at boot); 50 MiB cap checked before writing; unsupported file types rejected.Examples
examples/hello-c— ~30KB static C HTTP server (musl) exposed via gateway,key_provider: kmsexamples/heartbeat— zero-toolchain app (a shell script) with an embedded config file showing multi-file rootfs packingTesting
Verified end-to-end on a live CVM, including KMS + gateway, crash-restart, exec-bit/symlink extraction.