container-magic

Define your container workflow in a single YAML file. Container-magic generates a Dockerfile, build script, and run script that work with Docker or Podman - container-magic is not a dependency of your final product.

How It Works

You write a cm.yaml. Container-magic generates a Dockerfile, build.sh, and run.sh from it. These generated files are committed to your repository so that anyone can build and run the project with Docker or Podman.

For development, cm build and cm run read the config directly and handle workspace mounting, user identity mapping, and runtime features automatically.

Quick Start

pip install container-magic
cm init python:3.11 my-project
cd my-project
cm build
cm run python --version

On Nix, run it without installing - or add it as a flake input:

nix run github:markhedleyjones/container-magic -- init python:3.11 my-project
nix profile install github:markhedleyjones/container-magic   # install cm

The flake exposes packages.default / apps.default (the cm CLI) and an overlays.default for declarative setups.

A minimal cm.yaml:

names:
  image: my-project
  workspace: workspace
  user: nonroot

stages:
  base:
    from: python:3.11-slim
    steps:
      - apt-get:
          install:
            - git
            - build-essential
      - pip:
          install:
            - numpy
            - pandas

  development:
    from: base

  production:
    from: base

Build and run the production image:

./build.sh
./run.sh python workspace/train.py

Key Features

• Development and production from one config - live-mounted workspace in dev, baked-in code in prod
• Automatic user handling - host user identity in dev, dedicated user in prod, no manual setup
• GPU, display, and audio - NVIDIA GPU passthrough, X11/Wayland forwarding, PulseAudio/PipeWire
• Custom commands - define once, use in both dev and prod with port publishing and environment variables
• Multi-stage builds - share steps between stages, pip packages install into the base image's Python (no venv duplication)
• Transparent execution - run commands from anywhere in your repo with automatic path translation
• Data volumes - shorthand for sibling folders (outputs, cache) that persist across runs without entering the image
• AWS credential forwarding - mount host AWS config into the container
• Cached assets - download models and datasets once, reuse across builds
• Dev Container support - generate .devcontainer/devcontainer.json to open the same image in VS Code or Codespaces
• Reproducible, cacheable builds - opt-in base-image digest pinning and BuildKit cache mounts for faster, repeatable rebuilds
• Build secrets - pass credentials at build time via BuildKit secret mounts, never baked into image layers

Documentation

Full documentation is available at markhedleyjones.com/container-magic.

Page	Contents
Getting Started	Installation, first project, workflow
Configuration	Full YAML reference - names, runtime, stages, commands, build secrets, dev containers, digest pinning, cache mounts
Build Steps	Package managers, custom commands, layer caching
Cached Assets	Asset downloading, caching, and cache management
User Handling	Dev vs prod users, copy ownership, permissions
Troubleshooting	Common issues and solutions

Contributing

Contributions and feedback welcome! Open an issue or pull request on GitHub.