diff --git a/hal/ubuntu/README.md b/hal/ubuntu/README.md index e79d7503..7fdbb1c6 100644 --- a/hal/ubuntu/README.md +++ b/hal/ubuntu/README.md @@ -1,11 +1,151 @@ -# Ubuntu +# Ubuntu Cloud-Init for Cube AI -This directory contains the cloud-init configuration files for Cube AI. +This directory contains cloud-init configurations and QEMU launch scripts for running Cube AI in Ubuntu-based Confidential VMs (CVMs). -## After the first boot +## Directory Structure -For local development, replace the following IP address entries in `docker/.env` with the IP address of the qemu virtual machine as follows: +``` +hal/ubuntu/ + qemu.sh # QEMU launch script (local deployment) + user-data-tdx.yaml # Cloud-init for local QEMU TDX VMs (Ollama backend) + user-data-snp.yaml # Cloud-init for local QEMU SNP VMs (Ollama backend) + user-data-regular.yaml # Cloud-init for local QEMU regular VMs (Ollama backend) + user-data-vllm-tdx.yaml # Cloud-init for local QEMU TDX VMs (vLLM backend) + user-data-vllm-snp.yaml # Cloud-init for local QEMU SNP VMs (vLLM backend) + user-data-vllm-regular.yaml # Cloud-init for local QEMU regular VMs (vLLM backend) + debs/ # Custom kernel .deb packages (SNP with Coconut SVSM only, added manually) + cloud/ # Cloud provider configs (GCP, Azure) — see cloud/README.md +``` + +### Backend Selection + +Each CVM mode has two cloud-init variants: +- **Ollama** (default) — `user-data-{mode}.yaml` — lightweight, supports multiple models, good for development +- **vLLM** — `user-data-vllm-{mode}.yaml` — high-performance inference, OpenAI-compatible API, uses GPU if available + +The cube agent is backend-agnostic — it proxies requests to whatever `UV_CUBE_AGENT_TARGET_URL` is set to (Ollama on port 11434, vLLM on port 8000). + +## Local Deployment (QEMU) + +All files in the root of this directory are for **local QEMU-based deployment**. Use `qemu.sh` to launch VMs directly on a host machine with KVM. + +### Quick Start + +```bash +# Auto-detect CVM support (TDX or SNP) and launch +sudo ./qemu.sh start + +# Or force a specific mode +sudo ./qemu.sh start_tdx # Intel TDX +sudo ./qemu.sh start_regular # Regular KVM (no CVM) +``` + +For AMD SEV-SNP, a two-step process is required: + +```bash +sudo ./qemu.sh prepare_snp # Step 1: Install custom kernel via cloud-init (regular KVM) +sudo ./qemu.sh start_snp # Step 2: Boot the prepared image with IGVM/SNP +``` + +### Environment Variables + +```bash +ENABLE_CVM=tdx sudo ./qemu.sh start # Force CVM mode: auto, tdx, snp, none +RAM=32768M CPU=16 sudo ./qemu.sh start # Customize VM resources +IGVM=/path/to/coconut-qemu.igvm sudo ./qemu.sh start_snp # Override IGVM path +``` + +### Detect Available CVM Support + +```bash +sudo ./qemu.sh detect +``` + +### After First Boot + +Default SSH access: +- **Port**: 6190 (forwarded from guest port 22) +- **User**: ultraviolet +- **Password**: password + +## Cloud Deployment (GCP / Azure) + +For deploying on cloud providers, see [cloud/README.md](cloud/README.md). Cloud providers handle confidential computing at the hypervisor level, so no custom kernel, IGVM, or module loading is needed. + +## CVM Details + +### Intel TDX + +- Ubuntu 24.04 kernel has `CONFIG_INTEL_TDX_GUEST=y` enabled by default — no custom kernel needed +- **Local QEMU**: Uses `user-data-tdx.yaml` with `qemu.sh start_tdx`; loads `tdx_guest` module via modprobe +- **Cloud**: TDX module loading is handled by the cloud provider + +### AMD SEV-SNP + +- **Local QEMU with Coconut SVSM**: Requires a custom kernel and two-step boot process (see [SNP Custom Kernel](#snp-custom-kernel) below). Uses `user-data-snp.yaml` which loads `sev-guest` and `ccp` modules via modprobe +- **Cloud (GCP/Azure)**: SEV-SNP is enabled at the hypervisor level — no custom kernel, IGVM, or module loading needed. Use configs in `cloud/` + +#### SNP Custom Kernel + +A custom kernel is only required when the **local** SNP host runs Coconut SVSM. The standard Ubuntu 24.04 kernel does not support Coconut SVSM. + +A custom-built kernel is required with the following configuration options enabled: + +- `CONFIG_AMD_MEM_ENCRYPT=y` — AMD memory encryption support +- `CONFIG_SEV_GUEST=y` — SEV guest driver +- `CONFIG_TCG_PLATFORM=y` — vTPM support +- Coconut SVSM guest support patches applied + +The kernel must be packaged as `.deb` files (`linux-image-*.deb`, `linux-headers-*.deb`). + +**Installing the kernel into the seed image:** + +Place the `.deb` files in a `debs/` directory next to `qemu.sh`: + +``` +hal/ubuntu/ + qemu.sh + user-data-snp.yaml + debs/ + linux-image-*.deb + linux-headers-*.deb + linux-modules-*.deb (if needed) +``` + +The `prepare_snp` command automatically packages the debs along with `user-data` and `meta-data` into the seed ISO using `genisoimage`: ```bash - UV_CUBE_NEXTAUTH_URL=http://:${UI_PORT} +genisoimage -output seed.img -volid cidata -rock / ``` + +On first boot (`prepare_snp`), cloud-init mounts the seed ISO, installs the `.deb` packages, and runs `update-grub`. Then `start_snp` boots the prepared image with IGVM/SNP. + +**Dependency:** + +```bash +sudo apt-get install genisoimage +``` + +## Host Requirements (Local QEMU Only) + +These requirements apply only to local QEMU deployment. Cloud providers manage these at the infrastructure level. + +### For TDX VMs +- Intel CPU with TDX support (4th Gen Xeon Scalable or newer) +- TDX-enabled BIOS/firmware +- Host kernel with TDX module initialized + +### For SNP VMs +- AMD EPYC CPU with SEV-SNP support (Milan or newer) +- SEV-SNP enabled in BIOS +- Host kernel with SEV-SNP/SVSM support +- `/dev/sev` device available +- Coconut SVSM QEMU binary at `/qemu-svsm/bin/qemu-system-x86_64` +- IGVM file at `/etc/cocos/coconut-qemu.igvm` (or set `IGVM` env var) +- `genisoimage` installed (`apt-get install genisoimage`) +- Custom kernel `.deb` files in `debs/` (see [SNP Custom Kernel](#snp-custom-kernel)) + +### Common +- QEMU with confidential computing support +- OVMF firmware (for UEFI boot) +- KVM enabled diff --git a/hal/ubuntu/cloud/README.md b/hal/ubuntu/cloud/README.md new file mode 100644 index 00000000..b9a7fde9 --- /dev/null +++ b/hal/ubuntu/cloud/README.md @@ -0,0 +1,135 @@ +# Cloud Deployment for Cube AI + +This directory contains cloud-init configurations for deploying Cube AI on confidential VMs on GCP and Azure, using the [cocos-infra](https://github.com/ultravioletrs/cocos-infra) Terraform templates. + +## Files + +| File | Backend | Description | +|------|---------|-------------| +| `cube-agent-config.yml` | Ollama | Default configuration with Ollama for model management | +| `cube-agent-vllm-config.yml` | vLLM | High-performance configuration with vLLM | + +## Prerequisites + +- [OpenTofu](https://opentofu.org) or Terraform installed +- GCP or Azure account with permissions to create confidential VMs +- cocos-infra repository cloned: + +```bash +git clone https://github.com/ultravioletrs/cocos-infra.git +cd cocos-infra +``` + +## Deployment + +### Google Cloud Platform (GCP) + +#### Step 1: Deploy KMS (for disk encryption) + +```bash +cd gcp/kms +tofu init +tofu apply -var-file="../../terraform.tfvars" +``` + +Note the `disk_encryption_id` output — you'll need it in the next step. + +#### Step 2: Configure terraform.tfvars + +Create or update `terraform.tfvars` at the root of cocos-infra: + +```hcl +vm_name = "cube-ai-vm" +project_id = "your-gcp-project-id" +region = "us-central1" +min_cpu_platform = "AMD Milan" +confidential_instance_type = "SEV_SNP" +machine_type = "n2d-standard-4" +vm_id = "cube-vm-001" +workspace_id = "cube-workspace-001" +disk_encryption_id = "" # output from gcp/kms step above +cloud_init_config = "/path/to/cube/hal/ubuntu/cloud/cube-agent-config.yml" +``` + +#### Step 3: Deploy the VM + +```bash +cd gcp +tofu init +tofu apply -var-file="../terraform.tfvars" +``` + +--- + +### Microsoft Azure + +#### Step 1: Authenticate + +```bash +az login +``` + +#### Step 2: Deploy KMS (for disk encryption) + +```bash +cd azure/kms +tofu init +tofu apply -var-file="../../terraform.tfvars" +``` + +Note the `disk_encryption_id` output. + +#### Step 3: Configure terraform.tfvars + +```hcl +vm_name = "cube-ai-vm" +resource_group_name = "cube-ai-rg" +location = "westus" +subscription_id = "your-subscription-id" +machine_type = "Standard_DC4ads_v5" +vm_id = "cube-vm-001" +workspace_id = "cube-workspace-001" +disk_encryption_id = "" # output from azure/kms step above +cloud_init_config = "/path/to/cube/hal/ubuntu/cloud/cube-agent-config.yml" +``` + +#### Step 4: Deploy the VM + +```bash +cd azure +tofu init +tofu apply -var-file="../terraform.tfvars" +``` + +## Configuration + +### Choosing a Backend + +- Use `cube-agent-config.yml` for Ollama — good for multi-model setups and general use +- Use `cube-agent-vllm-config.yml` for vLLM — OpenAI-compatible API, better throughput for single-model production use + +### Environment Variables + +Set these in `vllm.env` or `agent.env` inside the VM to customize: + +| Variable | Default | Description | +|----------|---------|-------------| +| `CUBE_VLLM_MODEL` | `meta-llama/Llama-2-7b-hf` | HuggingFace model ID for vLLM | +| `UV_CUBE_AGENT_LOG_LEVEL` | `info` | Agent log level | +| `UV_CUBE_AGENT_CA_URL` | `` | Attestation Manager URL | + +## After Deployment + +SSH into the VM and check service status: + +```bash +# Check cloud-init completed +cloud-init status --wait + +# Check services +sudo systemctl status cube-agent +sudo systemctl status ollama # or vllm + +# Test agent health +curl http://localhost:7001/health +``` diff --git a/hal/ubuntu/cloud/cube-agent-config.yml b/hal/ubuntu/cloud/cube-agent-config.yml new file mode 100644 index 00000000..8c6f260e --- /dev/null +++ b/hal/ubuntu/cloud/cube-agent-config.yml @@ -0,0 +1,286 @@ +#cloud-config +# Cube AI Cloud-Init Configuration +# Copyright (c) Ultraviolet +# SPDX-License-Identifier: Apache-2.0 +# +# Cloud-init for Cube AI agent with Ollama backend on GCP or Azure confidential VMs. +# For local QEMU deployment, use user-data-*.yaml files in the parent directory instead. +# +# Usage: +# Clone https://github.com/ultravioletrs/cocos-infra.git and set +# cloud_init_config to this file path in terraform.tfvars. +# See cloud/README.md for full deployment instructions. +# +# For vLLM backend, see: hal/ubuntu/cloud/cube-agent-vllm-config.yml + +package_update: true +package_upgrade: false + +users: + - name: cubeadmin + plain_text_passwd: changeme + lock_passwd: false + sudo: ALL=(ALL) NOPASSWD:ALL + shell: /bin/bash + groups: [docker] + - name: ollama + system: true + home: /var/lib/ollama + shell: /usr/sbin/nologin + +ssh_pwauth: true + +packages: + - curl + - git + - golang-go + - build-essential + - ca-certificates + - jq + +write_files: + # Cube Agent environment configuration template + # Customize these values for your deployment + - path: /etc/cube/agent.env.template + content: | + # Cube Agent Configuration + # Copy to /etc/cube/agent.env and customize + + # Logging + UV_CUBE_AGENT_LOG_LEVEL=info + + # Network binding + UV_CUBE_AGENT_HOST=0.0.0.0 + UV_CUBE_AGENT_PORT=7001 + + # Instance identification + UV_CUBE_AGENT_INSTANCE_ID=cube-agent-01 + + # AI Backend URL (Ollama default) + UV_CUBE_AGENT_TARGET_URL=http://localhost:11434 + + # TLS Configuration (optional - uncomment for production) + # UV_CUBE_AGENT_SERVER_CERT=/etc/cube/certs/server.crt + # UV_CUBE_AGENT_SERVER_KEY=/etc/cube/certs/server.key + # UV_CUBE_AGENT_SERVER_CA_CERTS=/etc/cube/certs/ca.crt + + # Attestation Manager URL (optional) + # UV_CUBE_AGENT_CA_URL=https://cloud.prism.ultraviolet.rs/am-certs + permissions: '0644' + + # Default agent.env (minimal configuration) + - path: /etc/cube/agent.env + content: | + UV_CUBE_AGENT_LOG_LEVEL=info + UV_CUBE_AGENT_HOST=0.0.0.0 + UV_CUBE_AGENT_PORT=7001 + UV_CUBE_AGENT_INSTANCE_ID=cube-agent-01 + UV_CUBE_AGENT_TARGET_URL=http://localhost:11434 + permissions: '0644' + + # Ollama systemd service + - path: /etc/systemd/system/ollama.service + content: | + [Unit] + Description=Ollama Service + After=network-online.target + Wants=network-online.target + + [Service] + Type=simple + User=ollama + Group=ollama + Environment="OLLAMA_HOST=0.0.0.0:11434" + ExecStart=/usr/local/bin/ollama serve + Restart=always + RestartSec=3 + + [Install] + WantedBy=multi-user.target + permissions: '0644' + + # Cube Agent systemd service + - path: /etc/systemd/system/cube-agent.service + content: | + [Unit] + Description=Cube Agent Service + After=network-online.target ollama.service + Wants=network-online.target + + [Service] + Type=simple + EnvironmentFile=/etc/cube/agent.env + ExecStart=/usr/local/bin/cube-agent + Restart=on-failure + RestartSec=5 + StartLimitBurst=5 + StartLimitIntervalSec=60 + + [Install] + WantedBy=multi-user.target + permissions: '0644' + + # Model pull script for Ollama + - path: /usr/local/bin/pull-ollama-models.sh + content: | + #!/bin/bash + # Pull Ollama models after service is ready + # Default models: tinyllama:1.1b + # Override with CUBE_MODELS environment variable + + MODELS="${CUBE_MODELS:-tinyllama:1.1b}" + + # Wait for Ollama to be ready (max 2 minutes) + echo "Waiting for Ollama service..." + for i in $(seq 1 60); do + if curl -s http://localhost:11434/api/version > /dev/null 2>&1; then + echo "Ollama is ready" + break + fi + sleep 2 + done + + # Pull each model + IFS=',' read -ra MODEL_ARRAY <<< "$MODELS" + for model in "${MODEL_ARRAY[@]}"; do + model=$(echo "$model" | xargs) # Trim whitespace + echo "Pulling model: $model" + /usr/local/bin/ollama pull "$model" + done + + echo "Model pull complete" + permissions: '0755' + + # TLS certificate generation script (for development/testing) + - path: /usr/local/bin/generate-cube-certs.sh + content: | + #!/bin/bash + # Generate self-signed certificates for Cube Agent + # For production, replace with certificates from your CA + + CERT_DIR="/etc/cube/certs" + mkdir -p "$CERT_DIR" + + # Generate CA certificate + openssl req -x509 -newkey rsa:4096 \ + -keyout "$CERT_DIR/ca.key" \ + -out "$CERT_DIR/ca.crt" \ + -days 365 -nodes \ + -subj "/CN=Cube-CA" + + # Generate server certificate + openssl req -newkey rsa:4096 \ + -keyout "$CERT_DIR/server.key" \ + -out "$CERT_DIR/server.csr" \ + -nodes -subj "/CN=cube-agent" + + openssl x509 -req \ + -in "$CERT_DIR/server.csr" \ + -CA "$CERT_DIR/ca.crt" \ + -CAkey "$CERT_DIR/ca.key" \ + -CAcreateserial \ + -out "$CERT_DIR/server.crt" \ + -days 365 + + # Generate client certificate for mTLS + openssl req -newkey rsa:4096 \ + -keyout "$CERT_DIR/client.key" \ + -out "$CERT_DIR/client.csr" \ + -nodes -subj "/CN=cube-client" + + openssl x509 -req \ + -in "$CERT_DIR/client.csr" \ + -CA "$CERT_DIR/ca.crt" \ + -CAkey "$CERT_DIR/ca.key" \ + -CAcreateserial \ + -out "$CERT_DIR/client.crt" \ + -days 365 + + # Set permissions + chmod 600 "$CERT_DIR"/*.key + chmod 644 "$CERT_DIR"/*.crt + rm -f "$CERT_DIR"/*.csr + + echo "Certificates generated in $CERT_DIR" + permissions: '0755' + + # Optional: TLS certificate placeholders for production + # Uncomment and replace with your actual certificates + # + # - path: /etc/cube/certs/server.crt + # content: | + # -----BEGIN CERTIFICATE----- + # [Your server certificate here] + # -----END CERTIFICATE----- + # permissions: '0644' + # + # - path: /etc/cube/certs/server.key + # content: | + # -----BEGIN PRIVATE KEY----- + # [Your server private key here] + # -----END PRIVATE KEY----- + # permissions: '0600' + # + # - path: /etc/cube/certs/ca.crt + # content: | + # -----BEGIN CERTIFICATE----- + # [Your CA certificate here] + # -----END CERTIFICATE----- + # permissions: '0644' + +runcmd: + # Configure SSH for password authentication + - | + cat > /etc/ssh/sshd_config.d/60-cloudimg-settings.conf << 'SSHEOF' + PasswordAuthentication yes + SSHEOF + - systemctl restart sshd + + # Create Cube directories + - mkdir -p /etc/cube/certs + - mkdir -p /var/log/cube + + # Generate self-signed certificates (for development) + # Comment out for production if using real certificates + - /usr/local/bin/generate-cube-certs.sh + + # Set up Ollama directories + - mkdir -p /var/lib/ollama + - mkdir -p /home/ollama/.ollama + - chown -R ollama:ollama /var/lib/ollama + - chown -R ollama:ollama /home/ollama + + # Install Ollama + - curl -fsSL https://ollama.com/install.sh | sh + + # Install Cube Agent from latest release + - | + CUBE_VERSION="${CUBE_AGENT_VERSION:-latest}" + if [ "$CUBE_VERSION" = "latest" ]; then + DOWNLOAD_URL=$(curl -s https://api.github.com/repos/ultravioletrs/cube/releases/latest | jq -r '.assets[] | select(.name | contains("linux") and contains("amd64")) | .browser_download_url' | head -1) + else + DOWNLOAD_URL="https://github.com/ultravioletrs/cube/releases/download/${CUBE_VERSION}/cube-agent-linux-amd64" + fi + if [ -n "$DOWNLOAD_URL" ] && [ "$DOWNLOAD_URL" != "null" ]; then + curl -fsSL "$DOWNLOAD_URL" -o /usr/local/bin/cube-agent + chmod +x /usr/local/bin/cube-agent + else + echo "Could not find release, building from source..." + GO_VERSION=$(curl -fsSL https://go.dev/VERSION?m=text | head -1 | sed 's/go//') + curl -fsSL https://dl.google.com/go/go${GO_VERSION}.linux-amd64.tar.gz -o /tmp/go.tar.gz + rm -rf /usr/local/go && tar -C /usr/local -xzf /tmp/go.tar.gz && rm /tmp/go.tar.gz + git clone https://github.com/ultravioletrs/cube.git /tmp/cube + cd /tmp/cube && /usr/local/go/bin/go build -ldflags="-s -w" -o /usr/local/bin/cube-agent ./cmd/agent + fi + + # Enable and start services + - systemctl daemon-reload + - systemctl enable ollama.service + - systemctl start ollama.service + - systemctl enable cube-agent.service + - systemctl start cube-agent.service + + # Pull default models in background + - nohup /usr/local/bin/pull-ollama-models.sh > /var/log/cube/ollama-pull.log 2>&1 & + +final_message: "Cube AI deployment complete. Cube Agent running on port 7001." diff --git a/hal/ubuntu/cloud/cube-agent-vllm-config.yml b/hal/ubuntu/cloud/cube-agent-vllm-config.yml new file mode 100644 index 00000000..66474eaf --- /dev/null +++ b/hal/ubuntu/cloud/cube-agent-vllm-config.yml @@ -0,0 +1,258 @@ +#cloud-config +# Cube AI Cloud-Init Configuration - vLLM Backend +# Copyright (c) Ultraviolet +# SPDX-License-Identifier: Apache-2.0 +# +# This cloud-init configuration sets up a Cube AI agent with vLLM backend +# for high-performance inference on Ubuntu-based confidential VMs. +# +# vLLM Features: +# - Continuous batching for higher throughput +# - PagedAttention for efficient memory management +# - OpenAI-compatible API +# - Tensor parallelism for multi-GPU setups +# +# Usage: +# Clone https://github.com/ultravioletrs/cocos-infra.git, then: +# - GCP: set cloud_init_config to this file path in terraform.tfvars +# - Azure: set cloud_init_config to this file path in terraform.tfvars +# See cloud/README.md for full deployment instructions +# +# Recommended VM sizes: +# - GCP: n1-standard-8 with nvidia-tesla-t4 or better +# - Azure: Standard_NC6s_v3 (NVIDIA V100) or better + +package_update: true +package_upgrade: false + +users: + - name: cubeadmin + plain_text_passwd: changeme + lock_passwd: false + sudo: ALL=(ALL) NOPASSWD:ALL + shell: /bin/bash + groups: [docker] + - name: vllm + system: true + home: /var/lib/vllm + shell: /usr/sbin/nologin + +ssh_pwauth: true + +packages: + - curl + - git + - python3 + - python3-pip + - python3-venv + - build-essential + - ca-certificates + - jq +write_files: + # Cube Agent environment configuration for vLLM + - path: /etc/cube/agent.env.template + content: | + # Cube Agent Configuration for vLLM Backend + # Copy to /etc/cube/agent.env and customize + + # Logging + UV_CUBE_AGENT_LOG_LEVEL=info + + # Network binding + UV_CUBE_AGENT_HOST=0.0.0.0 + UV_CUBE_AGENT_PORT=7001 + + # Instance identification + UV_CUBE_AGENT_INSTANCE_ID=cube-agent-01 + + # vLLM Backend URL + UV_CUBE_AGENT_TARGET_URL=http://localhost:8000 + + # TLS Configuration (optional - uncomment for production) + # UV_CUBE_AGENT_SERVER_CERT=/etc/cube/certs/server.crt + # UV_CUBE_AGENT_SERVER_KEY=/etc/cube/certs/server.key + # UV_CUBE_AGENT_SERVER_CA_CERTS=/etc/cube/certs/ca.crt + + # Attestation Manager URL (optional) + # UV_CUBE_AGENT_CA_URL=https://prism.ultraviolet.rs/am-certs + permissions: '0644' + + # Default agent.env for vLLM + - path: /etc/cube/agent.env + content: | + UV_CUBE_AGENT_LOG_LEVEL=info + UV_CUBE_AGENT_HOST=0.0.0.0 + UV_CUBE_AGENT_PORT=7001 + UV_CUBE_AGENT_INSTANCE_ID=cube-agent-01 + UV_CUBE_AGENT_TARGET_URL=http://localhost:8000 + permissions: '0644' + + # vLLM environment configuration + - path: /etc/cube/vllm.env + content: | + # vLLM Configuration + # Model to serve (HuggingFace model ID) + VLLM_MODEL=${CUBE_VLLM_MODEL:-meta-llama/Llama-2-7b-hf} + + # Server configuration + VLLM_HOST=0.0.0.0 + VLLM_PORT=8000 + + # HuggingFace token (for gated models) + # HF_TOKEN=your_token_here + permissions: '0644' + + # vLLM systemd service + - path: /etc/systemd/system/vllm.service + content: | + [Unit] + Description=vLLM Inference Server + After=network-online.target + Wants=network-online.target + + [Service] + Type=simple + User=vllm + Group=vllm + EnvironmentFile=/etc/cube/vllm.env + WorkingDirectory=/var/lib/vllm + ExecStart=/var/lib/vllm/venv/bin/python -m vllm.entrypoints.openai.api_server \ + --model ${VLLM_MODEL} \ + --host ${VLLM_HOST} \ + --port ${VLLM_PORT} + Restart=on-failure + RestartSec=10 + StartLimitBurst=3 + StartLimitIntervalSec=60 + + [Install] + WantedBy=multi-user.target + permissions: '0644' + + # Cube Agent systemd service + - path: /etc/systemd/system/cube-agent.service + content: | + [Unit] + Description=Cube Agent Service + After=network-online.target vllm.service + Wants=network-online.target + + [Service] + Type=simple + EnvironmentFile=/etc/cube/agent.env + ExecStart=/usr/local/bin/cube-agent + Restart=on-failure + RestartSec=5 + StartLimitBurst=5 + StartLimitIntervalSec=60 + + [Install] + WantedBy=multi-user.target + permissions: '0644' + + # TLS certificate generation script + - path: /usr/local/bin/generate-cube-certs.sh + content: | + #!/bin/bash + # Generate self-signed certificates for Cube Agent + # For production, replace with certificates from your CA + + CERT_DIR="/etc/cube/certs" + mkdir -p "$CERT_DIR" + + # Generate CA certificate + openssl req -x509 -newkey rsa:4096 \ + -keyout "$CERT_DIR/ca.key" \ + -out "$CERT_DIR/ca.crt" \ + -days 365 -nodes \ + -subj "/CN=Cube-CA" + + # Generate server certificate + openssl req -newkey rsa:4096 \ + -keyout "$CERT_DIR/server.key" \ + -out "$CERT_DIR/server.csr" \ + -nodes -subj "/CN=cube-agent" + + openssl x509 -req \ + -in "$CERT_DIR/server.csr" \ + -CA "$CERT_DIR/ca.crt" \ + -CAkey "$CERT_DIR/ca.key" \ + -CAcreateserial \ + -out "$CERT_DIR/server.crt" \ + -days 365 + + # Generate client certificate for mTLS + openssl req -newkey rsa:4096 \ + -keyout "$CERT_DIR/client.key" \ + -out "$CERT_DIR/client.csr" \ + -nodes -subj "/CN=cube-client" + + openssl x509 -req \ + -in "$CERT_DIR/client.csr" \ + -CA "$CERT_DIR/ca.crt" \ + -CAkey "$CERT_DIR/ca.key" \ + -CAcreateserial \ + -out "$CERT_DIR/client.crt" \ + -days 365 + + # Set permissions + chmod 600 "$CERT_DIR"/*.key + chmod 644 "$CERT_DIR"/*.crt + rm -f "$CERT_DIR"/*.csr + + echo "Certificates generated in $CERT_DIR" + permissions: '0755' + +runcmd: + # Configure SSH for password authentication + - | + cat > /etc/ssh/sshd_config.d/60-cloudimg-settings.conf << 'SSHEOF' + PasswordAuthentication yes + SSHEOF + - systemctl restart sshd + + # Create Cube directories + - mkdir -p /etc/cube/certs + - mkdir -p /var/log/cube + - mkdir -p /var/lib/vllm + + # Generate self-signed certificates (for development) + - /usr/local/bin/generate-cube-certs.sh + + # Set up vLLM user and directories + - chown -R vllm:vllm /var/lib/vllm + + # Install vLLM in virtual environment + - | + sudo -u vllm python3 -m venv /var/lib/vllm/venv + sudo -u vllm /var/lib/vllm/venv/bin/pip install --upgrade pip + sudo -u vllm /var/lib/vllm/venv/bin/pip install vllm + + # Install Cube Agent from latest release + - | + CUBE_VERSION="${CUBE_AGENT_VERSION:-latest}" + if [ "$CUBE_VERSION" = "latest" ]; then + DOWNLOAD_URL=$(curl -s https://api.github.com/repos/ultravioletrs/cube/releases/latest | jq -r '.assets[] | select(.name | contains("linux") and contains("amd64")) | .browser_download_url' | head -1) + else + DOWNLOAD_URL="https://github.com/ultravioletrs/cube/releases/download/${CUBE_VERSION}/cube-agent-linux-amd64" + fi + if [ -n "$DOWNLOAD_URL" ] && [ "$DOWNLOAD_URL" != "null" ]; then + curl -fsSL "$DOWNLOAD_URL" -o /usr/local/bin/cube-agent + chmod +x /usr/local/bin/cube-agent + else + echo "Could not find release, building from source..." + GO_VERSION=$(curl -fsSL https://go.dev/VERSION?m=text | head -1 | sed 's/go//') + curl -fsSL https://dl.google.com/go/go${GO_VERSION}.linux-amd64.tar.gz -o /tmp/go.tar.gz + rm -rf /usr/local/go && tar -C /usr/local -xzf /tmp/go.tar.gz && rm /tmp/go.tar.gz + git clone https://github.com/ultravioletrs/cube.git /tmp/cube + cd /tmp/cube && /usr/local/go/bin/go build -ldflags="-s -w" -o /usr/local/bin/cube-agent ./cmd/agent + fi + + # Enable and start services + - systemctl daemon-reload + - systemctl enable vllm.service + - systemctl start vllm.service + - systemctl enable cube-agent.service + - systemctl start cube-agent.service + +final_message: "Cube AI (vLLM) deployment complete. Cube Agent running on port 7001, vLLM on port 8000." diff --git a/hal/ubuntu/qemu.sh b/hal/ubuntu/qemu.sh index b078516e..dbf47107 100755 --- a/hal/ubuntu/qemu.sh +++ b/hal/ubuntu/qemu.sh @@ -1,13 +1,17 @@ #!/bin/bash # Copyright (c) Ultraviolet # SPDX-License-Identifier: Apache-2.0 +# +# QEMU launch script for Ubuntu cloud images with CVM (TDX/SNP) support +set -e + +# Default configuration BASE_IMAGE_URL="https://cloud-images.ubuntu.com/noble/current/noble-server-cloudimg-amd64.img" BASE_IMAGE="ubuntu-base.qcow2" CUSTOM_IMAGE="ubuntu-custom.qcow2" DISK_SIZE="35G" SEED_IMAGE="seed.img" -USER_DATA="user-data" META_DATA="meta-data" VM_NAME="cube-ai-vm" RAM="16384M" @@ -17,264 +21,338 @@ PASSWORD="password" QEMU_BINARY="qemu-system-x86_64" OVMF_CODE="/usr/share/OVMF/OVMF_CODE.fd" OVMF_VARS="/usr/share/OVMF/OVMF_VARS.fd" -OVMF_VARS_COPY="OVMF_VARS.fd" # Per-VM copy of OVMF vars -ENABLE_CVM="${ENABLE_CVM:-auto}" # Options: auto, tdx, none - -if ! command -v wget &> /dev/null; then - echo "wget is not installed. Please install it and try again." - exit 1 -fi - -if ! command -v cloud-localds &> /dev/null; then - echo "cloud-localds is not installed. Please install it and try again." - exit 1 -fi - -if ! command -v qemu-system-x86_64 &> /dev/null; then - echo "qemu-system-x86_64 is not installed. Please install it and try again." - exit 1 -fi - -if [[ $EUID -ne 0 ]]; then - echo "This script must be run as root" 1>&2 - exit 1 -fi - -if [ ! -f $BASE_IMAGE ]; then - echo "Downloading base Ubuntu image..." - wget -q $BASE_IMAGE_URL -O $BASE_IMAGE -fi - -echo "Creating custom QEMU image..." -qemu-img create -f qcow2 -b $BASE_IMAGE -F qcow2 $CUSTOM_IMAGE $DISK_SIZE - -# Create a writable copy of OVMF_VARS for this VM instance -if [ ! -f $OVMF_VARS_COPY ]; then - echo "Creating OVMF vars copy..." - cp $OVMF_VARS $OVMF_VARS_COPY -fi - -# We don't upgrade the system since this changes initramfs -cat <<'EOF' > $USER_DATA -#cloud-config -package_update: true -package_upgrade: false - -users: - - name: ultraviolet - plain_text_passwd: password - lock_passwd: false - sudo: ALL=(ALL) NOPASSWD:ALL - shell: /bin/bash - - name: ollama - system: true - home: /var/lib/ollama - shell: /usr/sbin/nologin - -ssh_pwauth: true - -packages: - - curl - - git - - golang-go - - build-essential - -write_files: - - path: /etc/cube/agent.env - content: | - UV_CUBE_AGENT_LOG_LEVEL=info - UV_CUBE_AGENT_HOST=0.0.0.0 - UV_CUBE_AGENT_PORT=7001 - UV_CUBE_AGENT_INSTANCE_ID=cube-agent-01 - UV_CUBE_AGENT_TARGET_URL=http://localhost:11434 - UV_CUBE_AGENT_SERVER_CERT=/etc/cube/certs/server.crt - UV_CUBE_AGENT_SERVER_KEY=/etc/cube/certs/server.key - UV_CUBE_AGENT_SERVER_CA_CERTS=/etc/cube/certs/ca.crt - UV_CUBE_AGENT_CA_URL=https://prism.ultraviolet.rs/am-certs - permissions: '0644' - - path: /etc/systemd/system/ollama.service - content: | - [Unit] - Description=Ollama Service - After=network-online.target - Wants=network-online.target - - [Service] - Type=simple - User=ollama - Group=ollama - Environment="OLLAMA_HOST=0.0.0.0:11434" - ExecStart=/usr/local/bin/ollama serve - Restart=always - RestartSec=3 - - [Install] - WantedBy=multi-user.target - permissions: '0644' - - path: /etc/systemd/system/cube-agent.service - content: | - [Unit] - Description=Cube Agent Service - After=network-online.target ollama.service - Wants=network-online.target - - [Service] - Type=simple - EnvironmentFile=/etc/cube/agent.env - ExecStart=/usr/local/bin/cube-agent - Restart=on-failure - RestartSec=5 - StartLimitBurst=5 - StartLimitIntervalSec=60 - - [Install] - WantedBy=multi-user.target - permissions: '0644' - - path: /usr/local/bin/pull-ollama-models.sh - content: | - #!/bin/bash - # Wait for ollama to be ready - for i in $(seq 1 60); do - if curl -s http://localhost:11434/api/version > /dev/null 2>&1; then - break - fi - sleep 2 - done - # Pull models - /usr/local/bin/ollama pull llama3.2:3b - /usr/local/bin/ollama pull starcoder2:3b - /usr/local/bin/ollama pull nomic-embed-text:v1.5 - permissions: '0755' - -runcmd: - - echo 'ultraviolet:password' | chpasswd - - | - cat > /etc/ssh/sshd_config.d/60-cloudimg-settings.conf << 'SSHEOF' - PasswordAuthentication yes - SSHEOF - - systemctl restart sshd - - sleep 2 - - | - # Install TDX-capable kernel from Ubuntu's intel-tdx PPA - add-apt-repository -y ppa:kobuk-team/intel-tdx || echo "PPA add failed, trying canonical tdx" - apt-get update || true - apt-get install -y linux-image-generic linux-modules-extra-generic || echo "Kernel install failed" - # Try to load TDX guest module - modprobe tdx_guest 2>/dev/null || echo "tdx_guest module not yet available (may need reboot)" - # Add to modules to load at boot - mkdir -p /etc/modules-load.d - echo "tdx_guest" > /etc/modules-load.d/tdx.conf - - mkdir -p /etc/cube - - mkdir -p /etc/cube/certs - - | - # Generate CA certificate - openssl req -x509 -newkey rsa:4096 -keyout /etc/cube/certs/ca.key -out /etc/cube/certs/ca.crt -days 365 -nodes -subj "/CN=Cube-CA" - # Generate server certificate - openssl req -newkey rsa:4096 -keyout /etc/cube/certs/server.key -out /etc/cube/certs/server.csr -nodes -subj "/CN=cube-agent" - openssl x509 -req -in /etc/cube/certs/server.csr -CA /etc/cube/certs/ca.crt -CAkey /etc/cube/certs/ca.key -CAcreateserial -out /etc/cube/certs/server.crt -days 365 - # Generate client certificate for mTLS - openssl req -newkey rsa:4096 -keyout /etc/cube/certs/client.key -out /etc/cube/certs/client.csr -nodes -subj "/CN=cube-client" - openssl x509 -req -in /etc/cube/certs/client.csr -CA /etc/cube/certs/ca.crt -CAkey /etc/cube/certs/ca.key -CAcreateserial -out /etc/cube/certs/client.crt -days 365 - # Set permissions - chmod 600 /etc/cube/certs/*.key - chmod 644 /etc/cube/certs/*.crt - - mkdir -p /var/lib/ollama - - mkdir -p /home/ollama/.ollama - - chown -R ollama:ollama /var/lib/ollama - - chown -R ollama:ollama /home/ollama - - curl -fsSL https://ollama.com/install.sh | sh - - git clone https://github.com/ultravioletrs/cube.git /tmp/cube - - cd /tmp/cube && git fetch origin pull/88/head:pr-88 && git checkout pr-88 - - export HOME=/root - - cd /tmp/cube && /usr/bin/go build -ldflags="-s -w" -o /usr/local/bin/cube-agent ./cmd/agent - - systemctl daemon-reload - - systemctl enable ollama.service - - systemctl start ollama.service - - systemctl enable cube-agent.service - - systemctl start cube-agent.service - - nohup /usr/local/bin/pull-ollama-models.sh > /var/log/ollama-pull.log 2>&1 & - -final_message: "Cube Agent and Ollama services started." +OVMF_VARS_COPY="OVMF_VARS.fd" +SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)" + +# CVM mode: auto, tdx, snp, none +ENABLE_CVM="${ENABLE_CVM:-auto}" + +function check_dependencies() { + local missing=() + + if ! command -v wget &> /dev/null; then + missing+=("wget") + fi + + if ! command -v cloud-localds &> /dev/null; then + missing+=("cloud-localds (cloud-image-utils)") + fi + + if ! command -v qemu-system-x86_64 &> /dev/null; then + missing+=("qemu-system-x86_64") + fi + + if [ ${#missing[@]} -ne 0 ]; then + echo "Missing dependencies: ${missing[*]}" + echo "Please install them and try again." + exit 1 + fi +} + +function check_root() { + if [[ $EUID -ne 0 ]]; then + echo "This script must be run as root" 1>&2 + exit 1 + fi +} + +function detect_cvm_support() { + local tdx_available=false + local snp_available=false + + # Check for TDX host support + if dmesg 2>/dev/null | grep -q "virt/tdx: module initialized"; then + tdx_available=true + echo "TDX host support detected (module initialized)" + elif grep -q tdx /proc/cpuinfo 2>/dev/null; then + tdx_available=true + echo "TDX CPU support detected" + fi + + # Check for SEV-SNP host support + if [ -e /dev/sev ]; then + snp_available=true + echo "SEV device detected" + fi + if dmesg 2>/dev/null | grep -q "SEV-SNP supported"; then + snp_available=true + echo "SEV-SNP host support detected" + elif grep -q sev /proc/cpuinfo 2>/dev/null; then + snp_available=true + echo "SEV CPU support detected" + fi + + # Return detected support + if [ "$tdx_available" = true ]; then + echo "tdx" + elif [ "$snp_available" = true ]; then + echo "snp" + else + echo "none" + fi +} + +function download_base_image() { + if [ ! -f "$BASE_IMAGE" ]; then + echo "Downloading base Ubuntu image..." + wget -q --show-progress "$BASE_IMAGE_URL" -O "$BASE_IMAGE" + else + echo "Base image already exists: $BASE_IMAGE" + fi +} + +function create_custom_image() { + echo "Creating custom QEMU image..." + qemu-img create -f qcow2 -b "$BASE_IMAGE" -F qcow2 "$CUSTOM_IMAGE" "$DISK_SIZE" +} + +function create_ovmf_vars_copy() { + if [ ! -f "$OVMF_VARS_COPY" ]; then + echo "Creating OVMF vars copy..." + cp "$OVMF_VARS" "$OVMF_VARS_COPY" + fi +} + +function create_seed_image() { + local user_data_file="$1" + + echo "Creating seed image with $user_data_file..." + + # Create meta-data + cat < "$META_DATA" +instance-id: iid-${VM_NAME} +local-hostname: $VM_NAME EOF -cat < $META_DATA + cloud-localds "$SEED_IMAGE" "$user_data_file" "$META_DATA" +} + +function create_seed_image_snp() { + local user_data_file="$1" + + echo "Creating SNP seed image with $user_data_file..." + + if ! command -v genisoimage &> /dev/null; then + echo "genisoimage is not installed. Please install genisoimage and try again." + exit 1 + fi + + local cidata_dir + cidata_dir="$(mktemp -d)" + trap "rm -rf $cidata_dir" RETURN + + # Write meta-data + cat < "$cidata_dir/meta-data" instance-id: iid-${VM_NAME} local-hostname: $VM_NAME EOF -echo "Creating ubuntu seed image..." -cloud-localds $SEED_IMAGE $USER_DATA $META_DATA - -# Detect TDX support -TDX_AVAILABLE=false -if [ "$ENABLE_CVM" = "auto" ] || [ "$ENABLE_CVM" = "tdx" ]; then - # Check if TDX is initialized on the host (for creating guest VMs) - if dmesg | grep -q "virt/tdx: module initialized"; then - TDX_AVAILABLE=true - echo "TDX host support detected" - elif grep -q tdx /proc/cpuinfo; then - TDX_AVAILABLE=true - echo "TDX CPU support detected" - else - echo "TDX not available on host" - fi -fi - -# Override if explicitly set -if [ "$ENABLE_CVM" = "tdx" ]; then - TDX_AVAILABLE=true - echo "TDX mode forced via ENABLE_CVM=tdx" -elif [ "$ENABLE_CVM" = "none" ]; then - TDX_AVAILABLE=false - echo "CVM disabled via ENABLE_CVM=none" -fi - -# Build QEMU command based on TDX availability -QEMU_CMD="$QEMU_BINARY" -QEMU_OPTS="-name $VM_NAME" -QEMU_OPTS="$QEMU_OPTS -m $RAM" -QEMU_OPTS="$QEMU_OPTS -smp $CPU" -QEMU_OPTS="$QEMU_OPTS -enable-kvm" -QEMU_OPTS="$QEMU_OPTS -boot d" -QEMU_OPTS="$QEMU_OPTS -netdev user,id=vmnic,hostfwd=tcp::6190-:22,hostfwd=tcp::6191-:80,hostfwd=tcp::6192-:443,hostfwd=tcp::6193-:7001" -QEMU_OPTS="$QEMU_OPTS -nographic" -QEMU_OPTS="$QEMU_OPTS -no-reboot" -QEMU_OPTS="$QEMU_OPTS -drive file=$SEED_IMAGE,media=cdrom" -QEMU_OPTS="$QEMU_OPTS -drive file=$CUSTOM_IMAGE,if=none,id=disk0,format=qcow2" -QEMU_OPTS="$QEMU_OPTS -device virtio-scsi-pci,id=scsi,disable-legacy=on" -QEMU_OPTS="$QEMU_OPTS -device scsi-hd,drive=disk0" - -if [ "$TDX_AVAILABLE" = true ]; then - echo "Starting QEMU VM with Intel TDX (Confidential VM)..." - # Update the -name option to include process and debug-threads - QEMU_OPTS=$(echo "$QEMU_OPTS" | sed "s/-name $VM_NAME/-name $VM_NAME,process=$VM_NAME,debug-threads=on/") - # Remove -m and add memory-backend-memfd for TDX (critical!) - QEMU_OPTS=$(echo "$QEMU_OPTS" | sed "s/-m $RAM//") - QEMU_OPTS="$QEMU_OPTS -object memory-backend-memfd,id=ram1,size=$RAM,share=true,prealloc=false" - QEMU_OPTS="$QEMU_OPTS -m $RAM" - QEMU_OPTS="$QEMU_OPTS -cpu host,pmu=off" - # TDX guest object with quote generation socket - QEMU_OPTS="$QEMU_OPTS -object {\"qom-type\":\"tdx-guest\",\"id\":\"tdx0\",\"quote-generation-socket\":{\"type\":\"vsock\",\"cid\":\"2\",\"port\":\"4050\"}}" - QEMU_OPTS="$QEMU_OPTS -machine q35,confidential-guest-support=tdx0,memory-backend=ram1,kernel-irqchip=split,hpet=off" - # Use -bios for TDX boot - QEMU_OPTS="$QEMU_OPTS -bios /usr/share/ovmf/OVMF.fd" - # Disk boot (Ubuntu cloud image) - QEMU_OPTS="$QEMU_OPTS -device virtio-net-pci,disable-legacy=on,iommu_platform=true,netdev=vmnic,romfile=" - QEMU_OPTS="$QEMU_OPTS -nodefaults" - QEMU_OPTS="$QEMU_OPTS -nographic" - QEMU_OPTS="$QEMU_OPTS -serial mon:stdio" - QEMU_OPTS="$QEMU_OPTS -monitor pty" -else - echo "Starting QEMU VM in regular mode (no CVM)..." - QEMU_OPTS="$QEMU_OPTS -drive if=pflash,format=raw,unit=0,file=$OVMF_CODE,readonly=on" - QEMU_OPTS="$QEMU_OPTS -drive if=pflash,format=raw,unit=1,file=$OVMF_VARS_COPY" - QEMU_OPTS="$QEMU_OPTS -cpu host" - QEMU_OPTS="$QEMU_OPTS -machine q35" - QEMU_OPTS="$QEMU_OPTS -device virtio-net-pci,netdev=vmnic,romfile=" -fi - -# Execute QEMU (use eval to handle complex quoting) -echo "Full QEMU command:" -echo "$QEMU_CMD $QEMU_OPTS" -echo "" -$QEMU_CMD $QEMU_OPTS + # Copy user-data + cp "$user_data_file" "$cidata_dir/user-data" + + # Include kernel .deb files if present (required for SNP-compatible kernel) + if [ -d "${SCRIPT_DIR}/debs" ] && ls "${SCRIPT_DIR}/debs"/*.deb 2>/dev/null; then + mkdir -p "$cidata_dir/debs" + cp "${SCRIPT_DIR}/debs"/*.deb "$cidata_dir/debs/" + echo "Included kernel .deb files from ${SCRIPT_DIR}/debs/" + fi + + genisoimage -output "$SEED_IMAGE" -volid cidata -rock "$cidata_dir" +} + +function start_regular() { + echo "Starting QEMU VM in regular mode (no CVM)..." + + create_ovmf_vars_copy + create_seed_image "${SCRIPT_DIR}/user-data-regular.yaml" + + $QEMU_BINARY \ + -name "$VM_NAME" \ + -m "$RAM" \ + -smp "$CPU" \ + -enable-kvm \ + -boot d \ + -cpu host \ + -machine q35 \ + -drive if=pflash,format=raw,unit=0,file="$OVMF_CODE",readonly=on \ + -drive if=pflash,format=raw,unit=1,file="$OVMF_VARS_COPY" \ + -netdev user,id=vmnic,hostfwd=tcp::6190-:22,hostfwd=tcp::6191-:80,hostfwd=tcp::6192-:443,hostfwd=tcp::6193-:7001 \ + -device virtio-net-pci,netdev=vmnic,romfile= \ + -nographic \ + -no-reboot \ + -drive file="$SEED_IMAGE",media=cdrom \ + -drive file="$CUSTOM_IMAGE",if=none,id=disk0,format=qcow2 \ + -device virtio-scsi-pci,id=scsi,disable-legacy=on \ + -device scsi-hd,drive=disk0 +} + +function start_tdx() { + echo "Starting QEMU VM with Intel TDX (Confidential VM)..." + + create_seed_image "${SCRIPT_DIR}/user-data-tdx.yaml" + + $QEMU_BINARY \ + -name "$VM_NAME,process=$VM_NAME,debug-threads=on" \ + -m "$RAM" \ + -smp "$CPU" \ + -enable-kvm \ + -cpu host,pmu=off \ + -object memory-backend-memfd,id=ram1,size="$RAM",share=true,prealloc=false \ + -object '{"qom-type":"tdx-guest","id":"tdx0","quote-generation-socket":{"type":"vsock","cid":"2","port":"4050"}}' \ + -machine q35,confidential-guest-support=tdx0,memory-backend=ram1,kernel-irqchip=split,hpet=off \ + -bios /usr/share/ovmf/OVMF.fd \ + -netdev user,id=vmnic,hostfwd=tcp::6190-:22,hostfwd=tcp::6191-:80,hostfwd=tcp::6192-:443,hostfwd=tcp::6193-:7001 \ + -device virtio-net-pci,disable-legacy=on,iommu_platform=true,netdev=vmnic,romfile= \ + -nodefaults \ + -nographic \ + -serial mon:stdio \ + -monitor pty \ + -no-reboot \ + -drive file="$SEED_IMAGE",media=cdrom \ + -drive file="$CUSTOM_IMAGE",if=none,id=disk0,format=qcow2 \ + -device virtio-scsi-pci,id=scsi,disable-legacy=on,iommu_platform=true \ + -device scsi-hd,drive=disk0 \ + -device vhost-vsock-pci,guest-cid=3 +} + +function prepare_snp() { + echo "Preparing SNP image via cloud-init (regular KVM, no SNP)..." + echo "This installs the custom kernel into the image. Run start_snp after this completes." + + local SNP_QEMU="/qemu-svsm/bin/qemu-system-x86_64" + + create_seed_image_snp "${SCRIPT_DIR}/user-data-snp.yaml" + + $SNP_QEMU \ + -name "$VM_NAME" \ + -enable-kvm \ + -m 2048 \ + -smp 2 \ + -drive "file=$CUSTOM_IMAGE,if=virtio,format=qcow2" \ + -drive "file=$SEED_IMAGE,if=virtio,format=raw,readonly=on" \ + -netdev "user,id=n0,hostfwd=tcp::6190-:22" \ + -device virtio-net-pci,netdev=n0 \ + -nographic +} + +function start_snp() { + echo "Starting QEMU VM with AMD SEV-SNP (Confidential VM)..." + echo "Ensure prepare_snp has been run first to install the custom kernel." + + local SNP_QEMU="/qemu-svsm/bin/qemu-system-x86_64" + local IGVM="${IGVM:-/etc/cocos/coconut-qemu.igvm}" + + $SNP_QEMU \ + -name "$VM_NAME" \ + -enable-kvm \ + -cpu EPYC-v4 \ + -machine q35 \ + -smp "$CPU,sockets=1,threads=1" \ + -m "$RAM,slots=5,maxmem=40G" \ + -netdev user,id=vmnic,hostfwd=tcp::6190-:22,hostfwd=tcp::6191-:80,hostfwd=tcp::6192-:443,hostfwd=tcp::6193-:7001 \ + -device virtio-net-pci,disable-legacy=on,iommu_platform=true,netdev=vmnic,romfile= \ + -machine confidential-guest-support=sev0,memory-backend=ram1,igvm-cfg=igvm0 \ + -object memory-backend-memfd,id=ram1,size="$RAM",share=true,prealloc=false,reserve=false \ + -object sev-snp-guest,id=sev0,cbitpos=51,reduced-phys-bits=1 \ + -object igvm-cfg,id=igvm0,file="$IGVM" \ + -drive file="$CUSTOM_IMAGE",if=none,id=disk0,format=qcow2 \ + -device virtio-scsi-pci,id=scsi0,disable-legacy=on,iommu_platform=true \ + -device scsi-hd,drive=disk0,bus=scsi0.0 \ + -device vhost-vsock-pci,id=vhost-vsock-pci0,guest-cid=198 \ + -nographic \ + -no-reboot \ + -monitor pty +} + +function print_help() { + cat < /dev/null 2>&1; then + break + fi + sleep 2 + done + # Pull models + /usr/local/bin/ollama pull tinyllama:1.1b + /usr/local/bin/ollama pull starcoder2:3b + /usr/local/bin/ollama pull nomic-embed-text:v1.5 + permissions: '0755' + +runcmd: + - echo 'ultraviolet:password' | chpasswd + - | + cat > /etc/ssh/sshd_config.d/60-cloudimg-settings.conf << 'SSHEOF' + PasswordAuthentication yes + SSHEOF + - systemctl restart sshd + - sleep 2 + - mkdir -p /etc/cube + - mkdir -p /etc/cube/certs + - | + # Generate CA certificate + openssl req -x509 -newkey rsa:4096 -keyout /etc/cube/certs/ca.key -out /etc/cube/certs/ca.crt -days 365 -nodes -subj "/CN=Cube-CA" + # Generate server certificate + openssl req -newkey rsa:4096 -keyout /etc/cube/certs/server.key -out /etc/cube/certs/server.csr -nodes -subj "/CN=cube-agent" + openssl x509 -req -in /etc/cube/certs/server.csr -CA /etc/cube/certs/ca.crt -CAkey /etc/cube/certs/ca.key -CAcreateserial -out /etc/cube/certs/server.crt -days 365 + # Generate client certificate for mTLS + openssl req -newkey rsa:4096 -keyout /etc/cube/certs/client.key -out /etc/cube/certs/client.csr -nodes -subj "/CN=cube-client" + openssl x509 -req -in /etc/cube/certs/client.csr -CA /etc/cube/certs/ca.crt -CAkey /etc/cube/certs/ca.key -CAcreateserial -out /etc/cube/certs/client.crt -days 365 + # Set permissions + chmod 600 /etc/cube/certs/*.key + chmod 644 /etc/cube/certs/*.crt + - mkdir -p /var/lib/ollama + - mkdir -p /home/ollama/.ollama + - chown -R ollama:ollama /var/lib/ollama + - chown -R ollama:ollama /home/ollama + - curl -fsSL https://ollama.com/install.sh | sh + - | + # Install Go (required version from go.mod) + GO_VERSION=$(curl -fsSL https://go.dev/VERSION?m=text | head -1 | sed 's/go//') + curl -fsSL https://dl.google.com/go/go${GO_VERSION}.linux-amd64.tar.gz -o /tmp/go.tar.gz + rm -rf /usr/local/go + tar -C /usr/local -xzf /tmp/go.tar.gz + rm /tmp/go.tar.gz + - git clone https://github.com/ultravioletrs/cube.git /tmp/cube + - cd /tmp/cube && git fetch origin pull/88/head:pr-88 && git checkout pr-88 + - export HOME=/root + - cd /tmp/cube && /usr/local/go/bin/go build -ldflags="-s -w" -o /usr/local/bin/cube-agent ./cmd/agent + - systemctl daemon-reload + - systemctl enable ollama.service + - systemctl start ollama.service + - systemctl enable cube-agent.service + - systemctl start cube-agent.service + - nohup /usr/local/bin/pull-ollama-models.sh > /var/log/ollama-pull.log 2>&1 & + +final_message: "Cube Agent and Ollama services started." diff --git a/hal/ubuntu/user-data-snp.yaml b/hal/ubuntu/user-data-snp.yaml new file mode 100644 index 00000000..f1dc2368 --- /dev/null +++ b/hal/ubuntu/user-data-snp.yaml @@ -0,0 +1,185 @@ +#cloud-config +# Copyright (c) Ultraviolet +# SPDX-License-Identifier: Apache-2.0 +# +# Cloud-init for local QEMU AMD SEV-SNP VMs (Coconut SVSM) +# Used by: qemu.sh prepare_snp / start_snp +# For cloud deployments (GCP/Azure), see cloud/ directory instead + +package_update: true +package_upgrade: false + +growpart: + mode: auto + devices: ["/"] + ignore_growroot_disabled: false + +resize_rootfs: true + +users: + - name: ultraviolet + plain_text_passwd: password + lock_passwd: false + sudo: ALL=(ALL) NOPASSWD:ALL + shell: /bin/bash + - name: ollama + system: true + home: /var/lib/ollama + shell: /usr/sbin/nologin + +ssh_pwauth: true + +packages: + - curl + - git + - build-essential + +write_files: + - path: /etc/cube/agent.env + content: | + UV_CUBE_AGENT_LOG_LEVEL=info + UV_CUBE_AGENT_HOST=0.0.0.0 + UV_CUBE_AGENT_PORT=7001 + UV_CUBE_AGENT_INSTANCE_ID=cube-agent-01 + UV_CUBE_AGENT_TARGET_URL=http://localhost:11434 + UV_CUBE_AGENT_SERVER_CERT=/etc/cube/certs/server.crt + UV_CUBE_AGENT_SERVER_KEY=/etc/cube/certs/server.key + UV_CUBE_AGENT_SERVER_CA_CERTS=/etc/cube/certs/ca.crt + UV_CUBE_AGENT_CA_URL=https://prism.ultraviolet.rs/am-certs + # SNP-specific settings + AGENT_OS_TYPE=snp + AGENT_VMPL=2 + permissions: '0644' + - path: /etc/systemd/system/ollama.service + content: | + [Unit] + Description=Ollama Service + After=network-online.target + Wants=network-online.target + + [Service] + Type=simple + User=ollama + Group=ollama + Environment="OLLAMA_HOST=0.0.0.0:11434" + ExecStart=/usr/local/bin/ollama serve + Restart=always + RestartSec=3 + + [Install] + WantedBy=multi-user.target + permissions: '0644' + - path: /etc/systemd/system/cube-agent.service + content: | + [Unit] + Description=Cube Agent Service + After=network-online.target ollama.service + Wants=network-online.target + + [Service] + Type=simple + EnvironmentFile=/etc/cube/agent.env + ExecStart=/usr/local/bin/cube-agent + Restart=on-failure + RestartSec=5 + StartLimitBurst=5 + StartLimitIntervalSec=60 + + [Install] + WantedBy=multi-user.target + permissions: '0644' + - path: /usr/local/bin/pull-ollama-models.sh + content: | + #!/bin/bash + # Wait for ollama to be ready + for i in $(seq 1 60); do + if curl -s http://localhost:11434/api/version > /dev/null 2>&1; then + break + fi + sleep 2 + done + # Pull models + /usr/local/bin/ollama pull tinyllama:1.1b + /usr/local/bin/ollama pull starcoder2:3b + /usr/local/bin/ollama pull nomic-embed-text:v1.5 + permissions: '0755' + - path: /etc/modules-load.d/sev-snp.conf + content: | + # AMD SEV-SNP guest modules + sev-guest + ccp + permissions: '0644' + +runcmd: + - | + # Install custom SNP-compatible kernel from seed image if present + mkdir -p /mnt/seed + dev="$(blkid -L cidata || blkid -L CIDATA || true)" + if [ -n "$dev" ]; then + mount "$dev" /mnt/seed + if ls /mnt/seed/debs/*.deb 2>/dev/null; then + cp /mnt/seed/debs/*.deb /root/ + dpkg -i /root/*.deb || apt-get -y -f install + update-grub + sync + fi + umount /mnt/seed || true + fi + - echo 'ultraviolet:password' | chpasswd + - | + cat > /etc/ssh/sshd_config.d/60-cloudimg-settings.conf << 'SSHEOF' + PasswordAuthentication yes + SSHEOF + - systemctl restart sshd + - sleep 2 + # SEV-SNP Setup (local only — not needed on cloud providers like GCP/Azure + # which handle SEV-SNP at the hypervisor level) + - | + # Load SEV-SNP guest modules (local deployment only) + modprobe ccp 2>/dev/null || echo "CCP module not available" + modprobe sev-guest 2>/dev/null || echo "sev-guest module not available" + # Verify SEV-SNP guest support + if [ -e /dev/sev-guest ]; then + echo "SEV-SNP: Running inside an SEV-SNP protected VM" + echo "SEV-SNP attestation available via /dev/sev-guest" + else + echo "SEV-SNP: /dev/sev-guest not detected (expected on cloud providers)" + fi + - mkdir -p /etc/cube + - mkdir -p /etc/cube/certs + - | + # Generate CA certificate + openssl req -x509 -newkey rsa:4096 -keyout /etc/cube/certs/ca.key -out /etc/cube/certs/ca.crt -days 365 -nodes -subj "/CN=Cube-CA" + # Generate server certificate + openssl req -newkey rsa:4096 -keyout /etc/cube/certs/server.key -out /etc/cube/certs/server.csr -nodes -subj "/CN=cube-agent" + openssl x509 -req -in /etc/cube/certs/server.csr -CA /etc/cube/certs/ca.crt -CAkey /etc/cube/certs/ca.key -CAcreateserial -out /etc/cube/certs/server.crt -days 365 + # Generate client certificate for mTLS + openssl req -newkey rsa:4096 -keyout /etc/cube/certs/client.key -out /etc/cube/certs/client.csr -nodes -subj "/CN=cube-client" + openssl x509 -req -in /etc/cube/certs/client.csr -CA /etc/cube/certs/ca.crt -CAkey /etc/cube/certs/ca.key -CAcreateserial -out /etc/cube/certs/client.crt -days 365 + # Set permissions + chmod 600 /etc/cube/certs/*.key + chmod 644 /etc/cube/certs/*.crt + - mkdir -p /var/lib/ollama + - mkdir -p /home/ollama/.ollama + - chown -R ollama:ollama /var/lib/ollama + - chown -R ollama:ollama /home/ollama + - curl -fsSL https://ollama.com/install.sh | sh + - | + # Install Go (required version from go.mod) + GO_VERSION=$(curl -fsSL https://go.dev/VERSION?m=text | head -1 | sed 's/go//') + curl -fsSL https://dl.google.com/go/go${GO_VERSION}.linux-amd64.tar.gz -o /tmp/go.tar.gz + rm -rf /usr/local/go + tar -C /usr/local -xzf /tmp/go.tar.gz + rm /tmp/go.tar.gz + - git clone https://github.com/ultravioletrs/cube.git /tmp/cube + - cd /tmp/cube && git fetch origin pull/88/head:pr-88 && git checkout pr-88 + - export HOME=/root + - cd /tmp/cube && /usr/local/go/bin/go build -ldflags="-s -w" -o /usr/local/bin/cube-agent ./cmd/agent + - systemctl daemon-reload + - systemctl enable ollama.service + - systemctl start ollama.service + - systemctl enable cube-agent.service + - systemctl start cube-agent.service + - nohup /usr/local/bin/pull-ollama-models.sh > /var/log/ollama-pull.log 2>&1 & + +final_message: "Cube Agent (SEV-SNP) and Ollama services started." diff --git a/hal/ubuntu/user-data-tdx.yaml b/hal/ubuntu/user-data-tdx.yaml new file mode 100644 index 00000000..de6a52b1 --- /dev/null +++ b/hal/ubuntu/user-data-tdx.yaml @@ -0,0 +1,164 @@ +#cloud-config +# Copyright (c) Ultraviolet +# SPDX-License-Identifier: Apache-2.0 +# +# Cloud-init for local QEMU Intel TDX VMs +# Used by: qemu.sh start_tdx +# For cloud deployments (GCP/Azure), see cloud/ directory instead + +package_update: true +package_upgrade: false + +users: + - name: ultraviolet + plain_text_passwd: password + lock_passwd: false + sudo: ALL=(ALL) NOPASSWD:ALL + shell: /bin/bash + - name: ollama + system: true + home: /var/lib/ollama + shell: /usr/sbin/nologin + +ssh_pwauth: true + +packages: + - curl + - git + - build-essential + +write_files: + - path: /etc/cube/agent.env + content: | + UV_CUBE_AGENT_LOG_LEVEL=info + UV_CUBE_AGENT_HOST=0.0.0.0 + UV_CUBE_AGENT_PORT=7001 + UV_CUBE_AGENT_INSTANCE_ID=cube-agent-01 + UV_CUBE_AGENT_TARGET_URL=http://localhost:11434 + UV_CUBE_AGENT_SERVER_CERT=/etc/cube/certs/server.crt + UV_CUBE_AGENT_SERVER_KEY=/etc/cube/certs/server.key + UV_CUBE_AGENT_SERVER_CA_CERTS=/etc/cube/certs/ca.crt + UV_CUBE_AGENT_CA_URL=https://prism.ultraviolet.rs/am-certs + # TDX-specific settings + AGENT_OS_TYPE=tdx + permissions: '0644' + - path: /etc/systemd/system/ollama.service + content: | + [Unit] + Description=Ollama Service + After=network-online.target + Wants=network-online.target + + [Service] + Type=simple + User=ollama + Group=ollama + Environment="OLLAMA_HOST=0.0.0.0:11434" + ExecStart=/usr/local/bin/ollama serve + Restart=always + RestartSec=3 + + [Install] + WantedBy=multi-user.target + permissions: '0644' + - path: /etc/systemd/system/cube-agent.service + content: | + [Unit] + Description=Cube Agent Service + After=network-online.target ollama.service + Wants=network-online.target + + [Service] + Type=simple + EnvironmentFile=/etc/cube/agent.env + ExecStart=/usr/local/bin/cube-agent + Restart=on-failure + RestartSec=5 + StartLimitBurst=5 + StartLimitIntervalSec=60 + + [Install] + WantedBy=multi-user.target + permissions: '0644' + - path: /usr/local/bin/pull-ollama-models.sh + content: | + #!/bin/bash + # Wait for ollama to be ready + for i in $(seq 1 60); do + if curl -s http://localhost:11434/api/version > /dev/null 2>&1; then + break + fi + sleep 2 + done + # Pull models + /usr/local/bin/ollama pull tinyllama:1.1b + /usr/local/bin/ollama pull starcoder2:3b + /usr/local/bin/ollama pull nomic-embed-text:v1.5 + permissions: '0755' + +runcmd: + - echo 'ultraviolet:password' | chpasswd + - | + cat > /etc/ssh/sshd_config.d/60-cloudimg-settings.conf << 'SSHEOF' + PasswordAuthentication yes + SSHEOF + - systemctl restart sshd + - sleep 2 + # TDX Setup for Ubuntu 24.04+ + # Install TDX kernel modules and configure module loading + - | + KERNEL_VERSION=$(uname -r) + echo "Installing TDX modules for kernel ${KERNEL_VERSION}..." + apt-get update + apt-get install -y linux-modules-extra-${KERNEL_VERSION} || apt-get install -y linux-modules-extra-generic || echo "Module package install failed" + # Configure TDX module to load at boot + mkdir -p /etc/modules-load.d + echo "tdx_guest" > /etc/modules-load.d/tdx.conf + # Load TDX guest module immediately + modprobe tdx_guest 2>/dev/null || echo "tdx_guest module not available (will load after reboot)" + # Verify TDX device + if [ -e /dev/tdx_guest ]; then + echo "TDX: /dev/tdx_guest device available" + elif [ -e /sys/firmware/tdx ]; then + echo "TDX: Running inside a TDX Trust Domain" + else + echo "TDX: Module configured, will be available after reboot" + fi + - mkdir -p /etc/cube + - mkdir -p /etc/cube/certs + - | + # Generate CA certificate + openssl req -x509 -newkey rsa:4096 -keyout /etc/cube/certs/ca.key -out /etc/cube/certs/ca.crt -days 365 -nodes -subj "/CN=Cube-CA" + # Generate server certificate + openssl req -newkey rsa:4096 -keyout /etc/cube/certs/server.key -out /etc/cube/certs/server.csr -nodes -subj "/CN=cube-agent" + openssl x509 -req -in /etc/cube/certs/server.csr -CA /etc/cube/certs/ca.crt -CAkey /etc/cube/certs/ca.key -CAcreateserial -out /etc/cube/certs/server.crt -days 365 + # Generate client certificate for mTLS + openssl req -newkey rsa:4096 -keyout /etc/cube/certs/client.key -out /etc/cube/certs/client.csr -nodes -subj "/CN=cube-client" + openssl x509 -req -in /etc/cube/certs/client.csr -CA /etc/cube/certs/ca.crt -CAkey /etc/cube/certs/ca.key -CAcreateserial -out /etc/cube/certs/client.crt -days 365 + # Set permissions + chmod 600 /etc/cube/certs/*.key + chmod 644 /etc/cube/certs/*.crt + - mkdir -p /var/lib/ollama + - mkdir -p /home/ollama/.ollama + - chown -R ollama:ollama /var/lib/ollama + - chown -R ollama:ollama /home/ollama + - curl -fsSL https://ollama.com/install.sh | sh + - | + # Install Go (required version from go.mod) + GO_VERSION=$(curl -fsSL https://go.dev/VERSION?m=text | head -1 | sed 's/go//') + curl -fsSL https://dl.google.com/go/go${GO_VERSION}.linux-amd64.tar.gz -o /tmp/go.tar.gz + rm -rf /usr/local/go + tar -C /usr/local -xzf /tmp/go.tar.gz + rm /tmp/go.tar.gz + - git clone https://github.com/ultravioletrs/cube.git /tmp/cube + - cd /tmp/cube && git fetch origin pull/88/head:pr-88 && git checkout pr-88 + - export HOME=/root + - cd /tmp/cube && /usr/local/go/bin/go build -ldflags="-s -w" -o /usr/local/bin/cube-agent ./cmd/agent + - systemctl daemon-reload + - systemctl enable ollama.service + - systemctl start ollama.service + - systemctl enable cube-agent.service + - systemctl start cube-agent.service + - nohup /usr/local/bin/pull-ollama-models.sh > /var/log/ollama-pull.log 2>&1 & + +final_message: "Cube Agent (TDX) and Ollama services started." diff --git a/hal/ubuntu/user-data-vllm-regular.yaml b/hal/ubuntu/user-data-vllm-regular.yaml new file mode 100644 index 00000000..8d649038 --- /dev/null +++ b/hal/ubuntu/user-data-vllm-regular.yaml @@ -0,0 +1,143 @@ +#cloud-config +# Copyright (c) Ultraviolet +# SPDX-License-Identifier: Apache-2.0 +# +# Cloud-init for local QEMU regular (non-CVM) VMs with vLLM backend +# Used by: qemu.sh start_regular +# vLLM will use GPU automatically if available, otherwise falls back to CPU + +package_update: true +package_upgrade: false + +users: + - name: ultraviolet + plain_text_passwd: password + lock_passwd: false + sudo: ALL=(ALL) NOPASSWD:ALL + shell: /bin/bash + - name: vllm + system: true + home: /var/lib/vllm + shell: /usr/sbin/nologin + +ssh_pwauth: true + +packages: + - curl + - git + - build-essential + - python3 + - python3-pip + - python3-venv + +write_files: + - path: /etc/cube/agent.env + content: | + UV_CUBE_AGENT_LOG_LEVEL=info + UV_CUBE_AGENT_HOST=0.0.0.0 + UV_CUBE_AGENT_PORT=7001 + UV_CUBE_AGENT_INSTANCE_ID=cube-agent-01 + UV_CUBE_AGENT_TARGET_URL=http://localhost:8000 + UV_CUBE_AGENT_SERVER_CERT=/etc/cube/certs/server.crt + UV_CUBE_AGENT_SERVER_KEY=/etc/cube/certs/server.key + UV_CUBE_AGENT_SERVER_CA_CERTS=/etc/cube/certs/ca.crt + UV_CUBE_AGENT_CA_URL=https://prism.ultraviolet.rs/am-certs + permissions: '0644' + - path: /etc/cube/vllm.env + content: | + VLLM_MODEL=${CUBE_VLLM_MODEL:-meta-llama/Llama-2-7b-hf} + VLLM_HOST=0.0.0.0 + VLLM_PORT=8000 + permissions: '0644' + - path: /etc/systemd/system/vllm.service + content: | + [Unit] + Description=vLLM Inference Server + After=network-online.target + Wants=network-online.target + + [Service] + Type=simple + User=vllm + Group=vllm + EnvironmentFile=/etc/cube/vllm.env + WorkingDirectory=/var/lib/vllm + ExecStart=/var/lib/vllm/venv/bin/python -m vllm.entrypoints.openai.api_server \ + --model ${VLLM_MODEL} \ + --host ${VLLM_HOST} \ + --port ${VLLM_PORT} + Restart=on-failure + RestartSec=10 + StartLimitBurst=3 + StartLimitIntervalSec=60 + + [Install] + WantedBy=multi-user.target + permissions: '0644' + - path: /etc/systemd/system/cube-agent.service + content: | + [Unit] + Description=Cube Agent Service + After=network-online.target vllm.service + Wants=network-online.target + + [Service] + Type=simple + EnvironmentFile=/etc/cube/agent.env + ExecStart=/usr/local/bin/cube-agent + Restart=on-failure + RestartSec=5 + StartLimitBurst=5 + StartLimitIntervalSec=60 + + [Install] + WantedBy=multi-user.target + permissions: '0644' + +runcmd: + - echo 'ultraviolet:password' | chpasswd + - | + cat > /etc/ssh/sshd_config.d/60-cloudimg-settings.conf << 'SSHEOF' + PasswordAuthentication yes + SSHEOF + - systemctl restart sshd + - sleep 2 + - mkdir -p /etc/cube + - mkdir -p /etc/cube/certs + - | + # Generate CA certificate + openssl req -x509 -newkey rsa:4096 -keyout /etc/cube/certs/ca.key -out /etc/cube/certs/ca.crt -days 365 -nodes -subj "/CN=Cube-CA" + # Generate server certificate + openssl req -newkey rsa:4096 -keyout /etc/cube/certs/server.key -out /etc/cube/certs/server.csr -nodes -subj "/CN=cube-agent" + openssl x509 -req -in /etc/cube/certs/server.csr -CA /etc/cube/certs/ca.crt -CAkey /etc/cube/certs/ca.key -CAcreateserial -out /etc/cube/certs/server.crt -days 365 + # Generate client certificate for mTLS + openssl req -newkey rsa:4096 -keyout /etc/cube/certs/client.key -out /etc/cube/certs/client.csr -nodes -subj "/CN=cube-client" + openssl x509 -req -in /etc/cube/certs/client.csr -CA /etc/cube/certs/ca.crt -CAkey /etc/cube/certs/ca.key -CAcreateserial -out /etc/cube/certs/client.crt -days 365 + # Set permissions + chmod 600 /etc/cube/certs/*.key + chmod 644 /etc/cube/certs/*.crt + - mkdir -p /var/lib/vllm + - chown -R vllm:vllm /var/lib/vllm + - | + # Install vLLM in virtual environment + sudo -u vllm python3 -m venv /var/lib/vllm/venv + sudo -u vllm /var/lib/vllm/venv/bin/pip install --upgrade pip + sudo -u vllm /var/lib/vllm/venv/bin/pip install vllm + - | + # Install Go (required version from go.mod) + GO_VERSION=$(curl -fsSL https://go.dev/VERSION?m=text | head -1 | sed 's/go//') + curl -fsSL https://dl.google.com/go/go${GO_VERSION}.linux-amd64.tar.gz -o /tmp/go.tar.gz + rm -rf /usr/local/go + tar -C /usr/local -xzf /tmp/go.tar.gz + rm /tmp/go.tar.gz + - git clone https://github.com/ultravioletrs/cube.git /tmp/cube + - cd /tmp/cube && git fetch origin pull/88/head:pr-88 && git checkout pr-88 + - export HOME=/root + - cd /tmp/cube && /usr/local/go/bin/go build -ldflags="-s -w" -o /usr/local/bin/cube-agent ./cmd/agent + - systemctl daemon-reload + - systemctl enable vllm.service + - systemctl start vllm.service + - systemctl enable cube-agent.service + - systemctl start cube-agent.service + +final_message: "Cube Agent (vLLM) and vLLM services started." diff --git a/hal/ubuntu/user-data-vllm-snp.yaml b/hal/ubuntu/user-data-vllm-snp.yaml new file mode 100644 index 00000000..0f26edf7 --- /dev/null +++ b/hal/ubuntu/user-data-vllm-snp.yaml @@ -0,0 +1,187 @@ +#cloud-config +# Copyright (c) Ultraviolet +# SPDX-License-Identifier: Apache-2.0 +# +# Cloud-init for local QEMU AMD SEV-SNP VMs with vLLM backend (Coconut SVSM) +# Used by: qemu.sh prepare_snp / start_snp +# vLLM will use GPU automatically if available, otherwise falls back to CPU +# For cloud deployments (GCP/Azure), see cloud/ directory instead + +package_update: true +package_upgrade: false + +growpart: + mode: auto + devices: ["/"] + ignore_growroot_disabled: false + +resize_rootfs: true + +users: + - name: ultraviolet + plain_text_passwd: password + lock_passwd: false + sudo: ALL=(ALL) NOPASSWD:ALL + shell: /bin/bash + - name: vllm + system: true + home: /var/lib/vllm + shell: /usr/sbin/nologin + +ssh_pwauth: true + +packages: + - curl + - git + - build-essential + - python3 + - python3-pip + - python3-venv + +write_files: + - path: /etc/cube/agent.env + content: | + UV_CUBE_AGENT_LOG_LEVEL=info + UV_CUBE_AGENT_HOST=0.0.0.0 + UV_CUBE_AGENT_PORT=7001 + UV_CUBE_AGENT_INSTANCE_ID=cube-agent-01 + UV_CUBE_AGENT_TARGET_URL=http://localhost:8000 + UV_CUBE_AGENT_SERVER_CERT=/etc/cube/certs/server.crt + UV_CUBE_AGENT_SERVER_KEY=/etc/cube/certs/server.key + UV_CUBE_AGENT_SERVER_CA_CERTS=/etc/cube/certs/ca.crt + UV_CUBE_AGENT_CA_URL=https://prism.ultraviolet.rs/am-certs + # SNP-specific settings + AGENT_OS_TYPE=snp + AGENT_VMPL=2 + permissions: '0644' + - path: /etc/cube/vllm.env + content: | + VLLM_MODEL=${CUBE_VLLM_MODEL:-meta-llama/Llama-2-7b-hf} + VLLM_HOST=0.0.0.0 + VLLM_PORT=8000 + permissions: '0644' + - path: /etc/systemd/system/vllm.service + content: | + [Unit] + Description=vLLM Inference Server + After=network-online.target + Wants=network-online.target + + [Service] + Type=simple + User=vllm + Group=vllm + EnvironmentFile=/etc/cube/vllm.env + WorkingDirectory=/var/lib/vllm + ExecStart=/var/lib/vllm/venv/bin/python -m vllm.entrypoints.openai.api_server \ + --model ${VLLM_MODEL} \ + --host ${VLLM_HOST} \ + --port ${VLLM_PORT} + Restart=on-failure + RestartSec=10 + StartLimitBurst=3 + StartLimitIntervalSec=60 + + [Install] + WantedBy=multi-user.target + permissions: '0644' + - path: /etc/systemd/system/cube-agent.service + content: | + [Unit] + Description=Cube Agent Service + After=network-online.target vllm.service + Wants=network-online.target + + [Service] + Type=simple + EnvironmentFile=/etc/cube/agent.env + ExecStart=/usr/local/bin/cube-agent + Restart=on-failure + RestartSec=5 + StartLimitBurst=5 + StartLimitIntervalSec=60 + + [Install] + WantedBy=multi-user.target + permissions: '0644' + - path: /etc/modules-load.d/sev-snp.conf + content: | + # AMD SEV-SNP guest modules + sev-guest + ccp + permissions: '0644' + +runcmd: + - | + # Install custom SNP-compatible kernel from seed image if present + mkdir -p /mnt/seed + dev="$(blkid -L cidata || blkid -L CIDATA || true)" + if [ -n "$dev" ]; then + mount "$dev" /mnt/seed + if ls /mnt/seed/debs/*.deb 2>/dev/null; then + cp /mnt/seed/debs/*.deb /root/ + dpkg -i /root/*.deb || apt-get -y -f install + update-grub + sync + fi + umount /mnt/seed || true + fi + - echo 'ultraviolet:password' | chpasswd + - | + cat > /etc/ssh/sshd_config.d/60-cloudimg-settings.conf << 'SSHEOF' + PasswordAuthentication yes + SSHEOF + - systemctl restart sshd + - sleep 2 + # SEV-SNP Setup (local only — not needed on cloud providers like GCP/Azure + # which handle SEV-SNP at the hypervisor level) + - | + # Load SEV-SNP guest modules (local deployment only) + modprobe ccp 2>/dev/null || echo "CCP module not available" + modprobe sev-guest 2>/dev/null || echo "sev-guest module not available" + # Verify SEV-SNP guest support + if [ -e /dev/sev-guest ]; then + echo "SEV-SNP: Running inside an SEV-SNP protected VM" + echo "SEV-SNP attestation available via /dev/sev-guest" + else + echo "SEV-SNP: /dev/sev-guest not detected (expected on cloud providers)" + fi + - mkdir -p /etc/cube + - mkdir -p /etc/cube/certs + - | + # Generate CA certificate + openssl req -x509 -newkey rsa:4096 -keyout /etc/cube/certs/ca.key -out /etc/cube/certs/ca.crt -days 365 -nodes -subj "/CN=Cube-CA" + # Generate server certificate + openssl req -newkey rsa:4096 -keyout /etc/cube/certs/server.key -out /etc/cube/certs/server.csr -nodes -subj "/CN=cube-agent" + openssl x509 -req -in /etc/cube/certs/server.csr -CA /etc/cube/certs/ca.crt -CAkey /etc/cube/certs/ca.key -CAcreateserial -out /etc/cube/certs/server.crt -days 365 + # Generate client certificate for mTLS + openssl req -newkey rsa:4096 -keyout /etc/cube/certs/client.key -out /etc/cube/certs/client.csr -nodes -subj "/CN=cube-client" + openssl x509 -req -in /etc/cube/certs/client.csr -CA /etc/cube/certs/ca.crt -CAkey /etc/cube/certs/ca.key -CAcreateserial -out /etc/cube/certs/client.crt -days 365 + # Set permissions + chmod 600 /etc/cube/certs/*.key + chmod 644 /etc/cube/certs/*.crt + - mkdir -p /var/lib/vllm + - chown -R vllm:vllm /var/lib/vllm + - | + # Install vLLM in virtual environment + sudo -u vllm python3 -m venv /var/lib/vllm/venv + sudo -u vllm /var/lib/vllm/venv/bin/pip install --upgrade pip + sudo -u vllm /var/lib/vllm/venv/bin/pip install vllm + - | + # Install Go (required version from go.mod) + GO_VERSION=$(curl -fsSL https://go.dev/VERSION?m=text | head -1 | sed 's/go//') + curl -fsSL https://dl.google.com/go/go${GO_VERSION}.linux-amd64.tar.gz -o /tmp/go.tar.gz + rm -rf /usr/local/go + tar -C /usr/local -xzf /tmp/go.tar.gz + rm /tmp/go.tar.gz + - git clone https://github.com/ultravioletrs/cube.git /tmp/cube + - cd /tmp/cube && git fetch origin pull/88/head:pr-88 && git checkout pr-88 + - export HOME=/root + - cd /tmp/cube && /usr/local/go/bin/go build -ldflags="-s -w" -o /usr/local/bin/cube-agent ./cmd/agent + - systemctl daemon-reload + - systemctl enable vllm.service + - systemctl start vllm.service + - systemctl enable cube-agent.service + - systemctl start cube-agent.service + +final_message: "Cube Agent (SEV-SNP + vLLM) and vLLM services started." diff --git a/hal/ubuntu/user-data-vllm-tdx.yaml b/hal/ubuntu/user-data-vllm-tdx.yaml new file mode 100644 index 00000000..9cddcd10 --- /dev/null +++ b/hal/ubuntu/user-data-vllm-tdx.yaml @@ -0,0 +1,162 @@ +#cloud-config +# Copyright (c) Ultraviolet +# SPDX-License-Identifier: Apache-2.0 +# +# Cloud-init for local QEMU Intel TDX VMs with vLLM backend +# Used by: qemu.sh start_tdx +# vLLM will use GPU automatically if available, otherwise falls back to CPU +# For cloud deployments (GCP/Azure), see cloud/ directory instead + +package_update: true +package_upgrade: false + +users: + - name: ultraviolet + plain_text_passwd: password + lock_passwd: false + sudo: ALL=(ALL) NOPASSWD:ALL + shell: /bin/bash + - name: vllm + system: true + home: /var/lib/vllm + shell: /usr/sbin/nologin + +ssh_pwauth: true + +packages: + - curl + - git + - build-essential + - python3 + - python3-pip + - python3-venv + +write_files: + - path: /etc/cube/agent.env + content: | + UV_CUBE_AGENT_LOG_LEVEL=info + UV_CUBE_AGENT_HOST=0.0.0.0 + UV_CUBE_AGENT_PORT=7001 + UV_CUBE_AGENT_INSTANCE_ID=cube-agent-01 + UV_CUBE_AGENT_TARGET_URL=http://localhost:8000 + UV_CUBE_AGENT_SERVER_CERT=/etc/cube/certs/server.crt + UV_CUBE_AGENT_SERVER_KEY=/etc/cube/certs/server.key + UV_CUBE_AGENT_SERVER_CA_CERTS=/etc/cube/certs/ca.crt + UV_CUBE_AGENT_CA_URL=https://prism.ultraviolet.rs/am-certs + # TDX-specific settings + AGENT_OS_TYPE=tdx + permissions: '0644' + - path: /etc/cube/vllm.env + content: | + VLLM_MODEL=${CUBE_VLLM_MODEL:-meta-llama/Llama-2-7b-hf} + VLLM_HOST=0.0.0.0 + VLLM_PORT=8000 + permissions: '0644' + - path: /etc/systemd/system/vllm.service + content: | + [Unit] + Description=vLLM Inference Server + After=network-online.target + Wants=network-online.target + + [Service] + Type=simple + User=vllm + Group=vllm + EnvironmentFile=/etc/cube/vllm.env + WorkingDirectory=/var/lib/vllm + ExecStart=/var/lib/vllm/venv/bin/python -m vllm.entrypoints.openai.api_server \ + --model ${VLLM_MODEL} \ + --host ${VLLM_HOST} \ + --port ${VLLM_PORT} + Restart=on-failure + RestartSec=10 + StartLimitBurst=3 + StartLimitIntervalSec=60 + + [Install] + WantedBy=multi-user.target + permissions: '0644' + - path: /etc/systemd/system/cube-agent.service + content: | + [Unit] + Description=Cube Agent Service + After=network-online.target vllm.service + Wants=network-online.target + + [Service] + Type=simple + EnvironmentFile=/etc/cube/agent.env + ExecStart=/usr/local/bin/cube-agent + Restart=on-failure + RestartSec=5 + StartLimitBurst=5 + StartLimitIntervalSec=60 + + [Install] + WantedBy=multi-user.target + permissions: '0644' + +runcmd: + - echo 'ultraviolet:password' | chpasswd + - | + cat > /etc/ssh/sshd_config.d/60-cloudimg-settings.conf << 'SSHEOF' + PasswordAuthentication yes + SSHEOF + - systemctl restart sshd + - sleep 2 + # TDX Setup for Ubuntu 24.04+ + - | + KERNEL_VERSION=$(uname -r) + echo "Installing TDX modules for kernel ${KERNEL_VERSION}..." + apt-get update + apt-get install -y linux-modules-extra-${KERNEL_VERSION} || apt-get install -y linux-modules-extra-generic || echo "Module package install failed" + mkdir -p /etc/modules-load.d + echo "tdx_guest" > /etc/modules-load.d/tdx.conf + modprobe tdx_guest 2>/dev/null || echo "tdx_guest module not available (will load after reboot)" + if [ -e /dev/tdx_guest ]; then + echo "TDX: /dev/tdx_guest device available" + elif [ -e /sys/firmware/tdx ]; then + echo "TDX: Running inside a TDX Trust Domain" + else + echo "TDX: Module configured, will be available after reboot" + fi + - mkdir -p /etc/cube + - mkdir -p /etc/cube/certs + - | + # Generate CA certificate + openssl req -x509 -newkey rsa:4096 -keyout /etc/cube/certs/ca.key -out /etc/cube/certs/ca.crt -days 365 -nodes -subj "/CN=Cube-CA" + # Generate server certificate + openssl req -newkey rsa:4096 -keyout /etc/cube/certs/server.key -out /etc/cube/certs/server.csr -nodes -subj "/CN=cube-agent" + openssl x509 -req -in /etc/cube/certs/server.csr -CA /etc/cube/certs/ca.crt -CAkey /etc/cube/certs/ca.key -CAcreateserial -out /etc/cube/certs/server.crt -days 365 + # Generate client certificate for mTLS + openssl req -newkey rsa:4096 -keyout /etc/cube/certs/client.key -out /etc/cube/certs/client.csr -nodes -subj "/CN=cube-client" + openssl x509 -req -in /etc/cube/certs/client.csr -CA /etc/cube/certs/ca.crt -CAkey /etc/cube/certs/ca.key -CAcreateserial -out /etc/cube/certs/client.crt -days 365 + # Set permissions + chmod 600 /etc/cube/certs/*.key + chmod 644 /etc/cube/certs/*.crt + - mkdir -p /var/lib/vllm + - chown -R vllm:vllm /var/lib/vllm + - | + # Install vLLM in virtual environment + sudo -u vllm python3 -m venv /var/lib/vllm/venv + sudo -u vllm /var/lib/vllm/venv/bin/pip install --upgrade pip + sudo -u vllm /var/lib/vllm/venv/bin/pip install vllm + - | + # Install Go (required version from go.mod) + GO_VERSION=$(curl -fsSL https://go.dev/VERSION?m=text | head -1 | sed 's/go//') + curl -fsSL https://dl.google.com/go/go${GO_VERSION}.linux-amd64.tar.gz -o /tmp/go.tar.gz + rm -rf /usr/local/go + tar -C /usr/local -xzf /tmp/go.tar.gz + rm /tmp/go.tar.gz + - git clone https://github.com/ultravioletrs/cube.git /tmp/cube + - cd /tmp/cube && git fetch origin pull/88/head:pr-88 && git checkout pr-88 + - export HOME=/root + - cd /tmp/cube && /usr/local/go/bin/go build -ldflags="-s -w" -o /usr/local/bin/cube-agent ./cmd/agent + - systemctl daemon-reload + - systemctl enable vllm.service + - systemctl start vllm.service + - systemctl enable cube-agent.service + - systemctl start cube-agent.service + +final_message: "Cube Agent (TDX + vLLM) and vLLM services started."