Skip to content

README.md

Latest commit

 

History

History
227 lines (164 loc) · 4.96 KB

README.md

File metadata and controls

227 lines (164 loc) · 4.96 KB
Raw

Engram Deployment Guide

This guide covers deploying Engram to GPU-enabled infrastructure using Ansible with Quadlet (systemd-managed containers).

Prerequisites

Infrastructure

  • GPU server with NVIDIA GPU and drivers installed
  • Tailscale or equivalent mesh network configured (optional, for remote access)
  • ACME-compatible CA for TLS certificates (e.g., Let's Encrypt or step-ca)
  • Ansible control node with access to target server

Required Components

  • NVIDIA Container Toolkit installed on host
  • CDI (Container Device Interface) configured
  • Podman with systemd integration (Quadlet)
  • Caddy reverse proxy (deployed via your Caddy role)

Deployment Steps

1. Configure Secrets

Store credentials securely (e.g., using gopass, vault, or environment variables):

# Database password
gopass insert engram/db-password

# API key for authentication
gopass insert engram/api-key

2. Configure Group Variables

In your inventory group_vars file (e.g., group_vars/engram_servers.yml):

engram_db_password: "{{ lookup('community.general.gopass', 'engram/db-password') }}"
engram_api_key: "{{ lookup('community.general.gopass', 'engram/api-key') }}"

# Optional overrides
engram_embedding_model: "BAAI/bge-large-en-v1.5"
engram_log_level: "INFO"
engram_gpu_enabled: true

3. Deploy Infrastructure

From your ansible directory:

# Deploy Engram service
ansible-playbook -i inventory/hosts.yml playbooks/deploy-engram.yml

# Deploy Caddy reverse proxy
ansible-playbook -i inventory/hosts.yml playbooks/deploy-caddy.yml

4. Verify Deployment

Check service status:

# On target server
systemctl --user status engram-app.service
systemctl --user status engram-db.service
systemctl --user status caddy.service

# Check logs
journalctl --user -u engram-app.service -f

Test endpoint:

# Health check (direct)
curl http://localhost:8800/health

# Via Caddy reverse proxy
curl https://engram.your-domain.example.com/health

Caddy Configuration

Caddy acts as a reverse proxy with automatic TLS:

engram.your-domain.example.com {
    reverse_proxy localhost:8800

    tls {
        ca https://your-ca.example.com:9000/acme/acme/directory
    }
}

Certificate Renewal

Caddy automatically:

  1. Obtains certificates from your CA via ACME
  2. Renews certificates before expiration
  3. Reloads configuration without downtime

No manual intervention required for certificate lifecycle.

Container Architecture

Network Setup

  • Database and app communicate via an internal Docker/Podman network
  • External access via Caddy reverse proxy on host network

GPU Passthrough

  • Uses CDI for NVIDIA GPU access
  • Configured via --device nvidia.com/gpu=all
  • Requires nvidia-container-toolkit on host

Resource Limits

  • Engram App: 8GB RAM, 4 CPU cores (GPU workload)
  • PostgreSQL: 2GB RAM, 2 CPU cores
  • Caddy: 256MB RAM, 1 CPU core

Troubleshooting

GPU Not Available

Check NVIDIA container toolkit:

which nvidia-ctk
nvidia-ctk cdi list
ls -la /etc/cdi/nvidia.yaml

Generate CDI config if missing:

sudo nvidia-ctk cdi generate --output=/etc/cdi/nvidia.yaml

Database Connection Issues

Verify network and credentials:

podman exec engram-app env | grep DATABASE_URL
podman exec engram-db psql -U engram -c '\l'

Certificate Errors

Check CA connectivity:

curl -k https://your-ca.example.com:9000/health

View Caddy logs:

journalctl --user -u caddy.service -f

Service Won't Start

Check Quadlet configuration:

# View generated unit files
ls -la ~/.config/containers/systemd/

# Check for errors
systemctl --user daemon-reload
systemctl --user status engram-app.service
journalctl --user -u engram-app.service -n 50

Updating Engram

Application Updates

  1. Build new image with updated version tag
  2. Update engram_version in group_vars
  3. Re-run deployment playbook:
ansible-playbook -i inventory/hosts.yml playbooks/deploy-engram.yml

Quadlet will pull the new image and restart services.

Database Migrations

Migrations run automatically on startup via Alembic:

# Check migration status
podman exec engram-app alembic current
podman exec engram-app alembic history

Monitoring

Health Checks

  • Endpoint: http://localhost:8800/health
  • Interval: 30s
  • Start period: 60s (allows model download)

Logs

# Application logs
journalctl --user -u engram-app.service -f

# Database logs
journalctl --user -u engram-db.service -f

# Caddy logs
journalctl --user -u caddy.service -f

Resource Usage

podman stats engram-app engram-db
nvidia-smi  # GPU utilization

Configuration Reference

Key settings (configurable via environment variables):

  • engram_embedding_model: HuggingFace model name
  • engram_gpu_enabled: Enable/disable GPU passthrough
  • engram_port: Application port (default: 8800)
  • engram_workers: Uvicorn worker count
  • engram_db_shared_buffers: PostgreSQL memory tuning