Skip to content

maintenance: cuda and ci enchancements#32

Merged
ravnoor merged 25 commits intomainfrom
dev
Aug 21, 2025
Merged

maintenance: cuda and ci enchancements#32
ravnoor merged 25 commits intomainfrom
dev

Conversation

@ravnoor
Copy link
Copy Markdown
Contributor

@ravnoor ravnoor commented Aug 21, 2025

This pull request makes significant updates to the Docker images, CI/CD workflows, and documentation to modernize the environment, improve reproducibility, and streamline the build and test processes. The most important changes include upgrading the base images to CUDA 12.2/Ubuntu 22.04, switching from Miniconda to Miniforge for Conda environments, refactoring GitHub Actions workflows for more flexibility and efficiency, and updating documentation and scripts to reflect these changes.

Docker and Environment Modernization:

  • Upgraded both Dockerfile and ci/runner.Dockerfile to use nvidia/cuda:12.2.2-cudnn8-devel-ubuntu22.04 as the base image instead of the older CUDA 10/Ubuntu 18.04 images, ensuring compatibility with newer hardware and software stacks. [1] [2]
  • Replaced Miniconda with Miniforge for Conda environment management in both Dockerfiles, and updated the installation logic and environment variables accordingly. [1] [2]
  • Updated Conda environment creation steps to specify exact Python and pygpu versions, and ensured dependencies are installed in isolated environments. [1] [2]

CI/CD Workflow Refactoring:

  • Refactored GitHub Actions workflows to add support for manual triggers (workflow_dispatch), separate jobs for main and CI Docker images, and improved disk space management before builds. [1] [2]
  • Added cache usage to Docker builds for faster and more efficient CI runs, and improved tagging and metadata extraction for Docker images.
  • Updated Python build workflow to only test against Python 3.8 and upgraded actions to latest versions.
  • Improved test inference pipeline workflow by targeting only main and dev branches, switching to curl for dataset downloads, and running tests within the correct Conda environment. [1] [2]

Documentation and Usability Improvements:

  • Updated the README.md to recommend Miniforge over Miniconda, clarified Python and dependency versions, and revised setup instructions to match the new environment setup. [1] [2] [3]
  • Adjusted ci/runner.docker-compose.yml to use the new image naming and updated runner version and device IDs for compatibility with the new base image.
  • Fixed environment variable paths in ci/start-runner.sh to reflect the switch to Miniforge.

ravnoor and others added 25 commits August 15, 2025 23:29
…22.04

- support gpus with latest cuda compute capability
- last version with base py38 support available for download
- Version 3.7 was not found in the local cache
- Error: The version '3.7' with architecture 'x64' was not found for Ubuntu 24.04.
- The list of all available versions can be found here: https://raw.githubusercontent.com/actions/python-versions/main/versions-manifest.json
- default anaconda repo issues
docker: bump base os to ubuntu 22.04 with cuda 12.2 and cudnn 8
- and manual ad-hoc trigger
- no submodules needed for ci image
ci: build ci runner image in github actions
@ravnoor ravnoor self-assigned this Aug 21, 2025
@ravnoor ravnoor requested a review from Copilot August 21, 2025 02:28
Copy link
Copy Markdown
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull Request Overview

This pull request modernizes the deepFCD project's environment and CI/CD infrastructure by upgrading from CUDA 10/Ubuntu 18.04 to CUDA 12.2/Ubuntu 22.04, switching from Miniconda to Miniforge for package management, and enhancing GitHub Actions workflows for better efficiency and maintainability.

  • Upgraded base Docker images to use CUDA 12.2 with Ubuntu 22.04 for better hardware compatibility
  • Replaced Miniconda with Miniforge across all Docker images and documentation
  • Enhanced CI/CD workflows with manual triggers, improved caching, and separate jobs for main and CI images

Reviewed Changes

Copilot reviewed 9 out of 9 changed files in this pull request and generated 1 comment.

Show a summary per file
File Description
Dockerfile Upgraded to CUDA 12.2/Ubuntu 22.04 base image and replaced Miniconda with Miniforge
ci/runner.Dockerfile Modernized CI runner image with CUDA 12.2, simplified dependencies, and Miniforge installation
ci/start-runner.sh Updated PATH to reflect Miniforge installation location
ci/runner.docker-compose.yml Updated image references and runner version, adjusted GPU device configuration
README.md Updated installation instructions to recommend Miniforge over Miniconda with specific versions
.github/workflows/*.yml Enhanced workflows with manual triggers, caching, disk space management, and separate CI image builds

Tip: Customize your code reviews with copilot-instructions.md. Create the file or learn how to get started.

@ravnoor ravnoor merged commit e385c5c into main Aug 21, 2025
9 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants