pixi is unable to 'pixi lock' on an Mac OSX ARM64 for a recipe targetting linux-64 with cuda deps

[kratsg]

the following toml file -- out of the box, does not succeed when I try to pixi lock on either my mac or my linux-64 with a cuda lib

[workspace]
name = "ml-platform"
version = "2026.2.0"
description = "MaNIAC Lab ML platform environment"
channels = ["conda-forge"]
platforms = ["linux-64"]

[activation.env]
CONDA_OVERRIDE_CUDA = "12.8"

[dependencies]
# Python
python = "*"

# CERN ROOT
root = ">=6.32"

# Java (OpenJDK 8 in ml_base)
openjdk = "*"

# ML frameworks
tensorflow-gpu = "*"
keras = "*"

# Data science
numpy = "*"
pandas = "*"
scipy = "*"
scikit-learn = "*"
pyarrow = "*"
h5py = "*"
pytables = "*"
cython = "*"

# Visualization
matplotlib = "*"
seaborn = "*"
bokeh = "*"
graphviz = "*"

# Jupyter ecosystem
jupyterlab = "*"
jupyter = "*"
ipykernel = "*"
metakernel = "*"
ipywidgets = "*"
jupyterlab-git = "*"
rise = "*"

# HEP
uproot = "*"

# System tools (replacing apt-get from ml_base)
curl = "*"
wget = "*"
unzip = "*"
zip = "*"
vim = "*"
jq = "*"
rsync = "*"
git = "*"
make = "*"
compilers = "*"
pkg-config = "*"

# Libraries (replacing apt-get dev packages)
freetype = "*"
libpng = "*"
zeromq = "*"
hdf5 = "*"
tbb = "*"
zlib = "*"

# Other
requests = "*"
pillow = "*"
tqdm = "*"
imageio = "*"

[pypi-dependencies]
# Packages NOT on conda-forge (or easier from PyPI)
tensorflow-datasets = "*"
keras-tuner = "*"
plumbum = "*"
jupyter-bokeh = "*"
qtpy = "*"
elasticsearch = "*"
gymnasium = "*"          # gym is deprecated, gymnasium is the successor
JSAnimation = "*"
dask-labextension = "*"
atlasify = "*"
rucio-jupyterlab = "*"

the following above gives me

# pixi info
System
------------
       Pixi version: 0.63.2
        TLS backend: rustls
           Platform: linux-64
   Virtual packages: __unix=0=0
                   : __linux=6.10.14=0
                   : __glibc=2.39=0
                   : __cuda=12.8=0
                   : __archspec=1=x86_64
          Cache dir: /root/.cache/rattler/cache
       Auth storage: /root/.rattler/credentials.json
   Config locations: No config files found

Global
------------
            Bin dir: /root/.pixi/bin
    Environment dir: /root/.pixi/envs
       Manifest dir: /root/.pixi/manifests/pixi-global.toml

Workspace
------------
               Name: ml-platform
            Version: 2026.2.0
      Manifest file: /Users/kratsg/maniacdockerimages/ml_platform/pixi.toml

Environments
------------
        Environment: default
           Features: default
           Channels: conda-forge
   Dependency count: 46
       Dependencies: python, root, openjdk, tensorflow-gpu, keras, numpy, pandas, scipy, scikit-learn, pyarrow, h5py, pytables, cython, matplotlib, seaborn, bokeh, graphviz, jupyterlab, jupyter, ipykernel, metakernel, ipywidgets, jupyterlab-git, rise, uproot, curl, wget, unzip, zip, vim, jq, rsync, git, make, compilers, pkg-config, freetype, libpng, zeromq, hdf5, tbb, zlib, requests, pillow, tqdm, imageio
  PyPI Dependencies: tensorflow-datasets, keras-tuner, plumbum, jupyter-bokeh, qtpy, elasticsearch, gymnasium, JSAnimation, dask-labextension, atlasify, rucio-jupyterlab
   Target platforms: linux-64
    Prefix location: /Users/kratsg/maniacdockerimages/ml_platform/.pixi/envs/default

# pixi lock
Error:   x failed to solve requirements of environment 'default' for platform 'linux-64'
  |->   x failed to solve the environment
  |
  `-> Cannot solve the request because of: tensorflow-gpu * cannot be installed because there are no viable options:
      ├─ tensorflow-gpu 2.19.1 would require
      │  └─ tensorflow ==2.19.1 cuda128*_253, which cannot be installed because there are no viable options:
      │     └─ tensorflow 2.19.1 | 2.19.1 | 2.19.1 would require
      │        └─ __cuda *, for which no candidates were found.
      ├─ tensorflow-gpu 2.19.1 would require
      │  └─ tensorflow ==2.19.1 cuda129*_252, which cannot be installed because there are no viable options:
      │     └─ tensorflow 2.19.1 | 2.19.1 | 2.19.1 would require
      │        └─ __cuda *, for which no candidates were found.
      ├─ tensorflow-gpu 2.19.1 would require
      │  └─ tensorflow ==2.19.1 cuda128*_251, which cannot be installed because there are no viable options:
      │     └─ tensorflow 2.19.1 | 2.19.1 | 2.19.1 would require
      │        └─ __cuda *, for which no candidates were found.
      ├─ tensorflow-gpu 2.19.1 would require
      │  └─ tensorflow ==2.19.1 cuda128*_250, which cannot be installed because there are no viable options:
      │     └─ tensorflow 2.19.1 | 2.19.1 | 2.19.1 would require
      │        └─ __cuda *, for which no candidates were found.
      ├─ tensorflow-gpu 2.19.1 would require
      │  └─ tensorflow ==2.19.1 cuda128*_203, which cannot be installed because there are no viable options:
      │     └─ tensorflow 2.19.1 | 2.19.1 | 2.19.1 would require
      │        └─ __cuda *, for which no candidates were found.
...
lots of output
...

but if I manually pixi add each dependency one-by-one... I get this toml

[workspace]
authors = ["Giordon Stark <kratsg@gmail.com>"]
channels = ["conda-forge"]
name = "my_workspace"
platforms = ["linux-64"]
version = "0.1.0"

[tasks]

[dependencies]
tensorflow-gpu = ">=2.6.2,<3"
root = ">=6.26.4,<7"
python = ">=3.9.15,<3.10"
openjdk = ">=11.0.1,<12"
keras = ">=2.6.0,<3"
numpy = ">=1.19.5,<2"
pandas = ">=1.4.4,<2"
scipy = ">=1.9.1,<2"
scikit-learn = ">=1.6.1,<2"
pyarrow = ">=6.0.1,<7"
h5py = ">=3.1.0,<4"
pytables = ">=3.6.1,<4"
cython = ">=3.1.3,<4"
matplotlib = ">=3.3.2,<4"
seaborn = ">=0.12.2,<0.13"
bokeh = ">=3.3.4,<4"
graphviz = ">=3.0.0,<4"
jupyterlab = ">=4.4.6,<5"
jupyter = ">=1.1.1,<2"
ipykernel = ">=6.30.1,<7"
metakernel = ">=0.30.3,<0.31"
ipywidgets = ">=8.1.7,<9"
jupyterlab-git = ">=0.51.2,<0.52"
rise = ">=5.7.1,<6"
uproot = ">=5.2.2,<6"
curl = ">=7.86.0,<8"
wget = ">=1.20.3,<2"
unzip = ">=6.0,<7"
zip = ">=3.0,<4"
vim = ">=9.0.814,<10"
jq = ">=1.8.1,<2"
rsync = ">=3.2.7,<4"
git = ">=2.39.1,<3"
make = ">=4.4.1,<5"
compilers = ">=1.5.1,<2"
pkg-config = ">=0.29.2,<0.30"
freetype = ">=2.12.1,<3"
libpng = ">=1.6.43,<2"
zeromq = ">=4.3.5,<5"
hdf5 = ">=1.10.6,<2"
tbb = ">=2021.7.0,<2022"
zlib = ">=1.2.13,<2"
requests = ">=2.32.5,<3"
pillow = ">=9.2.0,<10"
tqdm = ">=4.67.1,<5"
imageio = ">=2.37.0,<3"

but it's not clear to me how to debug.

Here is the full pixi log error message for version conflict: pixi.log

[matthewfeickert]

@kratsg I think the issue is that you're not using system-requirements. CONDA_OVERRIDE_CUDA by itself is just mocking the existence of the virtual package on your platform, it doesn't do the same thing as setting a CUDA system-requirements. c.f. Section 6.2 CUDA hardware accelerated environment creation of https://doi.org/10.25080/nwuf8465 as well as the PyTorch Installation tutorial-like example.

For example, on a machine with

$ nvidia-smi --version
NVIDIA-SMI version  : 590.48.01
NVML version        : 590.48
DRIVER version      : 590.48.01
CUDA Version        : 13.1

$ pixi init debug && cd debug
$ pixi workspace system-requirements add cuda 12.8
$ pixi add tensorflow-gpu
✔ Added tensorflow-gpu >=2.19.1,<3

solves to the following Pixi manifest

[workspace]
channels = ["conda-forge"]
name = "debug"
platforms = ["linux-64"]
version = "0.1.0"

[tasks]

[dependencies]
tensorflow-gpu = ">=2.19.1,<3"

[system-requirements]
cuda = "12.8"

If I don't have a system-requirements table then Pixi will resolve to tensorflow-gpu v2.6.2

$ cd .. && rm -rf ./debug && pixi init debug && cd debug
$ pixi add tensorflow-gpu
✔ Added tensorflow-gpu >=2.6.2,<3

I'd start with repeating the commands above and if that solves seeing if you can get everything by adding a system-requirements table and then running

rm pixi.lock && pixi lock

with the first workspace you gave.

If that doesn't work, can you determine the minimal dependency set in an pixi add command that results in the error?

[kratsg]

[system-requirements] solves this in fact! This is really useful, and for some reason it isn't mentioned in the pixi documentation. I was looking [here]https://pixi.prefix.dev/latest/workspace/system_requirements/#available-override-options) but somehow missed the rest of it. In fact I'm a bit confused by this section and it's not clear to me why I would need the env variables when the toml can be modified as described above.

[matthewfeickert]

In fact I'm a bit confused by this section and it's not clear to me why I would need the env variables when the toml can be modified as described above.

@kratsg The system-requirements table is used to provide information for what Pixi expects the host system to be able to support in terms of requirements and execution time and is used at dependency resolve time to ensure that the correct versions are found based on the virtual packages. If a system that does not meet the system-requirements wants to force an override of the virtual packages present, such as for installation, then the CONDA_OVERRIDE_ environmental variables are necessary.

As an example, on a linux-64 machine without an NVIDIA GPU driver (i.e. the __cuda virtual package is not present) the following will happen

$ pixi init example && cd example
$ pixi workspace system-requirements add cuda 12.0
$ pixi add pytorch-gpu
Error:   x Cannot install environment 'default'
  `-> Virtual package '__cuda' does not match any of the available virtual packages on your machine: [__linux=6.8.0=0, __glibc=2.39=0, __unix=0=0, __archspec=1=skylake]
  help:  You can mock the virtual package by overriding the environment variable, e.g.: '`CONDA_OVERRIDE_CUDA=12.0`'

and when CONDA_OVERRIDE_CUDA=12.0 is used the environment is able to be resolved, even though there is no actual GPU driver.

$ CONDA_OVERRIDE_CUDA=12.0 pixi add pytorch-gpu
Added pytorch-gpu >=2.10.0,<3

system-requirements is necessary for proper/correct environment resolution given declared expected hardware-level dependencies. CONDA_OVERRIDE_ environmental variables allow for forcing the resolution as if the given override virtual package was present. These two capabilities of Pixi allow for full resolution of computing environments for machine configurations that you do not have access to, which is very powerful.

In addition to Section 6.2, c.f. Section 7 Deploying environments to remote compute of https://doi.org/10.25080/nwuf8465 for more information on this.