[BUG] Large make_regression call fails in CUDA 13 base image with cublasLtMatmul error

[csadorf]

Describe the bug

cuml.datasets.make_regression fails in a fresh process for a large, valid input shape in the RAPIDS 26.08 CUDA 13 base image.

The failure occurs inside cublasLtMatmul before any estimator is constructed or fitted.

Steps/Code to reproduce bug

Run this single command on a host with an NVIDIA GPU:

docker run --rm --gpus all --entrypoint python \
  rapidsai/base@sha256:7d3527573dcfec08e119774118f9edfaa96b542bf0f58d168ea883bd7356df03 \
  -c "import cupy as cp; from cuml.datasets import make_regression; X, y = make_regression(n_samples=168_000_000, n_features=16, random_state=42, dtype='float32'); cp.cuda.get_current_stream().synchronize(); print(X.shape, y.shape)"

Actual behavior

The call fails after approximately two seconds:

RuntimeError: cuBLAS error encountered at:
.../raft/linalg/detail/cublaslt_wrappers.hpp line=299:
call='cublasLtMatmul(...)',
Reason=13:CUBLAS_STATUS_EXECUTION_FAILED

Expected behavior

The dataset should be generated successfully:

(168000000, 16) (168000000, 1)

The same call succeeds on the same GPU using the identical cuML 26.08.00a169 build in a local conda environment with cuBLAS 13.1.1.

Environment details

• Environment: Docker on bare metal
• Image tag: rapidsai/base:26.08a-cuda13-py3.14
• Image digest: sha256:7d3527573dcfec08e119774118f9edfaa96b542bf0f58d168ea883bd7356df03
• Image OS: Ubuntu 24.04 x86_64
• Python: 3.14.6
• cuML: 26.08.00a169
• CuPy: 14.1.1
• CUDA runtime reported by CuPy: 13.2
• cuBLAS runtime: 13.6.0
• GPU: NVIDIA RTX PRO 6000 Blackwell Workstation Edition
• Driver: 595.84

Additional context

The following controls pass:

• make_regression(n_samples=1_000, n_features=16)
• make_regression(n_samples=120_000_000, n_features=16)
• make_regression(n_samples=168_000_000, n_features=1)
• The equivalent CuPy 168_000_000 × 16 by 16 × 1 matrix multiplication
• The original 168_000_000 × 16 call in a local conda environment with the same cuML 26.08.00a169 build and cuBLAS 13.1.1

This appears distinct from #6736. That issue failed in ElasticNet.fit through cublasaxpy; this reproducer fails in a clean process inside make_regression through cublasLtMatmul, before an estimator runs.

[csadorf]

I've filed NVIDIA/raft#3097 with the RAFT-only reproducer, exact 2^31 boundary, and cuBLASLt algorithm-selection results. The failure reproduces without cuML, so a fix will need to be made in RAFT or cuBLAS rather than in cuML.