feat[cuda]: export arrays to ArrowDeviceArray

[a10y]

This PR adds scaffolding for the first part of implementing export of Vortex Arrays directly to Arrow C Data Device Interface arrays, and implements the actual export for StructArray and PrimitiveArray

Setup

Arrow defines a standard for exchanging data in-process to native code using the C Data Interface, which converts an Arrow array into a struct of pointers.

There is an (experimental, but mostly stable) extension called the C Data Device Interface which includes extra information such as the GPU device type and device ID.

cudf knows how to import data exported this way using the from_arrow_device_column method for column_views.

We replicate the ArrowArray and ArrowDeviceArray structs from the spec.

These are very similar to the existing FFI_ArrowArray and FFI_ArrowSchema structs from the arrow-data crate, except that those use arrow Bytes and real pointers and we need to use CUDA device pointers and BufferHandles.

Testing

Rather than polluting the repo/CI with fetching and building cudf, I made a new repo https://github.com/vortex-data/cudf-test-harness/

This will load a cdylib, call its export_array() function, imports it into cudf and calls some simple functions, then frees it by calling the embedded release function pointer. This step is hooked up to CI now and only adds ~1m to the CUDA tests jobs.

[codspeed-hq]

Merging this PR will degrade performance by 33.69%

⚡ 3 improved benchmarks
❌ 1 regressed benchmark
✅ 1134 untouched benchmarks
⏩ 1265 skipped benchmarks¹

⚠️ Please fix the performance issues or acknowledge them on CodSpeed.

Performance Changes

	Mode	Benchmark	BASE	HEAD	Efficiency
❌	Simulation	chunked_bool_into_canonical[(1000, 10)]	41.8 µs	63 µs	-33.69%
⚡	Simulation	patched_take_10k_contiguous_not_patches	1.5 ms	1.2 ms	+18.49%
⚡	Simulation	patched_take_10k_random	2.4 ms	2.1 ms	+10.85%
⚡	Simulation	patched_take_10k_adversarial	2.3 ms	2 ms	+10.69%

---

_{Comparing aduffy/c-device-iface (9824f29) with develop (5648546)}

Footnotes

1. 1265 benchmarks were skipped, so the baseline results were used instead. If they were deleted from the codebase, click here and archive them to remove them from the performance reports. ↩

[0ax1]

Couple of questions / remarks

[0ax1]

How about sth along the lines of:

let futures = fields
    .iter()
    .map(|field| {
        let field = field.clone();
        async move {
            let cuda_field = field.execute_cuda(ctx).await?;
            export_canonical(cuda_field, ctx).await
        }
    })
    .collect::<Vec<_>>();

let results = join_all(futures).await;

If fields live on different streams, they run in parallel.

[a10y]

are fields going to live on different streams? if so, things might get a little weird with the SyncEvent stuff

[a10y]

Specifically: the contract for the returned ArrowDeviceArray is that its sync_event field, if not NULL, contains a pointer to a cudaStream_t which AFAIU can only be linked to one stream.

Similarly the CudaExecutionCtx is linked to only one global stream rn. So assuming that StructArray::execute_cuda(ctx) gets called, all of its fields should be getting executed with the same context, and thus the same stream.

[robert3005]

You can make a stream wait for another one but that’s a complexity we should figure out how to avoid.

[0ax1]

I think we have at most one stream per file during a scan rn (looking at vortex-file/src/tests gpu_scan)? Where the stream is retrieved from the stream pool and capped at max 4 (DEFAULT_STREAM_POOL_CAPACITY).

that’s a complexity we should figure out how to avoid.

We should manifest this in writing in code docs somewhere. The scenario where this can bite us is if we want to saturate a GPU with one large file, as there'd be no parallelism on a stream level in that case.

[0ax1]

We can drop the Sync impl I think.

[joseph-isaacs]

would this logic every differ from export arrow with a different execute function?

[a10y]

I'm not sure I understand the question

[joseph-isaacs]

Could we share logic between this exporter and the arrow export if we could pass around a different executor?

[a10y]

I'm not sure what you mean by passing around a different exporter, but no I don't think there's much shared logic between this and just execute_arrow

[0ax1]

👌

[0ax1]

If we await for all fields in the lines above to complete, why do we need the additional sync event? This should only be relevant if we want to hand out an arrowdevicearray where canonicalization might not be completed yet, and is still processing (c style) async right?