Skip to content

gin/gdaki: Transcribe efa dp direct version bump - #1373

Open
anshumang wants to merge 5 commits into
aws:masterfrom
anshumang:transcribe-bump-efa-dp-direct
Open

gin/gdaki: Transcribe efa dp direct version bump#1373
anshumang wants to merge 5 commits into
aws:masterfrom
anshumang:transcribe-bump-efa-dp-direct

Conversation

@anshumang

Copy link
Copy Markdown
Contributor

Adds backendVersion 2 to the GDAKI GIN path, matching efa-dp-direct's wide-WQE and 64-bit-request-ID support.

  • Freezes the existing QP/CQ device-struct layout as backendVersion 1
  • Adds the backendVersion 2 layout.
  • Requests the 128B WQE from libfabric for backendVersion 2, and refuses the endpoint if the queried SQ entry size is not 128.

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.

@anshumang
anshumang requested a review from a team as a code owner August 25, 2026 18:19
@anshumang
anshumang force-pushed the transcribe-bump-efa-dp-direct branch from ffbfd23 to 8682367 Compare August 29, 2026 02:36
@anshumang
anshumang requested a review from bwbarrett as a code owner August 29, 2026 02:36
@anshumang
anshumang force-pushed the transcribe-bump-efa-dp-direct branch from 8682367 to 00b58c2 Compare August 29, 2026 02:52
attrs.rq_entry_size = rq_attr.entry_size;
attrs.sq_max_inline_data = gdaki_wide_wqe_inline_size;
attrs.sq_max_rdma_sges = gdaki_max_rdma_sges;
attrs.sq_wq_caps = EFA_CUDA_WQ_CAPS_64_BIT_REQ_ID_V2;

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why dont we get attrs.sq_wq_caps from sq_attr.caps ?
I am assuming sq_attr.caps reports if the firmware actually supports 128bytes.

Right now we are always hardcoding that it supports req_id=64bit.

Sync the vendored CUDA datapath with efa-dp-direct Gerrit change 453817 patchset 1. This provides the version-dispatched host API, caller-owned descriptor storage, 64-bit request IDs, 128-byte inline RDMA writes, and runtime WQE geometry.

Signed-off-by: Anshuman Goswami <anshumgo@amazon.com>
Build GDAKI QP and CQ descriptors through the frozen v1 operation
table.

Signed-off-by: Anshuman Goswami <anshumgo@amazon.com>
Thread backendVersion from createContext through QP/CQ construction
and dispatch lifecycle operations through the matching efa-dp-direct
operation table.

Signed-off-by: Anshuman Goswami <anshumgo@amazon.com>
Build QP and CQ descriptors through efa_cuda_dp_v2 using the
provider-reported SQ entry size and 64-bit request-ID capability.
Keep backendVersion 1 on the v1 path.

Update the GDAKI GPU tests to use EfaCudaWrBuilder and verify a request
ID with nonzero high bits.

Signed-off-by: Anshuman Goswami <anshumgo@amazon.com>
Adopt efa-dp-direct 1.0.0's version-dispatched, caller-owned host API. Map GIN backend versions 1 and 2 to API majors 0 and 1, respectively, copy initialized queue descriptors through the plugin accelerator abstraction, and use the validated single-RDMA-SGE limit for the major-1 WQE layout.

The host library no longer calls CUDA directly, so build the unified host source and drop the libcuda dependency.

Signed-off-by: Anshuman Goswami <anshumgo@amazon.com>
@anshumang
anshumang force-pushed the transcribe-bump-efa-dp-direct branch from 5b0a26e to 8b2f0a9 Compare September 4, 2026 01:12
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants