[Bug]: Client initialization failure occurs when mooncake operates across different architectures. #1573
Description
Bug Report
When a client is registered on an x86 server, the mooncake master on an ARM server reports an initialization failure.
When an ARM server and an x86 server are used as the prefill and decode deployments respectively, and mooncake spans across the ARM architecture and the x86 architecture, an exception occurs during client initialization. It is suspected that under the GCC compiler of different architectures, if the mooncake developers do not strictly use methods such as #pragma pack(1) to restrict memory alignment (Memory Padding),
an anomaly may occur when the x86 master sends the 24-byte data it has packed to the ARM client, and the ARM client attempts to unpack the data according to the 32-byte format, which directly results in an out-of-bounds error. Please confirm whether this issue exists.
Logs related to vllm startup:
WARNING [OK[376535] [coro_rpc_client.hpp:1173] deserialize rpc result failed
E0227 09:58:39.023512 376535 master_client.cpp:183] RPC call failed: failed to deserialize rpc return value
WARNING [OK[376543] [coro_rpc_client.hpp:1173] deserialize rpc result failed
E0227 09:58:39.023550 376543 master_client.cpp:183] RPC call failed: failed to deserialize rpc return value
E0227 09:58:39.023697 375703 real_client.cpp:193] Failed to create client
WARNING [OK[375704] [coro_rpc_client.hpp:1173] deserialize rpc result failed
E0227 09:58:39.023797 375706 real_client.cpp:193] Failed to create client
WARNING [OK[375703] [coro_rpc_client.hpp:1173] deserialize rpc result failed
E0227 09:58:39.023771 375641 master_client.cpp:183] RPC call failed: failed to deserialize rpc return value
E0227 09:58:39.023851 375641 master_client.cpp:183] RPC call failed: failed to deserialize rpc return value
E0227 09:58:39.023883 375701 real_client.cpp:1997] Client is not initialized
Before submitting...
- Ensure you searched for relevant issues and read the [documentation]