feat(Storage): Enable full object checksum validation on JSON path#8825
Conversation
product-auto-label
bot
added
the
api: storage
thiyaguk09
force-pushed
the
feat/enable-full-checksum-validation
branch
2 times, most recently
from
d61822d to
4091fcc
Compare
Refactor Resumable, Streamable, and Multipart uploaders to ensure integrity headers (X-Goog-Hash) are only attached to the request when an upload is being finalized. - In StreamableUploader, introduced `$isFinalRequest` to track intent before writeSize recalculations. - In ResumableUploader, added a boundary check to only attach the hash when the current range matches the total file size. - Aligns with GCS best practices for resumable upload integrity.
thiyaguk09
force-pushed
the
feat/enable-full-checksum-validation
branch
from
4091fcc to
2ac9aad
Compare
bshaffer
left a comment
bshaffer
left a comment
There was a problem hiding this comment.
Just one question and a few code-cleanup nits! Otherwise this looks great
- Refactor Rest.php hash calculation to be more concise using array_filter. - Remove redundant array initializations in Rest.php by utilizing PHP autovivification. - Improve readability of X-Goog-Hash header generation.
bshaffer
left a comment
bshaffer
left a comment
There was a problem hiding this comment.
The failing error message is tricky but it's valid - you need to upgrade the minimum version of google/cloud-core in Storage/composer.json to be the future version which is going to be released, since it will rely on that version of the library. That version will be 1.72.0.
Once that's done, this will be good to go!
bshaffer
changed the title
Updates the minimum required version of cloud-core to ensure the new checksum validation logic is available in all environments, fixing the lowest-dependencies test failures.
I've bumped the |
2 participants
Enhanced Checksum Validation & Header Logic
This PR implements comprehensive MD5 and CRC32c checksum validation for object uploads, ensuring data integrity via the
X-Goog-Hashheader, improving data integrity verification. It refactors the upload architecture to handle hashes dynamically across different upload strategies.Key Technical Changes
1. Core Library Enhancements (
google-cloud-core)ResumableUploader&StreamableUploader: Added type-safe logic(int)($rangeEnd + 1) === (int)$sizeto ensureX-Goog-Hashis transmitted only on the final chunk/request, preventing intermediate validation errors.MultipartUploader: Standardized header merging to ensure hashes calculated by the connection layer are always included in single-shot uploads.restOptionsmerging to ensure custom metadata and encryption headers are preserved alongside checksums.2. Storage Package Improvements (
google-cloud-storage)MD5orCRC32chashes when thevalidateoption is enabled.Bucket::upload()to honor user-provided checksums and prevent redundant re-calculation.BucketTestandRestTestto verify hash behavior in resumable, streamable, and multipart scenarios.Note
CI "Lowest Dependencies" Failure: This failure occurs because the CI environment pulls the tagged version of
google-cloud-corefrom Packagist instead of using the local changes in this PR. This will resolve once the Core changes are merged.