Fix CDC upsert producing double DELETE instead of DELETE+INSERT#30903
Open
brian-vo1 wants to merge 1 commit intoyugabyte:masterfrom
Open
Fix CDC upsert producing double DELETE instead of DELETE+INSERT#30903brian-vo1 wants to merge 1 commit intoyugabyte:masterfrom
brian-vo1 wants to merge 1 commit intoyugabyte:masterfrom
Conversation
Contributor
There was a problem hiding this comment.
Code Review
This pull request modifies the CDC producer to ensure that DELETE operations on packed rows trigger a new CDC record when single record updates are enabled. It also includes an integration test to verify that upserts on existing rows correctly produce a DELETE followed by an INSERT. A review comment identifies a potential out-of-bounds access in the test's record processing loop and suggests adding an assertion to prevent crashes if more records than expected are returned.
✅ Deploy Preview for infallible-bardeen-164bc9 ready!Built without sensitive environment variables
To edit notification comments on pull requests, go to your Netlify project configuration. |
25106f9 to
f6dd846
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Fix a bug where an upsert (
INSERT ... ON CONFLICT DO UPDATE) on an existing row emits two DELETE records instead of a DELETE followed by an INSERT in CDC output, whenenable_single_record_updateand packed rows are enabled.Motivation
When a row is upserted, Yugabyte internally writes a tombstone (delete) followed by a packed row (insert) for the same key. The CDC intent processing logic in
PopulateCDCSDKIntentRecorddecides when to start a new CDC record based on key changes, tombstones, and timestamp differences - but it did not account for the case where a packed row follows a tombstone for the same key at the same timestamp.This caused both the tombstone and the packed row to be merged into a single record, producing two DELETEs instead of a DELETE + INSERT.
Changes
src/yb/cdc/cdcsdk_producer.ccAdded a condition to the
new_cdc_record_neededcheck: when the current value is a packed row and the previous record was a DELETE, force a new CDC record.This ensures the packed row (INSERT) is emitted as a separate record rather than being folded into the preceding DELETE.
src/yb/integration-tests/cdcsdk_ysql-test.ccAdded
UpsertOnExistingRowProducesDeleteThenInserttest that:Test plan
UpsertOnExistingRowProducesDeleteThenInsertvalidates correct DELETE+INSERT outputNote
Medium Risk
Touches core CDC intent-to-record grouping logic, which can affect downstream changefeed correctness; change is small and covered by a new integration test for the reported upsert case.
Overview
Fixes a CDCSDK bug where a packed-row write following a tombstone for the same key could be merged into the prior record under
FLAGS_enable_single_record_update, causingINSERT ... ON CONFLICT DO UPDATEto surface as twoDELETEs.Updates
PopulateCDCSDKIntentRecordto force a new CDC record when a packed row is encountered after aDELETEfor the same primary key, and adds an integration test (UpsertOnExistingRowProducesDeleteThenInsert) asserting the expectedDELETEthenINSERTsequence with packed rows enabled.Written by Cursor Bugbot for commit f6dd846. This will update automatically on new commits. Configure here.