Skip to content

Fix CDC upsert producing double DELETE instead of DELETE+INSERT#30903

Open
brian-vo1 wants to merge 1 commit intoyugabyte:masterfrom
Shopify:origin/upserts_double_deletes_fix
Open

Fix CDC upsert producing double DELETE instead of DELETE+INSERT#30903
brian-vo1 wants to merge 1 commit intoyugabyte:masterfrom
Shopify:origin/upserts_double_deletes_fix

Conversation

@brian-vo1
Copy link
Copy Markdown

@brian-vo1 brian-vo1 commented Mar 31, 2026

Summary

Fix a bug where an upsert (INSERT ... ON CONFLICT DO UPDATE) on an existing row emits two DELETE records instead of a DELETE followed by an INSERT in CDC output, when enable_single_record_update and packed rows are enabled.

Motivation

When a row is upserted, Yugabyte internally writes a tombstone (delete) followed by a packed row (insert) for the same key. The CDC intent processing logic in PopulateCDCSDKIntentRecord decides when to start a new CDC record based on key changes, tombstones, and timestamp differences - but it did not account for the case where a packed row follows a tombstone for the same key at the same timestamp.

This caused both the tombstone and the packed row to be merged into a single record, producing two DELETEs instead of a DELETE + INSERT.

Changes

src/yb/cdc/cdcsdk_producer.cc

Added a condition to the new_cdc_record_needed check: when the current value is a packed row and the previous record was a DELETE, force a new CDC record.
This ensures the packed row (INSERT) is emitted as a separate record rather than being folded into the preceding DELETE.

src/yb/integration-tests/cdcsdk_ysql-test.cc

Added UpsertOnExistingRowProducesDeleteThenInsert test that:

  • Inserts a row, then upserts the same key with a new value
  • Verifies CDC output contains exactly 1 DELETE + 1 INSERT (not 2 DELETEs)

Test plan

  • New test UpsertOnExistingRowProducesDeleteThenInsert validates correct DELETE+INSERT output
  • Existing CDC tests pass unchanged
  • End-to-end verified with a CDC consumer app against a local YugabyteDB build — upserts on existing rows produce DELETE followed by INSERT in CDC output

Note

Medium Risk
Touches core CDC intent-to-record grouping logic, which can affect downstream changefeed correctness; change is small and covered by a new integration test for the reported upsert case.

Overview
Fixes a CDCSDK bug where a packed-row write following a tombstone for the same key could be merged into the prior record under FLAGS_enable_single_record_update, causing INSERT ... ON CONFLICT DO UPDATE to surface as two DELETEs.

Updates PopulateCDCSDKIntentRecord to force a new CDC record when a packed row is encountered after a DELETE for the same primary key, and adds an integration test (UpsertOnExistingRowProducesDeleteThenInsert) asserting the expected DELETE then INSERT sequence with packed rows enabled.

Written by Cursor Bugbot for commit f6dd846. This will update automatically on new commits. Configure here.

@CLAassistant
Copy link
Copy Markdown

CLAassistant commented Mar 31, 2026

CLA assistant check
All committers have signed the CLA.

Copy link
Copy Markdown
Contributor

@gemini-code-assist gemini-code-assist bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review

This pull request modifies the CDC producer to ensure that DELETE operations on packed rows trigger a new CDC record when single record updates are enabled. It also includes an integration test to verify that upserts on existing rows correctly produce a DELETE followed by an INSERT. A review comment identifies a potential out-of-bounds access in the test's record processing loop and suggests adding an assertion to prevent crashes if more records than expected are returned.

@netlify
Copy link
Copy Markdown

netlify bot commented Mar 31, 2026

Deploy Preview for infallible-bardeen-164bc9 ready!

Built without sensitive environment variables

Name Link
🔨 Latest commit 25106f9
🔍 Latest deploy log https://app.netlify.com/projects/infallible-bardeen-164bc9/deploys/69cc2650469d9f0008d56003
😎 Deploy Preview https://deploy-preview-30903--infallible-bardeen-164bc9.netlify.app
📱 Preview on mobile
Toggle QR Code...

QR Code

Use your smartphone camera to open QR code link.

To edit notification comments on pull requests, go to your Netlify project configuration.

@brian-vo1 brian-vo1 force-pushed the origin/upserts_double_deletes_fix branch from 25106f9 to f6dd846 Compare March 31, 2026 20:01
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants