Skip to content

Fix id collisions between AssignUniqueId operators - #30955

Open
hqbhoho wants to merge 1 commit into
trinodb:masterfrom
hqbhoho:fix-assign-unique-id-per-task-pool
Open

Fix id collisions between AssignUniqueId operators#30955
hqbhoho wants to merge 1 commit into
trinodb:masterfrom
hqbhoho:fix-assign-unique-id-per-task-pool

Conversation

@hqbhoho

@hqbhoho hqbhoho commented Aug 31, 2026

Copy link
Copy Markdown
Contributor

Description

AssignUniqueId embeds only the stage id and partition id of the task in the id mask, while row ids were drawn from a per-factory pool starting at zero. Two AssignUniqueId nodes executing in the same task therefore assign the same ids to different rows.

MERGE hits this when the join is colocated with the partitioning of the target table (e.g. a bucket-partitioned Iceberg target with join_distribution_type=PARTITIONED): the target scan stays in the join stage, so the target and source AssignUniqueId nodes run in the same task. After the right join coalesces the two id columns, MarkDistinct over (unique_id, case_number) treats unrelated rows as duplicate matches and fails with a false MERGE_TARGET_ROW_MULTIPLE_MATCHES.

The fix shares a single row id pool (AtomicLong) across all AssignUniqueId operator factories in a task, held by LocalExecutionPlanContext and passed to every factory. Ids are then unique within the whole query: the stage/partition mask separates tasks, and the shared pool separates operators within a task. This follows the operator-level direction suggested by @Praveen2112 in the issue, and subsumes the factory-duplicate() pool sharing from commit 0e880c8.

There is no execution-time cost: the per-row path is unchanged, and the shared pool is touched once per 2^20 ids per driver (a single getAndAdd).

Validation:

  • New TestAssignUniqueIdOperator covers id disjointness for factories sharing a pool and for duplicated factories.
  • New TestIssue30639 reproduces the issue end to end; it fails on current master with One MERGE target table row matched more than one source row and passes with this change. The test pins bucket_execution_enabled and asserts both AssignUniqueId nodes plan into one fragment, so it cannot become vacuous if planner defaults change.
  • TestMerge passes unchanged; the plan shape is not affected.

Additional context and related issues

Release notes

( ) This is not user-visible or is docs only, and no release notes are required.
( ) Release notes are required. Please propose a release note for me.
(x) Release notes are required, with the following suggested text:

## General
* Fix false `MERGE_TARGET_ROW_MULTIPLE_MATCHES` failures for `MERGE` when the join is colocated with the partitioning of the target table. ({issue}`30639`)

@github-actions github-actions Bot added iceberg Iceberg connector cla-signed labels Aug 31, 2026
@hqbhoho
hqbhoho force-pushed the fix-assign-unique-id-per-task-pool branch 2 times, most recently from 33e185c to 38c28a3 Compare August 31, 2026 10:21
@hqbhoho hqbhoho changed the title Fix unique id collisions between AssignUniqueId operators in a task Fix id collisions between AssignUniqueId operators Aug 31, 2026
@hqbhoho
hqbhoho force-pushed the fix-assign-unique-id-per-task-pool branch 2 times, most recently from 279150e to ce61f4d Compare August 31, 2026 10:37
@hqbhoho
hqbhoho force-pushed the fix-assign-unique-id-per-task-pool branch from ce61f4d to 950dac7 Compare August 31, 2026 11:19
@hqbhoho
hqbhoho requested a review from Praveen2112 August 31, 2026 11:22
@hqbhoho
hqbhoho force-pushed the fix-assign-unique-id-per-task-pool branch from 950dac7 to 86f1a00 Compare September 2, 2026 07:56
Comment thread plugin/trino-iceberg/src/test/java/io/trino/plugin/iceberg/TestIssue30639.java Outdated
Comment thread plugin/trino-iceberg/src/test/java/io/trino/plugin/iceberg/TestIssue30639.java Outdated
The target and source AssignUniqueId operators of a MERGE can execute
in the same task and drew overlapping ids from separate pools, causing
false MERGE_TARGET_ROW_MULTIPLE_MATCHES failures.
@hqbhoho
hqbhoho force-pushed the fix-assign-unique-id-per-task-pool branch from dbe2637 to 39a1b71 Compare September 4, 2026 06:15
@hqbhoho
hqbhoho requested a review from ebyhr September 4, 2026 07:06
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

cla-signed iceberg Iceberg connector

Development

Successfully merging this pull request may close these issues.

MERGE fails on a Partitioned Iceberg tables without duplicates with MERGE_TARGET_ROW_MULTIPLE_MATCHES

3 participants