Skip to content

bug: deep merge operators (*=, *+) fail to deduplicate keys, resulting in duplicate mapping entries #2564

@ysmox

Description

@ysmox

Describe the bug
When merging two identical YAML files using *= or *+ operators, yq produces duplicate keys in the output instead of recognizing they are the same. This creates invalid YAML with duplicate mapping keys.

Version of yq: 4.51.1 (commit: f98028c)
Operating system: Debian GNU/Linux 13 (trixie), Kernel 6.12.48+deb13-amd64
Installed via: source build (go build, Go 1.25.5)

Also tested on: v4.47.2 and v4.50.1 (downloaded from releases page) - same issue occurs

Input Yaml
test/1.yaml:

files.associations:
  "**/templates/*.yaml": "helm"
  "**/templates/*.tpl": "helm"
  "charts/**/templates/*.yaml": "helm"

test/2.yaml:

files.associations:
  "**/templates/*.yaml": "helm"
  "**/templates/*.tpl": "helm"
  "charts/**/templates/*.yaml": "helm"

Command
The command you ran:

./yq '. *= load("test/1.yaml")' test/2.yaml

Also affects:

./yq '. *+ load("test/1.yaml")' test/2.yaml

Actual behaviour

files.associations:
  "**/templates/*.yaml": "helm"
  "**/templates/*.tpl": "helm"
  "**/templates/*.yaml": "helm"
  "charts/**/templates/*.yaml": "helm"

Note: "**/templates/*.yaml" appears twice (4 entries total instead of 3).

Expected behaviour
Since both files are identical, merging should not create duplicate keys:

files.associations:
  "**/templates/*.yaml": "helm"
  "**/templates/*.tpl": "helm"
  "charts/**/templates/*.yaml": "helm"

Additional context
Duplicate keys in YAML mappings are technically invalid according to the YAML spec (though many parsers tolerate them by taking the last value). This bug affects data integrity when programmatically merging YAML files.

Metadata

Metadata

Assignees

No one assigned

    Labels

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions