Skip to content

unsorted read_dir() in extra_component_dirs makes dependencies.lock non-reproducible across machines #432

Description

@vena

Bug description

when extra_components points at a parent directory, extra_component_dirs() enumerates it with comp_dir.read_dir() and no sort (build/native/cargo_driver/config.rs:148). that order can change across platforms/filesystems/whatever and reaches the component manager, which will hash based on that non-deterministic result.

so you can have two machines computing different values. component manager sees that as a mismatch, resolves everything from scratch, rewrites the lock, and a committed lock never constrains the build (mine is committed because esp-idf-sys wires it as the cmake DEPENDENCIES_LOCK prop, but this is a defensive suggestion in general, filed as a bug because it produces real issues)

  • Would you like to work on a fix? [y/n] sure if necessary, it's just .sort()'ing the read_dir() result

To Reproduce

  1. Project with two or more components that each have an idf_component.yml under one parent dir
[package.metadata.esp-idf-sys]
extra_components = [{ component_dirs = ["components"] }]
  1. build on machine A (say, macos), commit the generated components_.lock
  2. build the same commit on machine B (say, ubuntu, different filesystem/platform to provoke the order mismatch)
  3. diff the lock, manifest_hash has changed.

Expected behavior

component discovery order shouldn't depend on filesystem order. the same sources should produce the same dependencies.lock on any machine

Environment

  • Crate (esp-idf-sys) version: 0.38.1 (same code on master)
  • ESP-IDF branch or tag: v5.5.5
  • Target device (MCU): esp32p4
  • OS: macOS 26.7 (arm64) and Ubuntu 24.04.5

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    bugSomething isn't working

    Type

    No type

    Projects

    • Status
      Todo

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions