Skip to content

Auth files permanently deleted from S3 on service restart due to race condition #973

@shunkakinoki

Description

@shunkakinoki

Problem

When the objectstore bootstraps, syncAuthFromBucket() uses os.RemoveAll() to wipe the local auth directory before pulling from S3. This creates a race condition where the file watcher detects the deletions and propagates them to the remote S3 store before the pull completes.

Impact

  • Auth files are permanently deleted from S3 on every service restart
  • Users lose authentication configuration unexpectedly
  • Downstream integrations that depend on persistent auth files break

Reproduction

  1. Have auth files stored in S3
  2. Restart the service
  3. Observe auth files are deleted from S3

Proposed Solution

PR #859 addresses this by using incremental sync instead of delete-then-pull, preventing the file watcher from seeing spurious delete events.

Related

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions