Skip to content

miner: avoid recursive worker read lock - #2352

Merged
pratikspatil024 merged 1 commit into
developfrom
ppatil/fix-miner-lock-order
Aug 11, 2026
Merged

miner: avoid recursive worker read lock#2352
pratikspatil024 merged 1 commit into
developfrom
ppatil/fix-miner-lock-order

Conversation

@pratikspatil024

@pratikspatil024 pratikspatil024 commented Aug 10, 2026

Copy link
Copy Markdown
Member

Summary

Fixes the miner timeout in nightly race run 31292744499, where TestPrefetchRaceWithSetExtra ran until the package's 15-minute timeout.

This was a real Bor block-production concurrency bug exposed by the test, not a flaky assertion, CI instability, or an external dependency failure. prepareWork and runPrefetcher hold worker.mu for reading while calling makeHeader. The dynamic gas-limit helper attempted to acquire the same read lock again. When SetExtra queued for the write lock, Go's writer-preferring RWMutex blocked the nested reader; the writer could not proceed until the outer reader released, creating a deadlock.

The fix keeps the lock-taking gas-limit helper for standalone callers and adds a lock-held helper for makeHeader. A deterministic regression test queues a writer, confirms it is waiting, and verifies that header construction completes while its caller retains the read lock.

Executed tests

  • Pre-fix regression: TestMakeHeaderWithQueuedWriter failed with the expected blocked-header assertion.
  • go test -race ./miner -run '^TestMakeHeaderWithQueuedWriter$' -count=20
  • go test -race ./miner -run '^TestPrefetchRaceWithSetExtra$' -shuffle=1786249703675931145 -count=5
  • go test -race ./miner -run '^TestCalculateDesiredGasLimit' -count=10
  • go test -race ./miner -shuffle=1786249703675931145 -count=1 -timeout=15m
  • go vet ./miner
  • make lint (0 issues)
  • Diffguard using the PR workflow configuration with mutation testing: 100% (1/1 killed)
  • gosec ./miner/... found no issue on changed lines; it exited non-zero for 22 pre-existing findings elsewhere in miner.
  • PR CI: unit tests, integration tests, both e2e variants, lint, CodeQL, Diffguard, and Codecov passed.

Rollout notes

Backward-compatible locking fix. It does not change gas-limit calculation, header contents, consensus rules, configuration, or operator behavior.

The current develop dependency set has an unrelated govulncheck failure for GO-2026-6061 in google.golang.org/grpc v1.79.3; recent nightly govulncheck runs already fail on the same base SHA. That dependency update remains separate from this PR.

Remaining risk is limited to future call paths invoking makeHeader without holding worker.mu; all current production and test call sites were audited and hold the read lock. CI unit, integration, and both e2e variants passed.

Use a lock-held gas-limit helper from makeHeader so a queued writer cannot block a recursive RLock while the caller still holds the worker read lock. Add a deterministic queued-writer regression test.
@pratikspatil024
pratikspatil024 marked this pull request as ready for review August 10, 2026 11:27
Copilot AI lite review requested due to automatic review settings August 10, 2026 11:27

@claude claude Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Claude Code Review

This repository is configured for manual code reviews. Comment @claude review for a one-time review, or @claude review always to subscribe this PR to a review on every future push.

Tip: disable this comment in your organization's Code Review settings.

@pratikspatil024

Copy link
Copy Markdown
Member Author

codegenie review

@github-actions

github-actions Bot commented Aug 10, 2026

Copy link
Copy Markdown

🧞 Codegenie Review

✅ No credible findings.

Coverage

Reviewed 4/4 hunks.
Coverage levels: deep 3, normal 0, light 1, skip 0.

Stats

  • 🤖 Model: anthropic claude-opus-5 high
  • 🧞 Codegenie: v0.5.5 (58f82a9b2c)
  • Elapsed time: 3m 17s
  • Git: 0xPolygon/bor from develop to ppatil/fix-miner-lock-order (ae29ea3f17)
  • Review completeness: complete.
  • Usage: model calls 27, tokens 532776, cost $2.1665.
  • Effective caps: tokens 8000000.
  • Local context pressure: 7 degraded tool results.

✅ No Findings

No credible findings were found. Everything looks good.

View Workflow Job

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR fixes a miner deadlock caused by recursively acquiring worker.mu’s read lock during header construction while a writer was queued (Go’s writer-preferring RWMutex can block nested readers once a writer is waiting). The change makes header construction use a lock-free (but lock-required) gas-limit helper, and adds a deterministic regression test to ensure header creation completes even with a queued writer while the read lock is held.

Changes:

  • Split dynamic gas-limit calculation into a lock-taking wrapper and a w.mu-already-held helper, and use the locked helper from makeHeader.
  • Document makeHeader’s locking contract (caller must hold w.mu for reading) to avoid reintroducing recursive lock acquisition.
  • Add TestMakeHeaderWithQueuedWriter to reproduce and guard against the deadlock scenario under -race.

Reviewed changes

Copilot reviewed 2 out of 2 changed files in this pull request and generated no comments.

File Description
miner/worker.go Avoids recursive RWMutex read locking by introducing calculateDesiredGasLimitLocked and using it from makeHeader under an existing RLock.
miner/worker_test.go Adds a regression test that queues a writer and verifies makeHeader completes while the read lock remains held.

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

@codecov

codecov Bot commented Aug 10, 2026

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 54.24%. Comparing base (6604100) to head (ae29ea3).

Additional details and impacted files

Impacted file tree graph

@@             Coverage Diff             @@
##           develop    #2352      +/-   ##
===========================================
+ Coverage    54.18%   54.24%   +0.05%     
===========================================
  Files          908      908              
  Lines       162242   162243       +1     
===========================================
+ Hits         87909    88002      +93     
+ Misses       68891    68810      -81     
+ Partials      5442     5431      -11     
Files with missing lines Coverage Δ
miner/worker.go 78.08% <100.00%> (-0.19%) ⬇️

... and 26 files with indirect coverage changes

Files with missing lines Coverage Δ
miner/worker.go 78.08% <100.00%> (-0.19%) ⬇️

... and 26 files with indirect coverage changes

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

@pratikspatil024
pratikspatil024 merged commit bea1a20 into develop Aug 11, 2026
19 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants