Skip to content

fix(vlm): serialize Transformers predictor execution - #5377

Open
qdivan wants to merge 1 commit into
opendatalab:masterfrom
qdivan:fix/transformers-predictor-serialization
Open

qdivan wants to merge 1 commit into
opendatalab:masterfrom
qdivan:fix/transformers-predictor-serialization

Conversation

@qdivan

@qdivan qdivan commented Aug 10, 2026

Copy link
Copy Markdown

Motivation

MinerU shares one predictor per backend/model tuple, while mineru-api accepts concurrent requests. For the Transformers backend, overlapping model.generate() calls can overwrite Qwen2-VL's request-specific rope_deltas state. This causes the empty-delta shape failures reported in #4376 and #4906; reducing global API concurrency to one avoids the failure but also serializes unrelated document work.

Modification

  • Reuse MinerU's existing per-predictor execution guard for the Transformers backend, as is already done for MLX.
  • Detect both explicit backend selection and the concrete Transformers client module.
  • Keep vLLM, LMDeploy, HTTP, and async-vLLM predictors unlocked so their native concurrency is unchanged.
  • Add focused sync/async regression coverage for Transformers serialization, MLX preservation, lock reuse, and unlocked concurrent backends.

This serializes only calls into a shared local Transformers predictor. PDF rendering and post-processing remain concurrent.

Verification

On upstream master at 79d6d8d79fb8f3ddba5cc34c07a16f0ec36f56c7:

  • Base behavior: the focused regression had 2 failures and 1 error because Transformers predictors had no execution lock.
  • Patched behavior: python3 tests/unittest/test_vlm_predictor_execution_guard.py — 5 passed (repeated three times during independent audit).
  • python3 -m py_compile mineru/backend/vlm/vlm_analyze.py tests/unittest/test_vlm_predictor_execution_guard.py
  • git diff --check
  • Independent read-only audit: PASS.

No GPU/model run is required for this integration-level locking change: the regression directly exercises the production sync and async guard implementations without model downloads.

BC-breaking

No. Public APIs and output formats are unchanged. Only concurrent execution through a shared local Transformers predictor is serialized.

Checklist

  • The reported failure is covered by focused regression tests.
  • MLX behavior and concurrent vLLM/LMDeploy/HTTP behavior are covered.
  • Static checks and focused tests pass.
  • No documentation change is required for this internal concurrency fix.

Signed-off-by: qdivan <qdivan@users.noreply.github.com>
@github-actions

github-actions Bot commented Aug 10, 2026

Copy link
Copy Markdown
Contributor

All contributors have signed the CLA ✍️ ✅
Posted by the CLA Assistant Lite bot.

@qdivan

qdivan commented Aug 11, 2026

Copy link
Copy Markdown
Author

I have read the CLA Document and I hereby sign the CLA

github-actions Bot added a commit that referenced this pull request Aug 11, 2026
@qdivan
qdivan marked this pull request as ready for review August 11, 2026 10:42
@dosubot dosubot Bot added the size:S This PR changes 10-29 lines, ignoring generated files. label Aug 11, 2026
@qdivan

qdivan commented Sep 1, 2026

Copy link
Copy Markdown
Author

This is ready for maintainer review. The focused concurrency regression passes (5/5 across three consecutive runs), the CLA check is green, and the current head remains clean and mergeable. The patch narrowly serializes only the shared Transformers predictor path while preserving concurrency for other backends. Could a maintainer review it when available?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

size:S This PR changes 10-29 lines, ignoring generated files.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant