Skip to content

feat: add request priority scheduling for mineru-api - #5447

Open
isaac-fang-if wants to merge 1 commit into
opendatalab:masterfrom
isaac-fang-if:feat/api-priority-scheduling
Open

isaac-fang-if wants to merge 1 commit into
opendatalab:masterfrom
isaac-fang-if:feat/api-priority-scheduling

Conversation

@isaac-fang-if

Copy link
Copy Markdown

Motivation

mineru-api currently schedules parse tasks strictly first-come-first-served. In real deployments the queue often mixes very different workloads: interactive requests where a human is waiting on the response, and large batch jobs that can tolerate hours of delay. Today there is no way to let an urgent interactive request overtake a backlog of batch work — the only options are cancelling jobs or restarting the service.

This PR adds an opt-in per-request priority to POST /file_parse and POST /tasks so deployments can express scheduling preference, while keeping the default behavior byte-for-byte identical to today's FIFO. The priority semantics intentionally follow vLLM (larger = scheduled first) so they are familiar to anyone already running vLLM-based MinerU backends, and they leave a clean upgrade path to engine-level pass-through in a later PR.

Design context and the slot-first dispatcher analysis are in the Ideas discussion linked above.

Modification

  • mineru/cli/api_request.py
    • New optional integer form field priority (default 0) in the shared parse_request_form / ParseRequestOptions, so both mineru-api and mineru-router expose it from a single source of truth (the router proxies multipart fields transparently).
  • mineru/cli/fast_api.py
    • AsyncParseTask gains priority: int = 0; the value is echoed back in the task status payload.
    • AsyncTaskManager.queue changes from asyncio.Queue to asyncio.PriorityQueue, keyed on (-priority, submit_order, task_id): larger priority dequeues first; equal priorities keep submission order.
    • get_queued_ahead counts tasks with a strictly smaller key, so the reported queue position matches actual scheduling order.
    • The dispatcher is restructured to be slot-first: it acquires an execution slot (_request_semaphore) before taking the next task from the queue. This matters under sparse arrivals: with the previous dequeue-then-acquire structure, a freed slot is handed out by the semaphore's FIFO wake-up order to processors that were already spawned in submission order, silently nullifying priorities. The unit tests include a regression case that fails with the old structure. The semaphore is passed to _process_task and released in its finally, so slots cannot leak on task failure.
  • docs/en/usage/quick_usage.md, docs/zh/usage/quick_usage.md
    • Document the new priority field, its semantics, and the default.
  • tests/unittest/test_fast_api_priority_scheduling.py (new)
    • 6 unit tests: priority ordering under backlog, FIFO for equal priorities, queued_ahead semantics, the slot-first dispatcher under sparse arrivals (starvation regression), slot release on task failure, and the no-semaphore path.

No changes are required in mineru-router — it stages the multipart request and forwards form.multi_items() verbatim, so the new field flows through automatically.

BC-breaking (Optional)

None.

  • priority defaults to 0, and every task with the same priority keeps submission order — the default deployment schedules exactly as before.
  • The wire change is additive: one new optional form field, one new key in the status payload. Existing clients that never send priority are unaffected.
  • No public function signatures changed.

Use cases (Optional)

Mixed interactive + batch deployment (the motivating production case, validated on Ascend 910B + vLLM):

# Urgent interactive request jumps the backlog
curl -X POST http://127.0.0.1:8000/tasks \
  -F "files=@contract.pdf" \
  -F "return_md=true" \
  -F "priority=10"

# Overnight batch job keeps the default
curl -X POST http://127.0.0.1:8000/tasks \
  -F "files=@archive_scan.pdf" \
  -F "return_md=true"

Known trade-off (also noted in the discussion): there is no aging, so a permanently starved low-priority queue is possible if high-priority traffic never stops. This matches vLLM's behavior and keeps the scheduler simple; aging can be layered on later if maintainers want it.

Checklist

Before PR:

  • Pre-commit or other linting tools are used to fix the potential lint issues.
  • Bug fixes are fully covered by unit tests, the case that causes the bug should be added in the unit tests.
  • The modification is covered by complete unit tests. If not, please add more unit test to ensure the correctness.
  • The documentation has been modified accordingly, like docstring or example tutorials.

After PR:

  • If the modification has potential influence on downstream or other related projects, this PR should be tested with those projects.
  • CLA has been signed and all committers have signed the CLA in this PR.

Add an optional integer priority form field (default 0) to POST /file_parse and POST /tasks. Tasks are scheduled by (-priority, submit_order), so larger values run first while equal priorities keep FIFO, and the default preserves current behavior. The dispatcher acquires an execution slot before dequeuing so priorities survive sparse arrivals, and queued_ahead reflects the scheduled order. The field flows through mineru-router unchanged via the shared multipart form.

Co-Authored-By: Claude Code <noreply@anthropic.com>
@github-actions

github-actions Bot commented Sep 1, 2026

Copy link
Copy Markdown
Contributor

All contributors have signed the CLA ✍️ ✅
Posted by the CLA Assistant Lite bot.

@isaac-fang-if

Copy link
Copy Markdown
Author

I have read the CLA Document and I hereby sign the CLA

github-actions Bot added a commit that referenced this pull request Sep 1, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant