feat: add request priority scheduling for mineru-api - #5447
Open
isaac-fang-if wants to merge 1 commit into
Open
isaac-fang-if wants to merge 1 commit into
isaac-fang-if wants to merge 1 commit into
Conversation
Add an optional integer priority form field (default 0) to POST /file_parse and POST /tasks. Tasks are scheduled by (-priority, submit_order), so larger values run first while equal priorities keep FIFO, and the default preserves current behavior. The dispatcher acquires an execution slot before dequeuing so priorities survive sparse arrivals, and queued_ahead reflects the scheduled order. The field flows through mineru-router unchanged via the shared multipart form. Co-Authored-By: Claude Code <noreply@anthropic.com>
Contributor
|
All contributors have signed the CLA ✍️ ✅ |
Author
|
I have read the CLA Document and I hereby sign the CLA |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Motivation
mineru-apicurrently schedules parse tasks strictly first-come-first-served. In real deployments the queue often mixes very different workloads: interactive requests where a human is waiting on the response, and large batch jobs that can tolerate hours of delay. Today there is no way to let an urgent interactive request overtake a backlog of batch work — the only options are cancelling jobs or restarting the service.This PR adds an opt-in per-request
prioritytoPOST /file_parseandPOST /tasksso deployments can express scheduling preference, while keeping the default behavior byte-for-byte identical to today's FIFO. The priority semantics intentionally follow vLLM (larger = scheduled first) so they are familiar to anyone already running vLLM-based MinerU backends, and they leave a clean upgrade path to engine-level pass-through in a later PR.Design context and the slot-first dispatcher analysis are in the Ideas discussion linked above.
Modification
mineru/cli/api_request.pypriority(default0) in the sharedparse_request_form/ParseRequestOptions, so bothmineru-apiandmineru-routerexpose it from a single source of truth (the router proxies multipart fields transparently).mineru/cli/fast_api.pyAsyncParseTaskgainspriority: int = 0; the value is echoed back in the task status payload.AsyncTaskManager.queuechanges fromasyncio.Queuetoasyncio.PriorityQueue, keyed on(-priority, submit_order, task_id): larger priority dequeues first; equal priorities keep submission order.get_queued_aheadcounts tasks with a strictly smaller key, so the reported queue position matches actual scheduling order._request_semaphore) before taking the next task from the queue. This matters under sparse arrivals: with the previous dequeue-then-acquire structure, a freed slot is handed out by the semaphore's FIFO wake-up order to processors that were already spawned in submission order, silently nullifying priorities. The unit tests include a regression case that fails with the old structure. The semaphore is passed to_process_taskand released in itsfinally, so slots cannot leak on task failure.docs/en/usage/quick_usage.md,docs/zh/usage/quick_usage.mdpriorityfield, its semantics, and the default.tests/unittest/test_fast_api_priority_scheduling.py(new)queued_aheadsemantics, the slot-first dispatcher under sparse arrivals (starvation regression), slot release on task failure, and the no-semaphore path.No changes are required in
mineru-router— it stages the multipart request and forwardsform.multi_items()verbatim, so the new field flows through automatically.BC-breaking (Optional)
None.
prioritydefaults to0, and every task with the same priority keeps submission order — the default deployment schedules exactly as before.priorityare unaffected.Use cases (Optional)
Mixed interactive + batch deployment (the motivating production case, validated on Ascend 910B + vLLM):
Known trade-off (also noted in the discussion): there is no aging, so a permanently starved low-priority queue is possible if high-priority traffic never stops. This matches vLLM's behavior and keeps the scheduler simple; aging can be layered on later if maintainers want it.
Checklist
Before PR:
After PR: