QVAC-19798 feat[api]: add OpenCL GGML backend support for ocr-ggml EasyOCR and DocTR#2490
Open
Alok-Ranjan23 wants to merge 5 commits into
Open
QVAC-19798 feat[api]: add OpenCL GGML backend support for ocr-ggml EasyOCR and DocTR#2490Alok-Ranjan23 wants to merge 5 commits into
Alok-Ranjan23 wants to merge 5 commits into
Conversation
Extend the existing CPU/Vulkan/Metal backend selection with an OpenCL option
for both the EasyOCR and DocTR pipelines. The Vulkan/Metal framework already
injects a generic ggml_backend_dev_t into all four pipeline steps, so no
step-level changes are needed — this wires the selection + API + build + tests.
- OcrTypes.hpp: add BackendDevice::OPENCL.
- OcrBackendSelection.{hpp,cpp}: add isOpenCLBackendName() + an OPENCL case in
selectBackendDevice (trySelectGpu with rejectAdreno=false — OpenCL is Adreno's
sound GPU path, the inverse of the guarded Vulkan path). CPU fallback +
fallbackReason preserved.
- AddonJs.hpp / index.d.ts / index.js: accept backendDevice: 'opencl'.
- vcpkg.json: add the opencl port (platform: android) so the OpenCL loader is
present, mirroring vla-ggml / llm-llamacpp / embed-llamacpp.
- tests: opencl-backend.test.js (desktop opt-in: OpenCL selected OR explicit
CPU fallback) and android-opencl.test.js (Adreno path, accuracy-gated);
findOpenCLBackendLib() helper; mobile grouping + regenerated runner.
- README: document the 'opencl' option and Android/Adreno caveats.
CPU stays the default and all existing behaviour is unchanged. OpenCL is
Android/Adreno-focused, so real GPU execution is validated on Android CI.
![github-code-quality[bot]](https://avatars.githubusercontent.com/u/9919?s=60&v=4){kind=small}
Contributor
Tier-based Approval Status |
Alok-Ranjan23
added
verify
tier1
verified
Contributor
Mobile integration tests — @qvac/ocr-ggml (iOS)Result: passed
|
Contributor
Mobile integration tests — @qvac/ocr-ggml (Android)Result: passed
|
CI on PR #2490 showed that requesting backendDevice:'opencl' on Adreno correctly selects the OpenCL device (GPUOpenCL) but then GGML_ABORTs in the CRAFT detector: ggml's OpenCL backend does not implement POOL_2D (ggml-opencl.cpp: "op not supported node_22 (POOL_2D)" -> SIGABRT). ggml aborts on any unsupported op rather than failing gracefully, crashing the process. Add a POOL_2D op-support probe (ggml_backend_dev_supports_op) in OcrBackendSelection::trySelectGpu: a matched GPU device that cannot run the op family every OCR graph needs is rejected in favour of CPU with a clear fallbackReason, instead of crashing at inference. POOL_2D is used by all four OCR graphs and is exactly the op ggml's OpenCL backend is missing; the probe mirrors easyocr/craft.cpp maxpool_2x2 so full-featured Vulkan/Metal devices (which run OCR for real) always pass and are unaffected. Net: requesting 'opencl' is now safe (never crashes) and transparently resolves to CPU for OCR until ggml's OpenCL backend gains the required vision ops (POOL_2D / conv / upscale / transpose-conv) — tracked as a follow-up. README updated to document the op-support gate and CPU fallback.
…pencl-ggml-backend-ocr-ggml
…backend-ocr-ggml' into feat/QVAC-19798-add-opencl-ggml-backend-ocr-ggml
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Adds opt-in OpenCL GGML backend support to
@qvac/ocr-ggmlfor both the EasyOCR and DocTR pipelines, with transparent CPU fallback and backend-identity reporting — completing the GPU-backend matrix alongside the existing Vulkan/Metal support.The Vulkan/Metal work (QVAC-19797) already refactored all four pipeline steps (EasyOCR CRAFT + CRNN, DocTR DBNet + recognizer) to run on a generic injected
ggml_backend_dev_tresolved byOcrBackendSelection. So this change needs no step-level rewrites — it extends the selection layer, the JS/TS API, the build, and tests.What changed
OcrTypes.hpp— addBackendDevice::OPENCL.OcrBackendSelection.{hpp,cpp}— addisOpenCLBackendName()(matches"opencl", case-insensitive, e.g. ggml'sGPUOpenCLdevice /OpenCLreg name) and anOPENCLcase inselectBackendDevice(). It callstrySelectGpu(..., rejectAdreno=false): unlike Vulkan, Adreno is NOT skipped for OpenCL — OpenCL is Adreno's sound compute path (the inverse of the Vulkan Adreno guard). CPU fallback +fallbackReasonbehaviour is unchanged.AddonJs.hpp/index.d.ts/index.js— acceptbackendDevice: 'opencl'(the value is already forwarded generically; only validation/types/docs change).vcpkg.json— add theopenclport (platform: android) so the OpenCL loader/headers are present, mirroringvla-ggml/llm-llamacpp/embed-llamacpp. The existingCMakeLists.txtBACKEND_DL_LIBSloop already exports any builtggml-openclbackend intoprebuilds/.test/integration/opencl-backend.test.js: desktop opt-in contract — requesting'opencl'either selects an OpenCL GPU or reports an explicit CPU fallback; gated on a shippedlibggml-opencllib so it skips cleanly where the backend wasn't built.test/integration/android-opencl.test.js: Android/Adreno path, accuracy-gated (OCR output must be correct on whichever backend resolves).findOpenCLBackendLib()helper inutils.js; both wrappers registered intest/mobile/test-groups.json+ regeneratedintegration.auto.cjs.README.md— document the'opencl'option, the Adreno rationale, and the Android-focused build/driver caveats.Notes / scope
'opencl'is purely opt-in.openclvcpkg port is Android-only across the monorepo, and ggml-opencl isn't built for desktop Linux/Windows or Apple. So the desktop test exercises the fallback contract, while real OpenCL execution is validated on the Android device farm (Adreno: S25 Ultra / S26), which the existingon-pr-ocr-ggmlworkflow runs.Test plan
standard) clean on touched filesclang-format(LLVM 22) clean on changed C++ linescpp-lint(clang-format + clang-tidy)android-arm64with the OpenCL backend)android-openclexercises the Adreno OpenCL path with the accuracy gateAsana: https://app.asana.com/1/45238840754660/project/1214153063536860/task/1215288817867554