Add Moondream2 vision-language notebook with OpenVINO#3332
Add Moondream2 vision-language notebook with OpenVINO#3332LuciferDono wants to merge 3 commits intoopenvinotoolkit:latestfrom
Conversation
Demonstrate Moondream2 (2B param VLM) conversion to OpenVINO IR with INT4 weight compression. Covers image captioning, visual QA, object detection, and interactive Gradio demo.
|
Check out this pull request on See visual diffs & provide feedback on Jupyter Notebooks. Powered by ReviewNB |
|
hey @aleksandr-mokrov, just checking in on this. anything I should change or is there something blocking review? happy to adjust if needed. |
Thanks for the contribution! I tried running the notebook locally and hit an error during model conversion with optimum-cli. Could you please share the full conversion log from a successful run on your side? Also, which versions of optimum-intel and openvino did you use? |
|
Hey @aleksandr-mokrov, sorry for the wait. Turns out Moondream2 registers as moondream1, which isn't in optimum-intel's export registry. The trust_remote_code error was actually from a positional arg bug, but even with that fixed the real problem is that the architecture just isn't supported by optimum-cli. Looking into converting it with ov.convert_model directly instead. Will update the PR when I have something working. |
Summary
Details
Moondream2 is a compact VLM designed for edge deployment. This notebook shows how to optimize it with OpenVINO for efficient inference on Intel hardware. Uses
OVModelForVisualCausalLMfrom optimum-intel,device_widgetfor device selection, and INT4/FP16 compression options.Checklist