Skip to content

Add Moondream2 vision-language notebook with OpenVINO#3332

Open
LuciferDono wants to merge 3 commits intoopenvinotoolkit:latestfrom
LuciferDono:moondream2-vision
Open

Add Moondream2 vision-language notebook with OpenVINO#3332
LuciferDono wants to merge 3 commits intoopenvinotoolkit:latestfrom
LuciferDono:moondream2-vision

Conversation

@LuciferDono
Copy link

@LuciferDono LuciferDono commented Mar 7, 2026

Summary

  • Add new notebook demonstrating Moondream2 (2B parameter vision-language model) with OpenVINO
  • Model conversion to OpenVINO IR with INT4 weight compression via optimum-cli
  • Inference demos: image captioning, visual question answering, object detection
  • Interactive Gradio interface for uploading images and asking questions
  • Follows existing notebook conventions (SmolVLM2, Phi-4 multimodal patterns)

Details

Moondream2 is a compact VLM designed for edge deployment. This notebook shows how to optimize it with OpenVINO for efficient inference on Intel hardware. Uses OVModelForVisualCausalLM from optimum-intel, device_widget for device selection, and INT4/FP16 compression options.

Checklist

  • Notebook runs end-to-end
  • Follows repository conventions (telemetry, utils fetch, device widget, Gradio demo)
  • README with Scarf pixel
  • Cross-platform compatible (pathlib.Path, no hardcoded paths)
  • Cleanup cell included

Demonstrate Moondream2 (2B param VLM) conversion to OpenVINO IR
with INT4 weight compression. Covers image captioning, visual QA,
object detection, and interactive Gradio demo.
@review-notebook-app
Copy link

Check out this pull request on  ReviewNB

See visual diffs & provide feedback on Jupyter Notebooks.


Powered by ReviewNB

@LuciferDono
Copy link
Author

hey @aleksandr-mokrov, just checking in on this. anything I should change or is there something blocking review? happy to adjust if needed.

@aleksandr-mokrov
Copy link
Collaborator

hey @aleksandr-mokrov, just checking in on this. anything I should change or is there something blocking review? happy to adjust if needed.

Thanks for the contribution!

I tried running the notebook locally and hit an error during model conversion with optimum-cli. Could you please share the full conversion log from a successful run on your side? Also, which versions of optimum-intel and openvino did you use?

@LuciferDono
Copy link
Author

Hey @aleksandr-mokrov, sorry for the wait.

Turns out Moondream2 registers as moondream1, which isn't in optimum-intel's export registry. The trust_remote_code error was actually from a positional arg bug, but even with that fixed the real problem is that the architecture just isn't supported by optimum-cli.

Looking into converting it with ov.convert_model directly instead. Will update the PR when I have something working.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants