Skip to content

Support newer vision models from ollama #241

@johnsonjo4531

Description

@johnsonjo4531

I was hoping openui could support newer vision models other than just moondream and llava (which seem to be hard coded at the moment). Ollama now has newer vision models available.

Related Info:

Here are all the places llava is hardcoded of which this one seems really important to the functionality of actually using just llava and moondream. Also it appears this issue which was referenced in that file and given below has now been resolved:

# The Ollama OpenAPI compatibility layer doesn't support images
# see: https://github.com/ollama/ollama/issues/3690
# TODO: remove this when it does or make it configurable

Also, I'm pretty sure hardcoding the vision models is still the way to go unless ollama gets some sort of api for listing only the vision models. If they had such an api which they don't at the moment then you could build the list at build time or runtime (of which this issue is for listing all models is related, but not particularly for listing vision models and hasn't been implemented yet)

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions