[Feature]: Add VLLM support

[Noobzik]

Is your feature request related to a problem? Please describe.

Wanted to use Nvidia nemotrons models locally, so I spin up a VLLM instance in a Nvidia Docker Toolkit. However, I noticed the VLLM is not supported

Describe the solution you'd like

Support of VLLM by implementing a new configuration for it

Describe alternatives you've considered

No response

Additional context

No response

[linear]

RES-202

[srbhr]

Thanks, I'll look into it.

Were you able to use it with any other LLMs in the meantime?

[Noobzik]

Tried with Ollama via Docker Nvidia toolkit + docker compose of the project with no success of the backend communicating with both Ollama and front end (Using Nvidia DGX Spark as a remote, so no localhost or anything like 127. IP address, also using tails)

This is something I am still looking on my free time

[srbhr]

I think there's a port issue with the Docker Image because it uses local as the host, so it's inside the Docker host. But then Ollama runs as a different layer.

[Noobzik]

I have another issue with the docker-compose that I can't figure out the fix, I'll post it in discussion, since it's configuration related (and not related to here)