Docker image No module named 'ps3'

[Entr0opsy]

Running docker run --gpus all --ipc=host --ulimit memlock=-1 --ulimit stack=67108864 vila:latest
leads to the below error

=============
== PyTorch ==

NVIDIA Release 24.06 (build 96418707)
PyTorch Version 2.4.0a0+f70bd71
Container image Copyright (c) 2024, NVIDIA CORPORATION & AFFILIATES. All rights reserved.
Copyright (c) 2014-2024 Facebook Inc.
Copyright (c) 2011-2014 Idiap Research Institute (Ronan Collobert)
Copyright (c) 2012-2014 Deepmind Technologies (Koray Kavukcuoglu)
Copyright (c) 2011-2012 NEC Laboratories America (Koray Kavukcuoglu)
Copyright (c) 2011-2013 NYU (Clement Farabet)
Copyright (c) 2006-2010 NEC Laboratories America (Ronan Collobert, Leon Bottou, Iain Melvin, Jason Weston)
Copyright (c) 2006 Idiap Research Institute (Samy Bengio)
Copyright (c) 2001-2004 Idiap Research Institute (Ronan Collobert, Samy Bengio, Johnny Mariethoz)
Copyright (c) 2015 Google Inc.
Copyright (c) 2015 Yangqing Jia
Copyright (c) 2013-2016 The Caffe contributors
All rights reserved.

Various files include modifications (c) NVIDIA CORPORATION & AFFILIATES. All rights reserved.

This container image and its contents are governed by the NVIDIA Deep Learning Container License.
By pulling and using the container, you accept the terms and conditions of this license:
https://developer.nvidia.com/ngc/nvidia-deep-learning-container-license

NOTE: CUDA Forward Compatibility mode ENABLED.
Using CUDA 12.5 driver version 555.42.02 with kernel driver version 550.144.03.
See https://docs.nvidia.com/deploy/cuda-compatibility/ for details.

NOTE: The SHMEM allocation limit is set to the default of 64MB. This may be
insufficient for PyTorch. NVIDIA recommends the use of the following flags:
docker run --gpus all --ipc=host --ulimit memlock=-1 --ulimit stack=67108864 ...

[2025-07-02 05:26:38,356] [INFO] [real_accelerator.py:110:get_accelerator] Setting ds_accelerator to cuda (auto detect)
PS3 is not installed. Please install it using the following command:
pip install ps3
Traceback (most recent call last):
File "/app/server.py", line 31, in
from llava.model.builder import load_pretrained_model
File "/app/llava/model/init.py", line 1, in
from .language_model.llava_llama import LlavaLlamaConfig, LlavaLlamaModel
File "/app/llava/model/language_model/llava_llama.py", line 33, in
from ..llava_arch import LlavaMetaForCausalLM, LlavaMetaModel
File "/app/llava/model/llava_arch.py", line 42, in
from llava.model.multimodal_encoder.builder import build_vision_tower
File "/app/llava/model/multimodal_encoder/builder.py", line 25, in
from .ps3_encoder import PS3VisionTower
File "/app/llava/model/multimodal_encoder/ps3_encoder.py", line 21, in
from ps3 import PS3Config, PS3ImageProcessor, PS3VisionConfig, PS3VisionModel
ModuleNotFoundError: No module named 'ps3'
ERROR conda.cli.main_run:execute(127): conda run python -u -W ignore server.py failed. (See above for error)

also running pip install ps3 leads to
ERROR: Could not find a version that satisfies the requirement ps3 (from versions: none)
ERROR: No matching distribution found for ps3

I think it should be pip install ps3-torch

Has anyone successfully used docker to get inference??

[Gamapro]

Hey. After some trial and error I was able to run the docker.

For that particular error, you should add the line

pip install ps3-torch

As well, I found you should upgrade the triton dependency

pip install triton==3.3.1

---

Another error i found is the following

ValueError: Cannot use chat template functions because tokenizer.chat_template is not set and no template argument was passed! For information about writing templates and setting the tokenizer.chat_template attribute, please see the documentation at https://huggingface.co/docs/transformers/main/en/chat_templating

In order to solve this, you should add the following line to the tokenizer_config.json of the model you want to use, in my case it was Efficient-Large-Model/VILA1.5-3b

` "chat_template": "{% for message in messages %}\n{% if message['role'] == 'user' %}\n{{ '<|user|>\n' + message['content'] + eos_token }}\n{% elif message['role'] == 'system' %}\n{{ '<|system|>\n' + message['content'] + eos_token }}\n{% elif message['role'] == 'assistant' %}\n{{ '<|assistant|>\n'  + message['content'] + eos_token }}\n{% endif %}\n{% if loop.last and add_generation_prompt %}\n{{ '<|assistant|>' }}\n{% endif %}\n{% endfor %}",

In my setting, this file is stored in the following path

~/.cache/huggingface/hub/models--Efficient-Large-Model--VILA1.5-3b/snapshots/llm/tokenizer_config.json

---

Last error I found was related to a failure importing FloatPointQuantizeTorch.
For this, simply add the following line on top of the file with the import error.

sys.path.append(os.path.dirname(os.path.abspath(__file__)))