Using Multi GPU

[kdcyberdude]

Hi Team,

I'm running multiple ComfyUI backends on SwarmUI with several GPUs, as described here. When I use the Generate tab, images are produced in parallel across the available GPUs. However, when I serve the workflow using this extension, simultaneous POST requests do not utilize all available backends/GPUs.

Is multi-GPU support available in this extension, or is additional configuration required?
Thanks.

[frostming]

The gpu is configured here

But it is also possible to change from env var when you serve it: BENTOML_CONFIG_OVERRIDES='{"services": {"ComfyService": {"resources": {"gpu": 4}}}}'