add niivue to list of integrations

[hanayik]

nnInteractive is a fantastic model to work with. I'm one of the core NiiVue contributors and have created a reference application that integrates NiiVue with nnInteractive via a simple FastAPI server.

This demo application is meant to serve as a guide for others in the community.

The demo repo is here: https://github.com/niivue/niivue-nninteractive

[FabianIsensee]

Thanks for this work and for the nice words about nnInteractive. I just tried to test your integration but am getting an error upon starting the server:

2025-10-23 10:07:55,855 - uvicorn.error - ERROR - Traceback (most recent call last):
File "/home/isensee/git_repos/niivue-nninteractive/.pixi/envs/default/lib/python3.12/site-packages/starlette/routing.py", line 694, in lifespan
async with self.lifespan_context(app) as maybe_state:
^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/isensee/git_repos/niivue-nninteractive/.pixi/envs/default/lib/python3.12/contextlib.py", line 210, in aenter
return await anext(self.gen)
^^^^^^^^^^^^^^^^^^^^^
File "/home/isensee/git_repos/niivue-nninteractive/src/niivue_nninteractive/api.py", line 37, in lifespan
initialize_model()
File "/home/isensee/git_repos/niivue-nninteractive/src/niivue_nninteractive/api.py", line 134, in initialize_model
model_session.initialize_from_trained_model_folder(model_path)
File "/home/isensee/git_repos/niivue-nninteractive/.pixi/envs/default/lib/python3.12/site-packages/nnInteractive/inference/inference_session.py", line 666, in initialize_from_trained_model_folder
checkpoint = torch.load(join(model_training_output_dir, fold_folder, checkpoint_name),
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/isensee/git_repos/niivue-nninteractive/.pixi/envs/default/lib/python3.12/site-packages/torch/serialization.py", line 1525, in load
return _load(
^^^^^^
File "/home/isensee/git_repos/niivue-nninteractive/.pixi/envs/default/lib/python3.12/site-packages/torch/serialization.py", line 2114, in _load
result = unpickler.load()
^^^^^^^^^^^^^^^^
File "/home/isensee/git_repos/niivue-nninteractive/.pixi/envs/default/lib/python3.12/site-packages/torch/serialization.py", line 2078, in persistent_load
typed_storage = load_tensor(
^^^^^^^^^^^^
File "/home/isensee/git_repos/niivue-nninteractive/.pixi/envs/default/lib/python3.12/site-packages/torch/serialization.py", line 2044, in load_tensor
wrap_storage = restore_location(storage, location)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/isensee/git_repos/niivue-nninteractive/.pixi/envs/default/lib/python3.12/site-packages/torch/serialization.py", line 1859, in restore_location
return default_restore_location(storage, str(map_location))
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/isensee/git_repos/niivue-nninteractive/.pixi/envs/default/lib/python3.12/site-packages/torch/serialization.py", line 698, in default_restore_location
result = fn(storage, location)
^^^^^^^^^^^^^^^^^^^^^
File "/home/isensee/git_repos/niivue-nninteractive/.pixi/envs/default/lib/python3.12/site-packages/torch/serialization.py", line 636, in _deserialize
device = _validate_device(location, backend_name)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/isensee/git_repos/niivue-nninteractive/.pixi/envs/default/lib/python3.12/site-packages/torch/serialization.py", line 615, in _validate_device
raise RuntimeError(
RuntimeError: Attempting to deserialize object on CUDA device 5 but torch.cuda.device_count() is 1. Please use torch.load with map_location to map your storages to an existing device.

It is weird that this happens since it happens in the nnInteractive code - but it doesn't happen when running nnInteractive from within napari. Did you by any chance modify the checkpoint?

Best,
Fabian

[hanayik]

@FabianIsensee , I had set the device to cuda:5 on my GPU server when testing. I have pushed a commit that defaults to using cuda:0 now. When a machine only has one GPU available, it should now use the correct device.

[FabianIsensee]

I was able to start it now but there still seems to be an issue:

[hanayik]

Whoops, had the IP address hard coded. Updated now :)

https://github.com/niivue/niivue-nninteractive

Environment Setup

Be sure to set the VITE_API_URL variable in frontend/.env. This defaults to http://localhost:8000, (fine for running the demo app on your local machine) but will need to be modified if you are not running the demo application locally. If the application is running on another machine, use the IP address (or full domain name). This applies to connecting machines over a VPN too.

reference commit with changes: niivue/niivue-nninteractive@c54a75a

[FabianIsensee]

Thanks for the update. I was able to run it now!

The integrations we list on the nnInteractive website are generally GUI programs that are ready for end users to make annotations.

The NiiVue integration, however, still feels more like a proof of concept at this stage. The workflow available through the repo only exposes rudimentary GUI functionality (no way of deleting images, only scribbles are supported, no hotkeys, etc.).

The fact that getting it running also took some effort might point in the same direction — that this is primarily intended for developers rather than for end users.

Could you clarify what the goal is here? If it’s meant as a user-facing tool, it would need more work to reach that level of maturity. If it’s more of a component for other NiiVue developers, then it probably wouldn’t fit alongside the other integrations on the website and might need to be presented differently.