Non-deterministic inference results with identical input frames

[AxxhimK93]

I'm currently working on my master's thesis, where I'm researching real-time monocular depth estimation for endoscopic videos. I'm using your TensorRT C++ port of Depth Anything V2 – thank you for making this available!

During my experiments, I noticed unexpected behavior:
When running inference multiple times on the same input frame, I get different raw depth values each time I restart the inference.

Interestingly, Run 2 and Run 3 produced identical results, which suggests that the model architecture itself might be deterministic, but something in the pipeline introduces variability.

My setup:
Depth Anything V2 TensorRT implementation
TensorRT 10.13.2.6
CUDA Toolkit 12.9
FP16 precision
RTX 5090
Input resolution: 532x532

I suspect this might be related to TensorRT kernel/tactic selection at engine load time?

My questions:

1. Does the ONNX export process preserve determinism, or could there be issues there?
2. Are there specific TensorRT builder flags you'd recommend for deterministic results?
3. Have you observed similar non-determinism in your testing?

This variability contributes to temporal flickering in video applications, which is critical for my medical imaging use case.

Thank you for your work on this port!

Best regards,
Achim