🐛 [Bug] one ASR model can run with use_python_runtime=True while core dump on w/o it

[wenbingl]

Bug Description

After the fixing of this issue: #4107, there is one more issue on run the ASR model.

To Reproduce

Disable use_python_runtime will cause a core dump
model.decoder = torch_tensorrt.dynamo.compile(
decoder_ep,
arg_inputs=[],
kwarg_inputs=kwarg_inputs,
truncate_double=True,
min_block_size=4,
optimization_level=5,
decompose_attention=True,
use_python_runtime=True
)

Expected behavior

Environment

Build information about Torch-TensorRT can be found by turning on debug messages

• Torch-TensorRT Version (e.g. 1.0.0):
• PyTorch Version (e.g. 1.0):
• CPU Architecture:
• OS (e.g., Linux):
• How you installed PyTorch (conda, pip, libtorch, source):
• Build command you used (if compiling from source):
• Are you using local sources or building from archives:
• Python version:
• CUDA version:
• GPU models and configuration:
• Any other relevant information:

Additional context