why is the prompt_table in ModelRunner.generate passed in as npy file instead of a tensor ?

[yupbank]

is there any distributed constrains here? or due to the limitation of CLI? it feels strange to have user convert torch tensor to numpy array and then to disk npy file to pass it to the modelrunner.generate, and then within the generator, disk npy gets to convert back to torch tensor again.

torch tensor saved to disk https://github.com/NVIDIA/TensorRT-LLM/blob/main/examples/multimodal/run.py#L170
conver back to torch tensor https://github.com/NVIDIA/TensorRT-LLM/blob/main/tensorrt_llm/runtime/model_runner.py#L272