Deepspeed-FastGen: Missing Precision Evaluation

[m1mc]

In the existing doc of novel SplitFuse, the long prompt is decomposed for early token generation in forward pass. For pretrained LLMs from say huggingface, I think the decomposition will degrade the precision unless the LLM is also trained using SplitFuse, won’t it?