transposed_mode field in DeepSpeedInferenceConfig

[MrGeva]

Hi,

I noticed transposed_mode was added to the DeepSpeedInferenceConfig as part of the DeepSpeed Chat change
By default it is set to False, while HybridEngine is the only one to set it to True.
Why was this mode added and how do the weights become transposed when going through the HybridEngine flow, while when working on topology like bigscience/bloom w/o hybrid engine the weights are not transposed?