Parallel inference with multiple adapters can be rather slow

[XiXunX]

I trained several adapters on bert, which all work pretty well. However, when I tried to load more than two adapters and make inference, the speed is heavily affected. The more adapters I load, the slower the inference is.

I intend to load dozens of adapters parallelly in the future, but it seems impossible due to the speed issue.

Does anyone has solutions or alternative repositories？ That would be really helpful, thanks a lot.

[XiXunX]

@calpt could you please have a look at this? Thanks.