Bugs/1627 bug test vmap fails on multi node runs on hardware accelerators

Bugs/1627 bug test vmap fails on multi node runs on hardware accelerators #2213