[BUG] Failed to run mig.sh on MIG dataproc-2.1-ubuntu20

[yinqingh]

Describe the bug
Observed following error while running mig.sh on dataproc-2.1-ubuntu20 with runtime version "2.1.72-ubuntu20" and kernel version "5.15.0-1067-gcp".

 make -f ./scripts/Makefile.modpost
   sed 's/\.ko$/\.o/' /var/lib/dkms/nvidia/495.29.05/build/modules.order | scripts/mod/modpost -m -a  -o /var/lib/dkms/nvidia/495.29.05/build/Module.symvers -e -i Module.symvers   -T -
 ERROR: modpost: GPL-incompatible module nvidia.ko uses GPL-only symbol 'rcu_read_unlock_strict'
 make[2]: *** [scripts/Makefile.modpost:133: /var/lib/dkms/nvidia/495.29.05/build/Module.symvers] Error 1

Tried with some old dataproc runtime versions. It works with runtime version "2.1.40-ubuntu20" and kernel version "5.15.0-1049-gcp".

Steps/Code to reproduce bug

1. Create dataproc cluster using MIG with nvidia-tesla-a100 gpu and runtime version "2.1.72-ubuntu20"
2. ssh to gpu node
3. download mig.sh
4. sudo bash mig.sh

Expected behavior
succeed to run mig.sh

Environment details (please complete the following information)

• Environment location: Dataproc, version 2.1.72-ubuntu20

[pxLi]

thanks for the investigation!

@sameerz This is the reason why mig-on-dataproc-2.1-ubuntu20 has been failing to initialize recently.

[SurajAralihalli]

Hello @yinqingh, I think you're using a different version of /gpu/mig.sh
Can you try with /spark-rapids/mig.sh?

I’ll inform the repository maintainers about this inconsistency.

Edit: Created issue GoogleCloudDataproc/initialization-actions#1259