Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
[SPARK-37345][CORE] Add
java.security.jgss/sun.security.krb5
to DEF…
…AULT_MODULE_OPTIONS ### What changes were proposed in this pull request? Add `java.security.jgss/sun.security.krb5` to `DEFAULT_MODULE_OPTIONS`. ### Why are the changes needed? To support Spark runs on Java 17: ``` Exception in thread "main" java.lang.IllegalArgumentException: Can't get Kerberos realm at org.apache.hadoop.security.HadoopKerberosName.setConfiguration(HadoopKerberosName.java:65) at org.apache.hadoop.security.UserGroupInformation.initialize(UserGroupInformation.java:306) at org.apache.hadoop.security.UserGroupInformation.setConfiguration(UserGroupInformation.java:352) at org.apache.spark.deploy.SparkHadoopUtil.<init>(SparkHadoopUtil.scala:50) at org.apache.spark.deploy.SparkHadoopUtil$.instance$lzycompute(SparkHadoopUtil.scala:397) at org.apache.spark.deploy.SparkHadoopUtil$.instance(SparkHadoopUtil.scala:397) at org.apache.spark.deploy.SparkHadoopUtil$.get(SparkHadoopUtil.scala:418) at org.apache.spark.executor.CoarseGrainedExecutorBackend$.run(CoarseGrainedExecutorBackend.scala:423) at org.apache.spark.executor.YarnCoarseGrainedExecutorBackend$.main(YarnCoarseGrainedExecutorBackend.scala:81) at org.apache.spark.executor.YarnCoarseGrainedExecutorBackend.main(YarnCoarseGrainedExecutorBackend.scala) Caused by: java.lang.IllegalAccessException: class org.apache.hadoop.security.authentication.util.KerberosUtil cannot access class sun.security.krb5.Config (in module java.security.jgss) because module java.security.jgss does not export sun.security.krb5 to unnamed module 3a0baae5 at java.base/jdk.internal.reflect.Reflection.newIllegalAccessException(Reflection.java:392) at java.base/java.lang.reflect.AccessibleObject.checkAccess(AccessibleObject.java:674) at java.base/java.lang.reflect.Method.invoke(Method.java:560) at org.apache.hadoop.security.authentication.util.KerberosUtil.getDefaultRealm(KerberosUtil.java:85) at org.apache.hadoop.security.HadoopKerberosName.setConfiguration(HadoopKerberosName.java:63) ... 9 more ``` ### Does this PR introduce _any_ user-facing change? No. ### How was this patch tested? Manual test: ``` export JAVA_HOME=/home/yumwang/jdk-17.0.1 export PATH=${JAVA_HOME}/bin:${PATH} bin/spark-sql --conf spark.yarn.dist.archives=/home/yumwang/jdk-17.0.1_linux-x64_bin.tar.gz --conf spark.executorEnv.JAVA_HOME=./jdk-17.0.1_linux-x64_bin.tar.gz/jdk-17.0.1/ ``` Closes apache#34615 from wangyum/SPARK-37345. Authored-by: Yuming Wang <[email protected]> Signed-off-by: Yuming Wang <[email protected]>
- Loading branch information