Generate classes identical up to the shim package name [databricks] #11665
Conversation
Signed-off-by: Gera Shegalov <[email protected]>
Signed-off-by: Gera Shegalov <[email protected]>
gerashegalov
changed the title
Signed-off-by: Gera Shegalov <[email protected]>
|
build |
Signed-off-by: Gera Shegalov <[email protected]>
|
build |
Signed-off-by: Gera Shegalov <[email protected]>
|
build |
| spark-rapids-shim-json-lines ***/ | ||
| package com.nvidia.spark.rapids.spark320 | ||
| package com.nvidia.spark.rapids.$_spark.version.classifier_ |
There was a problem hiding this comment.
Are we regressing any functionality in the IDE's ability to understand what's going on with this change?
There was a problem hiding this comment.
I think the only change for IDE is that the developer needs a warning that the original file should be edited instead of the generated one. I'll add this warning
There was a problem hiding this comment.
But when editing the original file, I assume the IDE is still totally confused by this package statement and thus doesn't understand what package the file is originally in. That would be an issue if there were many classes in the package in question, since the IDE would not realize all of those classes don't need to be imported. But in this specific case, that's not a problem.
There was a problem hiding this comment.
There is undoubtedly room for confusion. But I think we can mitigate if the need arises. $spark.version.classifier is a legal package name. We could import such files directly and extend interpolation support to anywhere in the code.
Signed-off-by: Gera Shegalov <[email protected]>
Signed-off-by: Gera Shegalov <[email protected]>
|
build |
This PR adds the feature to shimplify to be able to define classes that are identical up to the package name requiring a spark.version.classifer interpolation in the package name
This allows to delete the per-shim RapidsShuffleManager