Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[FEA] it would be nice if we could support org.apache.spark.sql.catalyst.expressions.Slice #11607

Open
nvliyuan opened this issue Oct 15, 2024 · 1 comment
Labels
feature request New feature or request good first issue Good for newcomers

Comments

@nvliyuan
Copy link
Collaborator

it would be nice if we could support org.apache.spark.sql.catalyst.expressions.Slice

scala> spark.sql("select slice(split(text,','),1,1) from table").show
24/10/15 07:27:33 WARN GpuOverrides:
!Exec <CollectLimitExec> cannot run on GPU because the Exec CollectLimitExec has been disabled, and is disabled by default because Collect Limit replacement can be slower on the GPU, if huge number of rows in a batch it could help by limiting the number of rows transferred from GPU to CPU. Set spark.rapids.sql.exec.CollectLimitExec to true if you wish to enable it
  @Partitioning <SinglePartition$> could run on GPU
  !Exec <ProjectExec> cannot run on GPU because not all expressions can be replaced
    @Expression <Alias> cast(slice(split(value#142, ,, -1), 1, 1) as string) AS slice(split(text, ,, -1), 1, 1)#157 could run on GPU
      @Expression <Cast> cast(slice(split(value#142, ,, -1), 1, 1) as string) could run on GPU
        ! <Slice> slice(split(value#142, ,, -1), 1, 1) cannot run on GPU because GPU does not currently support the operator class org.apache.spark.sql.catalyst.expressions.Slice
@nvliyuan nvliyuan added ? - Needs Triage Need team to review and classify feature request New feature or request labels Oct 15, 2024
@nvliyuan
Copy link
Collaborator Author

low priority

@sameerz sameerz added good first issue Good for newcomers and removed ? - Needs Triage Need team to review and classify labels Oct 15, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
feature request New feature or request good first issue Good for newcomers
Projects
None yet
Development

No branches or pull requests

2 participants