Support for spark 3.5+ #14678
-
|
Good day, |
Beta Was this translation helpful? Give feedback.
Replies: 1 comment 3 replies
-
|
Hi @MartinLisy, Making Spark NLP compatible with higher Spark versions is definitely a high priority for us, and we are working on it! Spark 3.5.* works, but Spark 4 and up requires some substantial changes in the library. |
Beta Was this translation helpful? Give feedback.
-
|
Thank you @DevinTDHa for the quick response. I can confirm that Spark NLP works on Spark 3.5.0. However, I have ran into issues when running Spark 3.5.2, for instance "Py4JError: com.johnsnowlabs.nlp.DocumentAssembler does not exist in the JVM". Is this something You are aware of or does it recquire an issue thread? |
Beta Was this translation helpful? Give feedback.
-
|
Hi @MartinLisy, We are not actually aware of issues there. It'd very be helpful for us if you can open an issue thread with precise reconstruction steps. That way, we can help you the most effectively. Thanks a lot! |
Beta Was this translation helpful? Give feedback.
-
|
Hi @DevinTDHa, My issues are explained by the closed issue #14203. Thanks for the support! |
Beta Was this translation helpful? Give feedback.
Hi @MartinLisy,
Making Spark NLP compatible with higher Spark versions is definitely a high priority for us, and we are working on it!
Spark 3.5.* works, but Spark 4 and up requires some substantial changes in the library.