Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Upgrade scala 2.12 #342

Open
wants to merge 2 commits into
base: 2.4
Choose a base branch
from
Open

Upgrade scala 2.12 #342

wants to merge 2 commits into from

Conversation

maasg
Copy link

@maasg maasg commented Nov 15, 2019

This PR upgrades the build and the code to compile to Scala 2.12.
The code needed minor changes to disambiguate the SAM type inference.

I hope somebody could help out to support a multi-target build to produce 2.11 (if still needed) and 2.12.

Addresses #321 (but it doesn't preserve support for 2.11)

@msftclas
Copy link

msftclas commented Nov 15, 2019

CLA assistant check
All CLA requirements met.

@conorbmurphy
Copy link

Has there been any update on this? Spark 3 will likely be released later this month so Scala 2.12 support is crucial

@mpfishe2
Copy link

mpfishe2 commented Jul 6, 2020

Any updates? Also have projects targeting Spark 3.0 (which is now released) and need Scala 2.12 support. Will this get resolved soon?

@FabianMeiswinkel
Copy link
Member

We are planning to release a new version of the CosmosDB Spark connector targeting Spark 3.0 and Scala 2.12 in the second half of this calendar year.

@maasg maasg force-pushed the upgrade-scala-2.12 branch from cb4b662 to 83f96be Compare July 6, 2020 14:01
@maasg
Copy link
Author

maasg commented Jul 6, 2020

@FabianMeiswinkel I've refreshed the PR and ran all the tests.
The CI seems to have a problem.

The command "~/bin/install-jdk.sh --target "/home/travis/oraclejdk8" --workspace "/home/travis/.cache/install-jdk" --feature "8" --license "BCL"" failed and exited with 56 during .

@FabianMeiswinkel
Copy link
Member

@FabianMeiswinkel I've refreshed the PR and ran all the tests.
The CI seems to have a problem.

The command "~/bin/install-jdk.sh --target "/home/travis/oraclejdk8" --workspace "/home/travis/.cache/install-jdk" --feature "8" --license "BCL"" failed and exited with 56 during .

We have no plan of merging this PR into the 2.4 branch because Scala 2.11 and 2.12 aren't binary compatible - we aren't planning on maintaining to official versions of our connector targeting Spark 2.4 - instead we will create a new major version targeting Spark 3.0 and Scala 2.12 - and continue to maintain the one targeting Spark 2.4 and Scala 2.11.

You can of course use private builds based off of this PR for your purposes.

@maasg
Copy link
Author

maasg commented Jul 7, 2020

@FabianMeiswinkel Thanks for the info. Would you like me to close this PR or leave it as-is?
I created it b/c some people needed a Scala 2.12 version and I gave them a hand with that but I don't use it myself and might go stale again after a while.

@FabianMeiswinkel
Copy link
Member

@maasg - If you don't mind I would keep it open - I think it is a valuable help for anyone who needs to switch to Scala 2.12 even before the new major version of the CosmosDB Spark connector has been released.

@asyncoder
Copy link

Spark 3.0 w/ Scala 2.12 has been released but still Cosmos connector supports only Scala 2.11. Any updates please?

@jenden
Copy link

jenden commented Dec 8, 2020

Related to #405 - 15+ people asking for this change. Is this still on the roadmap for 2020?

@FabianMeiswinkel
Copy link
Member

We are planning to release a Spark 3 compatible connector before April 2021 - preview will start in Deb. 2021.

@SteffenMangold
Copy link

We are planning to release a Spark 3 compatible connector before April 2021 - preview will start in Deb. 2021.

Waiting strongly on this preview. Otherwise, we can not use the last DataBricks features. This really blocks us currently.

@duttong13
Copy link

Any update on this upgrade? Databricks will EoL support scala 2.11 clusters by July 2021 so we need this upgrade way before then

@EdgeyErn
Copy link

EdgeyErn commented Apr 1, 2021

Its April 1st, I haven't seen a preview or anything. Any update on this? Really expected more out of Microsoft.

@vinaykv1991
Copy link

Any update on this?

We need to have scala 2.12 build in spark 2.4.x

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.