diverging implicit expansion on squaredDistance

[pan3793]

Hi,

I'm trying to bump breeze from 1.0 to 1.2 in Apache Spark, but got such compile exceptions. And I noticed that breeze migrated to new codegen framework recently, is it relative?

[error] /Users/chengpan/Projects/apache-spark/mllib/src/test/scala/org/apache/spark/mllib/util/MLUtilsSuite.scala:60:46: diverging implicit expansion for type breeze.linalg.squaredDistance.Impl2[breeze.linalg.Vector[Double],breeze.linalg.Vector[Double],VR]
[error] starting with method castOps in trait VectorOps
[error]       val squaredDist = breezeSquaredDistance(v1.asBreeze, v2.asBreeze)
[error]                                              ^
[error] /Users/chengpan/Projects/apache-spark/mllib/src/test/scala/org/apache/spark/mllib/util/MLUtilsSuite.scala:66:47: diverging implicit expansion for type breeze.linalg.squaredDistance.Impl2[breeze.linalg.Vector[Double],breeze.linalg.Vector[Double],VR]
[error] starting with method castOps in trait VectorOps
[error]       val squaredDist2 = breezeSquaredDistance(v2.asBreeze, v3.asBreeze)
[error]                                               ^
[error] /Users/chengpan/Projects/apache-spark/mllib/src/test/scala/org/apache/spark/mllib/util/MLUtilsSuite.scala:74:48: diverging implicit expansion for type breeze.linalg.squaredDistance.Impl2[breeze.linalg.Vector[Double],breeze.linalg.Vector[Double],VR]
[error] starting with method castOps in trait VectorOps
[error]         val squaredDist = breezeSquaredDistance(v2.asBreeze, v4.asBreeze)
[error]                                                ^
[error] four errors found
[warn] In the last 9 seconds, 5.224 (60.3%) were spent in GC. [Heap: 2.85GB free of 3.80GB, max 3.80GB] Consider increasing the JVM heap using `-Xmx` or try a different collector, e.g. `-XX:+UseG1GC`, for better performance.
[error] (mllib / Test / compileIncremental) Compilation failed
[error] Total time: 341 s (05:41), completed 2021-5-4 20:10:48

The compile failed code is here https://github.com/pan3793/spark/blob/SPARK-35310/mllib/src/test/scala/org/apache/spark/mllib/util/MLUtilsSuite.scala

And to reproduce the issue, run build/mvn clean compile -DskipTests or build/sbt compile test:compile.

[dlwh]

Interesting. I imagine something got messed up. I'm currently doing a major refactor to try to get Scala 3 support in and I'm fighting a lot of issues like this. In particular the behavior around castOps is getting changed quite a lot, so I hope this will be fixed soon.

…

On Tue, May 4, 2021 at 5:44 AM Cheng Pan ***@***.***> wrote: Hi, I'm trying to bump breeze from 1.0 to 1.2 in Apache Spark, but got such compile exceptions. And I noticed that breeze migrated to new codegen framework recently, is it relative? [error] /Users/chengpan/Projects/apache-spark/mllib/src/test/scala/org/apache/spark/mllib/util/MLUtilsSuite.scala:60:46: diverging implicit expansion for type breeze.linalg.squaredDistance.Impl2[breeze.linalg.Vector[Double],breeze.linalg.Vector[Double],VR] [error] starting with method castOps in trait VectorOps [error] val squaredDist = breezeSquaredDistance(v1.asBreeze, v2.asBreeze) [error] ^ [error] /Users/chengpan/Projects/apache-spark/mllib/src/test/scala/org/apache/spark/mllib/util/MLUtilsSuite.scala:66:47: diverging implicit expansion for type breeze.linalg.squaredDistance.Impl2[breeze.linalg.Vector[Double],breeze.linalg.Vector[Double],VR] [error] starting with method castOps in trait VectorOps [error] val squaredDist2 = breezeSquaredDistance(v2.asBreeze, v3.asBreeze) [error] ^ [error] /Users/chengpan/Projects/apache-spark/mllib/src/test/scala/org/apache/spark/mllib/util/MLUtilsSuite.scala:74:48: diverging implicit expansion for type breeze.linalg.squaredDistance.Impl2[breeze.linalg.Vector[Double],breeze.linalg.Vector[Double],VR] [error] starting with method castOps in trait VectorOps [error] val squaredDist = breezeSquaredDistance(v2.asBreeze, v4.asBreeze) [error] ^ [error] four errors found [warn] In the last 9 seconds, 5.224 (60.3%) were spent in GC. [Heap: 2.85GB free of 3.80GB, max 3.80GB] Consider increasing the JVM heap using `-Xmx` or try a different collector, e.g. `-XX:+UseG1GC`, for better performance. [error] (mllib / Test / compileIncremental) Compilation failed [error] Total time: 341 s (05:41), completed 2021-5-4 20:10:48 The compile failed code is here https://github.com/pan3793/spark/blob/SPARK-35310/mllib/src/test/scala/org/apache/spark/mllib/util/MLUtilsSuite.scala And to reproduce the issue, run build/mvn clean compile -DskipTests or build/sbt compile test:compile. — You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub <#810>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AAACLIN47SGUB6IHRXAVDZDTL7TZHANCNFSM44CUNJXA> .