Automatically extract list of dependencies from the BOM

[gaeljw]

Context

Unless I missed something, the plugin's current usage requires manually defining all dependencies on which we want to "apply the BOM".

Case A

Given the following setup:

// project/Dependencies.scala
case class JacksonDependencies(platformBom: Bom) {
  val dependencies: Seq[ModuleID] = Seq(
    "com.fasterxml.jackson.core" % "jackson-databind" % platformBom
  )
}

// build.sbt
lazy val deps = Bom.read("com.fasterxml.jackson" % "jackson-bom" % "2.16.0")(bom => JacksonDependencies(bom))

lazy val `sbt-bom-example-github` = project
  .in(file("."))
  .settings(deps)
  .settings(
    name := "sbt-bom-example-github",
    resolvers := Resolver.DefaultMavenRepository +: resolvers.value,
    libraryDependencies += deps.key.value.dependencies
  )

The dependency tree is as follows:

[info] sbt-bom-example-github:sbt-bom-example-github_2.12:0.1.0-SNAPSHOT [S]
[info]   +-com.fasterxml.jackson.core:jackson-databind:2.16.0
[info]     +-com.fasterxml.jackson.core:jackson-annotations:2.16.0
[info]     +-com.fasterxml.jackson.core:jackson-core:2.16.0

So far, so good.

Case B

But now, let's take the following example of a more real-life project with many transitive dependencies:

// project/Dependencies.scala
case class JacksonDependencies(platformBom: Bom) {
  val dependencies: Seq[ModuleID] = Seq(
    "com.fasterxml.jackson.core" % "jackson-databind" % platformBom
  )
}

// build.sbt
lazy val deps = Bom.read("com.fasterxml.jackson" % "jackson-bom" % "2.16.0")(bom => JacksonDependencies(bom))

lazy val `sbt-bom-example-github` = project
  .in(file("."))
  .settings(deps)
  .settings(
    name := "sbt-bom-example-github",
    resolvers := Resolver.DefaultMavenRepository +: resolvers.value,
    libraryDependencies += "com.typesafe.play" %% "play" % "2.8.21",
    libraryDependencies += deps.key.value.dependencies
  )

The difference with case A is:

+ libraryDependencies += "com.typesafe.play" %% "play" % "2.8.21",
   libraryDependencies += deps.key.value.dependencies

Now the dependency tree is like this (extract to be easier to read):

info] sbt-bom-example-github:sbt-bom-example-github_2.12:0.1.0-SNAPSHOT [S]
[info]   +-com.fasterxml.jackson.core:jackson-databind:2.16.0
[info]   | +-com.fasterxml.jackson.core:jackson-annotations:2.16.0
[info]   | +-com.fasterxml.jackson.core:jackson-core:2.16.0
[info]   | 
[info]   +-com.typesafe.play:play_2.12:2.8.21 [S]
[info]     +-com.fasterxml.jackson.core:jackson-annotations:2.11.4 (evicted by: 2.16.0)
[info]     +-com.fasterxml.jackson.core:jackson-annotations:2.16.0
[info]     +-com.fasterxml.jackson.core:jackson-core:2.11.4 (evicted by: 2.16.0)
[info]     +-com.fasterxml.jackson.core:jackson-core:2.16.0
[info]     +-com.fasterxml.jackson.core:jackson-databind:2.11.4 (evicted by: 2.16.0)
[info]     +-com.fasterxml.jackson.core:jackson-databind:2.16.0
[info]     | +-com.fasterxml.jackson.core:jackson-annotations:2.16.0
[info]     | +-com.fasterxml.jackson.core:jackson-core:2.16.0
[info]     | 
[info]     +-com.fasterxml.jackson.datatype:jackson-datatype-jdk8:2.11.4
[info]     | +-com.fasterxml.jackson.core:jackson-core:2.11.4 (evicted by: 2.16.0)
[info]     | +-com.fasterxml.jackson.core:jackson-core:2.16.0
[info]     | +-com.fasterxml.jackson.core:jackson-databind:2.11.4 (evicted by: 2.16.0)
[info]     | +-com.fasterxml.jackson.core:jackson-databind:2.16.0
[info]     |   +-com.fasterxml.jackson.core:jackson-annotations:2.16.0
[info]     |   +-com.fasterxml.jackson.core:jackson-core:2.16.0
[info]     |   
[info]     +-com.fasterxml.jackson.datatype:jackson-datatype-jsr310:2.11.4
[info]     | +-com.fasterxml.jackson.core:jackson-annotations:2.11.4 (evicted by: 2.1..
[info]     | +-com.fasterxml.jackson.core:jackson-annotations:2.16.0
[info]     | +-com.fasterxml.jackson.core:jackson-core:2.11.4 (evicted by: 2.16.0)
[info]     | +-com.fasterxml.jackson.core:jackson-core:2.16.0
[info]     | +-com.fasterxml.jackson.core:jackson-databind:2.11.4 (evicted by: 2.16.0)
[info]     | +-com.fasterxml.jackson.core:jackson-databind:2.16.0
[info]     |   +-com.fasterxml.jackson.core:jackson-annotations:2.16.0
[info]     |   +-com.fasterxml.jackson.core:jackson-core:2.16.0
[info]     |   
...

As I haven't listed com.fasterxml.jackson.datatype:jackson-datatype-jdk8 or com.fasterxml.jackson.datatype:jackson-datatype-jsr310 explicitly in my JacksonDependencies class, they are not getting the BOM version: they are pulled in version 2.11.4 instead of the desired/expected 2.16.0.

This can be fixed by explicitly listing all dependencies but this can be long and not future-proof as you can relatively easily know the deps you're pulling today (to list them and add them explicitly) but you won't notice if someone adds another lib that transitively needs another Jackson lib that you hadn't listed yet.

Proposal

In my opinion, this is also why BOM exist: manage transitive dependencies versions.

I think we could have the sbt plugin parse the BOM and expose the dependencies listed in the BOM as a variable that can then be used in dependencyOverrides.

For instance, given the Jackson BOM (extract):

 <properties>
    <jackson.version>2.16.0</jackson.version>

    <jackson.version.annotations>${jackson.version}</jackson.version.annotations>
    <jackson.version.core>${jackson.version}</jackson.version.core>
    <jackson.version.databind>${jackson.version}</jackson.version.databind>
    <jackson.version.datatype>${jackson.version}</jackson.version.datatype>

   ...

  </properties>

  <dependencyManagement>
    <dependencies>

      <!-- Core -->
      <dependency>
        <groupId>com.fasterxml.jackson.core</groupId>
        <artifactId>jackson-annotations</artifactId>
        <version>${jackson.version.annotations}</version>
      </dependency>
      <dependency>
        <groupId>com.fasterxml.jackson.core</groupId>
        <artifactId>jackson-core</artifactId>
        <version>${jackson.version.core}</version>
      </dependency>
      <dependency>
        <groupId>com.fasterxml.jackson.core</groupId>
        <artifactId>jackson-databind</artifactId>
        <version>${jackson.version.databind}</version>
      </dependency>

      <!-- ... -->

      <dependency>
        <groupId>com.fasterxml.jackson.datatype</groupId>
        <artifactId>jackson-datatype-jdk8</artifactId>
        <version>${jackson.version.datatype}</version>
      </dependency>
      <dependency>
        <groupId>com.fasterxml.jackson.datatype</groupId>
        <artifactId>jackson-datatype-jsr310</artifactId>
        <version>${jackson.version.datatype}</version>
      </dependency>

...

We could have following usage in a SBT project:

// build.sbt

lazy val deps = Bom.read("com.fasterxml.jackson" % "jackson-bom" % "2.16.0")(bom => ???)

lazy val `sbt-bom-example-github` = project
  .in(file("."))
  .settings(deps)
  .settings(
    name := "sbt-bom-example-github",
    resolvers := Resolver.DefaultMavenRepository +: resolvers.value,
    libraryDependencies += "com.typesafe.play" %% "play" % "2.8.21",
    dependencyOverrides ++= deps.key.value.bomDependencies
  )

Where deps.key.value.bomDependencies is somehow automatically provided by the plugin and would contain:

bomDependencies: Seq[ModuleID] = Seq(
    "com.fasterxml.jackson.core" % "jackson-annotations" % "<theVersionFromTheBom>",
    "com.fasterxml.jackson.core" % "jackson-core" % "<theVersionFromTheBom>",
    "com.fasterxml.jackson.core" % "jackson-databind" % "<theVersionFromTheBom>",
    "com.fasterxml.jackson.datatype" % "jackson-datatype-jdk8" % "<theVersionFromTheBom>",
    "com.fasterxml.jackson.datatype" % "jackson-datatype-jsr310" % "<theVersionFromTheBom>",
   // and all others...
  )

That would be a closer experience to what we have in Maven.

How

It's unclear to me how this could be implemented, but I'd be happy to help if this sounds interesting to you.

[molekyla]

Hi @gaeljw. I think that your proposal is reasonable and should be implemented. The team gratefully accepts contributions via pull requests.

[gaeljw]

I'll have a look to the code in the coming days/weeks but if you already have some things in mind I should know or places I should look at in priority, please do :)

[molekyla]

I'll have a look to the code in the coming days/weeks but if you already have some things in mind I should know or places I should look at in priority, please do :)

I believe the list should be created in method BomReader.assembleBom():https://github.com/heremaps/here-sbt-bom/blob/master/plugin/src/main/scala/com/here/bom/internal/BomReader.scala#L134

And then propagated up to the client code

[gaeljw]

Here's a PR that solves this, any feedback welcome of course :) #13

[molekyla]

Closing the issue. See #13 for the details