Apply ZIO SBT Ecosystem Plugin #1091

khajavi · 2023-02-28T19:26:11Z

No description provided.

khajavi · 2023-03-04T15:47:49Z

This work is ready for review.

The failing CI seems related to this issue #1095

afsalthaj · 2023-03-05T12:58:14Z

Thanks Milad. Have been waiting for this plugin and great addition :-)

afsalthaj · 2023-03-05T12:59:25Z

.github/workflows/ci.yml

@@ -1,89 +1,215 @@
-name: CI
+# This file was autogenerated using `zio-sbt-ci` plugin via `sbt generateGithubWorkflow` 
+# task and should be included in the git repository. Please do not edit it manually.


afsalthaj · 2023-03-05T13:00:53Z

.scalafmt.conf

-  "core/src/test/scala/zio/config/ProductBuilderTest.scala"
+  "core/src/test/scala/zio/config/ProductBuilderTest.scala",
+  "core/shared/src/main/scala-3.x/*",
+  "magnolia/shared/src/main/scala-dotty/*"


I think this can also go into a custom module within ecosystem plugin that wraps scalafmt
And for custom settings, such as exclude, I could simply configure with in each module setting, excludeFmt

Thank you! Added to the backlog.

afsalthaj · 2023-03-05T13:01:37Z

build.sbt

-    developers := List(
+    name                       := "ZIO Config",
+    crossScalaVersions -= scala211.value,
+    ciEnabledBranches          := Seq("series/4.x"),


I think we should include master as well, since master works with latest zio

What about series/2.x? Doesn't it work for the latest version of ZIO?

I don't think we need to support 2.x going forward

afsalthaj · 2023-03-05T13:11:19Z

build.sbt

-    licenses := List("Apache-2.0" -> url("http://www.apache.org/licenses/LICENSE-2.0")),
-    developers := List(
+    name                       := "ZIO Config",
+    crossScalaVersions -= scala211.value,


I would rather just rely on ecosystem plugin to decide the common scala versions that an ecosystem library should support. Obviously, ecosystem libraries can choose to exclude from this common list. This implies there will not be specific inThisBuild settings for crossScalaVersions. The exclusion, should be within each module.
Ecosystem Plugin can also choose to pre-built excluded list.

Currently, zio-sbt provides default scala versions, then we can add/remove and customize that for the whole modules or any submodule.
I'm not sure I got what you said, here I excluded scala 211 from all modules, although we can do that one by one.

afsalthaj · 2023-03-05T13:12:23Z

build.sbt

  )
 )

-addCommandAlias("fmt", "; scalafmtSbt; scalafmt; test:scalafmt")
-addCommandAlias("fix", "; all compile:scalafix test:scalafix; all scalafmtSbt scalafmtAll")
 addCommandAlias("compileAll", "; ++2.12.16; root2-12/compile; ++2.13.8!; root2-13/compile")


I would delete this as well. CommandAlias can be a pain. I would either choose it as sbtSettings that are not based on String. Even better is within ecosystem plugin

afsalthaj · 2023-03-05T13:12:55Z

build.sbt

  )
 )

-addCommandAlias("fmt", "; scalafmtSbt; scalafmt; test:scalafmt")
-addCommandAlias("fix", "; all compile:scalafix test:scalafix; all scalafmtSbt scalafmtAll")
 addCommandAlias("compileAll", "; ++2.12.16; root2-12/compile; ++2.13.8!; root2-13/compile")
 addCommandAlias("testAll", "; ++2.12.16; root2-12/test; ++2.13.8!; root2-13/test")


I would rather choose the same Same as above

afsalthaj · 2023-03-05T13:13:55Z

build.sbt

@@ -65,13 +67,14 @@ lazy val magnoliaDependencies =

 lazy val refinedDependencies =
  libraryDependencies ++= {
-    if (scalaBinaryVersion.value == "2.11") Seq.empty // Just to make IntelliJ happy
+    if (scalaVersion.value == scala211.value) Seq.empty // Just to make IntelliJ happy


This can happen several times. One setting can be addDependency which takes simply takes a list of scala versions to be excluded. Otherwise, different projects may have different ways to handle dependency settings.

Also refinedDependencies is an unnecessary redirection, that it could be inlined as well

That's a good idea, but it is the first time I saw such a case. I'm doing the same for other ecosystem projects. If I saw that the common case, I'll make a general utility inside zio-sbt.

Just wrote these utility functions:

def addDependenciesFor(scalaBinaryVersions: String*)(dependencies: ModuleID*) = libraryDependencies ++= { if (scalaBinaryVersions.contains(scalaBinaryVersion.value)) dependencies else Seq.empty } def addDependenciesExceptFor(scalaBinaryVersions: String*)(dependencies: ModuleID*) = libraryDependencies ++= { if (scalaBinaryVersions.contains(scalaBinaryVersion.value)) Seq.empty else dependencies } def addScalacOptionsFor(scalaBinaryVersions: String*)(options: String*) = scalacOptions ++= { if (scalaBinaryVersions.contains(scalaBinaryVersion.value)) options else Seq.empty } def addScalacOptionsExceptFor(scalaBinaryVersions: String*)(options: String*) = scalacOptions ++= { if (scalaBinaryVersions.contains(scalaBinaryVersion.value)) Seq.empty else options }

afsalthaj · 2023-03-05T13:16:10Z

build.sbt

-    ),
-    testFrameworks := Seq(new TestFramework("zio.test.sbt.ZTestFramework"))
+      "com.amazonaws" % "aws-java-sdk-ssm" % awsVersion
+    )


Much better for any new contributor, is to be able to say

lazy val zioConfigAws = jvmModule("aws").withDependencies().dependOn(zioConfig)

As mentioned above withDependencies can take care of weird cases, such as excluding if Scala-version is x etc.

The ecosystem plugin mandates the name of the root project (say zio-config), and any module definition simply append the file path name. This way every project becomes a standard

I agree, will work on this!

afsalthaj · 2023-03-05T13:20:29Z

build.sbt

-  .settings(stdSettings("zio-config-scalaz"))
-  .settings(crossProjectSettings)
+  .settings(stdSettings(name = "zio-config-scalaz", enableCrossProject = true))
+  .settings(enableZIO())


I think its mostly to help defining modules easily without relying on adhoc sbt settings. As mentioned above, as a maintainer I am far relieved if I can just simply define module(...) using the plugin, or if needed jvmModule, jsModule etc.

afsalthaj · 2023-03-05T13:29:38Z

I have put some comments, but all of it boils down to 1 suggestion, that being able to define module based on the plugin,

lazy val myModule = 
   module("aws")
    .withDependencies("dev.zio" %%% "izumi-reflect" % "2.2.2")
    .dependsOn(core)

Adding customisations becomes even simpler

lazy val myModule = 
   module("aws")
    .skipScala211
    .withDependencies("dev.zio" %%% "izumi-reflect" % "2.2.2")
    .dependsOn(core)

This is much easier for a maintainer. Example: In the above case, core can support 211, but aws cannot, and there is no need of an indirection such as having a big Map of each module and scala versions supported

khajavi · 2023-03-05T13:52:24Z

@afsalthaj
That is an excellent DSL for writing builds without any boilerplate. I agree with such language. The first iteration aimed to factor out BuildHelper and make it maintainable.
I am excited to work on implementing your suggestions for the next phase.

khajavi · 2023-03-05T14:07:01Z

In the above case, core can support 211, but aws cannot, and there is no need of an indirection such as having a big Map of each module and scala versions supported

The name of supportedScalaVersions is somehow confusing, I will rename it to something like ciScalaVersionMatrix. We just use that to create the test matrix. This is just a workaround because I lack deep knowledge of how to iterate through sbt submodules and create matrix map pragmatically.

So ideally I would like to have something like this:

val ciScalaVersionsMatrix: Map[String, Seq[String]] = 
  listOfAllModules.map { module =>
      (module / thisProject).value.id   -> (module / crossScalaVersions).value
  }

But it will not compile due to complicated macro-based sbt internals. We need to find sbt specific solution to traverse modules and pull out settings.

khajavi · 2023-03-09T14:54:55Z

Thrilled to see the geeeen color 😅
@afsalthaj Can you please have a look?

we should monitor and investigate to find the best value.

with 4, it takes about one hour and 14 minutes to run all jobs.

…dJVM

Error message: [error] -- Error: /home/runner/work/zio-config/zio-config/magnolia/shared/src/test/scala-dotty/zio/config/magnolia/MarkdownSpec.scala:14:44 [error] 14 | generateDocs(deriveConfig[RawConfig]).toTable.toGithubFlavouredMarkdown [error] | ^ [error] | given instance derived is declared as `inline`, but was not inlined [error] | [error] | Try increasing `-Xmax-inlines` above 32

To reproduce the following error, run `sbt zioConfigRefinedJVM/Test/compile` [error] -- [E008] Not Found Error: /home/runner/work/zio-config/zio-config/refined/shared/src/test/scala/zio/config/refined/NumericSupportTest.scala:3:26 [error] 3 |import eu.timepit.refined.W [error] | ^ [error] | value W is not a member of eu.timepit.refined

khajavi force-pushed the apply-zio-ecosystem-plugin branch from 757e449 to 96facd1 Compare March 1, 2023 18:25

khajavi marked this pull request as ready for review March 4, 2023 15:46

khajavi requested a review from a team as a code owner March 4, 2023 15:46

afsalthaj reviewed Mar 5, 2023

View reviewed changes

khajavi mentioned this pull request Mar 5, 2023

Support for Scalafmt zio/zio-sbt#147

Open

khajavi force-pushed the apply-zio-ecosystem-plugin branch from 354e474 to 2ffa644 Compare March 7, 2023 11:21

khajavi added 10 commits March 10, 2023 21:40

initial work.

8027461

zio config settings.

f0bf7a0

js, jvm and native settings.

81df7f9

zioConfigAws.

8ab053f

enable streaming.

2541259

zioConfigAwsJVM.

ac4b135

zioConfigAws.

20f5eec

more setting migration.

19ef6f3

more settings.

da087eb

move most settings to new version.

1cc26da

khajavi added 14 commits March 10, 2023 21:40

generate ci.

6e35536

remove extra commands.

db67037

fmt

23fd7ba

update ci.

bda9921

include master branch for ci trigger.

d324327

increase initial and maximum heap size.

a1fa92d

set max parallel test to 4.

f72a709

we should monitor and investigate to find the best value.

set max parallel test to 8.

a6d0131

with 4, it takes about one hour and 14 minutes to run all jobs.

scalafix.

8e1f60e

fix docs generation warning.

eb80de3

fmt.

43c2bd1

scalafix.

3deac09

add back scala 3 support for zioConfigMagnoliaJVM and zioConfigRefine…

f287178

…dJVM

fmt.

b2636a8

khajavi force-pushed the apply-zio-ecosystem-plugin branch from 1c5b6ce to b2636a8 Compare March 10, 2023 18:17

khajavi added 10 commits March 11, 2023 20:08

update zio-sbt plugins.

a9a0a06

add utility functions for add dependencies and scalac options.

707ef7a

fix name for docs project.

2326962

fix build errors.

73aeffc

update zio-sbt plugins.

a1b4dc1

scalafix.

25b80bf

remove CheckAllCodeCompiles step from ci.

e37453f

refined: add support for scala 3.

dfbb8df

khajavi requested a review from afsalthaj March 14, 2023 11:26

afsalthaj approved these changes Mar 24, 2023

View reviewed changes

Merge branch 'series/4.x' into apply-zio-ecosystem-plugin

be7e985

tPl0ch mentioned this pull request May 23, 2023

Removed silencer SBT plugin #1171

Merged

Merge branch 'series/4.x' into apply-zio-ecosystem-plugin

1d8ee07

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Apply ZIO SBT Ecosystem Plugin #1091

Apply ZIO SBT Ecosystem Plugin #1091

khajavi commented Feb 28, 2023

khajavi commented Mar 4, 2023

afsalthaj commented Mar 5, 2023

afsalthaj Mar 5, 2023

afsalthaj Mar 5, 2023

khajavi Mar 5, 2023

afsalthaj Mar 5, 2023

khajavi Mar 5, 2023

afsalthaj Mar 6, 2023

khajavi Mar 8, 2023

afsalthaj Mar 5, 2023

khajavi Mar 7, 2023

afsalthaj Mar 5, 2023

khajavi Mar 8, 2023

afsalthaj Mar 5, 2023

khajavi Mar 8, 2023

afsalthaj Mar 5, 2023

khajavi Mar 7, 2023

khajavi Mar 12, 2023

afsalthaj Mar 5, 2023 •

edited

Loading

khajavi Mar 7, 2023

afsalthaj Mar 5, 2023

afsalthaj commented Mar 5, 2023 •

edited

Loading

khajavi commented Mar 5, 2023

khajavi commented Mar 5, 2023

khajavi commented Mar 9, 2023

Apply ZIO SBT Ecosystem Plugin #1091

Are you sure you want to change the base?

Apply ZIO SBT Ecosystem Plugin #1091

Conversation

khajavi commented Feb 28, 2023

khajavi commented Mar 4, 2023

afsalthaj commented Mar 5, 2023

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

afsalthaj Mar 5, 2023 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

afsalthaj commented Mar 5, 2023 • edited Loading

khajavi commented Mar 5, 2023

khajavi commented Mar 5, 2023

khajavi commented Mar 9, 2023

afsalthaj Mar 5, 2023 •

edited

Loading

afsalthaj commented Mar 5, 2023 •

edited

Loading