Skip to content

Commit

Permalink
Rename netty-server-loom to netty-server-sync (#3704)
Browse files Browse the repository at this point in the history
  • Loading branch information
kciesielski authored Apr 19, 2024
1 parent 20a526c commit 4dbbb06
Show file tree
Hide file tree
Showing 23 changed files with 70 additions and 64 deletions.
29 changes: 16 additions & 13 deletions build.sbt
Original file line number Diff line number Diff line change
Expand Up @@ -219,7 +219,7 @@ lazy val rawAllAggregates = core.projectRefs ++
vertxServerZio.projectRefs ++
jdkhttpServer.projectRefs ++
nettyServer.projectRefs ++
nettyServerLoom.projectRefs ++
nettyServerSync.projectRefs ++
nettyServerCats.projectRefs ++
nettyServerZio.projectRefs ++
nimaServer.projectRefs ++
Expand Down Expand Up @@ -249,8 +249,14 @@ lazy val rawAllAggregates = core.projectRefs ++
derevo.projectRefs ++
awsCdk.projectRefs

def buildWithLoom(project: String): Boolean =
project.contains("Loom") || project.contains("nima") || project.contains("perfTests") || project.contains("examples3")
lazy val loomProjects: Seq[String] = Seq(nettyServerSync, nimaServer, examples).flatMap(_.projectRefs).flatMap(projectId)

def projectId(projectRef: ProjectReference): Option[String] =
projectRef match {
case ProjectRef(_, id) => Some(id)
case LocalProject(id) => Some(id)
case _ => None
}

lazy val allAggregates: Seq[ProjectReference] = {
val filteredByNative = if (sys.env.isDefinedAt("STTP_NATIVE")) {
Expand All @@ -262,15 +268,14 @@ lazy val allAggregates: Seq[ProjectReference] = {
}
if (sys.env.isDefinedAt("ONLY_LOOM")) {
println("[info] ONLY_LOOM defined, including only loom-based projects")
filteredByNative.filter(p => buildWithLoom(p.toString))
filteredByNative.filter(p => projectId(p).forall(loomProjects.contains))
} else if (sys.env.isDefinedAt("ALSO_LOOM")) {
println("[info] ALSO_LOOM defined, including also loom-based projects")
filteredByNative
} else {
println("[info] ONLY_LOOM *not* defined, *not* including loom-based-projects")
filteredByNative.filterNot(p => buildWithLoom(p.toString))
filteredByNative.filterNot(p => projectId(p).forall(loomProjects.contains))
}

}

// separating testing into different Scala versions so that it's not all done at once, as it causes memory problems on CI
Expand Down Expand Up @@ -1450,18 +1455,16 @@ lazy val nettyServer: ProjectMatrix = (projectMatrix in file("server/netty-serve
.jvmPlatform(scalaVersions = scala2And3Versions)
.dependsOn(serverCore, serverTests % Test)

lazy val nettyServerLoom: ProjectMatrix =
ProjectMatrix("nettyServerLoom", file("server/netty-server/loom"))
lazy val nettyServerSync: ProjectMatrix =
ProjectMatrix("nettyServerSync", file("server/netty-server/sync"))
.settings(commonJvmSettings)
.settings(
name := "tapir-netty-server-loom",
name := "tapir-netty-server-sync",
// needed because of https://github.com/coursier/coursier/issues/2016
useCoursier := false,
Test / run / fork := true,
libraryDependencies ++= Seq(
"com.softwaremill.ox" %% "core" % Versions.ox,
"org.reactivestreams" % "reactive-streams-tck" % Versions.reactiveStreams % Test,
"com.disneystreaming" %% "weaver-cats" % "0.8.4" % Test
"com.softwaremill.ox" %% "core" % Versions.ox
)
)
.jvmPlatform(scalaVersions = List(scala3))
Expand Down Expand Up @@ -2149,7 +2152,7 @@ lazy val examples: ProjectMatrix = (projectMatrix in file("examples"))
sttpClient,
swaggerUiBundle,
http4sServerZio,
nettyServerLoom,
nettyServerSync,
nettyServerZio,
zioHttpServer,
zioJson,
Expand Down
4 changes: 2 additions & 2 deletions doc/contributing.md
Original file line number Diff line number Diff line change
Expand Up @@ -20,11 +20,11 @@ Let's use the name `enumeration` in Tapir codebase to represent these "true" enu

## JDK version

To ensure that Tapir can be used in a wide range of projects, the CI job uses JDK11 for most of the modules. There are exceptions (like `netty-server-loom` and `nima-server`) which require JDK version >= 21. This requirement is adressed by the build matrix in `.github/workflows/ci.yml`, which runs separate builds on a newer Java version, and sets a `ONLY_LOOM` env variable, used by build.sbt to recognise that it should limit the scope of an aggregated task to these projects only.
To ensure that Tapir can be used in a wide range of projects, the CI job uses JDK11 for most of the modules. There are exceptions (like `netty-server-sync` and `nima-server`) which require JDK version >= 21. This requirement is adressed by the build matrix in `.github/workflows/ci.yml`, which runs separate builds on a newer Java version, and sets a `ONLY_LOOM` env variable, used by build.sbt to recognise that it should limit the scope of an aggregated task to these projects only.
For local development, feel free to use any JDK >= 11. You can be on JDK 21, then with missing `ONLY_LOOM` variable you can still run sbt tasks on projects excluded from aggegate build, for example:
```scala
nimaServer/Test/test
nettyServerLoom/compile
nettyServerSync3/compile
// etc.
```

Expand Down
4 changes: 4 additions & 0 deletions doc/migrating.md
Original file line number Diff line number Diff line change
@@ -1,5 +1,9 @@
# Migrating

## From 10.0.4 to 10.0.5

- `tapir-server-netty-loom` has been renamed to `tapir-netty-server-sync`, and is availavble only for Scala 3. Use imports from `sttp.tapir.server.netty.sync`, and start your server using `NettySyncServer()`. See [examples/HelloWorldNettySyncServer.scala](https://github.com/softwaremill/tapir/blob/master/examples/src/main/scala/sttp/tapir/examples/HelloWorldNettySyncServer.scala) for a full example.

## From 1.9.3 to 1.9.4

- `NettyConfig.defaultNoStreaming` has been removed, use `NettyConfig.default`.
Expand Down
6 changes: 3 additions & 3 deletions doc/server/netty.md
Original file line number Diff line number Diff line change
Expand Up @@ -7,7 +7,7 @@ To expose an endpoint using a [Netty](https://netty.io)-based server, first add
"com.softwaremill.sttp.tapir" %% "tapir-netty-server" % "@VERSION@"

// if you want to use Java 21 Loom virtual threads in direct style:
"com.softwaremill.sttp.tapir" %% "tapir-netty-server-loom" % "@VERSION@"
"com.softwaremill.sttp.tapir" %% "tapir-netty-server-sync" % "@VERSION@"

// if you are using cats-effect:
"com.softwaremill.sttp.tapir" %% "tapir-netty-server-cats" % "@VERSION@"
Expand Down Expand Up @@ -44,7 +44,7 @@ val binding: Future[NettyFutureServerBinding] =
NettyFutureServer().addEndpoint(helloWorld).start()
```

The `tapir-netty-server-loom` server uses `Id[T]` as its wrapper effect for compatibility, while `Id[A]` means in fact just `A`, representing direct style. It is
The `tapir-netty-server-sync` server uses `Id[T]` as its wrapper effect for compatibility, while `Id[A]` means in fact just `A`, representing direct style. It is
available only for Scala 3.
See [examples/HelloWorldNettySyncServer.scala](https://github.com/softwaremill/tapir/blob/master/examples/src/main/scala/sttp/tapir/examples/HelloWorldNettySyncServer.scala) for a full example.
To learn more about handling concurrency with Ox, see the [documentation](https://ox.softwaremill.com/).
Expand Down Expand Up @@ -141,7 +141,7 @@ object WebSocketsNettyCatsServer extends ResourceApp.Forever {
}
```

### tapir-netty-server-loom
### tapir-netty-server-sync

In the Loom-based backend, Tapir uses [Ox](https://ox.softwaremill.com) to manage concurrency, and your transformation pipeline should be represented as `Ox ?=> Source[A] => Source[B]`. Any forks started within this function will be run under a safely isolated internal scope.
See [examples/websocket/WebSocketNettySyncServer.scala](https://github.com/softwaremill/tapir/blob/master/examples/src/main/scala/sttp/tapir/examples/websocket/WebSocketNettySyncServer.scala) for a full example.
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,7 @@ package sttp.tapir.examples

import ox.*
import sttp.tapir.*
import sttp.tapir.server.netty.loom.{Id, NettySyncServer}
import sttp.tapir.server.netty.sync.{Id, NettySyncServer}

object HelloWorldNettySyncServer:
val helloWorld = endpoint.get
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -4,10 +4,10 @@ import ox.*
import ox.channels.*
import sttp.capabilities.WebSockets
import sttp.tapir.*
import sttp.tapir.server.netty.loom.Id
import sttp.tapir.server.netty.loom.OxStreams
import sttp.tapir.server.netty.loom.OxStreams.Pipe // alias for Ox ?=> Source[A] => Source[B]
import sttp.tapir.server.netty.loom.NettySyncServer
import sttp.tapir.server.netty.sync.Id
import sttp.tapir.server.netty.sync.OxStreams
import sttp.tapir.server.netty.sync.OxStreams.Pipe // alias for Ox ?=> Source[A] => Source[B]
import sttp.tapir.server.netty.sync.NettySyncServer
import sttp.ws.WebSocketFrame

import scala.concurrent.duration.*
Expand Down
2 changes: 1 addition & 1 deletion generated-doc/out/contributing.md
Original file line number Diff line number Diff line change
Expand Up @@ -20,7 +20,7 @@ Let's use the name `enumeration` in Tapir codebase to represent these "true" enu

## JDK version

To ensure that Tapir can be used in a wide range of projects, the CI job uses JDK11 for most of the modules. There are exceptions (like `netty-server-loom` and `nima-server`) which require JDK version >= 21. This requirement is adressed by the build matrix in `.github/workflows/ci.yml`, which runs separate builds on a newer Java version, and sets a `ONLY_LOOM` env variable, used by build.sbt to recognise that it should limit the scope of an aggregated task to these projects only.
To ensure that Tapir can be used in a wide range of projects, the CI job uses JDK11 for most of the modules. There are exceptions (like `netty-server-sync` and `nima-server`) which require JDK version >= 21. This requirement is adressed by the build matrix in `.github/workflows/ci.yml`, which runs separate builds on a newer Java version, and sets a `ONLY_LOOM` env variable, used by build.sbt to recognise that it should limit the scope of an aggregated task to these projects only.
For local development, feel free to use any JDK >= 11. You can be on JDK 21, then with missing `ONLY_LOOM` variable you can still run sbt tasks on projects excluded from aggegate build, for example:
```scala
nimaServer/Test/test
Expand Down
4 changes: 2 additions & 2 deletions generated-doc/out/server/netty.md
Original file line number Diff line number Diff line change
Expand Up @@ -7,7 +7,7 @@ To expose an endpoint using a [Netty](https://netty.io)-based server, first add
"com.softwaremill.sttp.tapir" %% "tapir-netty-server" % "1.10.4"

// if you want to use Java 21 Loom virtual threads in direct style:
"com.softwaremill.sttp.tapir" %% "tapir-netty-server-loom" % "1.10.4"
"com.softwaremill.sttp.tapir" %% "tapir-netty-server-sync" % "1.10.4"

// if you are using cats-effect:
"com.softwaremill.sttp.tapir" %% "tapir-netty-server-cats" % "1.10.4"
Expand Down Expand Up @@ -44,7 +44,7 @@ val binding: Future[NettyFutureServerBinding] =
NettyFutureServer().addEndpoint(helloWorld).start()
```

The `tapir-netty-server-loom` server uses `Id[T]` as its wrapper effect for compatibility, while `Id[A]` means in fact just `A`, representing direct style.
The `tapir-netty-server-sync` server uses `Id[T]` as its wrapper effect for compatibility, while `Id[A]` means in fact just `A`, representing direct style.

```scala
import sttp.tapir._
Expand Down
4 changes: 2 additions & 2 deletions perf-tests/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -122,9 +122,9 @@ For WebSockets we want to measure latency distribution, not throughput, so use g
```
perfTests/runMain sttp.tapir.perf.apis.ServerRunner http4s.Tapir
```
If you're testing `NettySyncServer` (tapir-server-netty-loom), its server runner is located elsewhere:
If you're testing `NettySyncServer` (tapir-server-netty-sync), its server runner is located elsewhere:
```
nettyServerLoom3/Test/runMain sttp.tapir.netty.loom.perf.NettySyncServerRunner
nettyServerSync3/Test/runMain sttp.tapir.netty.sync.perf.NettySyncServerRunner
```
This is caused by `perf-tests` using Scala 2.13 forced by Gatling, while `NettySyncServer` is written excluisively for Scala 3.

Expand Down
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
package sttp.tapir.server.netty.loom
package sttp.tapir.server.netty.sync

import ox.Ox
import ox.channels.Source
Expand Down
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
package sttp.tapir.server.netty.loom
package sttp.tapir.server.netty.sync

import ox.*
import internal.ox.OxDispatcher
Expand All @@ -25,7 +25,7 @@ import scala.util.control.NonFatal
* [[NettySyncServerEndpointListOverriddenOptions]] is an intermediary helper type representing added endpoints, which have custom server
* options.
*/
private[loom] case class NettySyncServerEndpointListOverridenOptions(
private[sync] case class NettySyncServerEndpointListOverridenOptions(
ses: List[ServerEndpoint[OxStreams & WebSockets, Id]],
overridenOptions: NettySyncServerOptions
)
Expand Down
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
package sttp.tapir.server.netty.loom
package sttp.tapir.server.netty.sync

import internal.{NettySyncRequestBody, NettySyncToResponseBody}
import internal.ox.OxDispatcher
Expand Down
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
package sttp.tapir.server.netty.loom
package sttp.tapir.server.netty.sync

import org.slf4j.LoggerFactory
import sttp.tapir.model.ServerRequest
Expand Down
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
package sttp.tapir.server.netty.loom.internal
package sttp.tapir.server.netty.sync.internal

import io.netty.handler.codec.http.HttpContent
import org.playframework.netty.http.StreamedHttpRequest
Expand All @@ -9,9 +9,9 @@ import sttp.tapir.TapirFile
import sttp.tapir.model.ServerRequest
import sttp.tapir.server.netty.internal.NettyRequestBody
import sttp.tapir.server.netty.internal.reactivestreams.{FileWriterSubscriber, SimpleSubscriber}
import sttp.tapir.server.netty.loom.*
import sttp.tapir.server.netty.sync.*

private[loom] class NettySyncRequestBody(val createFile: ServerRequest => TapirFile) extends NettyRequestBody[Id, OxStreams]:
private[sync] class NettySyncRequestBody(val createFile: ServerRequest => TapirFile) extends NettyRequestBody[Id, OxStreams]:

override given monad: MonadError[Id] = idMonad
override val streams: capabilities.Streams[OxStreams] = OxStreams
Expand Down
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
package sttp.tapir.server.netty.loom.internal
package sttp.tapir.server.netty.sync.internal

import _root_.ox.*
import io.netty.channel.ChannelHandlerContext
Expand All @@ -9,13 +9,13 @@ import sttp.tapir.server.interpreter.ToResponseBody
import sttp.tapir.server.netty.NettyResponse
import sttp.tapir.server.netty.NettyResponseContent.ReactiveWebSocketProcessorNettyResponseContent
import sttp.tapir.server.netty.internal.{NettyToResponseBody, RunAsync}
import sttp.tapir.server.netty.loom._
import sttp.tapir.server.netty.loom.internal.ox.OxDispatcher
import sttp.tapir.server.netty.sync._
import sttp.tapir.server.netty.sync.internal.ox.OxDispatcher
import sttp.tapir.*

import java.nio.charset.Charset

private[loom] class NettySyncToResponseBody(runAsync: RunAsync[Id], oxDispatcher: OxDispatcher)(using me: MonadError[Id])
private[sync] class NettySyncToResponseBody(runAsync: RunAsync[Id], oxDispatcher: OxDispatcher)(using me: MonadError[Id])
extends ToResponseBody[NettyResponse, OxStreams]:

val delegate = new NettyToResponseBody(runAsync)(me)
Expand Down
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
package sttp.tapir.server.netty.loom.internal.ox
package sttp.tapir.server.netty.sync.internal.ox

import ox.*
import ox.channels.Actor
Expand All @@ -18,7 +18,7 @@ import ox.channels.Actor
* @param ox
* concurrency scope where a fork will be run, using a nested scope to isolate failures.
*/
private[loom] class OxDispatcher()(using ox: Ox):
private[sync] class OxDispatcher()(using ox: Ox):
private class Runner:
def runAsync(thunk: Ox ?=> Unit, onError: Throwable => Unit): Unit =
fork {
Expand Down
Original file line number Diff line number Diff line change
@@ -1,11 +1,11 @@
package sttp.tapir.server.netty.loom.internal.reactivestreams
package sttp.tapir.server.netty.sync.internal.reactivestreams

import org.reactivestreams.{Subscriber, Subscription}
import ox.*
import ox.channels.*

/** Can be used together with an [[OxProcessor]] to read from a Source when there's demand. */
private[loom] class ChannelSubscription[A](
private[sync] class ChannelSubscription[A](
subscriber: Subscriber[? >: A],
source: Source[A]
) extends Subscription:
Expand Down
Original file line number Diff line number Diff line change
@@ -1,12 +1,12 @@
package sttp.tapir.server.netty.loom.internal.reactivestreams
package sttp.tapir.server.netty.sync.internal.reactivestreams

import org.reactivestreams.Subscriber
import ox.*
import ox.channels.*
import org.reactivestreams.Subscription
import org.reactivestreams.Processor
import sttp.tapir.server.netty.loom.internal.ox.OxDispatcher
import sttp.tapir.server.netty.loom.OxStreams
import sttp.tapir.server.netty.sync.internal.ox.OxDispatcher
import sttp.tapir.server.netty.sync.OxStreams

/** A reactive Processor, which is both a Publisher and a Subscriber
*
Expand All @@ -18,7 +18,7 @@ import sttp.tapir.server.netty.loom.OxStreams
* an optional function allowing wrapping external subscribers, can be used to intercept onNext, onComplete and onError with custom
* handling. Can be just identity.
*/
private[loom] class OxProcessor[A, B](
private[sync] class OxProcessor[A, B](
oxDispatcher: OxDispatcher,
pipeline: OxStreams.Pipe[A, B],
wrapSubscriber: Subscriber[? >: B] => Subscriber[? >: B]
Expand Down
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
package sttp.tapir.server.netty.loom.internal.ws
package sttp.tapir.server.netty.sync.internal.ws

import io.netty.channel.ChannelHandlerContext
import io.netty.handler.codec.http.websocketx.{CloseWebSocketFrame, WebSocketCloseStatus, WebSocketFrame => NettyWebSocketFrame}
Expand All @@ -8,14 +8,14 @@ import ox.*
import ox.channels.{ChannelClosedException, Source}
import sttp.tapir.model.WebSocketFrameDecodeFailure
import sttp.tapir.server.netty.internal.ws.WebSocketFrameConverters._
import sttp.tapir.server.netty.loom.OxStreams
import sttp.tapir.server.netty.loom.internal.ox.OxDispatcher
import sttp.tapir.server.netty.loom.internal.reactivestreams.OxProcessor
import sttp.tapir.server.netty.sync.OxStreams
import sttp.tapir.server.netty.sync.internal.ox.OxDispatcher
import sttp.tapir.server.netty.sync.internal.reactivestreams.OxProcessor
import sttp.tapir.{DecodeResult, WebSocketBodyOutput}
import sttp.ws.WebSocketFrame
import java.io.IOException

private[loom] object OxSourceWebSocketProcessor:
private[sync] object OxSourceWebSocketProcessor:

def apply[REQ, RESP](
oxDispatcher: OxDispatcher,
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -2,11 +2,11 @@ package sttp.tapir.server.netty

import sttp.monad.MonadError

package object loom:
package object sync:
type Id[X] = X
type IdRoute = Route[Id]

private[loom] implicit val idMonad: MonadError[Id] = new MonadError[Id] {
private[sync] implicit val idMonad: MonadError[Id] = new MonadError[Id] {
override def unit[T](t: T): Id[T] = t
override def map[T, T2](fa: Id[T])(f: T => T2): Id[T2] = f(fa)
override def flatMap[T, T2](fa: Id[T])(f: T => Id[T2]): Id[T2] = f(fa)
Expand Down
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
package sttp.tapir.server.netty.loom
package sttp.tapir.server.netty.sync

import cats.data.NonEmptyList
import cats.effect.unsafe.implicits.global
Expand Down
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
package sttp.tapir.server.netty.loom
package sttp.tapir.server.netty.sync

import cats.data.NonEmptyList
import cats.effect.{IO, Resource}
Expand Down Expand Up @@ -72,6 +72,5 @@ class NettySyncTestServerInterpreter(eventLoopGroup: NioEventLoopGroup)
NettyConfig.default.eventLoopGroup(eventLoopGroup).randomPort.withDontShutdownEventLoopGroupOnClose.noGracefulShutdown
val customizedConfig = gracefulShutdownTimeout.map(config.withGracefulShutdownTimeout).getOrElse(config)
val options = NettySyncServerOptions.default
val interpreter = NettySyncServerInterpreter(options)
useInScope(NettySyncServer(options, customizedConfig).addEndpoints(endpoints.toList).start())(_.stop())
}
Original file line number Diff line number Diff line change
@@ -1,16 +1,16 @@
package sttp.tapir.server.netty.loom.perf
package sttp.tapir.server.netty.sync.perf

import ox.*
import ox.channels.*
import sttp.tapir.server.netty.loom.NettySyncServerOptions
import sttp.tapir.server.netty.loom.NettySyncServerBinding
import sttp.tapir.server.netty.loom.NettySyncServer
import sttp.tapir.server.netty.sync.NettySyncServerOptions
import sttp.tapir.server.netty.sync.NettySyncServerBinding
import sttp.tapir.server.netty.sync.NettySyncServer

import sttp.tapir.*
import sttp.tapir.server.netty.loom.Id
import sttp.tapir.server.netty.sync.Id
import sttp.tapir.server.ServerEndpoint
import sttp.tapir.server.model.EndpointExtensions.*
import sttp.tapir.server.netty.loom.OxStreams
import sttp.tapir.server.netty.sync.OxStreams
import sttp.tapir.Endpoint
import sttp.capabilities.WebSockets
import scala.concurrent.duration._
Expand Down

0 comments on commit 4dbbb06

Please sign in to comment.