Skip to content

Commit

Permalink
Merge pull request #13 from BenWhitehead/finch-0.9
Browse files Browse the repository at this point in the history
Refactor to support Finch 0.9.x
  • Loading branch information
BenWhitehead committed Jan 8, 2016
2 parents 346948c + e2bf9f9 commit 0a6d580
Show file tree
Hide file tree
Showing 14 changed files with 186 additions and 419 deletions.
40 changes: 14 additions & 26 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -3,13 +3,13 @@ finch-server

Finch Server is a library that merges together the great libraries [finch](https://github.com/finagle/finch), and [twitter-server](https://github.com/twitter/twitter-server).

Twitter has done a great job of providing great features in twitter-server, but there are some subtleties in utilizing these features, this project aims to made it as easy as possible to use finch and twitter-server together.
Twitter has done a great job of providing great features in twitter-server, but there are some subtleties in utilizing these features, this project aims to make it as easy as possible to use finch and twitter-server together.

# Features

## Config Flags

All confiurable aspects of a finch-server can be configured with command line flags using twitter-utils [Flag](https://github.com/twitter/util/blob/master/util-app/src/main/scala/com/twitter/app/Flag.scala). The following is a list of flags that are available to configure a finch-server.
All configurable aspects of a finch-server can be configured with command line flags using twitter-utils [Flag](https://github.com/twitter/util/blob/master/util-app/src/main/scala/com/twitter/app/Flag.scala). The following is a list of flags that are available to configure a finch-server.

The motivation behind using flags is that when running applications on Mesos or in the Cloud, having a self contained task is much better than depending on other resource to configure your app (such as properties/config files). By having all configuration take place with command line flags it becomes trivial to run your server on any host.

Expand All @@ -26,11 +26,6 @@ The motivation behind using flags is that when running applications on Mesos or

# Filters

## Exception Handling
Finch-server provides a default exception handler that will handle any exception that your application hasn't handled. When an unhandled exception is encountered a metric will be incremented for the configured StatsReceiver the server boots with.

This allows for easier monitoring of the number of occurrences of particular exceptions in your application.

## Route Histograms
Finch Proves a great API for declaring route handlers, finch-server adds a filter to the server to record latency and request count per route and report all metrics to the configured StatsReceiver.

Expand All @@ -43,32 +38,25 @@ Twitter-server provides a great AdminHttpServer that can be used to gain insight
# SLF4J Logging
By default all twitter libraries use Java Util Logging, finch-server has been configured to use SLF4J for all logging and automatically sets up all SLF4J bridges and re-configures Java Util Logging.

No SLF4J Backed is declared as a dependency, so feel free to pick wichever backend you like. The unit tests however use logback.
No SLF4J Backed is declared as a dependency, so feel free to pick whichever backend you like. The unit tests however use logback.

An example `logback.xml` for how to configure access-log file and rolling is available in `example-logback.xml` in the repo or bundled in the jar.

# Usage
finch-server is very easy to use, all you need to create an echo server are the two following components. With these two components you will now have a full server running your application on port 7070 and the Admin Server on 9990.

### Finch Endpoint
```scala
object Echo extends HttpEndpoint {
def service(echo: String) = new Service[HttpRequest, HttpResponse] {
def apply(request: HttpRequest): Future[HttpResponse] = {
Ok(echo).toFuture
}
}
def route = {
case Method.Get -> Root / "echo" / echo => service(echo)
}
}
```
finch-server is very easy to use, all you need to create an echo server is the following object. With this object you will now have a full server running your application on port 7070 and the Admin Server on 9990.

### Server Object
```scala
object TestingServer extends SimpleFinchServer {
import io.finch._

object EchoServer extends FinchServer {
override lazy val serverName = "echo"
def endpoint = Echo

val echo: Endpoint[String] = get("echo" / string) { (phrase: String) =>
Ok(phrase)
}

def service = echo.toService
}
```

Expand Down Expand Up @@ -100,6 +88,6 @@ Artifacts for finch-server are currently hosed in a google storage bucket, so yo
resolvers += "finch-server" at "http://storage.googleapis.com/benwhitehead_me/maven/public"
libraryDependencies ++= Seq(
"io.github.benwhitehead.finch" %% "finch-server" % "0.7.3"
"io.github.benwhitehead.finch" %% "finch-server" % "0.9.0"
)
```
16 changes: 6 additions & 10 deletions build.sbt
Original file line number Diff line number Diff line change
Expand Up @@ -2,25 +2,21 @@ organization := "io.github.benwhitehead.finch"

name := "finch-server"

version := "0.7.4-SNAPSHOT"
version := "0.9.0"

crossScalaVersions := Seq("2.10.3", "2.11.4")
scalaVersion := "2.11.7"

scalacOptions ++= Seq("-unchecked", "-deprecation")

javacOptions ++= Seq("-source", "1.7", "-target", "1.7")

javacOptions in doc := Seq("-source", "1.7")

resolvers += "Twitter" at "http://maven.twttr.com/"

libraryDependencies ++= Seq(
"com.github.finagle" %% "finch-core" % "0.4.0",
"com.github.finagle" %% "finch-json" % "0.4.0",
"com.github.finagle" %% "finch-jackson" % "0.4.0" % "test",
"com.twitter" %% "finagle-stats" % "6.24.0",
"com.twitter" %% "finagle-httpx" % "6.24.0",
"com.twitter" %% "twitter-server" % "1.9.0",
"com.github.finagle" %% "finch-core" % "0.9.2",
"com.twitter" %% "finagle-stats" % "6.31.0",
"com.twitter" %% "finagle-http" % "6.31.0",
"com.twitter" %% "twitter-server" % "1.16.0",
"org.slf4j" % "slf4j-api" % "1.7.10",
"org.slf4j" % "jul-to-slf4j" % "1.7.10",
"org.slf4j" % "jcl-over-slf4j" % "1.7.10",
Expand Down
91 changes: 38 additions & 53 deletions src/main/scala/io/github/benwhitehead/finch/FinchServer.scala
Original file line number Diff line number Diff line change
Expand Up @@ -15,23 +15,23 @@

package io.github.benwhitehead.finch

import java.io.{File, FileNotFoundException, FileOutputStream}
import java.lang.management.ManagementFactory
import java.net.{InetSocketAddress, SocketAddress}

import com.twitter.app.App
import com.twitter.conversions.storage.intToStorageUnitableWholeNumber
import com.twitter.finagle._
import com.twitter.finagle.http.{Request, Response}
import com.twitter.finagle.netty3.Netty3ListenerTLSConfig
import com.twitter.finagle.param.Label
import com.twitter.finagle.ssl.Ssl
import com.twitter.server.Lifecycle.Warmup
import com.twitter.server.{Admin, AdminHttpServer, Lifecycle, Stats}
import com.twitter.util.Await
import io.finch._
import com.twitter.util.{Await, CountDownLatch}
import io.github.benwhitehead.finch.filters._
import io.github.benwhitehead.finch.request.DelegateService

import java.io.{File, FileNotFoundException, FileOutputStream}
import java.lang.management.ManagementFactory
import java.net.{InetSocketAddress, SocketAddress}

trait FinchServer[Request] extends App
trait FinchServer extends App
with SLF4JLogging
with AdminHttpServer
with Admin
Expand All @@ -42,34 +42,32 @@ trait FinchServer[Request] extends App

lazy val pid: String = ManagementFactory.getRuntimeMXBean.getName.split('@').head

implicit val stats = statsReceiver

case class Config(
port: Int = 7070,
pidPath: String = "",
httpsPort: Int = 7443,
certificatePath: String = "",
keyPath: String = "",
maxRequestSize: Int = 5/*,
decompressionEnabled: Boolean = false,
compressionLevel: Int = 6*/
maxRequestSize: Int = 5
)
object DefaultConfig extends Config(
httpPort(),
pidFile(),
httpsPort(),
certificatePath(),
keyPath(),
maxRequestSize()/*,
decompressionEnabled(),
compressionLevel()*/
maxRequestSize()
)

def serverName: String = "finch"
def endpoint: Endpoint[Request, HttpResponse]
def filter: Filter[HttpRequest, HttpResponse, Request, HttpResponse]
def service: Service[Request, Response]
lazy val config: Config = DefaultConfig

@volatile private var server: Option[ListeningServer] = None
@volatile private var tlsServer: Option[ListeningServer] = None
private val cdl = new CountDownLatch(1)

def writePidFile() {
val pidFile = new File(config.pidPath)
Expand All @@ -92,7 +90,7 @@ trait FinchServer[Request] extends App
logger.info(s"admin http server started on: ${adminHttpServer.boundAddress}")
server = Some(startServer())
logger.info(s"http server started on: ${(server map {_.boundAddress}).get}")
server map { closeOnExit(_) }
server foreach { closeOnExit(_) }

if (!config.certificatePath.isEmpty && !config.keyPath.isEmpty) {
verifyFileReadable(config.certificatePath, "SSL Certificate")
Expand All @@ -101,7 +99,8 @@ trait FinchServer[Request] extends App
logger.info(s"https server started on: ${(tlsServer map {_.boundAddress}).get}")
}

tlsServer map { closeOnExit(_) }
tlsServer foreach { closeOnExit(_) }
cdl.countDown()

(server, tlsServer) match {
case (Some(s), Some(ts)) => Await.all(s, ts)
Expand All @@ -111,55 +110,45 @@ trait FinchServer[Request] extends App
}
}

def serverPort: Int = (server map { case s => getPort(s.boundAddress) }).get
def tlsServerPort: Int = (tlsServer map { case s => getPort(s.boundAddress) }).get
def awaitServerStartup(): Unit = {
cdl.await()
}

def serverPort: Int = {
assert(cdl.isZero, "Server not yet started")
(server map { case s => getPort(s.boundAddress) }).get
}
def tlsServerPort: Int = {
assert(cdl.isZero, "TLS Server not yet started")
(tlsServer map { case s => getPort(s.boundAddress) }).get
}

def startServer(): ListeningServer = {
val name = s"http/$serverName"
Httpx.server
.configured(param.Label(name))
.configured(Httpx.param.MaxRequestSize(config.maxRequestSize.megabytes))
// TODO: Figure out how to add back compression support
Http.server
.configured(Label(name))
.configured(Http.param.MaxRequestSize(config.maxRequestSize.megabytes))
.serve(new InetSocketAddress(config.port), getService(s"srv/$name"))
}

def startTlsServer(): ListeningServer = {
val name = s"https/$serverName"
Httpx.server
.configured(param.Label(name))
.configured(Httpx.param.MaxRequestSize(config.maxRequestSize.megabytes))
Http.server
.configured(Label(name))
.configured(Http.param.MaxRequestSize(config.maxRequestSize.megabytes))
.withTls(Netty3ListenerTLSConfig(() => Ssl.server(config.certificatePath, config.keyPath, null, null, null)))
// TODO: Figure out how to add back compression support
.serve(new InetSocketAddress(config.httpsPort), getService(s"srv/$name"))
}

def getService(serviceName: String): Service[HttpRequest, HttpResponse] = {
AccessLog(new StatsFilter(serviceName), accessLog()) !
errorHandler(serviceName) !
filter !
(endpoint orElse NotFound(serviceName))
def getService(serviceName: String): Service[Request, Response] = {
AccessLog(new StatsFilter(serviceName), accessLog()) andThen
service
}

def errorHandler(serviceName: String): Filter[HttpRequest, HttpResponse, HttpRequest, HttpResponse] = new HandleExceptions(serviceName)

onExit {
removePidFile()
}

def NotFound(serviceName: String) = new Endpoint[Request, HttpResponse] {
lazy val underlying = DelegateService {
io.finch.response.NotFound()
}
val stats404 = {
val stats = {
if (serviceName.nonEmpty) statsReceiver.scope(s"$serviceName/error")
else statsReceiver.scope("error")
}
stats.counter("404")
}
def route = { case _ => stats404.incr(); underlying }
}

private def getPort(s: SocketAddress): Int = {
s match {
case inet: InetSocketAddress => inet.getPort
Expand All @@ -174,7 +163,3 @@ trait FinchServer[Request] extends App
}
}
}

trait SimpleFinchServer extends FinchServer[HttpRequest] {
override def filter = Filter.identity
}
3 changes: 0 additions & 3 deletions src/main/scala/io/github/benwhitehead/finch/Flags.scala
Original file line number Diff line number Diff line change
Expand Up @@ -8,7 +8,4 @@ object httpsPort extends GlobalFlag[Int](7443, "the TCP port for the https serve
object certificatePath extends GlobalFlag[String]("", "Path to PEM format SSL certificate file")
object keyPath extends GlobalFlag[String]("", "Path to SSL Key file")
object maxRequestSize extends GlobalFlag[Int](5, "Max request size (in megabytes)")
// TODO: Figure out how to add back compression support
//object decompressionEnabled extends GlobalFlag[Boolean](false, "Enables deflate,gzip Content-Encoding handling")
//object compressionLevel extends GlobalFlag[Int](6, "Enables deflate,gzip Content-Encoding handling")
object accessLog extends GlobalFlag[String]("access-log", "Whether to add an Access Log Filter, and if so which type [off|access-log{default}|access-log-combined]. Any value other than the listed 3 will be treated as off.")
Original file line number Diff line number Diff line change
Expand Up @@ -25,6 +25,7 @@ import java.lang.reflect.{ParameterizedType, Type}
/**
* Found at http://stackoverflow.com/a/14166997
*/
@deprecated("will be removed in 0.10.0", "0.9.0")
object JacksonWrapper {
implicit val mapper = new ObjectMapper()
.registerModule(DefaultScalaModule)
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -23,7 +23,7 @@ import org.slf4j.bridge.SLF4JBridgeHandler
trait SLF4JLogging { self: App =>
init {
// Turn off Java util logging so that slf4j can configure it
LogManager.getLogManager.getLogger("").getHandlers.toList.map { l =>
LogManager.getLogManager.getLogger("").getHandlers.toList.foreach { l =>
l.setLevel(Level.OFF)
}
org.slf4j.LoggerFactory.getLogger("slf4j-logging").debug("Installing SLF4JLogging")
Expand Down
Original file line number Diff line number Diff line change
@@ -0,0 +1,56 @@
package io.github.benwhitehead.finch.filters

import java.text.SimpleDateFormat
import java.util.Date

import com.twitter.finagle.http.Version.{Http10, Http11}
import com.twitter.finagle.{Filter, Service, SimpleFilter}
import com.twitter.finagle.http.{Version, Response, Request}
import com.twitter.util.Future

class AccessLog(combined: Boolean) extends SimpleFilter[Request, Response] {
lazy val accessLog = org.slf4j.LoggerFactory.getLogger("access-log")
def apply(request: Request, service: Service[Request, Response]) = {
if (accessLog.isTraceEnabled) {
service(request) flatMap { case resp =>
val reqHeaders = request.headerMap
val remoteHost = request.remoteHost
val identd = "-"
val user = "-"
val requestEndTime = new SimpleDateFormat("dd/MM/yyyy:hh:mm:ss Z").format(new Date())
val reqResource = s"${request.method.toString.toUpperCase} ${request.uri} ${versionString(request.version)}"
val statusCode = resp.statusCode
val responseBytes = resp.headerMap.getOrElse("Content-Length", "-")

if (!combined) {
accessLog.trace(f"""$remoteHost%s $identd%s $user%s [$requestEndTime%s] "$reqResource%s" $statusCode%d $responseBytes%s""")
} else {
val referer = reqHeaders.getOrElse("Referer", "-")
val userAgent = reqHeaders.getOrElse("User-Agent", "-")
accessLog.trace(f"""$remoteHost%s $identd%s $user%s [$requestEndTime%s] "$reqResource%s" $statusCode%d $responseBytes%s "$referer%s" "$userAgent%s"""")
}
Future.value(resp)
}
} else {
service(request)
}
}

def versionString(v: Version) = v match {
case Http11 => "HTTP/1.1"
case Http10 => "HTTP/1.0"
}
}
object AccessLog {
lazy val accessLog = org.slf4j.LoggerFactory.getLogger("access-log")
type httpFilter = Filter[Request, Response, Request, Response]
def apply(preFilter: httpFilter, logType: String): httpFilter = {
logType match {
case "access-log" => preFilter andThen new AccessLog(false)
case "access-log-combined" => preFilter andThen new AccessLog(true)
case _ =>
accessLog.trace("access log disabled")
preFilter
}
}
}
Original file line number Diff line number Diff line change
@@ -0,0 +1,21 @@
package io.github.benwhitehead.finch.filters

import com.twitter.finagle.{Service, SimpleFilter}
import com.twitter.finagle.http.{Response, Request}
import com.twitter.finagle.stats.{Stat, StatsReceiver}
import com.twitter.util.Future

class StatsFilter(baseScope: String = "")(implicit statsReceiver: StatsReceiver) extends SimpleFilter[Request, Response] {
val stats = {
if (baseScope.nonEmpty) statsReceiver.scope(s"$baseScope/route")
else statsReceiver.scope("route")
}
def apply(request: Request, service: Service[Request, Response]): Future[Response] = {
val label = s"${request.method.toString.toUpperCase}/ROOT/${request.path.stripPrefix("/")}"
Stat.timeFuture(stats.stat(label)) {
val f = service(request)
stats.counter(label).incr()
f
}
}
}
Loading

0 comments on commit 0a6d580

Please sign in to comment.