Finatra 19.2.0

Added

• finatra-kafka: Expose timeout duration in FinagleKafkaConsumerBuilder dest(). abd68ddf
• finatra-kafka-streams: Expose all existing RocksDb configurations. See
  c.t.f.k.config.FinatraRocksDBConfig for details on flag names,
  descriptions and default values. 1454867c
• finatra-kafka-streams: Added two RocksDB flags related to block cache tuning,
  cache_index_and_filter_blocks and pin_l0_filter_and_index_blocks_in_cache.
  3b0931e3
• finatra-kafka: Adding an implicit implementation of
  c.t.app.Flaggable[c.t.finatra.kafka.domain.SeekStrategy]
  and c.t.app.Flaggable[org.apache.kafka.clients.consumer.OffsetResetStrategy].
  ef071e54
• finatra-http: Added support to serve c.t.io.Reader as a streaming response in
  c.t.finatra.http.internal.marshalling.CallbackConverter. e5bda446
• finatra-kafka: Expose endOffsets() in FinagleKafkaConsumer. bcbb5774
• finatra-kafka-streams: Adding missing ScalaDocs. Adding metric for elapsed state
  restore time. RocksDB configuration now contains a flag for adjusting the number
  of cache shard bits, rocksdb.block.cache.shard.bits. afd9a17c
• finatra-jackson: Added @pattern annotation to support finatra/jackson for regex pattern
  validation on string values. 862f0ab1

Changed

• finatra-kafka-streams: Refactor package names. All classes moved from
  com.twitter.finatra.streams to com.twitter.finatra.kafkastreams. a2ad0ef3
• finatra-kafka-streams: Delete deprecated and unused classes. ee948398
• finatra-kafka-streams: c.t.finatra.streams.transformer.domain.Time is now the canonical
  representation of time for watermarks and timers. RichLong implicit from
  com.twitter.finatra.streams.converters.time has been renamed to RichFinatraKafkaStreamsLong.
  093b31b8
• finatra-jackson: Fix CaseClassField annotation reflection for Scala 2.12. 3747c1ab
• finatra-kafka-streams: Combine FinatraTransformer with FinatraTransformerV2. cd455c43
• finatra-thrift: The return type of ReqRepDarkTrafficFilterModule#newFilter has been changed from
  DarkTrafficFilter[MethodIface] to Filter.TypeAgnostic. 50184f1b
• finatra-kafka: Add lookupBootstrapServers function that takes timeout as a parameter.
  e3426fb9
• finatra-thrift: If a Controller is not configured with exactly one endpoint
  per method, it will throw an AssertionError instead of logging an error message.
  An attempt to use non-legacy functionality with a legacy Controller will throw
  an AssertionError. d1d6d1e0
• finatra-kafka: Add flags for controlling rocksdb internal LOG file growth.
  • rocksdb.log.info.level Allows the setting of rocks log levels
    DEBUG_LEVEL, INFO_LEVEL, WARN_LEVEL, ERROR_LEVEL, FATAL_LEVEL,
    HEADER_LEVEL.
  • rocksdb.log.max.file.size The maximal size of the info log file.
  • rocksdb.log.keep.file.num Maximal info log files to be kept.
    c03a497c
• finatra-kafka: Add admin routes for properties and topology information
  • /admin/kafka/streams/properties Dumps the
    KafkaStreamsTwitterServer#properties as plain text in the TwitterServer
    admin page.
  • /admin/kafka/streams/topology Dumps the KafkaStreamsTwitterServer#topology as plain text in the TwitterServer admin page.
    ecf2e54f
• inject-server: EmbeddedTwitterServer that fails to start will now continue to
  throw the startup failure on calls to methods that require a successfully started server.
  3ca4437c

Fixed

• finatra-kafka-streams: FinatraTopologyTester did not set
  TopologyTestDriver#initialWallClockTimeMs on initialization causing diverging wall clock time
  when TopologyTestDriver#advanceWallClockTime advanced time. The divergence was between
  system time set by org.joda.time.DateTimeUtils.setCurrentMillisFixed and internal mock timer
  TopologyTestDriver#mockWallClockTime. FinatraTopologyTester.inMemoryStatsReceiver is reset on
  TopologyFeatureTest#beforeEach for all test that extend TopologyFeatureTest.
  3b93a7d7
• finatra-kafka-streams: Improve watermark assignment/propagation upon reading the first
  message and when caching key value stores are used. 9aa12b8d
• finatra-jackson: Support inherited annotations in case class deserialization. Case class
  deserialization support does not properly find inherited Jackson annotations. This means
  that code like this:
  trait MyTrait { @JsonProperty("differentName") def name: String } case class MyCaseClass(name: String) extends MyTrait
  would not properly expect an incoming field with name differentName to parse into the
  case class name field. This commit provides support for capturing inherited annotations
  on case class fields. Annotations processed in order, thus if the same annotation appears
  in the class hierarchy multiple times, the value configured on the class will win otherwise
  will be in the order of trait linearization with the "last" declaration prevailing.
  6237ff86
• finatra: Remove extraneous dependency on old javax.servlet ServletAPI dependency.
  The fixes #478. 85100952

Finatra 19.1.0

Added

• finatra-kafka-streams: SumAggregator and CompositeSumAggregator only support enhanced window
  aggregations for the sum operation. Deprecate SumAggregator and CompositeSumAggregator and create
  an AggregatorTransformer class that can perform arbitrary aggregations. f588970e

• finatra-streams: Open-source Finatra Streams. Finatra Streams is an integration
  between Kafka Streams and Finatra which we've been using internally at Twitter
  for the last year. The library is not currently open-source.
  47cce546

• inject-server: Add lint rule to alert when deprecated util-logging JUL flags from the
  c.t.inject.server.DeprecatedLogging trait are user defined. This trait was mixed-in
  only for backwards compatibility when TwitterServer was moved to the slf4j-api and the flags are
  not expected to be configured. By default, util-app based applications will fail to start if
  they are passed a flag value at startup which they do not define. Users should instead configure
  their chosen slf4j-api logging implementation directly. 388bf8f9

• finatra-thrift: c.t.finatra.thrift.Controllers now support per-method filtering and
  access to headers via c.t.scrooge.{Request, Response} wrappers. To use this new
  functionality, create a Controller which extends the
  c.t.finatra.thrift.Controller(SomeThriftService) abstract class instead of constructing a
  Controller that mixes in the SomeThriftService.BaseServiceIface trait. With this, you can now
  provide implementations in form of c.t.scrooge.Request/c.t.scrooge.Response wrappers by calling
  the handle(ThriftMethod) method. Note that a Controller constructed this way cannot also
  extend a BaseServiceIface.

  handle(SomeMethod).filtered(someFilter).withFn { req: Request[SomeMethod.Args] =>

  : val requestHeaders = req.headers
  // .. implementation here

  // response: Future\[Response\[SomeMethod.SuccessType\]\]
  

  }

  Note that if Request/Response based implementations are used the types on any
  existing ExceptionMappers should be adjusted accordingly. Also, if a DarkTrafficFilterModule
  was previously used, it must be swapped out for a ReqRepDarkTrafficFilterModule
  9d891cd1

Changed

• inject-core, inject-server: Remove deprecated @Bind support from test mixins. Users should
  instead prefer using the bind[T]
  DSL in tests. 841f6974

• inject-app: Remove deprecated bind[T] DSL methods from c.t.inject.app.BindDSL.

  Instead of:

  injector.bind[T](instance)
  injector.bind[T, Ann](instance)
  injector.bind[T](ann, instance)
  

  Users should instead use the more expressive forms of these methods, e.g.,:

  injector.bind[T].toInstance(instance)
  injector.bind[T].annotatedWith[Ann].toInstance(instance)
  injector.bind[T].annotatedWith(ann).toInstance(instance)
  

  which more closely mirrors the scala-guice binding DSL. 2690003d

• finatra-thrift: For services that wish to support dark traffic over
  c.t.scrooge.Request/c.t.scrooge.Response-based services, a new dark traffic module is
  available: c.t.finatra.thrift.modules.ReqRepDarkTrafficFilterModule 9d891cd1

• finatra-thrift: Creating a c.t.finatra.thrift.Controller that extends a
  ThriftService.BaseServiceIface has been deprecated. See the related bullet point in "Added" with
  the corresponding PHAB_ID to this one for how to migrate. 9d891cd1

• inject-core, inject-server: Remove deprecated WordSpec testing utilities. The framework
  default ScalaTest testing style is FunSuite though users are free to mix their testing
  style of choice with the framework provided test mixins as per the
  documentation.
  41767c6e

• finatra-thrift: Instead of failing (potentially silently)
  c.t.finatra.thrift.routing.ThriftWarmup now explicitly checks that it is
  using a properly configured c.t.finatra.thrift.routing.Router e2dc8b30

• finatra-inject: c.t.finatra.inject.server.PortUtils has been modified to
  work with c.t.f.ListeningServer only. Methods which worked with the
  now-removed c.t.f.b.Server have been modified or removed.
  642d7260

• finatra-kafka-streams: Finatra Queryable State methods currently require the window size
  to be passed into query methods for windowed key value stores. This is unnecessary, as
  the queryable state class can be passed the window size at construction time. We also now
  save off all FinatraKeyValueStores in a global manager class to allow query services
  (e.g. thrift) to access the same KeyValueStore implementation that the FinatraTransformer
  is using. c51e174b

Fixed

• finatra-kafka-streams: Fix bug where KeyValueStore#isOpen was throwing an
  exception when called on an uninitialized key value store
  d3f833a1

Closed

Finatra 18.12.0

18.12.0

Added

Changed

• finatra-thrift: c.t.finatra.thrift.exceptions.FinatraThriftExceptionMapper and
  c.t.finatra.thrift.exceptions.FinatraJavaThriftExceptionMapper now extend
  ExceptionManager[Throwable, Nothing] since the return type was never used. They are
  now also final. dc894547
• finatra-thrift: Remove c.t.finatra.thrift.routing.JavaThriftRouter#beforeFilter. This method
  adds too much confusion to the Router API and users are encouraged to instead apply their
  TypeAgnostic Filters directly to the resultant Service[-R, +R] by overriding the
  c.t.finatra.thrift.AbstractThriftServer#configureService method instead. b0cb8eaf
• finatra-thrift: c.t.finagle.Filter.TypeAgnostic filters are now the standard type of filter
  that can be added by configuring a ThriftRouter. c.t.finatra.thrift.ThriftFilter has been
  deprecated. 6e93b9cc
• finatra-thrift: c.t.finatra.thrift.ThriftRequest has been deprecated. All of the information
  contained in a ThriftRequest can be found in other ways:
  methodName -> Method.current.get.name
  traceId -> Trace.id
  clientId -> ClientId.current
  6e93b9cc

Fixed

• finatra-http: Validate headers to prevent header injection vulnerability. 8a925000

Closed

Finatra 18.11.0

Changed

• finatra-thrift: (BREAKING API CHANGE) Update DarkTrafficFilter#handleFailedInvocation to accept
  the request type for more fidelity in handling the failure. 20bd33ac
• finatra-http: Move request.ContentType and response.Mustache Java annotations to
  com.twitter.finatra.http package namespace. ef135610
• finatra-jackson: Move away from deprecated code and update error handling and exceptions post
  Jackson 2.9.x upgrade. f1e5c96e
• inject-core: (BREAKING API CHANGE) Remove c.t.inject.TestMixin#sleep. We do not want to
  promote this usage of Thread blocking in testing utilities. Add a new testing function:
  c.t.inject.TestMixin#await which will perform Await.result on a given c.t.util.Awaitable.
  This function was duplicated across tests in the codebase. We also introduce an overridable default
  timeout on the underlying Await.result call: c.t.inject.TestMixin#defaultAwaitTimeout.
  4aee1051

Fixed

• finatra-http: Fix registration of HTTP Routes in the Library registry to properly account
  for Routes that duplicate a URI with a different HTTP verb. That is, a Route should be considered
  unique per URI + HTTP verb combination. 6a715075

Finatra 18.10.0

Fixed

• finatra-thrift: Set the bound StatsReceiver in the underlying Finagle ThriftMux server
  in the c.t.finatra.thrift.ThriftServer. This prevented testing of underlying Finagle server
  stats as the InMemoryStatsReceiver used by the EmbeddedThriftServer was not properly passed
  all the way through the stack. 33d0524b

Changed

• finatra-http, finatra-thrift: Make HTTP and Thrift StatsFitlers "Response Classification"
  aware. 4085d40c

• finatra-http, finatra-thrift: (BREAKING API CHANGE) Update the DarkTrafficFilterModule in
  both HTTP and Thrift to allow for specifying further configuration of the underlying Finagle client.
  This allows users the ability to set Finagle client concerns like ResponseClassification or other
  configuration not expressed by the DarkTrafficFilterModule's API.

  Additionally, the Thrift DarkTrafficFilterModule has been updated to be ThriftMux only.
  For more information on mux see: What is ThriftMux.

  We also update the enableSampling method to accept a c.t.inject.Injector to aid in the
  decision-making for if a given request should be "sampled" by the filter. d7486843

• finatra-thrift: (BREAKING API CHANGE) Update c.t.finatra.thrift.routing.ThriftRouter API for
  adding Java Thrift controllers. The service: Class[_] was rendered unnecessary some time ago
  but not removed from the API signature. Because this parameter is useless and it shadows
  another variable inside of the code we remove it from the signature altogether rather than
  deprecating the API. c2378cc7

• finatra-thrift: Rename defaultFinatraThriftPort to defaultThriftPort.
  5910fd23

Finatra 18.9.1

Changed

• http/thrift: Update Library registry route information to include controller class name. ffb644e5

Finatra 18.9.0

18.9.0

Added

Changed

• inject-core: Remove unnecessary Await.result Future.Value in
  TestMixin. 1616188c
• finatra-http: (BREAKING API CHANGE) c.t.io.Reader and c.t.io.Writer are now abstracted over
  the type they produce/consume (Reader[A] and Writer[A]) and are no longer fixed to Buf.
  d56244d1

Fixed

Closed

Finatra 18.8.0

finatra-18.8.0 (2018-08-06)

Added

Changed

• finatra-http: (BREAKING API CHANGE) Typical TLS Configuration for an HTTPS server has been moved
  into a trait, c.t.finatra.http.Tls which also defines the relevant flags (and overridable
  defaults) for specifying the SSL cert and key paths. Users can choose to mix this trait into their
  c.t.finatra.http.HttpServer classes in order to specify an HTTPS server. Users who wish to maintain
  the current HTTPS functionality SHOULD mix in the Tls trait to their HttpServer: e.g.,

  class FooService extends HttpServer with Tls {
    ...
  }
  

  Additionally, TLS transport configuration for the underlying Finagle c.t.finagle.Http.Server is
  no longer done by default when creating and running an HTTPS server. This is to allow for more
  flexible configuration on the underlying c.t.finagle.Http.Server when setting up TLS. Thus it is
  recommended that users ensure to either mix in the provided Tls trait or provide the correct
  c.t.finagle.Http.Server transport configuration via the configureHttpsServer method.
  3c19b2df

• finatra-http: Rename defaultFinatraHttpPort to defaultHttpPort. 6fe4a3bf

• finatra-utils: Remove deprecated c.t.f.utils.Handler. 1088ca4b

Fixed

Closed

Finatra 18.7.0

finatra-18.7.0 (2018-07-10)

Added

• inject-utils: Add 'toLoggable' implicit from Array[Byte] to String. 0eece86a

Fixed

• finatra-http: Fix infinite loop introduced by PHAB D180166. Fix underlying issue of the
  ResponseBuilder requiring a stored RouteInfo for classifying exceptions for stating.
  57a02570

• finatra-http: Fix FailureExceptionMapper handling of wrapped exceptions. Unwrap cause for all
  c.t.finagle.Failure exceptions, regardless of flags and add a try-catch to ExceptionManager
  to remap exceptions thrown by ExceptionMappers 8521d980

Finatra 18.6.0

Added

• finatra: Add HTTP route, Thrift method, and Filter information to the Library
  registry. e12fd996

• finatra-inject/inject-logback: Add an c.t.inject.logback.AsyncAppender to
  provide metrics about the underlying queue. 8fa73604

Changed

• inject-slf4j: Move the SLF4J API logging bridges from inject-slf4j to inject-app
  and inject-server. This allows code in the inject framework to be mostly useful in
  environments where having the bridges on the classpath causes issues. 9e57deca

Fixed

• finatra-http: Fail startup for incorrect Controller callback functions. Controller route callback
  functions that do not specify an input parameter or specify an incorrect input parameter should
  fail server startup but were not correctly detected when building routes in the CallbackConverter.
  The route building logic has been patched to correctly detect these routes which would fail at
  runtime to ensure we fail fast at server startup (and can thus be caught by StartupTests).
  7b983dc9

• finatra-http: Change exceptions emitted from c.t.f.http.filter.HttpNackFilter to not extend
  from HttpException and add a specific mapper over HttpNackException such that Nack
  exceptions are handled distinctly from HttpExceptions and thus more specifically. Handling of
  Nack exceptions should not be conflated with handling of the more generic HttpExceptions and
  it should be clear if a new mapper is desired that it is specifically for changing how Nack
  exceptions are handled. f95bf28f