Akka Serialization Helper

Serialization toolbox for Akka messages, events and persistent state that helps achieve compile-time guarantee on serializability. There are some Akka serialization caveats this tool can help with:

1. Missing serialization binding
2. Incompatibility of persistent data
3. Jackson Akka Serializer
4. Missing Codec registration

⚠️ Check pekko-serialization-helper for Pekko ⚠️

Install

Add the following line to plugins.sbt (take Version from the above Maven badge or GitHub Releases):

addSbtPlugin("org.virtuslab.ash" % "sbt-akka-serialization-helper" % Version)

and enable the sbt plugin in the target project:

lazy val app = (project in file("app"))
  .enablePlugins(AkkaSerializationHelperPlugin)

Missing serialization binding

To serialize message, persistent state or event in Akka, Scala trait needs to be defined:

package org
trait MySer

Also, a serializer needs to be bound to this trait in a configuration file:

akka.actor {
  serializers {
    jackson-json = "akka.serialization.jackson.JacksonJsonSerializer"
  }
  serialization-bindings {
    "org.MySer" = jackson-json
  }
}

The problem occurs if a class is not extended with the base trait bound to the serializer:

trait MySer
case class MyMessage() // extends MySer

akka-serialization-helper to the rescue! It detects messages, events and persistent states, and checks whether they extend the given base trait and report an error when they don't. This ensures that the specified serializer is used by Akka and protects against an unintended fallback to Java serialization or outright serialization failure.

To use, base trait should be annotated with @org.virtuslab.ash.SerializabilityTrait:

@SerializabilityTrait
trait MySerializable

It allows catching errors like these:

import akka.actor.typed.Behavior

object BehaviorTest {
  sealed trait Command //extends MySerializable
  def method(msg: Command): Behavior[Command] = ???
}

And results in a compile error, preventing non-runtime-safe code from being executed:

test0.scala:7: error: org.random.project.BehaviorTest.Command is used as Akka message
but does not extend a trait annotated with org.virtuslab.ash.annotation.SerializabilityTrait.
Passing an object of a class that does NOT extend a trait annotated with SerializabilityTrait as a message may cause Akka to
fall back to Java serialization during runtime.


  def method(msg: Command): Behavior[Command] = ???
                            ^
test0.scala:6: error: Make sure this type is itself annotated, or extends a type annotated
with  @org.virtuslab.ash.annotation.SerializabilityTrait.
  sealed trait Command extends MySerializable
               ^

The compiler plugin only checks the classes in the sbt modules where AkkaSerializationHelperPlugin is explicitly enabled. It may happen that the base trait (like MySerializable in the example) lives in an sbt module like core where the plugin should not be enabled (e.g. for compilation performance reasons). However, MySerializable needs to be annotated with org.virtuslab.ash.SerializabilityTrait. In order to have access to the SerializabilityTrait annotation without enabling the entire suite of compiler plugins, add AkkaSerializationHelperPlugin.annotation to libraryDependencies:

import org.virtuslab.ash.AkkaSerializationHelperPlugin

lazy val core = (project in file("core"))
  .settings(libraryDependencies += AkkaSerializationHelperPlugin.annotation)

Incompatibility of persistent data

A typical problem with a persistence is when the already persisted data is not compatible with the schemas defined in a new version of the application.

To solve this, a mix of a compiler plugin and an sbt task can be used for dumping schema of akka-persistence to a file. It can be used for detecting accidental changes of events (journal) and states (snapshots) with a simple diff.

To dump persistence schema for each sbt module where AkkaSerializationHelperPlugin is enabled, run:

sbt ashDumpPersistenceSchema

Default file is target/<sbt-module-name>-dump-persistence-schema-<version>.yaml (target/ of top-level module!) but it can be changed using sbt keys:

ashDumpPersistenceSchemaOutputFilename := "file.yaml" // Changes filename
ashDumpPersistenceSchemaOutputDirectoryPath := "~" // Changes directory

Example dump

- name: org.random.project.Data
  typeSymbol: trait
- name: org.random.project.Data.ClassTest
  typeSymbol: class
  fields:
  - name: a
    typeName: java.lang.String
  - name: b
    typeName: scala.Int
  - name: c
    typeName: scala.Double
  parents:
  - org.random.project.Data
- name: org.random.project.Data.ClassWithAdditionData
  typeSymbol: class
  fields:
  - name: ad
    typeName: org.random.project.Data.AdditionalData
  parents:
  - org.random.project.Data

A diff command can be used to check the difference between the version of a schema from develop/main branch and the version from the current commit.

Jackson Akka Serializer

Using Jackson Serializer for akka-persistence is also one of the pitfalls and this plugin provides an alternative by using a serializer that uses Circe.

Dangerous code for Jackson:

case class Message(animal: Animal) extends MySer

sealed trait Animal

final case class Lion(name: String) extends Animal
final case class Tiger(name: String) extends Animal

To make this code work, a lot of Jackson annotations should be added:

case class Message(animal: Animal) extends MultiDocPrintService

@JsonTypeInfo(use = JsonTypeInfo.Id.NAME, property = "type")
@JsonSubTypes(
  Array(
    new JsonSubTypes.Type(value = classOf[Lion], name = "lion"),
    new JsonSubTypes.Type(value = classOf[Tiger], name = "tiger")))
sealed trait Animal

final case class Lion(name: String) extends Animal
final case class Tiger(name: String) extends Animal

Also if an object is defined:

case object Tick

There will not be exceptions during serialization but Jackson will create another instance of Tick instead of restoring the object Tick's underlying singleton.

actorRef ! Tick

// Inside the actor:
def receive = {
  case Tick => // this won't get matched
} // message will be unhandled

A Circe-based Akka serializer can be used. It uses Circe codecs, derived using Shapeless, that are generated during compile time (so serializer won't crash during runtime as reflection-based serializers may do). For a comparison of Circe with other serializers, read Appendix A.

Note that it is not obligatory to use this serializer for the other features (serializability checker, persistence schema dump) to work. They work as well with e.g. when Jackson serializer is selected.

Usage

Add the following to project dependencies:

import org.virtuslab.ash.AkkaSerializationHelperPlugin

lazy val app = (project in file("app"))
  // ...
  .settings(libraryDependencies += AkkaSerializationHelperPlugin.circeAkkaSerializer)

Note that enabling AkkaSerializationHelperPlugin does not add Circe serializer to the classpath automatically, as this sbt plugin can be used to improve safety of other serializers as well.

Create a custom serializer by extending CirceAkkaSerializer base class:

import org.virtuslab.ash.circe.CirceAkkaSerializer

class ExampleSerializer(actorSystem: ExtendedActorSystem)
    extends CirceAkkaSerializer[MySerializable](actorSystem) {

  override def identifier: Int = 41

  override lazy val codecs = Seq(Register[CommandOne], Register[CommandTwo])

  override lazy val manifestMigrations = Nil

  override lazy val packagePrefix = "org.project"
}

CirceAkkaSerializer can be configured to use Gzip compression when serializing payloads greater than defined size (default is without compression).

See default reference.conf file with comments for more details about CirceAkkaSerializer configuration.

For more guidelines on how to use the serializer, read Akka documentation about serialization, CirceAkkaSerializer Scaladoc and look at the examples.

Missing Codec registration

If a codec is not registered, a runtime exception will occur.

import org.virtuslab.ash.circe.CirceAkkaSerializer
import org.virtuslab.ash.circe.Register

class ExampleSerializer(actorSystem: ExtendedActorSystem)
  extends CirceAkkaSerializer[MySerializable](actorSystem) {
  // ...
  override lazy val codecs = Seq(Register[CommandOne]) // WHOOPS someone forgot to register CommandTwo...
}

java.lang.RuntimeException: Serialization of [CommandTwo] failed. Call Register[A]
for this class or its supertype and append result to `def codecs`.

To solve that, an annotation @org.virtuslab.ash.Serializer can be used.

During compilation, the plugin gathers all direct descendants of the class marked with @org.virtuslab.ash.SerializabilityTrait and checks the body of classes annotated with @org.virtuslab.ash.Serializer if they reference all these direct descendants in any way.

In practice, this is used for checking a class extending CirceAkkaSerializer, like this:

import org.virtuslab.ash.circe.CirceAkkaSerializer
import org.virtuslab.ash.circe.Register

@Serializer(
  classOf[MySerializable],
  typeRegexPattern = Register.REGISTRATION_REGEX)
class ExampleSerializer(actorSystem: ExtendedActorSystem)
  extends CirceAkkaSerializer[MySerializable](actorSystem) {
    // ...
    override lazy val codecs = Seq(Register[CommandOne]) // WHOOPS someone forgot to register CommandTwo...
    // ... but Codec Registration Checker will throw a compilation error here:
    // `No codec for `CommandOne` is registered in a class annotated with @org.virtuslab.ash.annotation.Serializer`
}

Note that as with Serializability Checker and Dump Persistence Schema, this compiler plugin only runs in the sbt modules where AkkaSerializationHelperPlugin is explicitly enabled.

For more information, read @Serializer scaladoc.

Additional configuration for compiler plugins

All compiler plugins and their verbose modes can be enabled/disabled using two sbt keys:

ashCompilerPluginEnable := false // default is true
ashCompilerPluginVerbose := true // default is false

This can be done for all compiler plugins, like above, or just one:

ashCodecRegistrationCheckerCompilerPlugin / ashCompilerPluginEnable := false
ashDumpPersistenceSchemaCompilerPlugin / ashCompilerPluginVerbose := true

Additionally, Compile and Test scope can be specified:

Compile / ashDumpPersistenceSchemaCompilerPlugin / ashCompilerPluginVerbose := true
Test / ashCompilerPluginEnable := false

For full list of sbt keys, check org.virtuslab.ash.AkkaSerializationHelperKeys.

Example applications

The simplest example is the akka-cluster application which uses Akka Serialization Helper: akka-cluster-app.

The second example is the akka-persistence application which shows usage of the Dump Persistence Schema Compiler Plugin: akka-persistence-app.

Step-by-step guide

See full step-by-step guide on Akka Serialization Helper usage.

Contributing Guide

If you want to contribute to this project, see Contributing Guide.

Appendix A: Comparison of available Akka Serializers

Serializer	Jackson	Circe	Protobuf v3	Avro	Borer	Kryo
Data formats	JSON or CBOR	JSON	JSON or custom binary	JSON or custom binary	JSON or CBOR	custom binary
Scala support	very poor, even with jackson-module-scala: poor support for Scala objects, without configuration (without adding ScalaObjectDeserializerModule usage) creates new instances of singleton types (Foo$), breaking pattern matching lacks support of basic scala types like Unit without explicit annotation doesn't work with generics extending AnyVal	perfect out of the box	perfect with ScalaPB	perfect with Avro4s	perfect out of the box	perfect out of the box
Akka support	akka-serialization-jackson	serializer provided by this project	used by akka-remote internally	requires custom serializer	requires custom serializer	akka-kryo
Compile-time mechanics	nothing happens in compile time; everything based on runtime reflection	derives codecs via Shapeless	with ScalaPB, generates Scala classes based on *.proto files	with Avro4s, derives Avro schemas using Magnolia	derives codecs without Magnolia	with akka-kryo, optionally derives codecs in compile time, but otherwise uses reflection in runtime
Runtime safety	none, uses reflection	encoders and decoders are created during compilation	*.proto files are validated before compilation	Avro schema is created during compilation	encoders and decoders are created during compilation	depends on whether codecs were derived in compile time (then standard for Scala code), or not (than none)
Boilerplate	a lot: ADTs requires amount of annotation equal to or exceeding the actual type definitions requires explicit serializers and deserializers in certain cases (e.g. enums)	every top-level sealed trait must be registered manually - but see Codec Registration Checker	in case of custom types, a second layer of models is needed	sometimes requires annotations	every top-level sealed trait must be registered manually; every transitively included class must have an explicitly defined codec	every top-level sealed trait must be registered manually
Schema evolution	removing field adding optional field with JacksonMigration: adding mandatory field renaming field renaming class support of forward versioning for rolling updates	adding optional field removing optional field adding required field with default value removing required field renaming field reordering fields transforming data before deserialization	adding optional field removing optional field adding required field with default value removing required field renaming field reordering fields changing between compatible types	reordering fields renaming fields adding optional field adding required field with default value removing field with default value	renaming fields transforming data before deserialization	adding field removing field renaming field renaming class

Appendix B: what happens with serialization of Messages / Events / States that do not extend a base trait bound to a serializer

In runtime, if given Message/Event/State class does not extend a base trait bound to a serializer, the following problems will occur (depending on Akka version and settings):

Note - the default setting for Akka 2.5 is akka.actor.allow-java-serialization=on, whereas akka.actor.allow-java-serialization=off is the default setting for Akka 2.6

Akka version	akka.actor.allow-java-serialization=off	akka.actor.allow-java-serialization=on
2.5	Serialization will fail with logs like below: [WARN] [...] Outgoing message attempted to use Java Serialization even though `akka.actor.allow-java-serialization = off` was set! Message type was: [class sample.cluster.transformation.BackendRegistration$] [ERROR] [...] Failed to serialize remote message [class akka.actor.ActorSelectionMessage] using serializer [class akka.remote.serialization.MessageContainerSerializer]. Transient association error (association remains live) akka.remote.MessageSerializer$SerializationException: Failed to serialize remote message [class akka.actor.ActorSelectionMessage] using serializer [class akka.remote.serialization.MessageContainerSerializer]. at akka.remote.MessageSerializer$.serialize(MessageSerializer.scala:67) at akka.remote.EndpointWriter.$anonfun$serializeMessage$1(Endpoint.scala:1021) at scala.util.DynamicVariable.withValue(DynamicVariable.scala:62) at akka.remote.EndpointWriter.serializeMessage(Endpoint.scala:1021) at akka.remote.EndpointWriter.writeSend(Endpoint.scala:887) at akka.remote.EndpointWriter$$anonfun$4.applyOrElse(Endpoint.scala:859) at akka.actor.Actor.aroundReceive(Actor.scala:539) at akka.actor.Actor.aroundReceive$(Actor.scala:537) at akka.remote.EndpointActor.aroundReceive(Endpoint.scala:536) at akka.actor.ActorCell.receiveMessage(ActorCell.scala:612) at akka.actor.ActorCell.invoke(ActorCell.scala:581) at akka.dispatch.Mailbox.processMailbox(Mailbox.scala:268) at akka.dispatch.Mailbox.run(Mailbox.scala:229) at akka.dispatch.Mailbox.exec(Mailbox.scala:241) at akka.dispatch.forkjoin.ForkJoinTask.doExec(ForkJoinTask.java:260) at akka.dispatch.forkjoin.ForkJoinPool$WorkQueue.runTask(ForkJoinPool.java:1339) at akka.dispatch.forkjoin.ForkJoinPool.runWorker(ForkJoinPool.java:1979) at akka.dispatch.forkjoin.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:107) Caused by: akka.serialization.DisabledJavaSerializer$JavaSerializationException: Attempted to serialize message using Java serialization while `akka.actor.allow-java-serialization` was disabled. Check WARNING logs for more details.	Serialization succeeds - but using Java serialization (which is not a good choice for production). Warning log like below will appear on the startup: [WARN] [...] Using the default Java serializer for class [sample.cluster.transformation.TransformationResult] which is not recommended because of performance implications. Use another serializer or disable this warning using the setting 'akka.actor.warn-about-java-serializer-usage'
2.6	Serialization will fail with logs like below: [WARN] [...] - Outgoing message attempted to use Java Serialization even though`akka.actor.allow-java-serialization = off` was set! Message type was: [class sample.cluster.transformation.Worker$TransformText] [ERROR] [...] - Failed to serialize message [sample.cluster.transformation.Worker$TransformText]. akka.serialization.DisabledJavaSerializer$JavaSerializationException: Attempted to serialize message using Java serialization while `akka.actor.allow-java-serialization` was disabled. Check WARNING logs for more details	Serialization succeeds - but using Java serialization (which is not a good choice for production). Warning log like below will appear on the startup: [WARN] [...] Using the default Java serializer for class [sample.cluster.transformation.TransformationResult] which is not recommended because of performance implications. Use another serializer or disable this warning using the setting 'akka.actor.warn-about-java-serializer-usage'