Oj Serializers

Faster JSON serializers for Ruby, built on top of the powerful oj library.

Why? 🤔

ActiveModel::Serializer has a nice DSL, but it allocates many objects leading to memory bloat, time spent on GC, and lower performance.

Oj::Serializer provides a similar API, with better performance.

Learn more about how this library achieves its performance.

Features ⚡️

• Intuitive declaration syntax, supporting mixins and inheritance
• Reduced memory allocation and improved performance
• Generate TypeScript interfaces automatically
• Support for has_one and has_many, compose with flat_one
• Useful development checks to avoid typos and mistakes
• Migrate easily from Active Model Serializers

Installation 💿

Add this line to your application's Gemfile:

gem 'oj_serializers'

And then run:

$ bundle install

Usage 🚀

You can define a serializer by subclassing Oj::Serializer, and specify which attributes should be serialized.

class AlbumSerializer < Oj::Serializer
  attributes :name, :genres

  attribute :release do
    album.release_date.strftime('%B %d, %Y')
  end

  has_many :songs, serializer: SongSerializer
end

Example Output

{
  name: "Abraxas",
  genres: [
    "Pyschodelic Rock",
    "Blues Rock",
    "Jazz Fusion",
    "Latin Rock",
  ],
  release: "September 23, 1970",
  songs: [
    {
      track: 1,
      name: "Sing Winds, Crying Beasts",
      composers: ["Michael Carabello"],
    },
    {
      track: 2,
      name: "Black Magic Woman / Gypsy Queen",
      composers: ["Peter Green", "Gábor Szabó"],
    },
    {
      track: 3,
      name: "Oye como va",
      composers: ["Tito Puente"],
    },
    {
      track: 4,
      name: "Incident at Neshabur",
      composers: ["Alberto Gianquinto", "Carlos Santana"],
    },
    {
      track: 5,
      name: "Se acabó",
      composers: ["José Areas"],
    },
    {
      track: 6,
      name: "Mother's Daughter",
      composers: ["Gregg Rolie"],
    },
    {
      track: 7,
      name: "Samba pa ti",
      composers: ["Santana"],
    },
    {
      track: 8,
      name: "Hope You're Feeling Better",
      composers: ["Rolie"],
    },
    {
      track: 9,
      name: "El Nicoya",
      composers: ["Areas"],
    },
  ],
}

You can then use your new serializer to render an object or collection:

class AlbumsController < ApplicationController
  def show
    render json: AlbumSerializer.one(album)
  end

  def index
    render json: { albums: AlbumSerializer.many(albums) }
  end
end

Active Model Serializers style

require "oj_serializers/sugar" # In an initializer

class AlbumsController < ApplicationController
  def show
    render json: album, serializer: AlbumSerializer
  end
  
  def index
    render json: albums, root: :albums, each_serializer: AlbumSerializer
  end
end

Rendering 🖨

Use one to serialize objects, and many to serialize enumerables:

render json: {
  favorite_album: AlbumSerializer.one(album),
  purchased_albums: AlbumSerializer.many(albums),
}

Serializers can be rendered arrays, hashes, or even inside ActiveModel::Serializer by using a method in the serializer, making it very easy to combine with other libraries and migrate incrementally.

render is a shortcut for one and many:

render json: {
  favorite_album: AlbumSerializer.render(album),
  purchased_albums: AlbumSerializer.render(albums),
}

Attributes DSL 🪄

Specify which attributes should be rendered by calling a method in the object to serialize.

class PlayerSerializer < Oj::Serializer
  attributes :first_name, :last_name, :full_name
end

You can serialize custom values by specifying that a method is an attribute:

class PlayerSerializer < Oj::Serializer
  attribute :name do
    "#{player.first_name} #{player.last_name}"
  end

  # or

  attribute
  def name
    "#{player.first_name} #{player.last_name}"
  end
end

Note

In this example, player was inferred from PlayerSerializer.

You can customize this by using object_as.

Associations 🔗

Use has_one to serialize individual objects, and has_many to serialize a collection.

You must specificy which serializer to use with the serializer option.

class SongSerializer < Oj::Serializer
  has_one :album, serializer: AlbumSerializer
  has_many :composers, serializer: ComposerSerializer
end

Specify a different value for the association by providing a block:

class SongSerializer < Oj::Serializer
  has_one :album, serializer: AlbumSerializer do
    Album.find_by(song_ids: song.id)
  end
end

In case you need to pass options, you can call the serializer manually:

class SongSerializer < Oj::Serializer
  attribute :album do
    AlbumSerializer.one(song.album, for_song: song)
  end
end

Aliasing or renaming attributes ↔️

You can pass as when defining an attribute or association to serialize it using a different key:

class SongSerializer < Oj::Serializer
  has_one :album, as: :first_release, serializer: AlbumSerializer

  attributes title: {as: :name}

  # or as a shortcut
  attributes title: :name
end

Conditional attributes ❔

You can render attributes and associations conditionally by using :if.

class PlayerSerializer < Oj::Serializer
  attributes :first_name, :last_name, if: -> { player.display_name? }

  has_one :album, serializer: AlbumSerializer, if: -> { player.album }
end

This is useful in cases where you don't want to null values to be in the response.

Advanced Usage 🧙‍♂️

Using a different alias for the internal object

In most cases, the default alias for the object will be convenient enough.

However, if you would like to specify it manually, use object_as:

class DiscographySerializer < Oj::Serializer
  object_as :artist

  # Now we can use `artist` instead of `object` or `discography`.
  attribute
  def latest_albums
    artist.albums.desc(:year)
  end
end

Identifier attributes

The identifier method allows you to only include an identifier if the record or document has been persisted.

class AlbumSerializer < Oj::Serializer
  identifier

  # or if it's a different field
  identifier :uuid
end

Additionally, identifier fields are always rendered first, even when sorting fields alphabetically.

Transforming attribute keys 🗝

When serialized data will be consumed from a client language that has different naming conventions, it can be convenient to transform keys accordingly.

For example, when rendering an API to be consumed from the browser via JavaScript, where properties are traditionally named using camel case.

Use transform_keys to handle that conversion.

class BaseSerializer < Oj::Serializer
  transform_keys :camelize

  # shortcut for
  transform_keys -> (key) { key.to_s.camelize(:lower) }
end

This has no performance impact, as keys will be transformed at load time.

Sorting attributes 📶

By default attributes are rendered in the order they are defined.

If you would like to sort attributes alphabetically, you can specify it at a serializer level:

class BaseSerializer < Oj::Serializer
  sort_attributes_by :name # or a Proc
end

This has no performance impact, as attributes will be sorted at load time.

Path helpers 🛣

In case you need to access path helpers in your serializers, you can use the following:

class BaseSerializer < Oj::Serializer
  include Rails.application.routes.url_helpers

  def default_url_options
    Rails.application.routes.default_url_options
  end
end

One slight variation that might make it easier to maintain in the long term is to use a separate singleton service to provide the url helpers and options, and make it available as urls.

Generating TypeScript automatically 🤖

It's easy for the backend and the frontend to become out of sync. Traditionally, preventing bugs requires writing extensive integration tests.

TypeScript is a great tool to catch this kind of bugs and mistakes, as it can detect incorrect usages and missing fields, but writing types manually is cumbersome, and they can become stale over time, giving a false sense of confidence.

types_from_serializers extends this library to allow embedding type information, as well as inferring types from the SQL schema when available, and uses this information to automatically generate TypeScript interfaces from your serializers.

As a result, it's posible to easily detect mismatches between the backend and the frontend, as well as make the fields more discoverable and provide great autocompletion in the frontend, without having to manually write the types.

Composing serializers 🧱

There are three options to compose serializers: inheritance, mixins, and flat_one.

Use flat_one to include all attributes from a different serializer:

class AttachmentSerializer < BaseSerializer
  identifier

  class BlobSerializer < BaseSerializer
    attributes :filename, :byte_size, :content_type, :created_at
  end

  flat_one :blob, serializer: BlobSerializer
end

Think of it as has_one without a "root", all the attributes are added directly.

Example Output

{
  id: 5,
  filename: "image.jpg,
  byte_size: 256074,
  content_type: "image/jpeg",
  created_at: "2022-08-04T17:25:12.637-07:00",
}

This is especially convenient when using types_from_serializers, as it enables automatic type inference for the included attributes.

Memoization & local state

Serializers are designed to be stateless so that an instanced can be reused, but sometimes it's convenient to store intermediate calculations.

Use memo for memoization and storing temporary information.

class DownloadSerializer < Oj::Serializer
  attributes :filename, :size

  attribute
  def progress
    "#{ last_event&.progress || 0 }%"
  end

private

  def last_event
    memo.fetch(:last_event) {
      download.events.desc(:created_at).first
    }
  end
end

hash_attributes 🚀

Very convenient when serializing Hash-like structures, this strategy uses the [] operator.

class PersonSerializer < Oj::Serializer
  hash_attributes 'first_name', :last_name
end

PersonSerializer.one('first_name' => 'Mary', :middle_name => 'Jane', :last_name => 'Watson')
# {first_name: "Mary", last_name: "Watson"}

mongo_attributes 🚀

Reads data directly from attributes in a Mongoid document.

By skipping type casting, coercion, and defaults, it achieves the best performance.

Although there are some downsides, depending on how consistent your schema is, and which kind of consumer the API has, it can be really powerful.

class AlbumSerializer < Oj::Serializer
  mongo_attributes :id, :name
end

Caching 📦

Usually rendering is so fast that turning caching on can be slower.

However, in cases of deeply nested structures, unpredictable query patterns, or methods that take a long time to run, caching can improve performance.

To enable caching, use cached, which calls cache_key in the object:

class CachedUserSerializer < UserSerializer
  cached
end

You can also provide a lambda to cached_with_key to define a custom key:

class CachedUserSerializer < UserSerializer
  cached_with_key ->(user) {
    "#{ user.id }/#{ user.current_sign_in_at }"
  }
end

It will leverage fetch_multi when serializing a collection with many or has_many, to minimize the amount of round trips needed to read and write all items to cache.

This works specially well if your cache store also supports write_multi.

Writing to JSON

In some corner cases it might be faster to serialize using a Oj::StringWriter, which you can access by using one_as_json and many_as_json.

Alternatively, you can toggle this mode at a serializer level by using default_format :json, or configure it globally from your base serializer:

class BaseSerializer < Oj::Serializer
  default_format :json
end

This will change the default shortcuts (render, one, one_if, and many), so that the serializer writes directly to JSON instead of returning a Hash.

Even when using this mode, you can still use rendered values inside arrays, hashes, and other serializers, thanks to the raw_json extensions.

Example Output

{
  "name": "Abraxas",
  "genres": [
    "Pyschodelic Rock",
    "Blues Rock",
    "Jazz Fusion",
    "Latin Rock"
  ],
  "release": "September 23, 1970",
  "songs": [
    {
      "track": 1,
      "name": "Sing Winds, Crying Beasts",
      "composers": [
        "Michael Carabello"
      ]
    },
    {
      "track": 2,
      "name": "Black Magic Woman / Gypsy Queen",
      "composers": [
        "Peter Green",
        "Gábor Szabó"
      ]
    },
    {
      "track": 3,
      "name": "Oye como va",
      "composers": [
        "Tito Puente"
      ]
    },
    {
      "track": 4,
      "name": "Incident at Neshabur",
      "composers": [
        "Alberto Gianquinto",
        "Carlos Santana"
      ]
    },
    {
      "track": 5,
      "name": "Se acabó",
      "composers": [
        "José Areas"
      ]
    },
    {
      "track": 6,
      "name": "Mother's Daughter",
      "composers": [
        "Gregg Rolie"
      ]
    },
    {
      "track": 7,
      "name": "Samba pa ti",
      "composers": [
        "Santana"
      ]
    },
    {
      "track": 8,
      "name": "Hope You're Feeling Better",
      "composers": [
        "Rolie"
      ]
    },
    {
      "track": 9,
      "name": "El Nicoya",
      "composers": [
        "Areas"
      ]
    }
  ]
}

Design 📐

Unlike ActiveModel::Serializer, which builds a Hash that then gets encoded to JSON, this implementation can use Oj::StringWriter to write JSON directly, greatly reducing the overhead of allocating and garbage collecting the hashes.

It also allocates a single instance per serializer class, which makes it easy to use, while keeping memory usage under control.

The internal design is simple and extensible, and because the library is written in Ruby, creating new serialization strategies requires very little code. Please open a Discussion if you need help 😃

Comparison with other libraries

ActiveModel::Serializer instantiates one serializer object per item to be serialized.

Other libraries such as blueprinter jsonapi-serializer evaluate serializers in the context of a class instead of an instance of a class. The downside is that you can't use instance methods or local memoization, and any mixins must be applied to the class itself.

panko-serializer also uses Oj::StringWriter, but it has the big downside of having to own the entire render tree. Putting a serializer inside a Hash or an Active Model Serializer and serializing that to JSON doesn't work, making a gradual migration harder to achieve. Also, it's optimized for Active Record but I needed good Mongoid support.

Oj::Serializer combines some of these ideas, by using instances, but reusing them to avoid object allocations. Serializing 10,000 items instantiates a single serializer. Unlike panko-serializer, it doesn't suffer from double encoding problems so it's easier to use.

Follow this discussion to find out more about the raw_json extensions that made this high level of interoperability possible.

As a result, migrating from active_model_serializers is relatively straightforward because instance methods, inheritance, and mixins work as usual.

Benchmarks 📊

This library includes some benchmarks to compare performance with similar libraries.

See this pull request for a quick comparison, or check the CI to see the latest results.

Migrating from other libraries

Please refer to the migration guide for a full discussion of the compatibility modes available to make it easier to migrate from active_model_serializers and similar libraries.

Formatting 📏

Even though most of the examples above use a single-line style to be succint, I highly recommend writing one attribute per line, sorting them alphabetically (most editors can do it for you), and always using a trailing comma.

class AlbumSerializer < Oj::Serializer
  attributes(
    :genres,
    :name,
    :release_date,
  )
end

It will make things clearer, minimize the amount of git conflicts, and keep the history a lot cleaner and more meaningful when using git blame.

Special Thanks 🙏

This library wouldn't be possible without the wonderful and performant oj library. Thanks Peter! 😃

Also, thanks to the libraries that inspired this one:

• active_model_serializers: For the DSL
• panko-serializer: For validating that using Oj::StringWriter was indeed fast

License

The gem is available as open source under the terms of the MIT License.