Skip to content
This repository has been archived by the owner on Jan 24, 2025. It is now read-only.

This Background Jobs style guide is a list of best practices working with Ruby background jobs.

Notifications You must be signed in to change notification settings

toptal/active-job-style-guide

Folders and files

NameName
Last commit message
Last commit date

Latest commit

author
Filipp Pirozhkov
Aug 7, 2019
7513976 · Aug 7, 2019

History

9 Commits
Aug 7, 2019

Repository files navigation

Active Job Style Guide

This style guide is a list of best practices working with Ruby background jobs using Active Job with Sidekiq backend.

Despite the common belief, they work quite well together if you follow the guidelines.

Active Record Models as Arguments

Pass Active Record models as arguments; do not pass by id. Active Job automatically serializes and deserializes Active Record models using GlobalID, and manual deserialization of the models is not necessary.

GlobalID handles model class mismatches properly.

Deserialization errors are reported to error tracking.

# bad - passing by id
# Deserialization error is reported, the job *is* scheduled for retry.
class SomeJob < ApplicationJob
  def perform(model_id)
    model = Model.find(model_id)
    do_something_with(model)
  end
end

# bad - model mismatch
class SomeJob < ApplicationJob
  def perform(model_id)
    Model.find(model_id)
    # ...
  end
end

# Will try to fetch a Model using another model class, e.g. User's id.
SomeJob.perform_later(user.id)

# acceptable - passing by id
# Deserialization error is reported, the job is *not* scheduled for retry.
class SomeJob < ApplicationJob
  def perform(model_id)
    model = Model.find(model_id)
    do_something_with(model)
  rescue ActiveRecord::RecordNotFound
    Rollbar.warning('Not found')
  end
end

# good - passing with GlobalID
# Deserialization error is reported, the job is *not* scheduled for retry.
class SomeJob < ApplicationJob
  def perform(model)
    do_something_with(model)
  end
end
Warning
Do not replace one style with another, use a transitional period to let all jobs scheduled with ids to be processed. Use a helper to temporarily support both numeric and GlobalID arguments.
class SomeJob < ApplicationJob
  include TransitionHelper

  def perform(model)
    # TODO: remove this when all jobs with numeric id arguments are processed
    model = fetch(model, Model)
    do_something_with(model)
  end
end

module TransitionHelper
  def fetch(id_or_object, model_class)
    case id_or_object
    when Numeric
      model_class.find(id_or_object)
    when model_class
      id_or_object
    else
      fail "Object type mismatch #{model_class}, #{id_or_object}"
    end
  end
end

Queue Assignments

Explicitly specify a queue to be used in job classes. Make sure the queue is on the list of processed queues.

Putting all jobs into one basket comes with a risk of more urgent jobs being executed with a significant delay. Do not put slow and fast jobs together in one queue. Do not put urgent and non-urgent jobs together in one queue.

# bad - no queue specified
class SomeJob < ApplicationJob
  def perform
    # ...
  end
end

# bad - the wrong queue specified
class SomeJob < ApplicationJob
  queue_as :hgh_prioriti # nonexistent queue specified

  def perform
    # ...
  end
end

# good
class SomeJob < ApplicationJob
  queue_as :high_priority

  def perform
    # ...
  end
end

Idempotency

Ideally, jobs should be idempotent, meaning there should be no bad side effects of them running more than once. Sidekiq only guarantees that the jobs will run at least once, but not necessarily exactly once.

Even jobs that do not fail due to errors might be interrupted during non-rolling-release deployments.

class UserNotificationJob < ApplicationJob
  def perform(user)
    send_email_to(user) unless already_notified?(user)
  end
end

Threads

Do not use threads in your jobs. Spawn jobs instead. Spinning up a thread in a job leads to opening a new database connection, and the connections are easily exhausted, up to the point when the webserver is down.

# bad - consumes all available connections
class SomeJob < ApplicationJob
  def perform
    User.find_each |user|
      Thread.new do
        ExternalService.update(user)
      end
    end
  end
end

# good
class SomeJob < ApplicationJob
  def perform(user)
    ExternalService.update(user)
  end
end

User.find_each |user|
  SomeJob.perform_later(user)
end

Retries

Avoid using ActiveJob’s built-in retry_on or ActiveJob::Retry (activejob-retry gem). Use Sidekiq retries, which are also available from within Active Job with Sidekiq 6+.

Do not hide or extract job retry mechanisms. Keep retries directives visible in the jobs.

# bad - makes three attempts without submitting to Rollbar,
# fails and relies on Sidekiq's retry that would also make several
# retry attempts, submitting each of the failures to Rollbar.
class SomeJob < ApplicationJob
  retry_on ThirdParty::Api::Errors::SomeError, wait: 1.minute, attempts: 3

  def perform(user)
    # ...
  end
end

# bad - it's not clear upfront if the job will be retried or not
class SomeJob < ApplicationJob
  include ReliableJob

  def perform(user)
    # ...
  end
end

# good - Sidekiq deals with retries
class SomeJob < ApplicationJob
  sidekiq_options retry: 3

  def perform(user)
    # ...
  end
end

Batches

Always use retries for jobs that are executed in batches, otherwise, the batch will never succeed.

Mind Transactions

Background processing of a scheduled job may happen sooner than you expect. Make sure to only schedule jobs when the transaction has been committed.

# bad - job may perform earlier than the transaction is committed
User.transaction do
  users_params.each do |user_params|
    user = User.create!(user_params)
    NotifyUserJob.perform_later(user)
  end
end

# good
users = User.transaction do
          users_params.map do |user_params|
            User.create!(user_params)
          end
        end
users.each { |user| NotifyUserJob.perform_later(user) }

About

This Background Jobs style guide is a list of best practices working with Ruby background jobs.

Topics

Resources

Stars

Watchers

Forks

Releases

No releases published

Packages

No packages published