This style guide is a list of best practices working with Ruby background jobs using Active Job with Sidekiq backend.
Despite the common belief, they work quite well together if you follow the guidelines.
Pass Active Record models as arguments; do not pass by id. Active Job automatically serializes and deserializes Active Record models using GlobalID, and manual deserialization of the models is not necessary.
GlobalID handles model class mismatches properly.
Deserialization errors are reported to error tracking.
# bad - passing by id
# Deserialization error is reported, the job *is* scheduled for retry.
class SomeJob < ApplicationJob
def perform(model_id)
model = Model.find(model_id)
do_something_with(model)
end
end
# bad - model mismatch
class SomeJob < ApplicationJob
def perform(model_id)
Model.find(model_id)
# ...
end
end
# Will try to fetch a Model using another model class, e.g. User's id.
SomeJob.perform_later(user.id)
# acceptable - passing by id
# Deserialization error is reported, the job is *not* scheduled for retry.
class SomeJob < ApplicationJob
def perform(model_id)
model = Model.find(model_id)
do_something_with(model)
rescue ActiveRecord::RecordNotFound
Rollbar.warning('Not found')
end
end
# good - passing with GlobalID
# Deserialization error is reported, the job is *not* scheduled for retry.
class SomeJob < ApplicationJob
def perform(model)
do_something_with(model)
end
end
Warning
|
Do not replace one style with another, use a transitional period to let all jobs scheduled with ids to be processed. Use a helper to temporarily support both numeric and GlobalID arguments. |
class SomeJob < ApplicationJob
include TransitionHelper
def perform(model)
# TODO: remove this when all jobs with numeric id arguments are processed
model = fetch(model, Model)
do_something_with(model)
end
end
module TransitionHelper
def fetch(id_or_object, model_class)
case id_or_object
when Numeric
model_class.find(id_or_object)
when model_class
id_or_object
else
fail "Object type mismatch #{model_class}, #{id_or_object}"
end
end
end
Explicitly specify a queue to be used in job classes. Make sure the queue is on the list of processed queues.
Putting all jobs into one basket comes with a risk of more urgent jobs being executed with a significant delay. Do not put slow and fast jobs together in one queue. Do not put urgent and non-urgent jobs together in one queue.
# bad - no queue specified
class SomeJob < ApplicationJob
def perform
# ...
end
end
# bad - the wrong queue specified
class SomeJob < ApplicationJob
queue_as :hgh_prioriti # nonexistent queue specified
def perform
# ...
end
end
# good
class SomeJob < ApplicationJob
queue_as :high_priority
def perform
# ...
end
end
Ideally, jobs should be idempotent, meaning there should be no bad side effects of them running more than once. Sidekiq only guarantees that the jobs will run at least once, but not necessarily exactly once.
Even jobs that do not fail due to errors might be interrupted during non-rolling-release deployments.
class UserNotificationJob < ApplicationJob
def perform(user)
send_email_to(user) unless already_notified?(user)
end
end
Do not use threads in your jobs. Spawn jobs instead. Spinning up a thread in a job leads to opening a new database connection, and the connections are easily exhausted, up to the point when the webserver is down.
# bad - consumes all available connections
class SomeJob < ApplicationJob
def perform
User.find_each |user|
Thread.new do
ExternalService.update(user)
end
end
end
end
# good
class SomeJob < ApplicationJob
def perform(user)
ExternalService.update(user)
end
end
User.find_each |user|
SomeJob.perform_later(user)
end
Avoid using ActiveJob’s built-in retry_on
or ActiveJob::Retry
(activejob-retry
gem).
Use Sidekiq retries, which are also available from within Active Job with Sidekiq 6+.
Do not hide or extract job retry mechanisms. Keep retries directives visible in the jobs.
# bad - makes three attempts without submitting to Rollbar,
# fails and relies on Sidekiq's retry that would also make several
# retry attempts, submitting each of the failures to Rollbar.
class SomeJob < ApplicationJob
retry_on ThirdParty::Api::Errors::SomeError, wait: 1.minute, attempts: 3
def perform(user)
# ...
end
end
# bad - it's not clear upfront if the job will be retried or not
class SomeJob < ApplicationJob
include ReliableJob
def perform(user)
# ...
end
end
# good - Sidekiq deals with retries
class SomeJob < ApplicationJob
sidekiq_options retry: 3
def perform(user)
# ...
end
end
Background processing of a scheduled job may happen sooner than you expect. Make sure to only schedule jobs when the transaction has been committed.
# bad - job may perform earlier than the transaction is committed
User.transaction do
users_params.each do |user_params|
user = User.create!(user_params)
NotifyUserJob.perform_later(user)
end
end
# good
users = User.transaction do
users_params.map do |user_params|
User.create!(user_params)
end
end
users.each { |user| NotifyUserJob.perform_later(user) }