Better Defaults and Run-time expectations

[drush]

Having used many background job frameworks for rails over the years, I was excited by the pitch for GoodJob, however after looking under the covers it appears that some of the design assumptions are less than ideal.

GJ should be designed with failures assumed. GJ rows should be updated with status information on failure/retry. The notion of delete and re-insert on failure is a critical design flaw and makes it possible for failed jobs to simply go missing. Specifically, I'd recommend:

• default to GoodJob.preserve_job_records = :on_unhandled_error (No one needs to know about successful jobs at high volume - but every failure needs visibility until it is resolved)

• Move error_count to a column rather that embedding it in the serialized_params

• Ensure error stores the last error or add a new last_error column if additional state is needed

• Retry should be an explicit method on the model. The user story here is that a job has failed because of an underlying system bug. After a fix and code deploy, the job is explicitly retried by devops to resolve the failure.

• Add :last_error_backtrace to help better diagnose errors

• never destroy failed jobs - only update job rows with additional status information on retry/failures.

[bensheldon]

@drush thank you for the feedback!

I think you're describing moving the current paradigm of an immutable record for each run/execution of an ActiveJob job, to one in which there is a single database record that represents an ActiveJob job that is updated with new data if the job is retried or fails/dead.

That's something that is on my mind and I appreciate you sharing some additional details.

Could you describe some of your workflow for identifying jobs? I generally rely upon external reporting tools (Sentry) and separate business objects for tracking lifecycle, which is why I'm curious about your needs and usecases. I wonder if the current database schema could meet your needs with different queries.

Also, I'm curious if you have ideas for how to ensure that a novice GoodJob user does not end up overflowing their database with stored records; that is why GoodJob does not store failed/dead jobs by default.

Lastly, "Retry should be an explicit method on the model" seems like a good feature if you wanted to contribute it.

[bensheldon]

@drush one more question: if the backtrace were to be stored on the run, do you have a preference for how it is stored? The unserialized data is an array of strings. For example, storing it as:

• jsonb
• postgres array
• serialized string