-
-
Notifications
You must be signed in to change notification settings - Fork 308
Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
- Loading branch information
Showing
5 changed files
with
110 additions
and
236 deletions.
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -1,284 +1,122 @@ | ||
# Changelog | ||
# Changelog for Oban v2.12 | ||
|
||
_🌟 Looking for changes to Web or Pro? Check the [Oban.Pro Changelog][opc] or | ||
the [Oban.Web Changelog][owc]. 🌟_ | ||
|
||
Oban v2.11 requires a v11 migration, Elixir v1.11+ and Postgres v10.0+ | ||
[Oban v2.12 Upgrade Guide](v2-12.html) | ||
|
||
[Oban v2.11 Upgrade Guide][upg] | ||
Oban v2.12 was dedicated to enriching the testing experience and expanding | ||
config, plugin, and queue validation across all environments. | ||
|
||
Oban v2.11 focused on reducing database load, bolstering telemetry-powered | ||
introspection, and improving the production experience for all users. To that | ||
end, we've extracted functionality from Oban Pro and switched to a new global | ||
coordination model. | ||
## Testing Modes | ||
|
||
## Leadership | ||
Testing modes bring a new, vastly improved, way to configure Oban for testing. | ||
The new `testing` option makes it explicit that Oban should operate in a | ||
restricted mode for the given environment. | ||
|
||
Coordination between nodes running Oban is crucial to how many plugins operate. | ||
Staging jobs once a second from multiple nodes is wasteful, as is pruning, | ||
rescuing, or scheduling cron jobs. Prior Oban versions used transactional | ||
advisory locks to prevent plugins from running concurrently, but there were some | ||
issues: | ||
Behind the scenes, the new testing modes rely on layers of validation within | ||
Oban's `Config` module. Now production configuration is validated automatically | ||
during test runs. Even though queues and plugins aren't _started_ in the test | ||
environment, their configuration is still validated. | ||
|
||
* Plugins don't know if they'll take the advisory lock, so they still need to | ||
run a query periodically. | ||
|
||
* Nodes don't usually start simultaneously, and time drifts between machines. | ||
There's no guarantee that the top of the minute for one node is the same as | ||
another's—chances are, they don't match. | ||
|
||
Oban 2.11 introduces a table-based leadership mechanism that guarantees only one | ||
node in a cluster, where "cluster" means a bunch of nodes connected to the same | ||
Postgres database, will run plugins. Leadership is transparent and designed for | ||
resiliency with minimum chatter between nodes. | ||
|
||
See the [Upgrade Guide][upg] for instructions on how to create the peers table | ||
and get started with leadership. If you're curious about the implementation | ||
details or want to use leadership in your application, take a look at docs for | ||
`Oban.Peer`. | ||
|
||
## Alternative PG (Process Groups) Notifier | ||
|
||
Oban relies heavily on PubSub, and until now it only provided a Postgres | ||
adapter. Postres is amazing, and has a highly performant PubSub option, but it | ||
doesn't work in every environment (we're looking at you, PG Bouncer). | ||
|
||
Fortunately, many Elixir applications run in a cluster connected by distributed | ||
Erlang. That means Process Groups, aka PG, is available for many applications. | ||
|
||
So, we pulled Oban Pro's PG notifier into Oban to make it available for | ||
everyone! If your app runs in a proper cluster, you can switch over to the PG | ||
notifier: | ||
To switch, stop overriding `plugins` and `queues` and enable a testing mode | ||
in your `test.exs` config: | ||
|
||
```elixir | ||
config :my_app, Oban, | ||
notifier: Oban.Notifiers.PG, | ||
... | ||
config :my_app, Oban, testing: :manual | ||
``` | ||
|
||
Now there are two notifiers to choose from, each with their own strengths and | ||
weaknesses: | ||
|
||
* `Oban.Notifiers.Postgres` — Pros: Doesn't require distributed erlang, | ||
publishes `insert` events to trigger queues; Cons: Doesn't work with PGBouncer | ||
intransaction mode, Doesn't work in tests because of the sandbox. | ||
|
||
* `Oban.Notifiers.PG` — Pros: Works PG Bouncer in transaction mode, Works in | ||
tests; Cons: Requires distributed Erlang, Doesn't publish `insert` events. | ||
|
||
## Basic Lifeline Plugin | ||
|
||
When a queue's producer crashes or a node shuts down before a job finishes | ||
executing, the job may be left in an `executing` state. The worst part is that | ||
these jobs—which we call "orphans"—are completely invisible until you go | ||
searching through the jobs table. | ||
Testing in `:manual` mode is identical to testing in older versions of Oban: | ||
jobs won't run automatically so you can use helpers like `assert_enqueued` and | ||
execute them manually with `Oban.drain_queue/2`. | ||
|
||
Oban Pro has awlays had a "Lifeline" plugin for just this ocassion—and now we've | ||
brought a basic `Lifeline` plugin to Oban. | ||
|
||
To automatically rescue orphaned jobs that are still `executing`, include the | ||
`Oban.Plugins.Lifeline` in your configuration: | ||
An alternate `:inline` allows Oban to bypass all database interaction and run | ||
jobs _immediately in the process that enqueued them_. | ||
|
||
```elixir | ||
config :my_app, Oban, | ||
plugins: [Oban.Plugins.Lifeline], | ||
... | ||
config :my_app, Oban, testing: :inline | ||
``` | ||
|
||
Now the plugin will search and rescue orphans after they've lingered for 60 | ||
minutes. | ||
Finally, new [testing guides][tst] cover test setup, unit [testing | ||
workers][tsw], integration [testing queues][tsq], and testing [dynamic | ||
configuration][tsc]. | ||
|
||
_🌟 Note: The `Lifeline` plugin may transition jobs that are genuinely | ||
`executing` and cause duplicate execution. For more accurate rescuing or to | ||
rescue jobs that have exhausted retry attempts see the `DynamicLifeline` plugin | ||
in [Oban Pro][pro]._ | ||
[tst]: testing.html | ||
[tsw]: testing_workers.html | ||
[tsq]: testing_queues.html | ||
[tsc]: testing_config.html | ||
|
||
[pro]: https://getoban.pro | ||
## Global Peer Module | ||
|
||
## Reindexer Plugin | ||
Oban v2.11 introduced centralized leadership via Postgres tables. However, | ||
Postgres based leadership isn't always a good fit. For example, an ephemeral | ||
leadership mechanism is preferred for integration testing. | ||
|
||
Over time various Oban indexes (heck, any indexes) may grow without `VACUUM` | ||
cleaning them up properly. When this happens, rebuilding the indexes will | ||
release bloat and free up space in your Postgres instance. | ||
|
||
The new `Reindexer` plugin makes index maintenance painless and automatic by | ||
periodically rebuilding all of your Oban indexes concurrently, without any | ||
locks. | ||
|
||
By default, reindexing happens once a day at midnight UTC, but it's configurable | ||
with a standard cron expression (and timezone). | ||
In that case, you can make use of the new `:global` powered peer module for | ||
leadership: | ||
|
||
```elixir | ||
config :my_app, Oban, | ||
plugins: [Oban.Plugins.Reindexer], | ||
peer: Oban.Peers.Global, | ||
... | ||
``` | ||
|
||
See `Oban.Plugins.Reindexer` for complete options and implementation details. | ||
|
||
## Improved Telemetry and Logging | ||
|
||
The default telemetry backed logger includes more job fields and metadata about | ||
execution. Most notably, the execution state and formatted error reports when | ||
jobs fail. | ||
|
||
Here's an example of the default output for a successful job: | ||
|
||
```json | ||
{ | ||
"args":{"action":"OK","ref":1}, | ||
"attempt":1, | ||
"duration":4327295, | ||
"event":"job:stop", | ||
"id":123, | ||
"max_attempts":20, | ||
"meta":{}, | ||
"queue":"alpha", | ||
"queue_time":3127905, | ||
"source":"oban", | ||
"state":"success", | ||
"tags":[], | ||
"worker":"Oban.Integration.Worker" | ||
} | ||
``` | ||
|
||
Now, here's an sample where the job has encountered an error: | ||
|
||
```json | ||
{ | ||
"attempt": 1, | ||
"duration": 5432, | ||
"error": "** (Oban.PerformError) Oban.Integration.Worker failed with {:error, \"ERROR\"}", | ||
"event": "job:exception", | ||
"state": "failure", | ||
"worker": "Oban.Integration.Worker" | ||
} | ||
``` | ||
|
||
## 2.11.2 — 2022-02-25 | ||
|
||
### Bug Fixes | ||
|
||
- [Peer] Retain election schedule timing after a peer shuts down. | ||
|
||
A bug in the Peer module's "down" handler overwrote the election scheduling | ||
interval to `0`. As soon as the leader crashed all other peers in the cluster | ||
would start trying to acquire leadership as fast as possible. That caused | ||
excessive database load and churn. | ||
|
||
In addition to the interval fix, this expands the scheduling interval to 30s, | ||
and warns on any unknown messages to aid debugging in the future. | ||
|
||
- [Notifier.Postgres] Prevent crashing after reconnect. | ||
|
||
The `handle_result` callback no longer sends an errorneous reply after a | ||
disconnection. | ||
|
||
- [Job] Guard against typos and unknown options passed to `new/2`. Passing an | ||
unrecognized option, such as `:scheduled_in` instead of `:schedule_in`, will | ||
make a job invalid with a helpful base error. | ||
|
||
Previously, passing an unknown option was silently ignored without any warning. | ||
|
||
## 2.11.1 — 2022-02-24 | ||
## 2.12.0 — 2022-04-19 | ||
|
||
### Enhancements | ||
|
||
- [Oban] Validate the configured `repo` by checking for `config/0`, rather than | ||
the more obscure `__adapter__/0` callback. This change improves integration | ||
with Repo wrappers such as `fly_postgres`. | ||
- [Oban] Replace queue, plugin, and peer test configuration with a single | ||
`:testing` option. Now configuring Oban for testing only requires one change, | ||
setting the test mode to either `:inline` or `:manual`. | ||
|
||
- [Cron] Expose `parse/1` to facilitate testing that cron expressions are valid | ||
and usable in a crontab. | ||
- `:inline`—jobs execute immediately within the calling process and without | ||
touching the database. This mode is simple and may not be suitable for apps | ||
with complex jobs. | ||
- `:manual`—jobs are inserted into the database where they can be verified and | ||
executed when desired. This mode is more advanced and trades simplicity for | ||
flexibility. | ||
|
||
### Bug Fixes | ||
|
||
- [Notifier.Postgres] Overwrite configured repo name when configuring the | ||
long-lived Postgres connection. | ||
|
||
- [Lifeline] Fix rescuing when using a custom prefix. The previous | ||
implementation assumed that there was an `oban_jobs_state` enum in the public | ||
prefix. | ||
- [Testing] Add `with_testing_mode/2` to temporarily change testing modes | ||
within the context of a function. | ||
|
||
- [Lifeline] Set `discarded_at` when discarding exhausted jobs. | ||
|
||
## 2.11.0 — 2022-02-13 | ||
|
||
### Enhancements | ||
Once the application starts in a particular testing mode it can't be changed. | ||
That's inconvenient if you're running in `:inline` mode and don't want a | ||
particular job to execute inline. | ||
|
||
- [Migration] Change the order of fields in the base index used for the primary | ||
Oban queries. | ||
- [Config] Add `validate/1` to aid in testing dynamic Oban configuration. | ||
|
||
The new order is much faster for frequent queries such as scheduled job | ||
staging. Check the v2.11 upgrade guide for instructions on swapping the | ||
index in existing applications. | ||
- [Config] Validate full plugin and queue options on init, without the need | ||
to start plugins or queues. | ||
|
||
- [Worker] Avoid spawning a separate task for workers that use timeouts. | ||
- [Peers.Global] Add an alternate `:global` powered peer module. | ||
|
||
- [Engine] Add `insert_job`, `insert_all_jobs`, `retry_job`, and | ||
`retry_all_jobs` as required callbacks for all engines. | ||
- [Plugin] A new `Oban.Plugin` behaviour formalizes starting and validating | ||
plugins. The behaviour is implemented by all plugins, and is the foundation of | ||
enhanced config validation. | ||
|
||
- [Oban] Raise more informative error messages for missing or malformed plugins. | ||
|
||
Now missing plugins have a different error from invalid plugins or invalid | ||
options. | ||
|
||
- [Telemetry] Normalize telemetry metadata for all engine operations: | ||
|
||
- Include `changeset` for `insert` | ||
- Include `changesets` for `insert_all` | ||
- Include `job` for `complete_job`, `discard_job`, etc | ||
|
||
- [Repo] Include `[oban_conf: conf]` in `telemetry_options` for all Repo | ||
operations. | ||
|
||
With this change it's possible to differentiate between database calls made by | ||
Oban versus the rest of your application. | ||
- [Plugin] Emit `[:oban, :plugin, :init]` event on init from every plugin. | ||
|
||
### Bug Fixes | ||
|
||
- [Telemetry] Emit `discard` rather than `error` events when a job exhausts all retries. | ||
|
||
Previously `discard_job` was only called for manual discards, i.e., when a job | ||
returned `:discard` or `{:discard, reason}`. Discarding for exhausted attempts | ||
was done within `error_job` in error cases. | ||
|
||
- [Cron] Respect the current timezone for `@reboot` jobs. Previously, `@reboot` | ||
expressions were evaluated on boot without the timezone applied. In that case | ||
the expression may not match the calculated time and jobs wouldn't trigger. | ||
|
||
- [Cron] Delay CRON evaluation until the next minute after initialization. Now | ||
all cron scheduling ocurrs reliably at the top of the minute. | ||
|
||
- [Drainer] Introduce `discard` accumulator for draining results. Now exhausted | ||
jobs along with manual discards count as a `discard` rather than a `failure` | ||
or `success`. | ||
|
||
- [Oban] Expand changeset wrapper within multi function. | ||
|
||
Previously, `Oban.insert_all` could handle a list of changesets, a wrapper map | ||
with a `:changesets` key, or a function. However, the function had to return a | ||
list of changesets rather than a changeset wrapper. This was unexpected and | ||
made some multi's awkward. | ||
|
||
- [Testing] Preserve `attempted_at/scheduled_at` in `perform_job/3` rather than | ||
overwriting them with the current time. | ||
- [Executor ] Skip timeout check with an unknown worker | ||
|
||
- [Oban] Include `false` as a viable `queue` or `plugin` option in typespecs | ||
When the worker can't be resolved we don't need to check the timeout. Doing so | ||
prevents returning a helpful "unknown worker" message, and instead causes a | ||
function error for `nil.timeout/1`. | ||
|
||
### Deprecations | ||
- [Testing] Include `log` and `prefix` in generated conf for `perform_job`. | ||
|
||
- [Telemetry] Hard deprecate `Telemetry.span/3`, previously it was | ||
soft-deprecated. | ||
The opts, and subsequent conf, built for `perform_job` didn't include the | ||
`prefix` or `log` options. That prevented functions that depend on a job's | ||
`conf` within `perform/1` from running with the correct options. | ||
|
||
### Removals | ||
- [Drainer] Retain the currently configured engine while draining a queue. | ||
|
||
- [Telemetry] Remove circuit breaker event documentation because `:circuit` | ||
events aren't emitted anymore. | ||
- [Watchman] Skip pausing queues when shutdown is immediate. This prevents | ||
queue's from interacting with the database during short test runs. | ||
|
||
For changes prior to v2.11 see the [v2.10][prv] docs. | ||
For changes prior to v2.12 see the [v2.11][prv] docs. | ||
|
||
[opc]: https://getoban.pro/docs/pro/changelog.html | ||
[owc]: https://getoban.pro/docs/web/changelog.html | ||
[prv]: https://hexdocs.pm/oban/2.10.1/changelog.html | ||
[upg]: https://hexdocs.pm/oban/v2-11.html | ||
[prv]: https://hexdocs.pm/oban/2.11.2/changelog.html |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
|
@@ -6,7 +6,7 @@ dependencies in `mix.exs`: | |
```elixir | ||
def deps do | ||
[ | ||
{:oban, "~> 2.11"} | ||
{:oban, "~> 2.12"} | ||
] | ||
end | ||
``` | ||
|
Oops, something went wrong.