stageleft v0.6.0

New Features

• extract initial Hydroflow+ utilities into a standard library

Bug Fixes

• restrict lifetime parameters to be actually invariant
  Our lifetimes were accidentally made covariant when the lifetime 'a
  was removed from the process/cluster tag type. This fixes that typing
  hole, and also loosens some restrictions on the lifetime of deploy
  environments.

Refactor

• generalize quorum logic

Documentation

• add repository field to Cargo.tomls, fix #1452
  #1452

  Will trigger new releases of the following:
  unchanged = 'hydroflow_deploy_integration', 'variadics', 'variadics_macro', 'pusherator'

  (All other crates already have changes, so would be released anyway)

• update CHANGELOG.mds for big rename
  Generated before rename per RELEASING.md instructions.

New Features (BREAKING)

• mark non-deterministic operators as unsafe and introduce timestamped streams
  Big PR.

  First big change is we introduce a Timestamped location. This is a bit
  of a hybrid between top-level locations and Tick locations. The idea
  is that you choose where timestamps are generated, and then have a
  guarantee that everything after that will be atomically computed (useful
  for making sure we add payloads to the log before ack-ing).

  The contract is that an operator or module that takes a Timestamped
  input must still be deterministic regardless of the stamps on messages
  (which are hidden unless you tick_batch). But unlike a top-level
  stream (which has the same constraints), you have the atomicity
  guarantee. Right now the guarantee is trivial since we have one global
  tick for everything. But in the future when we want to apply
  @davidchuyaya's optimizations this will be helpful to know when there
  are causal dependencies on when data can be sent to others.

  Second change is we mark every non-deterministic operator (modulo
  explicit annotations such as NoOrder) with Rust's unsafe keyword.
  This makes it super clear where non-determinism is taking place.

  I've used this to put unsafe blocks throughout our example code and
  add SAFETY annotations that argue why the non-determinism is safe (or
  point out that we've explicitly documented / expect non-determinism). I
  also added #![warn(unsafe_op_in_unsafe_fn)] to the examples and the
  template, since this forces good hygiene of annotating sources of
  non-determinism even inside a module that is intentionally
  non-deterministic.

  Paxos changes are mostly refactors, and I verified that the performance
  is the same as before.

• allow cluster self ID to be referenced as a global constant
  This eliminates the need to store cluster.self_id() in a local
  variable first, instead you can directly reference CLUSTER_SELF_ID.

Commit Statistics

• 8 commits contributed to the release.
• 45 days passed between releases.
• 7 commits were understood as conventional.
• 6 unique issues were worked on: #1501, #1559, #1574, #1583, #1584, #1591

Commit Details

view details

• #1501
  • Add repository field to Cargo.tomls, fix #1452 (28cd220)
• #1559
  • Restrict lifetime parameters to be actually invariant (f6989ba)
• #1574
  • Allow cluster self ID to be referenced as a global constant (a93a5e5)
• #1583
  • Generalize quorum logic (ec55910)
• #1584
  • Mark non-deterministic operators as unsafe and introduce timestamped streams (9393899)
• #1591
  • Extract initial Hydroflow+ utilities into a standard library (53da4c1)
• Uncategorized
  • Release dfir_lang v0.11.0, dfir_datalog_core v0.11.0, dfir_datalog v0.11.0, dfir_macro v0.11.0, hydroflow_deploy_integration v0.11.0, lattices_macro v0.5.8, variadics v0.0.8, variadics_macro v0.5.6, lattices v0.5.9, multiplatform_test v0.4.0, pusherator v0.0.10, dfir_rs v0.11.0, hydro_deploy v0.11.0, stageleft_macro v0.5.0, stageleft v0.6.0, stageleft_tool v0.5.0, hydro_lang v0.11.0, hydro_std v0.11.0, hydro_cli v0.11.0, safety bump 6 crates (9a7e486)
  • Update CHANGELOG.mds for big rename (e1a08e5)