From e1a08e5d165fbc80da2ae695e507078a97a9031f Mon Sep 17 00:00:00 2001 From: Mingwei Samuel Date: Mon, 23 Dec 2024 12:09:03 -0800 Subject: [PATCH] docs: update `CHANGELOG.md`s for big rename Generated before rename per `RELEASING.md` instructions. --- dfir_datalog/CHANGELOG.md | 92 +++-- dfir_datalog_core/CHANGELOG.md | 223 +++++++----- dfir_lang/CHANGELOG.md | 287 ++++++++++------ dfir_macro/CHANGELOG.md | 121 +++++-- dfir_rs/CHANGELOG.md | 357 ++++++++++++------- hydro_deploy/core/CHANGELOG.md | 74 +++- hydro_lang/CHANGELOG.md | 602 ++++++++++++++++++++++++--------- stageleft/CHANGELOG.md | 90 ++++- stageleft_macro/CHANGELOG.md | 39 ++- stageleft_tool/CHANGELOG.md | 40 ++- 10 files changed, 1402 insertions(+), 523 deletions(-) diff --git a/dfir_datalog/CHANGELOG.md b/dfir_datalog/CHANGELOG.md index c8dd77928a2f..6457bf6fd760 100644 --- a/dfir_datalog/CHANGELOG.md +++ b/dfir_datalog/CHANGELOG.md @@ -5,22 +5,54 @@ All notable changes to this project will be documented in this file. The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.0.0/), and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html). +## Unreleased + +### Refactor (BREAKING) + + - use `cfg(nightly)` instead of feature, remove `-Z` flag, use `Diagnostic::try_emit` + Previous PR (#1587) website build did not work because `panic = "abort"` + is set on wasm, leading to aborts for `proc_macro2::Span::unwrap()` + calls. + + All tests except trybuild seem to pass on stable, WIP #1587 next + +### `hydroflow_datalog` Commit Statistics + + + + - 1 commit contributed to the release. + - 38 days passed between releases. + - 1 commit was understood as [conventional](https://www.conventionalcommits.org). + - 1 unique issue was worked on: [#1606](https://github.com/hydro-project/hydroflow/issues/1606) + +### `hydroflow_datalog` Commit Details + + + +
view details + + * **[#1606](https://github.com/hydro-project/hydroflow/issues/1606)** + - Use `cfg(nightly)` instead of feature, remove `-Z` flag, use `Diagnostic::try_emit` ([`251b103`](https://github.com/hydro-project/hydroflow/commit/251b1039c71d45d3f86123dba1926026ded80824)) +
+ ## 0.10.0 (2024-11-08) + + ### Chore - update pinned rust version, clippy lints, remove some dead code -### Commit Statistics +### `hydroflow_datalog` Commit Statistics - - 1 commit contributed to the release. + - 2 commits contributed to the release. - 69 days passed between releases. - 1 commit was understood as [conventional](https://www.conventionalcommits.org). - 1 unique issue was worked on: [#1444](https://github.com/hydro-project/hydroflow/issues/1444) -### Commit Details +### `hydroflow_datalog` Commit Details @@ -28,6 +60,8 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0 * **[#1444](https://github.com/hydro-project/hydroflow/issues/1444)** - Update pinned rust version, clippy lints, remove some dead code ([`d567760`](https://github.com/hydro-project/hydroflow/commit/d5677604e93c07a5392f4229af94a0b736eca382)) + * **Uncategorized** + - Release hydroflow_lang v0.10.0, hydroflow_datalog_core v0.10.0, hydroflow_datalog v0.10.0, hydroflow_deploy_integration v0.10.0, hydroflow_macro v0.10.0, lattices_macro v0.5.7, variadics v0.0.7, variadics_macro v0.5.5, lattices v0.5.8, multiplatform_test v0.3.0, pusherator v0.0.9, hydroflow v0.10.0, hydro_deploy v0.10.0, stageleft_macro v0.4.0, stageleft v0.5.0, stageleft_tool v0.4.0, hydroflow_plus v0.10.0, hydro_cli v0.10.0, safety bump 8 crates ([`dcd48fc`](https://github.com/hydro-project/hydroflow/commit/dcd48fc7ee805898d9b5ef0d082870e30615e95b)) ## 0.9.0 (2024-08-30) @@ -37,11 +71,11 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0 ### Chore - lower min dependency versions where possible, update `Cargo.lock` - Moved from #1418 - + Moved from #1418 + --------- -### Commit Statistics +### `hydroflow_datalog` Commit Statistics @@ -50,7 +84,7 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0 - 1 commit was understood as [conventional](https://www.conventionalcommits.org). - 1 unique issue was worked on: [#1423](https://github.com/hydro-project/hydroflow/issues/1423) -### Commit Details +### `hydroflow_datalog` Commit Details @@ -72,7 +106,7 @@ Unchanged from previous release. - mark `hydroflow_datalog` and `hydroflow_macro` as unchanged for 0.8.0 release -### Commit Statistics +### `hydroflow_datalog` Commit Statistics @@ -81,7 +115,7 @@ Unchanged from previous release. - 1 commit was understood as [conventional](https://www.conventionalcommits.org). - 0 issues like '(#ID)' were seen in commit messages -### Commit Details +### `hydroflow_datalog` Commit Details @@ -102,7 +136,7 @@ Unchanged from previous release. - mark `hydroflow_datalog` as unchanged for 0.7 release -### Commit Statistics +### `hydroflow_datalog` Commit Statistics @@ -111,7 +145,7 @@ Unchanged from previous release. - 1 commit was understood as [conventional](https://www.conventionalcommits.org). - 0 issues like '(#ID)' were seen in commit messages -### Commit Details +### `hydroflow_datalog` Commit Details @@ -132,7 +166,7 @@ Unchanged from previous release. - mark hydroflow_datalog, hydroflow_macro as unchanged for release -### Commit Statistics +### `hydroflow_datalog` Commit Statistics @@ -141,7 +175,7 @@ Unchanged from previous release. - 1 commit was understood as [conventional](https://www.conventionalcommits.org). - 0 issues like '(#ID)' were seen in commit messages -### Commit Details +### `hydroflow_datalog` Commit Details @@ -163,7 +197,7 @@ Unchanged from previous release. https://github.com/frewsxcv/rust-crates-index/issues/159 is messing with smart-release -### Commit Statistics +### `hydroflow_datalog` Commit Statistics @@ -172,7 +206,7 @@ Unchanged from previous release. - 1 commit was understood as [conventional](https://www.conventionalcommits.org). - 0 issues like '(#ID)' were seen in commit messages -### Commit Details +### `hydroflow_datalog` Commit Details @@ -191,7 +225,7 @@ Unchanged from previous release. - bump proc-macro2 min version to 1.0.63 -### Commit Statistics +### `hydroflow_datalog` Commit Statistics @@ -200,7 +234,7 @@ Unchanged from previous release. - 1 commit was understood as [conventional](https://www.conventionalcommits.org). - 0 issues like '(#ID)' were seen in commit messages -### Commit Details +### `hydroflow_datalog` Commit Details @@ -221,7 +255,7 @@ Unchanged from previous release. - mark hydro_datalog as unchanged for 0.4 release -### Commit Statistics +### `hydroflow_datalog` Commit Statistics @@ -230,7 +264,7 @@ Unchanged from previous release. - 1 commit was understood as [conventional](https://www.conventionalcommits.org). - 0 issues like '(#ID)' were seen in commit messages -### Commit Details +### `hydroflow_datalog` Commit Details @@ -254,7 +288,7 @@ Unchanged from previous release. *latest = 2023-06-28 or something -### Commit Statistics +### `hydroflow_datalog` Commit Statistics @@ -263,7 +297,7 @@ Unchanged from previous release. - 2 commits were understood as [conventional](https://www.conventionalcommits.org). - 2 unique issues were worked on: [#780](https://github.com/hydro-project/hydroflow/issues/780), [#801](https://github.com/hydro-project/hydroflow/issues/801) -### Commit Details +### `hydroflow_datalog` Commit Details @@ -285,7 +319,7 @@ Unchanged from previous release. - manually bump versions for v0.2.0 release -### Commit Statistics +### `hydroflow_datalog` Commit Statistics @@ -294,7 +328,7 @@ Unchanged from previous release. - 1 commit was understood as [conventional](https://www.conventionalcommits.org). - 0 issues like '(#ID)' were seen in commit messages -### Commit Details +### `hydroflow_datalog` Commit Details @@ -318,7 +352,7 @@ Unchanged from previous release. - Add rustdocs to hydroflow's proc macros -### Commit Statistics +### `hydroflow_datalog` Commit Statistics @@ -327,7 +361,7 @@ Unchanged from previous release. - 2 commits were understood as [conventional](https://www.conventionalcommits.org). - 2 unique issues were worked on: [#677](https://github.com/hydro-project/hydroflow/issues/677), [#684](https://github.com/hydro-project/hydroflow/issues/684) -### Commit Details +### `hydroflow_datalog` Commit Details @@ -349,7 +383,7 @@ Unchanged from previous release. - rustfmt group imports -### Commit Statistics +### `hydroflow_datalog` Commit Statistics @@ -358,7 +392,7 @@ Unchanged from previous release. - 1 commit was understood as [conventional](https://www.conventionalcommits.org). - 1 unique issue was worked on: [#660](https://github.com/hydro-project/hydroflow/issues/660) -### Commit Details +### `hydroflow_datalog` Commit Details @@ -372,7 +406,7 @@ Unchanged from previous release. ## 0.0.0 (2023-04-25) -### Commit Statistics +### `hydroflow_datalog` Commit Statistics @@ -380,7 +414,7 @@ Unchanged from previous release. - 0 commits were understood as [conventional](https://www.conventionalcommits.org). - 17 unique issues were worked on: [#155](https://github.com/hydro-project/hydroflow/issues/155), [#184](https://github.com/hydro-project/hydroflow/issues/184), [#187](https://github.com/hydro-project/hydroflow/issues/187), [#204](https://github.com/hydro-project/hydroflow/issues/204), [#223](https://github.com/hydro-project/hydroflow/issues/223), [#232](https://github.com/hydro-project/hydroflow/issues/232), [#284](https://github.com/hydro-project/hydroflow/issues/284), [#302](https://github.com/hydro-project/hydroflow/issues/302), [#320](https://github.com/hydro-project/hydroflow/issues/320), [#321](https://github.com/hydro-project/hydroflow/issues/321), [#329](https://github.com/hydro-project/hydroflow/issues/329), [#360](https://github.com/hydro-project/hydroflow/issues/360), [#371](https://github.com/hydro-project/hydroflow/issues/371), [#467](https://github.com/hydro-project/hydroflow/issues/467), [#518](https://github.com/hydro-project/hydroflow/issues/518), [#609](https://github.com/hydro-project/hydroflow/issues/609), [#617](https://github.com/hydro-project/hydroflow/issues/617) -### Commit Details +### `hydroflow_datalog` Commit Details diff --git a/dfir_datalog_core/CHANGELOG.md b/dfir_datalog_core/CHANGELOG.md index 64d92a4d7c41..e916a5f56e98 100644 --- a/dfir_datalog_core/CHANGELOG.md +++ b/dfir_datalog_core/CHANGELOG.md @@ -5,12 +5,67 @@ All notable changes to this project will be documented in this file. The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.0.0/), and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html). +## Unreleased + +### New Features + + - initial flo implementation + Basic first pass implementation, without changing the current scheduler, + no breaking behavior. Adds the flo syntax (`loop { ... }`), basic graph + structure checks, nested loop blocks, and two basic windowing operators + (`batch()` and `all_once()`). + + Next steps: + - Needs scheduler changes: + - Implement un-windowing operators + - Implement windowing operator `repeat_n()` - + - Pipeline flags: Needs checking of bounded vs unbounded for `batch()` + vs `all_once()` + - Needs checking that all inputs into a loop agree + - State type negotiations (for `batch()` and handoffs (?) - don't always + use `Vec`) + - Dag performance optimizations (lots of things) + +### Refactor (BREAKING) + + - use `cfg(nightly)` instead of feature, remove `-Z` flag, use `Diagnostic::try_emit` + Previous PR (#1587) website build did not work because `panic = "abort"` + is set on wasm, leading to aborts for `proc_macro2::Span::unwrap()` + calls. + + All tests except trybuild seem to pass on stable, WIP #1587 next + +### `hydroflow_datalog_core` Commit Statistics + + + + - 2 commits contributed to the release. + - 38 days passed between releases. + - 2 commits were understood as [conventional](https://www.conventionalcommits.org). + - 2 unique issues were worked on: [#1585](https://github.com/hydro-project/hydroflow/issues/1585), [#1606](https://github.com/hydro-project/hydroflow/issues/1606) + +### `hydroflow_datalog_core` Commit Details + + + +
view details + + * **[#1585](https://github.com/hydro-project/hydroflow/issues/1585)** + - Initial flo implementation ([`b8acd84`](https://github.com/hydro-project/hydroflow/commit/b8acd843bdbcfb445bf942e697447f6bf58a10da)) + * **[#1606](https://github.com/hydro-project/hydroflow/issues/1606)** + - Use `cfg(nightly)` instead of feature, remove `-Z` flag, use `Diagnostic::try_emit` ([`251b103`](https://github.com/hydro-project/hydroflow/commit/251b1039c71d45d3f86123dba1926026ded80824)) +
+ ## 0.10.0 (2024-11-08) + + + + ### Chore - update rust-sitter - The latest Rust Sitter drops the dependency on `tree-sitter-cli`, which + The latest Rust Sitter drops the dependency on `tree-sitter-cli`, which eliminates many transitive dependencies. - update pinned rust version, clippy lints, remove some dead code @@ -18,16 +73,16 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0 - fixes for latest nightly clippy -### Commit Statistics +### `hydroflow_datalog_core` Commit Statistics - - 3 commits contributed to the release. + - 4 commits contributed to the release. - 69 days passed between releases. - 3 commits were understood as [conventional](https://www.conventionalcommits.org). - 3 unique issues were worked on: [#1444](https://github.com/hydro-project/hydroflow/issues/1444), [#1495](https://github.com/hydro-project/hydroflow/issues/1495), [#1537](https://github.com/hydro-project/hydroflow/issues/1537) -### Commit Details +### `hydroflow_datalog_core` Commit Details @@ -39,6 +94,8 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0 - Update rust-sitter ([`4f6d400`](https://github.com/hydro-project/hydroflow/commit/4f6d400d0992594f8f12992fb6939378b12fadd6)) * **[#1537](https://github.com/hydro-project/hydroflow/issues/1537)** - Fixes for latest nightly clippy ([`8442d1b`](https://github.com/hydro-project/hydroflow/commit/8442d1b524621a9f8b43372a9c25991efb33c25e)) + * **Uncategorized** + - Release hydroflow_lang v0.10.0, hydroflow_datalog_core v0.10.0, hydroflow_datalog v0.10.0, hydroflow_deploy_integration v0.10.0, hydroflow_macro v0.10.0, lattices_macro v0.5.7, variadics v0.0.7, variadics_macro v0.5.5, lattices v0.5.8, multiplatform_test v0.3.0, pusherator v0.0.9, hydroflow v0.10.0, hydro_deploy v0.10.0, stageleft_macro v0.4.0, stageleft v0.5.0, stageleft_tool v0.4.0, hydroflow_plus v0.10.0, hydro_cli v0.10.0, safety bump 8 crates ([`dcd48fc`](https://github.com/hydro-project/hydroflow/commit/dcd48fc7ee805898d9b5ef0d082870e30615e95b)) ## 0.9.0 (2024-08-30) @@ -50,8 +107,8 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0 ### Chore - lower min dependency versions where possible, update `Cargo.lock` - Moved from #1418 - + Moved from #1418 + --------- - drop unused dependencies @@ -63,7 +120,7 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0 - cleanup handling of span locations #1268, workaround fix #729 -### Commit Statistics +### `hydroflow_datalog_core` Commit Statistics @@ -72,7 +129,7 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0 - 4 commits were understood as [conventional](https://www.conventionalcommits.org). - 4 unique issues were worked on: [#1417](https://github.com/hydro-project/hydroflow/issues/1417), [#1420](https://github.com/hydro-project/hydroflow/issues/1420), [#1423](https://github.com/hydro-project/hydroflow/issues/1423), [#1432](https://github.com/hydro-project/hydroflow/issues/1432) -### Commit Details +### `hydroflow_datalog_core` Commit Details @@ -97,29 +154,29 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0 ### New Features - add `#[derive(Lattice)]` derive macros, fix #1247 - This adds derive macros to allow user-created macros. Each field must be - a lattice. - - Example usage: - ```rust - struct MyLattice - where - KeySet: Collection, - Epoch: Ord, - { - keys: SetUnion, - epoch: Max, - } - ``` - - Uses `#[derive(Lattice)]` for the `lattices` library `Pair` lattice. + This adds derive macros to allow user-created macros. Each field must be + a lattice. + + Example usage: + ```rust + struct MyLattice + where + KeySet: Collection, + Epoch: Ord, + { + keys: SetUnion, + epoch: Max, + } + ``` + + Uses `#[derive(Lattice)]` for the `lattices` library `Pair` lattice. Also contains some cleanup in the `lattices` crate. ### Bug Fixes - add `add_state_tick` to state API, reset at end of each tick, fix #1298 - Option 2 of #1298 - + Option 2 of #1298 + * Main feature in the title is in `src/scheduled/{context,graph}.rs` * Codegen for some stateful operators (those which can be used as singletons, and some others) is changed to use the new API. * Add a test `test_cartesian_product_tick_state` for #1298 @@ -128,35 +185,35 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0 ### Bug Fixes (BREAKING) - remove singleton referencer `persist::<'static>()` insertion - Also enables singletons for `persist()` and ensures that only the - `'static` lifetime is used - - Singletons are supposed to act like `cross_join()`. I.e. if we have this - code: - ```rust - stream -> filter(|item| ... #y ...) -> ... - ``` - It should behave equivalently to - ```rust - stream -> cj[0]; - y -> cj[1]; - cj = cross_join() -> filter(|(item, y)| ...) -> ... - ``` - - This has a very unintuitive replaying behavior, if `y` receives multiple - updates: + Also enables singletons for `persist()` and ensures that only the + `'static` lifetime is used + + Singletons are supposed to act like `cross_join()`. I.e. if we have this + code: + ```rust + stream -> filter(|item| ... #y ...) -> ... + ``` + It should behave equivalently to + ```rust + stream -> cj[0]; + y -> cj[1]; + cj = cross_join() -> filter(|(item, y)| ...) -> ... + ``` + + This has a very unintuitive replaying behavior, if `y` receives multiple + updates: 1. `y` receives an item `10` 2. `stream` receives an item `20` 3. `(10, 20)` is emitted 4. `y` receives an item `11` -5. `(11, 20)` is emitted +5. `(11, 20)` is emitted In this case the item `20` gets emitted twice. ### Refactor (BREAKING) - require lifetime on `perist*()` operators -### Commit Statistics +### `hydroflow_datalog_core` Commit Statistics @@ -165,7 +222,7 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0 - 4 commits were understood as [conventional](https://www.conventionalcommits.org). - 4 unique issues were worked on: [#1250](https://github.com/hydro-project/hydroflow/issues/1250), [#1295](https://github.com/hydro-project/hydroflow/issues/1295), [#1300](https://github.com/hydro-project/hydroflow/issues/1300), [#1332](https://github.com/hydro-project/hydroflow/issues/1332) -### Commit Details +### `hydroflow_datalog_core` Commit Details @@ -192,7 +249,7 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0 ### Chore - expect custom config names to prevent warnings - See + See https://doc.rust-lang.org/nightly/cargo/reference/build-scripts.html#rustc-check-cfg - update Rust Sitter chore(hydroflow_datalog): update Rust Sitter @@ -209,7 +266,7 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0 - add detupling syntax and allow interleaving with flattening -### Commit Statistics +### `hydroflow_datalog_core` Commit Statistics @@ -218,7 +275,7 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0 - 5 commits were understood as [conventional](https://www.conventionalcommits.org). - 5 unique issues were worked on: [#1160](https://github.com/hydro-project/hydroflow/issues/1160), [#1166](https://github.com/hydro-project/hydroflow/issues/1166), [#1168](https://github.com/hydro-project/hydroflow/issues/1168), [#1176](https://github.com/hydro-project/hydroflow/issues/1176), [#1192](https://github.com/hydro-project/hydroflow/issues/1192) -### Commit Details +### `hydroflow_datalog_core` Commit Details @@ -249,21 +306,21 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0 - initial proof-of-concept for singletons - include subgraph ID in pivot_run codegen - add syntax for "splatting" columns - feat(hydroflow_datalog): add syntax for "splatting" columns - - The * operator can be used on the left-hand side of a rule to indicate - that each row should be duplicated into many rows with that column being - replaced by every value in iterable corresponding to the original + feat(hydroflow_datalog): add syntax for "splatting" columns + + The * operator can be used on the left-hand side of a rule to indicate + that each row should be duplicated into many rows with that column being + replaced by every value in iterable corresponding to the original expression. - add aggregation for collecting values into a vector - feat(hydroflow_datalog): add aggregation for collecting values into a + feat(hydroflow_datalog): add aggregation for collecting values into a vector ### Bug Fixes - leftover comments - fix(hydroflow_datalog): leftover comments - + fix(hydroflow_datalog): leftover comments + Merged too soon. - #1050 for `fold_keyed` @@ -272,7 +329,7 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0 - add `#[repr(transparent)]` to type guard codegen theoretically could help performance but is probably negligible -### Commit Statistics +### `hydroflow_datalog_core` Commit Statistics @@ -281,7 +338,7 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0 - 8 commits were understood as [conventional](https://www.conventionalcommits.org). - 7 unique issues were worked on: [#1086](https://github.com/hydro-project/hydroflow/issues/1086), [#1091](https://github.com/hydro-project/hydroflow/issues/1091), [#1094](https://github.com/hydro-project/hydroflow/issues/1094), [#1132](https://github.com/hydro-project/hydroflow/issues/1132), [#1135](https://github.com/hydro-project/hydroflow/issues/1135), [#1136](https://github.com/hydro-project/hydroflow/issues/1136), [#1137](https://github.com/hydro-project/hydroflow/issues/1137) -### Commit Details +### `hydroflow_datalog_core` Commit Details @@ -314,7 +371,7 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0 - fix imports -### Commit Statistics +### `hydroflow_datalog_core` Commit Statistics @@ -323,7 +380,7 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0 - 1 commit was understood as [conventional](https://www.conventionalcommits.org). - 0 issues like '(#ID)' were seen in commit messages -### Commit Details +### `hydroflow_datalog_core` Commit Details @@ -348,7 +405,7 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0 - ensure other graph methods handle edge types (#929) -### Commit Statistics +### `hydroflow_datalog_core` Commit Statistics @@ -357,7 +414,7 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0 - 2 commits were understood as [conventional](https://www.conventionalcommits.org). - 1 unique issue was worked on: [#1041](https://github.com/hydro-project/hydroflow/issues/1041) -### Commit Details +### `hydroflow_datalog_core` Commit Details @@ -395,10 +452,10 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0 ### Refactor - emit prologue code before all subgraph code - Before, prologue code would be emitted before its subgraph, resulting in + Before, prologue code would be emitted before its subgraph, resulting in interleaving between subgraphs. -### Commit Statistics +### `hydroflow_datalog_core` Commit Statistics @@ -407,7 +464,7 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0 - 6 commits were understood as [conventional](https://www.conventionalcommits.org). - 5 unique issues were worked on: [#1016](https://github.com/hydro-project/hydroflow/issues/1016), [#1023](https://github.com/hydro-project/hydroflow/issues/1023), [#1033](https://github.com/hydro-project/hydroflow/issues/1033), [#945](https://github.com/hydro-project/hydroflow/issues/945), [#989](https://github.com/hydro-project/hydroflow/issues/989) -### Commit Details +### `hydroflow_datalog_core` Commit Details @@ -455,7 +512,7 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0 - update snapshots -### Commit Statistics +### `hydroflow_datalog_core` Commit Statistics @@ -464,7 +521,7 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0 - 10 commits were understood as [conventional](https://www.conventionalcommits.org). - 7 unique issues were worked on: [#882](https://github.com/hydro-project/hydroflow/issues/882), [#893](https://github.com/hydro-project/hydroflow/issues/893), [#896](https://github.com/hydro-project/hydroflow/issues/896), [#898](https://github.com/hydro-project/hydroflow/issues/898), [#906](https://github.com/hydro-project/hydroflow/issues/906), [#924](https://github.com/hydro-project/hydroflow/issues/924), [#926](https://github.com/hydro-project/hydroflow/issues/926) -### Commit Details +### `hydroflow_datalog_core` Commit Details @@ -507,7 +564,7 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0 - unify antijoin and difference with set and multiset semantics * fix: unify antijoin and difference with set and multiset semantics -* fix: replay semantics for antijoin and difference now work +* fix: replay semantics for antijoin and difference now work also added cross_join_multiset * fix: enforce sort for tests of anti_join and difference using assert_eq * fix: advance __borrow_ident beyond the current tick to prevent replay loops @@ -523,7 +580,7 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0 - fix new clippy lints on latest nightly 1.73.0-nightly (db7ff98a7 2023-07-31) -### Commit Statistics +### `hydroflow_datalog_core` Commit Statistics @@ -532,7 +589,7 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0 - 5 commits were understood as [conventional](https://www.conventionalcommits.org). - 4 unique issues were worked on: [#833](https://github.com/hydro-project/hydroflow/issues/833), [#845](https://github.com/hydro-project/hydroflow/issues/845), [#870](https://github.com/hydro-project/hydroflow/issues/870), [#872](https://github.com/hydro-project/hydroflow/issues/872) -### Commit Details +### `hydroflow_datalog_core` Commit Details @@ -564,7 +621,7 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0 *latest = 2023-06-28 or something -### Commit Statistics +### `hydroflow_datalog_core` Commit Statistics @@ -573,7 +630,7 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0 - 2 commits were understood as [conventional](https://www.conventionalcommits.org). - 2 unique issues were worked on: [#780](https://github.com/hydro-project/hydroflow/issues/780), [#801](https://github.com/hydro-project/hydroflow/issues/801) -### Commit Details +### `hydroflow_datalog_core` Commit Details @@ -595,7 +652,7 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0 - manually bump versions for v0.2.0 release -### Commit Statistics +### `hydroflow_datalog_core` Commit Statistics @@ -604,7 +661,7 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0 - 1 commit was understood as [conventional](https://www.conventionalcommits.org). - 0 issues like '(#ID)' were seen in commit messages -### Commit Details +### `hydroflow_datalog_core` Commit Details @@ -641,7 +698,7 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0 * remove group_by -### Commit Statistics +### `hydroflow_datalog_core` Commit Statistics @@ -650,7 +707,7 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0 - 4 commits were understood as [conventional](https://www.conventionalcommits.org). - 4 unique issues were worked on: [#697](https://github.com/hydro-project/hydroflow/issues/697), [#702](https://github.com/hydro-project/hydroflow/issues/702), [#714](https://github.com/hydro-project/hydroflow/issues/714), [#716](https://github.com/hydro-project/hydroflow/issues/716) -### Commit Details +### `hydroflow_datalog_core` Commit Details @@ -682,7 +739,7 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0 - remove `hydroflow::lang` module, move `Clear`, `MonotonicMap` to `hydroflow::util` instead -### Commit Statistics +### `hydroflow_datalog_core` Commit Statistics @@ -691,7 +748,7 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0 - 2 commits were understood as [conventional](https://www.conventionalcommits.org). - 3 unique issues were worked on: [#673](https://github.com/hydro-project/hydroflow/issues/673), [#677](https://github.com/hydro-project/hydroflow/issues/677), [#684](https://github.com/hydro-project/hydroflow/issues/684) -### Commit Details +### `hydroflow_datalog_core` Commit Details @@ -719,7 +776,7 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0 - rustfmt group imports - rustfmt prescribe flat-module `use` format -### Commit Statistics +### `hydroflow_datalog_core` Commit Statistics @@ -728,7 +785,7 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0 - 3 commits were understood as [conventional](https://www.conventionalcommits.org). - 3 unique issues were worked on: [#639](https://github.com/hydro-project/hydroflow/issues/639), [#642](https://github.com/hydro-project/hydroflow/issues/642), [#660](https://github.com/hydro-project/hydroflow/issues/660) -### Commit Details +### `hydroflow_datalog_core` Commit Details @@ -748,7 +805,7 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0 ## 0.0.0 (2023-04-25) -### Commit Statistics +### `hydroflow_datalog_core` Commit Statistics @@ -756,7 +813,7 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0 - 0 commits were understood as [conventional](https://www.conventionalcommits.org). - 59 unique issues were worked on: [#209](https://github.com/hydro-project/hydroflow/issues/209), [#254](https://github.com/hydro-project/hydroflow/issues/254), [#329](https://github.com/hydro-project/hydroflow/issues/329), [#371](https://github.com/hydro-project/hydroflow/issues/371), [#374](https://github.com/hydro-project/hydroflow/issues/374), [#376](https://github.com/hydro-project/hydroflow/issues/376), [#383](https://github.com/hydro-project/hydroflow/issues/383), [#388](https://github.com/hydro-project/hydroflow/issues/388), [#403](https://github.com/hydro-project/hydroflow/issues/403), [#413](https://github.com/hydro-project/hydroflow/issues/413), [#419](https://github.com/hydro-project/hydroflow/issues/419), [#425](https://github.com/hydro-project/hydroflow/issues/425), [#431](https://github.com/hydro-project/hydroflow/issues/431), [#441 11/14](https://github.com/hydro-project/hydroflow/issues/441 11/14), [#441 12/14](https://github.com/hydro-project/hydroflow/issues/441 12/14), [#441 14/14](https://github.com/hydro-project/hydroflow/issues/441 14/14), [#442](https://github.com/hydro-project/hydroflow/issues/442), [#443](https://github.com/hydro-project/hydroflow/issues/443), [#444](https://github.com/hydro-project/hydroflow/issues/444), [#455](https://github.com/hydro-project/hydroflow/issues/455), [#459](https://github.com/hydro-project/hydroflow/issues/459), [#467](https://github.com/hydro-project/hydroflow/issues/467), [#469](https://github.com/hydro-project/hydroflow/issues/469), [#470](https://github.com/hydro-project/hydroflow/issues/470), [#475](https://github.com/hydro-project/hydroflow/issues/475), [#486](https://github.com/hydro-project/hydroflow/issues/486), [#499](https://github.com/hydro-project/hydroflow/issues/499), [#500](https://github.com/hydro-project/hydroflow/issues/500), [#501](https://github.com/hydro-project/hydroflow/issues/501), [#502](https://github.com/hydro-project/hydroflow/issues/502), [#505](https://github.com/hydro-project/hydroflow/issues/505), [#507](https://github.com/hydro-project/hydroflow/issues/507), [#509](https://github.com/hydro-project/hydroflow/issues/509), [#516](https://github.com/hydro-project/hydroflow/issues/516), [#518](https://github.com/hydro-project/hydroflow/issues/518), [#522](https://github.com/hydro-project/hydroflow/issues/522), [#540](https://github.com/hydro-project/hydroflow/issues/540), [#543](https://github.com/hydro-project/hydroflow/issues/543), [#547](https://github.com/hydro-project/hydroflow/issues/547), [#549](https://github.com/hydro-project/hydroflow/issues/549), [#551](https://github.com/hydro-project/hydroflow/issues/551), [#554](https://github.com/hydro-project/hydroflow/issues/554), [#555](https://github.com/hydro-project/hydroflow/issues/555), [#556](https://github.com/hydro-project/hydroflow/issues/556), [#558](https://github.com/hydro-project/hydroflow/issues/558), [#559](https://github.com/hydro-project/hydroflow/issues/559), [#565](https://github.com/hydro-project/hydroflow/issues/565), [#566](https://github.com/hydro-project/hydroflow/issues/566), [#567](https://github.com/hydro-project/hydroflow/issues/567), [#568](https://github.com/hydro-project/hydroflow/issues/568), [#571](https://github.com/hydro-project/hydroflow/issues/571), [#572](https://github.com/hydro-project/hydroflow/issues/572), [#573](https://github.com/hydro-project/hydroflow/issues/573), [#576](https://github.com/hydro-project/hydroflow/issues/576), [#598](https://github.com/hydro-project/hydroflow/issues/598), [#604](https://github.com/hydro-project/hydroflow/issues/604), [#609](https://github.com/hydro-project/hydroflow/issues/609), [#617](https://github.com/hydro-project/hydroflow/issues/617), [#618](https://github.com/hydro-project/hydroflow/issues/618) -### Commit Details +### `hydroflow_datalog_core` Commit Details diff --git a/dfir_lang/CHANGELOG.md b/dfir_lang/CHANGELOG.md index 5bf10ccccd08..009dd35ddc70 100644 --- a/dfir_lang/CHANGELOG.md +++ b/dfir_lang/CHANGELOG.md @@ -5,8 +5,93 @@ All notable changes to this project will be documented in this file. The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.0.0/), and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html). +## Unreleased + +### New Features + + - initial flo implementation + Basic first pass implementation, without changing the current scheduler, + no breaking behavior. Adds the flo syntax (`loop { ... }`), basic graph + structure checks, nested loop blocks, and two basic windowing operators + (`batch()` and `all_once()`). + + Next steps: + - Needs scheduler changes: + - Implement un-windowing operators + - Implement windowing operator `repeat_n()` - + - Pipeline flags: Needs checking of bounded vs unbounded for `batch()` + vs `all_once()` + - Needs checking that all inputs into a loop agree + - State type negotiations (for `batch()` and handoffs (?) - don't always + use `Vec`) + - Dag performance optimizations (lots of things) + +### Other + + - fix stable tests + Sets `TRYBUILD=overwrite` on stable, as the output messages will often + not match `pinned-nightly` + + Adds env `HYDROFLOW_EXPECT_WARNINGS=ignore` to allow warning tests to + pass + +### New Features (BREAKING) + + - remove `import!`, fix #1110 + in prep for rust stable #1587 + + No good way to resolve the source file paths on stable + + No way to get good diagnostics on external files in general, at all + +### Bug Fixes (BREAKING) + + - rename `union` to `chain` and restrict LHS to be bounded + Returning a `Stream` from `union` on unbounded streams was unsound, + since the order of outputs is not deterministic. + +### Refactor (BREAKING) + + - use `cfg(nightly)` instead of feature, remove `-Z` flag, use `Diagnostic::try_emit` + Previous PR (#1587) website build did not work because `panic = "abort"` + is set on wasm, leading to aborts for `proc_macro2::Span::unwrap()` + calls. + + All tests except trybuild seem to pass on stable, WIP #1587 next + +### `hydroflow_lang` Commit Statistics + + + + - 5 commits contributed to the release. + - 38 days passed between releases. + - 5 commits were understood as [conventional](https://www.conventionalcommits.org). + - 5 unique issues were worked on: [#1565](https://github.com/hydro-project/hydroflow/issues/1565), [#1585](https://github.com/hydro-project/hydroflow/issues/1585), [#1600](https://github.com/hydro-project/hydroflow/issues/1600), [#1606](https://github.com/hydro-project/hydroflow/issues/1606), [#1608](https://github.com/hydro-project/hydroflow/issues/1608) + +### `hydroflow_lang` Commit Details + + + +
view details + + * **[#1565](https://github.com/hydro-project/hydroflow/issues/1565)** + - Rename `union` to `chain` and restrict LHS to be bounded ([`eb1ad3a`](https://github.com/hydro-project/hydroflow/commit/eb1ad3a54705efb06ee3f0647deaa9a52731ae6e)) + * **[#1585](https://github.com/hydro-project/hydroflow/issues/1585)** + - Initial flo implementation ([`b8acd84`](https://github.com/hydro-project/hydroflow/commit/b8acd843bdbcfb445bf942e697447f6bf58a10da)) + * **[#1600](https://github.com/hydro-project/hydroflow/issues/1600)** + - Remove `import!`, fix #1110 ([`f2a4bee`](https://github.com/hydro-project/hydroflow/commit/f2a4bee8cd6945937bed5bc22fd85efd8d0aef0a)) + * **[#1606](https://github.com/hydro-project/hydroflow/issues/1606)** + - Use `cfg(nightly)` instead of feature, remove `-Z` flag, use `Diagnostic::try_emit` ([`251b103`](https://github.com/hydro-project/hydroflow/commit/251b1039c71d45d3f86123dba1926026ded80824)) + * **[#1608](https://github.com/hydro-project/hydroflow/issues/1608)** + - Fix stable tests ([`7dea92b`](https://github.com/hydro-project/hydroflow/commit/7dea92b19e6b86566cc27babb457415896b6b608)) +
+ ## 0.10.0 (2024-11-08) + + + + ### Chore - update pinned rust version, clippy lints, remove some dead code @@ -14,13 +99,13 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0 ### New Features - generalized hash trie indexes for relational tuples - Generalized Hash Tries are part of the SIGMOD '23 FreeJoin - [paper](https://dl.acm.org/doi/abs/10.1145/3589295) by - Wang/Willsey/Suciu. They provide a compressed ("factorized") - representation of relations. By operating in the factorized domain, join - algorithms can defer cross-products and achieve asymptotically optimal - performance. - + Generalized Hash Tries are part of the SIGMOD '23 FreeJoin + [paper](https://dl.acm.org/doi/abs/10.1145/3589295) by + Wang/Willsey/Suciu. They provide a compressed ("factorized") + representation of relations. By operating in the factorized domain, join + algorithms can defer cross-products and achieve asymptotically optimal + performance. + --------- - Added state_by operator. For https://github.com/hydro-project/hydroflow/issues/1467 @@ -28,10 +113,10 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0 ### Bug Fixes - `cross_singleton()` forgot value if multiple runs in a tick, fix #1518 - Adds the minimal reproducer test from @shadaj - - Note this may have negative performance implications, as the singleton value now is stored in the state API (heap) instead of locally. If we use singleton syntax this duplicate allocation could probably be avoided. - + Adds the minimal reproducer test from @shadaj + + Note this may have negative performance implications, as the singleton value now is stored in the state API (heap) instead of locally. If we use singleton syntax this duplicate allocation could probably be avoided. + > Confirmed that this fixed the bugs in our Paxos implementation, no noticeable performance impact. @shadj ### Refactor @@ -41,28 +126,28 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0 ### Style - fixes for nightly clippy - a couple few spurious `too_many_arguments` and a spurious - `zombie_processes` still on current nightly (`clippy 0.1.84 (4392847410 + a couple few spurious `too_many_arguments` and a spurious + `zombie_processes` still on current nightly (`clippy 0.1.84 (4392847410 2024-10-21)`) ### Bug Fixes (BREAKING) - fix #1401 `lattice_bimorphism()` double-emit, add docs - Fixes the issue by combining the all values generated per subgraph - execution into one, which effectively de-duplicates the values. - + Fixes the issue by combining the all values generated per subgraph + execution into one, which effectively de-duplicates the values. + Adds basic docs. -### Commit Statistics +### `hydroflow_lang` Commit Statistics - - 7 commits contributed to the release. + - 8 commits contributed to the release. - 69 days passed between releases. - 7 commits were understood as [conventional](https://www.conventionalcommits.org). - 7 unique issues were worked on: [#1444](https://github.com/hydro-project/hydroflow/issues/1444), [#1469](https://github.com/hydro-project/hydroflow/issues/1469), [#1503](https://github.com/hydro-project/hydroflow/issues/1503), [#1505](https://github.com/hydro-project/hydroflow/issues/1505), [#1512](https://github.com/hydro-project/hydroflow/issues/1512), [#1520](https://github.com/hydro-project/hydroflow/issues/1520), [#1522](https://github.com/hydro-project/hydroflow/issues/1522) -### Commit Details +### `hydroflow_lang` Commit Details @@ -82,6 +167,8 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0 - `cross_singleton()` forgot value if multiple runs in a tick, fix #1518 ([`16b730c`](https://github.com/hydro-project/hydroflow/commit/16b730c75cfca79ea5f869308b1e1e14b3e9c155)) * **[#1522](https://github.com/hydro-project/hydroflow/issues/1522)** - Fix #1401 `lattice_bimorphism()` double-emit, add docs ([`e796200`](https://github.com/hydro-project/hydroflow/commit/e796200743f2cc2da5a0e91c492f016ca98008e8)) + * **Uncategorized** + - Release hydroflow_lang v0.10.0, hydroflow_datalog_core v0.10.0, hydroflow_datalog v0.10.0, hydroflow_deploy_integration v0.10.0, hydroflow_macro v0.10.0, lattices_macro v0.5.7, variadics v0.0.7, variadics_macro v0.5.5, lattices v0.5.8, multiplatform_test v0.3.0, pusherator v0.0.9, hydroflow v0.10.0, hydro_deploy v0.10.0, stageleft_macro v0.4.0, stageleft v0.5.0, stageleft_tool v0.4.0, hydroflow_plus v0.10.0, hydro_cli v0.10.0, safety bump 8 crates ([`dcd48fc`](https://github.com/hydro-project/hydroflow/commit/dcd48fc7ee805898d9b5ef0d082870e30615e95b)) ## 0.9.0 (2024-08-30) @@ -93,8 +180,8 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0 ### Chore - lower min dependency versions where possible, update `Cargo.lock` - Moved from #1418 - + Moved from #1418 + --------- - drop unused dependencies @@ -106,9 +193,9 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0 - allow `demux_enum` to have any number of outputs, fix #1329 - add `cross_singleton` operator - This operator is necessary to eliminate performance bottlenecks in Paxos - where cross-products result in unnecessary cloning and lack of - short-circuit behavior results in values being cloned out of the + This operator is necessary to eliminate performance bottlenecks in Paxos + where cross-products result in unnecessary cloning and lack of + short-circuit behavior results in values being cloned out of the internal state of `reduce_keyed`. ### Bug Fixes @@ -125,7 +212,7 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0 - change `state()` to output a stream of only de-duplicated items -### Commit Statistics +### `hydroflow_lang` Commit Statistics @@ -134,7 +221,7 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0 - 10 commits were understood as [conventional](https://www.conventionalcommits.org). - 10 unique issues were worked on: [#1362](https://github.com/hydro-project/hydroflow/issues/1362), [#1373](https://github.com/hydro-project/hydroflow/issues/1373), [#1407](https://github.com/hydro-project/hydroflow/issues/1407), [#1409](https://github.com/hydro-project/hydroflow/issues/1409), [#1412](https://github.com/hydro-project/hydroflow/issues/1412), [#1417](https://github.com/hydro-project/hydroflow/issues/1417), [#1420](https://github.com/hydro-project/hydroflow/issues/1420), [#1423](https://github.com/hydro-project/hydroflow/issues/1423), [#1428](https://github.com/hydro-project/hydroflow/issues/1428), [#1432](https://github.com/hydro-project/hydroflow/issues/1432) -### Commit Details +### `hydroflow_lang` Commit Details @@ -182,19 +269,19 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0 - allow `ensure_singleton_referencers_succeed_persist` to ignore `identity`/`tee`/`union` operators, helps #1290 - Adds tests for `ensure_singleton_referencers_succeed_persist` persist - insertion behavior - - Fixes some cases of #1290, when just identity operators - (union/tee/identity) are between the singleton referencer and its - preceding flow state. - + Adds tests for `ensure_singleton_referencers_succeed_persist` persist + insertion behavior + + Fixes some cases of #1290, when just identity operators + (union/tee/identity) are between the singleton referencer and its + preceding flow state. + TODO track per-operator instead: #1311 - improve spanning of singleton substitution code, add compile-fail test, fix #1294 Another small spanning improvement - add `add_state_tick` to state API, reset at end of each tick, fix #1298 - Option 2 of #1298 - + Option 2 of #1298 + * Main feature in the title is in `src/scheduled/{context,graph}.rs` * Codegen for some stateful operators (those which can be used as singletons, and some others) is changed to use the new API. * Add a test `test_cartesian_product_tick_state` for #1298 @@ -203,24 +290,24 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0 ### Refactor - improve diagnostics for missing generic params - Improves diagnostic messages a bit for when no generic params are - supplied but some are expected. Previously this would span to the entire + Improves diagnostic messages a bit for when no generic params are + supplied but some are expected. Previously this would span to the entire macro invocation. - improve diagnostics by re-spanning `#root` - Inspired by fixing the spans in `demux_enum` in #1271 - * re-span `#root` to `op_span` for better diagnostics - * use better span `func.inputs` in `demux` and `demux_enum` - * clippy fixups in `source_json`, `source_stdin` + Inspired by fixing the spans in `demux_enum` in #1271 + * re-span `#root` to `op_span` for better diagnostics + * use better span `func.inputs` in `demux` and `demux_enum` + * clippy fixups in `source_json`, `source_stdin` * fix #1201 (for the most part) ### Reverted - "feat(hydroflow): Added poll_futures and poll_futures_async operators.", fix #1183 - This reverts commit 997d90a76db9a4e05dbac35073a09548750ce342. - - We have been trying to figure out the semantics a bit, and want to give - it more thought before we commit to maintaining it - + This reverts commit 997d90a76db9a4e05dbac35073a09548750ce342. + + We have been trying to figure out the semantics a bit, and want to give + it more thought before we commit to maintaining it + Can un-revert and adjust the semantics later when we use it ### Style @@ -230,35 +317,35 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0 ### Bug Fixes (BREAKING) - remove singleton referencer `persist::<'static>()` insertion - Also enables singletons for `persist()` and ensures that only the - `'static` lifetime is used - - Singletons are supposed to act like `cross_join()`. I.e. if we have this - code: - ```rust - stream -> filter(|item| ... #y ...) -> ... - ``` - It should behave equivalently to - ```rust - stream -> cj[0]; - y -> cj[1]; - cj = cross_join() -> filter(|(item, y)| ...) -> ... - ``` - - This has a very unintuitive replaying behavior, if `y` receives multiple - updates: + Also enables singletons for `persist()` and ensures that only the + `'static` lifetime is used + + Singletons are supposed to act like `cross_join()`. I.e. if we have this + code: + ```rust + stream -> filter(|item| ... #y ...) -> ... + ``` + It should behave equivalently to + ```rust + stream -> cj[0]; + y -> cj[1]; + cj = cross_join() -> filter(|(item, y)| ...) -> ... + ``` + + This has a very unintuitive replaying behavior, if `y` receives multiple + updates: 1. `y` receives an item `10` 2. `stream` receives an item `20` 3. `(10, 20)` is emitted 4. `y` receives an item `11` -5. `(11, 20)` is emitted +5. `(11, 20)` is emitted In this case the item `20` gets emitted twice. ### Refactor (BREAKING) - require lifetime on `perist*()` operators -### Commit Statistics +### `hydroflow_lang` Commit Statistics @@ -267,7 +354,7 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0 - 13 commits were understood as [conventional](https://www.conventionalcommits.org). - 13 unique issues were worked on: [#1143](https://github.com/hydro-project/hydroflow/issues/1143), [#1216](https://github.com/hydro-project/hydroflow/issues/1216), [#1260](https://github.com/hydro-project/hydroflow/issues/1260), [#1266](https://github.com/hydro-project/hydroflow/issues/1266), [#1271](https://github.com/hydro-project/hydroflow/issues/1271), [#1280](https://github.com/hydro-project/hydroflow/issues/1280), [#1285](https://github.com/hydro-project/hydroflow/issues/1285), [#1295](https://github.com/hydro-project/hydroflow/issues/1295), [#1296](https://github.com/hydro-project/hydroflow/issues/1296), [#1297](https://github.com/hydro-project/hydroflow/issues/1297), [#1300](https://github.com/hydro-project/hydroflow/issues/1300), [#1312](https://github.com/hydro-project/hydroflow/issues/1312), [#1332](https://github.com/hydro-project/hydroflow/issues/1332) -### Commit Details +### `hydroflow_lang` Commit Details @@ -321,7 +408,7 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0 ### Chore - expect custom config names to prevent warnings - See + See https://doc.rust-lang.org/nightly/cargo/reference/build-scripts.html#rustc-check-cfg - update pyo3, silence warnings in generated code @@ -366,7 +453,7 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0 - change `lattice_bimorphism` to take state via singleton arguments #969 #1058 - replace `state()` with `state_ref()` implementation -### Commit Statistics +### `hydroflow_lang` Commit Statistics @@ -375,7 +462,7 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0 - 21 commits were understood as [conventional](https://www.conventionalcommits.org). - 15 unique issues were worked on: [#1143](https://github.com/hydro-project/hydroflow/issues/1143), [#1152](https://github.com/hydro-project/hydroflow/issues/1152), [#1157](https://github.com/hydro-project/hydroflow/issues/1157), [#1159](https://github.com/hydro-project/hydroflow/issues/1159), [#1160](https://github.com/hydro-project/hydroflow/issues/1160), [#1167](https://github.com/hydro-project/hydroflow/issues/1167), [#1171](https://github.com/hydro-project/hydroflow/issues/1171), [#1176](https://github.com/hydro-project/hydroflow/issues/1176), [#1182](https://github.com/hydro-project/hydroflow/issues/1182), [#1190](https://github.com/hydro-project/hydroflow/issues/1190), [#1192](https://github.com/hydro-project/hydroflow/issues/1192), [#1193](https://github.com/hydro-project/hydroflow/issues/1193), [#1198](https://github.com/hydro-project/hydroflow/issues/1198), [#1204](https://github.com/hydro-project/hydroflow/issues/1204), [#1232](https://github.com/hydro-project/hydroflow/issues/1232) -### Commit Details +### `hydroflow_lang` Commit Details @@ -438,7 +525,7 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0 - fix singleton reference edges being backwards, fix #1147 - Fix handling of single-node push subgraphs -### Commit Statistics +### `hydroflow_lang` Commit Statistics @@ -447,7 +534,7 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0 - 5 commits were understood as [conventional](https://www.conventionalcommits.org). - 3 unique issues were worked on: [#1134](https://github.com/hydro-project/hydroflow/issues/1134), [#1148](https://github.com/hydro-project/hydroflow/issues/1148), [#1150](https://github.com/hydro-project/hydroflow/issues/1150) -### Commit Details +### `hydroflow_lang` Commit Details @@ -507,7 +594,7 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0 - qualified path cleanups for clippy -### Commit Statistics +### `hydroflow_lang` Commit Statistics @@ -516,7 +603,7 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0 - 17 commits were understood as [conventional](https://www.conventionalcommits.org). - 13 unique issues were worked on: [#1086](https://github.com/hydro-project/hydroflow/issues/1086), [#1087](https://github.com/hydro-project/hydroflow/issues/1087), [#1089](https://github.com/hydro-project/hydroflow/issues/1089), [#1090](https://github.com/hydro-project/hydroflow/issues/1090), [#1091](https://github.com/hydro-project/hydroflow/issues/1091), [#1109](https://github.com/hydro-project/hydroflow/issues/1109), [#1128](https://github.com/hydro-project/hydroflow/issues/1128), [#1130](https://github.com/hydro-project/hydroflow/issues/1130), [#1133](https://github.com/hydro-project/hydroflow/issues/1133), [#1136](https://github.com/hydro-project/hydroflow/issues/1136), [#1137](https://github.com/hydro-project/hydroflow/issues/1137), [#1145](https://github.com/hydro-project/hydroflow/issues/1145), [#1146](https://github.com/hydro-project/hydroflow/issues/1146) -### Commit Details +### `hydroflow_lang` Commit Details @@ -564,7 +651,7 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0 - use an IR before lowering to Hydroflow Makes it possible to write custom optimization passes. -### Commit Statistics +### `hydroflow_lang` Commit Statistics @@ -573,7 +660,7 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0 - 2 commits were understood as [conventional](https://www.conventionalcommits.org). - 2 unique issues were worked on: [#1061](https://github.com/hydro-project/hydroflow/issues/1061), [#1070](https://github.com/hydro-project/hydroflow/issues/1070) -### Commit Details +### `hydroflow_lang` Commit Details @@ -614,7 +701,7 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0 - ensure other graph methods handle edge types (#929) -### Commit Statistics +### `hydroflow_lang` Commit Statistics @@ -623,7 +710,7 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0 - 8 commits were understood as [conventional](https://www.conventionalcommits.org). - 3 unique issues were worked on: [#1041](https://github.com/hydro-project/hydroflow/issues/1041), [#1053](https://github.com/hydro-project/hydroflow/issues/1053), [#1055](https://github.com/hydro-project/hydroflow/issues/1055) -### Commit Details +### `hydroflow_lang` Commit Details @@ -672,8 +759,8 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0 Also fixes type inference issues with reduce the same way as we did for fold. - Add initial structure for by-reference edge types - add initial test using Hydro CLI from Hydroflow+ - This also required a change to Hydroflow core to make it possible to run the dataflow itself on a single thread (using a LocalSet), even if the surrounding runtime is not single-threaded (required to work around deadlocks because we can't use async APIs inside Hydroflow+). This requires us to spawn any Hydroflow tasks (only for `dest_sink` at the moment) right next to when we run the dataflow rather than when the Hydroflow graph is initialized. From a conceptual perspective, this seems _more right_, since now creating a Hydroflow program will not result in any actual tasks running. - + This also required a change to Hydroflow core to make it possible to run the dataflow itself on a single thread (using a LocalSet), even if the surrounding runtime is not single-threaded (required to work around deadlocks because we can't use async APIs inside Hydroflow+). This requires us to spawn any Hydroflow tasks (only for `dest_sink` at the moment) right next to when we run the dataflow rather than when the Hydroflow graph is initialized. From a conceptual perspective, this seems _more right_, since now creating a Hydroflow program will not result in any actual tasks running. + In the third PR of this series, I aim to add a new Hydroflow+ operator that will automate the setup of a `dest_sink`/`source_stream` pair that span nodes. - add a functional surface syntax using staging - add defer() which is the same as defer_tick() except that it is lazy @@ -701,7 +788,7 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0 - unify node coloring code - emit prologue code before all subgraph code - Before, prologue code would be emitted before its subgraph, resulting in + Before, prologue code would be emitted before its subgraph, resulting in interleaving between subgraphs. - remove old unused structured `FlowProperties` @@ -709,7 +796,7 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0 - fold takes initial value by closure rather than by value -### Commit Statistics +### `hydroflow_lang` Commit Statistics @@ -718,7 +805,7 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0 - 26 commits were understood as [conventional](https://www.conventionalcommits.org). - 20 unique issues were worked on: [#1005](https://github.com/hydro-project/hydroflow/issues/1005), [#1009](https://github.com/hydro-project/hydroflow/issues/1009), [#1016](https://github.com/hydro-project/hydroflow/issues/1016), [#1017](https://github.com/hydro-project/hydroflow/issues/1017), [#1021](https://github.com/hydro-project/hydroflow/issues/1021), [#1023](https://github.com/hydro-project/hydroflow/issues/1023), [#1026](https://github.com/hydro-project/hydroflow/issues/1026), [#1033](https://github.com/hydro-project/hydroflow/issues/1033), [#1036](https://github.com/hydro-project/hydroflow/issues/1036), [#1040](https://github.com/hydro-project/hydroflow/issues/1040), [#899](https://github.com/hydro-project/hydroflow/issues/899), [#945](https://github.com/hydro-project/hydroflow/issues/945), [#947](https://github.com/hydro-project/hydroflow/issues/947), [#948](https://github.com/hydro-project/hydroflow/issues/948), [#949](https://github.com/hydro-project/hydroflow/issues/949), [#950](https://github.com/hydro-project/hydroflow/issues/950), [#959](https://github.com/hydro-project/hydroflow/issues/959), [#960](https://github.com/hydro-project/hydroflow/issues/960), [#978](https://github.com/hydro-project/hydroflow/issues/978), [#989](https://github.com/hydro-project/hydroflow/issues/989) -### Commit Details +### `hydroflow_lang` Commit Details @@ -843,7 +930,7 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0 - fix demux error messages and add tests - clippy redundant `to_string()` in `print!` lints - Clean up degenerate subgraph error message for consistency - Makes the pinned and latest nightly version have the same stderr output + Makes the pinned and latest nightly version have the same stderr output for consistent testing. - separate internal compiler operators in docs name/category/sort order @@ -858,7 +945,7 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0 - Simplify graph printing code, add delta/cumul green edges, allow hiding of vars/subgraphs -### Commit Statistics +### `hydroflow_lang` Commit Statistics @@ -867,7 +954,7 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0 - 41 commits were understood as [conventional](https://www.conventionalcommits.org). - 14 unique issues were worked on: [#882](https://github.com/hydro-project/hydroflow/issues/882), [#883](https://github.com/hydro-project/hydroflow/issues/883), [#884](https://github.com/hydro-project/hydroflow/issues/884), [#892](https://github.com/hydro-project/hydroflow/issues/892), [#896](https://github.com/hydro-project/hydroflow/issues/896), [#898](https://github.com/hydro-project/hydroflow/issues/898), [#902](https://github.com/hydro-project/hydroflow/issues/902), [#906](https://github.com/hydro-project/hydroflow/issues/906), [#923](https://github.com/hydro-project/hydroflow/issues/923), [#924](https://github.com/hydro-project/hydroflow/issues/924), [#926](https://github.com/hydro-project/hydroflow/issues/926), [#932](https://github.com/hydro-project/hydroflow/issues/932), [#933](https://github.com/hydro-project/hydroflow/issues/933), [#935](https://github.com/hydro-project/hydroflow/issues/935) -### Commit Details +### `hydroflow_lang` Commit Details @@ -965,7 +1052,7 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0 - unify antijoin and difference with set and multiset semantics * fix: unify antijoin and difference with set and multiset semantics -* fix: replay semantics for antijoin and difference now work +* fix: replay semantics for antijoin and difference now work also added cross_join_multiset * fix: enforce sort for tests of anti_join and difference using assert_eq * fix: advance __borrow_ident beyond the current tick to prevent replay loops @@ -984,7 +1071,7 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0 * address comments * fix clippy -### Commit Statistics +### `hydroflow_lang` Commit Statistics @@ -993,7 +1080,7 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0 - 14 commits were understood as [conventional](https://www.conventionalcommits.org). - 16 unique issues were worked on: [#820](https://github.com/hydro-project/hydroflow/issues/820), [#821](https://github.com/hydro-project/hydroflow/issues/821), [#822](https://github.com/hydro-project/hydroflow/issues/822), [#823](https://github.com/hydro-project/hydroflow/issues/823), [#833](https://github.com/hydro-project/hydroflow/issues/833), [#835](https://github.com/hydro-project/hydroflow/issues/835), [#840](https://github.com/hydro-project/hydroflow/issues/840), [#843](https://github.com/hydro-project/hydroflow/issues/843), [#844](https://github.com/hydro-project/hydroflow/issues/844), [#845](https://github.com/hydro-project/hydroflow/issues/845), [#851](https://github.com/hydro-project/hydroflow/issues/851), [#853](https://github.com/hydro-project/hydroflow/issues/853), [#861](https://github.com/hydro-project/hydroflow/issues/861), [#870](https://github.com/hydro-project/hydroflow/issues/870), [#872](https://github.com/hydro-project/hydroflow/issues/872), [#873](https://github.com/hydro-project/hydroflow/issues/873) -### Commit Details +### `hydroflow_lang` Commit Details @@ -1095,7 +1182,7 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0 - Rename `ConvertFrom::from` -> `LatticeFrom::lattice_from` -### Commit Statistics +### `hydroflow_lang` Commit Statistics @@ -1104,7 +1191,7 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0 - 14 commits were understood as [conventional](https://www.conventionalcommits.org). - 15 unique issues were worked on: [#741](https://github.com/hydro-project/hydroflow/issues/741), [#765](https://github.com/hydro-project/hydroflow/issues/765), [#773](https://github.com/hydro-project/hydroflow/issues/773), [#774](https://github.com/hydro-project/hydroflow/issues/774), [#775](https://github.com/hydro-project/hydroflow/issues/775), [#778](https://github.com/hydro-project/hydroflow/issues/778), [#780](https://github.com/hydro-project/hydroflow/issues/780), [#784](https://github.com/hydro-project/hydroflow/issues/784), [#789](https://github.com/hydro-project/hydroflow/issues/789), [#792](https://github.com/hydro-project/hydroflow/issues/792), [#799](https://github.com/hydro-project/hydroflow/issues/799), [#801](https://github.com/hydro-project/hydroflow/issues/801), [#803](https://github.com/hydro-project/hydroflow/issues/803), [#804](https://github.com/hydro-project/hydroflow/issues/804), [#809](https://github.com/hydro-project/hydroflow/issues/809) -### Commit Details +### `hydroflow_lang` Commit Details @@ -1166,7 +1253,7 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0 - remove `hydroflow_internalmacro`, use `hydroflow_lang/build.rs` instead -### Commit Statistics +### `hydroflow_lang` Commit Statistics @@ -1175,7 +1262,7 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0 - 4 commits were understood as [conventional](https://www.conventionalcommits.org). - 2 unique issues were worked on: [#728](https://github.com/hydro-project/hydroflow/issues/728), [#730](https://github.com/hydro-project/hydroflow/issues/730) -### Commit Details +### `hydroflow_lang` Commit Details @@ -1236,7 +1323,7 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0 * remove group_by -### Commit Statistics +### `hydroflow_lang` Commit Statistics @@ -1245,7 +1332,7 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0 - 12 commits were understood as [conventional](https://www.conventionalcommits.org). - 8 unique issues were worked on: [#696](https://github.com/hydro-project/hydroflow/issues/696), [#697](https://github.com/hydro-project/hydroflow/issues/697), [#702](https://github.com/hydro-project/hydroflow/issues/702), [#704](https://github.com/hydro-project/hydroflow/issues/704), [#706](https://github.com/hydro-project/hydroflow/issues/706), [#714](https://github.com/hydro-project/hydroflow/issues/714), [#716](https://github.com/hydro-project/hydroflow/issues/716), [#719](https://github.com/hydro-project/hydroflow/issues/719) -### Commit Details +### `hydroflow_lang` Commit Details @@ -1289,7 +1376,7 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0 - remove `hydroflow::lang` module, move `Clear`, `MonotonicMap` to `hydroflow::util` instead -### Commit Statistics +### `hydroflow_lang` Commit Statistics @@ -1298,7 +1385,7 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0 - 2 commits were understood as [conventional](https://www.conventionalcommits.org). - 5 unique issues were worked on: [#661](https://github.com/hydro-project/hydroflow/issues/661), [#673](https://github.com/hydro-project/hydroflow/issues/673), [#676](https://github.com/hydro-project/hydroflow/issues/676), [#677](https://github.com/hydro-project/hydroflow/issues/677), [#684](https://github.com/hydro-project/hydroflow/issues/684) -### Commit Details +### `hydroflow_lang` Commit Details @@ -1337,7 +1424,7 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0 - rustfmt group imports - rustfmt prescribe flat-module `use` format -### Commit Statistics +### `hydroflow_lang` Commit Statistics @@ -1346,7 +1433,7 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0 - 5 commits were understood as [conventional](https://www.conventionalcommits.org). - 7 unique issues were worked on: [#638](https://github.com/hydro-project/hydroflow/issues/638), [#639](https://github.com/hydro-project/hydroflow/issues/639), [#642](https://github.com/hydro-project/hydroflow/issues/642), [#649](https://github.com/hydro-project/hydroflow/issues/649), [#654](https://github.com/hydro-project/hydroflow/issues/654), [#660](https://github.com/hydro-project/hydroflow/issues/660), [#667](https://github.com/hydro-project/hydroflow/issues/667) -### Commit Details +### `hydroflow_lang` Commit Details @@ -1385,7 +1472,7 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0 - :<'static> now replays #143 #364 - :<'static> now replays #143 #364 -### Commit Statistics +### `hydroflow_lang` Commit Statistics @@ -1393,7 +1480,7 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0 - 3 commits were understood as [conventional](https://www.conventionalcommits.org). - 102 unique issues were worked on: [#155](https://github.com/hydro-project/hydroflow/issues/155), [#162](https://github.com/hydro-project/hydroflow/issues/162), [#211](https://github.com/hydro-project/hydroflow/issues/211), [#213](https://github.com/hydro-project/hydroflow/issues/213), [#230](https://github.com/hydro-project/hydroflow/issues/230), [#236](https://github.com/hydro-project/hydroflow/issues/236), [#239](https://github.com/hydro-project/hydroflow/issues/239), [#245](https://github.com/hydro-project/hydroflow/issues/245), [#250](https://github.com/hydro-project/hydroflow/issues/250), [#259](https://github.com/hydro-project/hydroflow/issues/259), [#261](https://github.com/hydro-project/hydroflow/issues/261), [#277](https://github.com/hydro-project/hydroflow/issues/277), [#278](https://github.com/hydro-project/hydroflow/issues/278), [#282](https://github.com/hydro-project/hydroflow/issues/282), [#284](https://github.com/hydro-project/hydroflow/issues/284), [#285](https://github.com/hydro-project/hydroflow/issues/285), [#295](https://github.com/hydro-project/hydroflow/issues/295), [#296](https://github.com/hydro-project/hydroflow/issues/296), [#298](https://github.com/hydro-project/hydroflow/issues/298), [#301](https://github.com/hydro-project/hydroflow/issues/301), [#309](https://github.com/hydro-project/hydroflow/issues/309), [#311](https://github.com/hydro-project/hydroflow/issues/311), [#320](https://github.com/hydro-project/hydroflow/issues/320), [#321](https://github.com/hydro-project/hydroflow/issues/321), [#323](https://github.com/hydro-project/hydroflow/issues/323), [#329](https://github.com/hydro-project/hydroflow/issues/329), [#331](https://github.com/hydro-project/hydroflow/issues/331), [#334](https://github.com/hydro-project/hydroflow/issues/334), [#337](https://github.com/hydro-project/hydroflow/issues/337), [#338](https://github.com/hydro-project/hydroflow/issues/338), [#350](https://github.com/hydro-project/hydroflow/issues/350), [#360](https://github.com/hydro-project/hydroflow/issues/360), [#363](https://github.com/hydro-project/hydroflow/issues/363), [#381](https://github.com/hydro-project/hydroflow/issues/381), [#382](https://github.com/hydro-project/hydroflow/issues/382), [#383](https://github.com/hydro-project/hydroflow/issues/383), [#399](https://github.com/hydro-project/hydroflow/issues/399), [#404](https://github.com/hydro-project/hydroflow/issues/404), [#405](https://github.com/hydro-project/hydroflow/issues/405), [#412](https://github.com/hydro-project/hydroflow/issues/412), [#419](https://github.com/hydro-project/hydroflow/issues/419), [#425](https://github.com/hydro-project/hydroflow/issues/425), [#431](https://github.com/hydro-project/hydroflow/issues/431), [#434](https://github.com/hydro-project/hydroflow/issues/434), [#441 1/14](https://github.com/hydro-project/hydroflow/issues/441 1/14), [#441 10/14](https://github.com/hydro-project/hydroflow/issues/441 10/14), [#441 11/14](https://github.com/hydro-project/hydroflow/issues/441 11/14), [#441 12/14](https://github.com/hydro-project/hydroflow/issues/441 12/14), [#441 13/14](https://github.com/hydro-project/hydroflow/issues/441 13/14), [#441 14/14](https://github.com/hydro-project/hydroflow/issues/441 14/14), [#441 2/14](https://github.com/hydro-project/hydroflow/issues/441 2/14), [#441 3/14](https://github.com/hydro-project/hydroflow/issues/441 3/14), [#441 4/14](https://github.com/hydro-project/hydroflow/issues/441 4/14), [#441 5/14](https://github.com/hydro-project/hydroflow/issues/441 5/14), [#441 6/14](https://github.com/hydro-project/hydroflow/issues/441 6/14), [#441 7/14](https://github.com/hydro-project/hydroflow/issues/441 7/14), [#441 8/14](https://github.com/hydro-project/hydroflow/issues/441 8/14), [#441 9/14](https://github.com/hydro-project/hydroflow/issues/441 9/14), [#444](https://github.com/hydro-project/hydroflow/issues/444), [#445](https://github.com/hydro-project/hydroflow/issues/445), [#448 1/2](https://github.com/hydro-project/hydroflow/issues/448 1/2), [#455](https://github.com/hydro-project/hydroflow/issues/455), [#459](https://github.com/hydro-project/hydroflow/issues/459), [#465](https://github.com/hydro-project/hydroflow/issues/465), [#468](https://github.com/hydro-project/hydroflow/issues/468), [#471](https://github.com/hydro-project/hydroflow/issues/471), [#475](https://github.com/hydro-project/hydroflow/issues/475), [#488](https://github.com/hydro-project/hydroflow/issues/488), [#490](https://github.com/hydro-project/hydroflow/issues/490), [#491](https://github.com/hydro-project/hydroflow/issues/491), [#493](https://github.com/hydro-project/hydroflow/issues/493), [#497](https://github.com/hydro-project/hydroflow/issues/497), [#499](https://github.com/hydro-project/hydroflow/issues/499), [#501](https://github.com/hydro-project/hydroflow/issues/501), [#508](https://github.com/hydro-project/hydroflow/issues/508), [#509](https://github.com/hydro-project/hydroflow/issues/509), [#511](https://github.com/hydro-project/hydroflow/issues/511), [#512](https://github.com/hydro-project/hydroflow/issues/512), [#518](https://github.com/hydro-project/hydroflow/issues/518), [#523](https://github.com/hydro-project/hydroflow/issues/523), [#524](https://github.com/hydro-project/hydroflow/issues/524), [#526](https://github.com/hydro-project/hydroflow/issues/526), [#530](https://github.com/hydro-project/hydroflow/issues/530), [#538](https://github.com/hydro-project/hydroflow/issues/538), [#547](https://github.com/hydro-project/hydroflow/issues/547), [#550](https://github.com/hydro-project/hydroflow/issues/550), [#555](https://github.com/hydro-project/hydroflow/issues/555), [#556](https://github.com/hydro-project/hydroflow/issues/556), [#559](https://github.com/hydro-project/hydroflow/issues/559), [#566](https://github.com/hydro-project/hydroflow/issues/566), [#573](https://github.com/hydro-project/hydroflow/issues/573), [#579](https://github.com/hydro-project/hydroflow/issues/579), [#591](https://github.com/hydro-project/hydroflow/issues/591), [#598](https://github.com/hydro-project/hydroflow/issues/598), [#599](https://github.com/hydro-project/hydroflow/issues/599), [#602](https://github.com/hydro-project/hydroflow/issues/602), [#604](https://github.com/hydro-project/hydroflow/issues/604), [#605](https://github.com/hydro-project/hydroflow/issues/605), [#607](https://github.com/hydro-project/hydroflow/issues/607), [#609](https://github.com/hydro-project/hydroflow/issues/609), [#610](https://github.com/hydro-project/hydroflow/issues/610), [#617](https://github.com/hydro-project/hydroflow/issues/617) -### Commit Details +### `hydroflow_lang` Commit Details diff --git a/dfir_macro/CHANGELOG.md b/dfir_macro/CHANGELOG.md index 574b4e2cd328..f0d6cda0de6d 100644 --- a/dfir_macro/CHANGELOG.md +++ b/dfir_macro/CHANGELOG.md @@ -5,22 +5,73 @@ All notable changes to this project will be documented in this file. The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.0.0/), and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html). +## Unreleased + +### Chore + + - upgrade to Docusaurus v3 + Main breaking change is MDX parsing, which trips up on unescaped `<` in + the generated docs, so we have to adjust the generator logic. + +### New Features (BREAKING) + + - remove `import!`, fix #1110 + in prep for rust stable #1587 + + No good way to resolve the source file paths on stable + + No way to get good diagnostics on external files in general, at all + +### Refactor (BREAKING) + + - use `cfg(nightly)` instead of feature, remove `-Z` flag, use `Diagnostic::try_emit` + Previous PR (#1587) website build did not work because `panic = "abort"` + is set on wasm, leading to aborts for `proc_macro2::Span::unwrap()` + calls. + + All tests except trybuild seem to pass on stable, WIP #1587 next + +### `hydroflow_macro` Commit Statistics + + + + - 3 commits contributed to the release. + - 38 days passed between releases. + - 3 commits were understood as [conventional](https://www.conventionalcommits.org). + - 3 unique issues were worked on: [#1558](https://github.com/hydro-project/hydroflow/issues/1558), [#1600](https://github.com/hydro-project/hydroflow/issues/1600), [#1606](https://github.com/hydro-project/hydroflow/issues/1606) + +### `hydroflow_macro` Commit Details + + + +
view details + + * **[#1558](https://github.com/hydro-project/hydroflow/issues/1558)** + - Upgrade to Docusaurus v3 ([`c58f13c`](https://github.com/hydro-project/hydroflow/commit/c58f13cff39b838fa283fae2711501c8b7894ff4)) + * **[#1600](https://github.com/hydro-project/hydroflow/issues/1600)** + - Remove `import!`, fix #1110 ([`f2a4bee`](https://github.com/hydro-project/hydroflow/commit/f2a4bee8cd6945937bed5bc22fd85efd8d0aef0a)) + * **[#1606](https://github.com/hydro-project/hydroflow/issues/1606)** + - Use `cfg(nightly)` instead of feature, remove `-Z` flag, use `Diagnostic::try_emit` ([`251b103`](https://github.com/hydro-project/hydroflow/commit/251b1039c71d45d3f86123dba1926026ded80824)) +
+ ## 0.10.0 (2024-11-08) + + ### Chore - update pinned rust version, clippy lints, remove some dead code -### Commit Statistics +### `hydroflow_macro` Commit Statistics - - 1 commit contributed to the release. + - 2 commits contributed to the release. - 69 days passed between releases. - 1 commit was understood as [conventional](https://www.conventionalcommits.org). - 1 unique issue was worked on: [#1444](https://github.com/hydro-project/hydroflow/issues/1444) -### Commit Details +### `hydroflow_macro` Commit Details @@ -28,6 +79,8 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0 * **[#1444](https://github.com/hydro-project/hydroflow/issues/1444)** - Update pinned rust version, clippy lints, remove some dead code ([`d567760`](https://github.com/hydro-project/hydroflow/commit/d5677604e93c07a5392f4229af94a0b736eca382)) + * **Uncategorized** + - Release hydroflow_lang v0.10.0, hydroflow_datalog_core v0.10.0, hydroflow_datalog v0.10.0, hydroflow_deploy_integration v0.10.0, hydroflow_macro v0.10.0, lattices_macro v0.5.7, variadics v0.0.7, variadics_macro v0.5.5, lattices v0.5.8, multiplatform_test v0.3.0, pusherator v0.0.9, hydroflow v0.10.0, hydro_deploy v0.10.0, stageleft_macro v0.4.0, stageleft v0.5.0, stageleft_tool v0.4.0, hydroflow_plus v0.10.0, hydro_cli v0.10.0, safety bump 8 crates ([`dcd48fc`](https://github.com/hydro-project/hydroflow/commit/dcd48fc7ee805898d9b5ef0d082870e30615e95b)) ## 0.9.0 (2024-08-30) @@ -37,15 +90,15 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0 ### Chore - lower min dependency versions where possible, update `Cargo.lock` - Moved from #1418 - + Moved from #1418 + --------- ### New Features - allow `demux_enum` to have any number of outputs, fix #1329 -### Commit Statistics +### `hydroflow_macro` Commit Statistics @@ -54,7 +107,7 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0 - 2 commits were understood as [conventional](https://www.conventionalcommits.org). - 2 unique issues were worked on: [#1409](https://github.com/hydro-project/hydroflow/issues/1409), [#1423](https://github.com/hydro-project/hydroflow/issues/1423) -### Commit Details +### `hydroflow_macro` Commit Details @@ -78,7 +131,7 @@ Unchanged from previous release. - mark `hydroflow_datalog` and `hydroflow_macro` as unchanged for 0.8.0 release -### Commit Statistics +### `hydroflow_macro` Commit Statistics @@ -87,7 +140,7 @@ Unchanged from previous release. - 1 commit was understood as [conventional](https://www.conventionalcommits.org). - 0 issues like '(#ID)' were seen in commit messages -### Commit Details +### `hydroflow_macro` Commit Details @@ -106,7 +159,7 @@ Unchanged from previous release. - simplify `demux_enum()`, somewhat improves error messages #1201 -### Commit Statistics +### `hydroflow_macro` Commit Statistics @@ -115,7 +168,7 @@ Unchanged from previous release. - 1 commit was understood as [conventional](https://www.conventionalcommits.org). - 1 unique issue was worked on: [#1204](https://github.com/hydro-project/hydroflow/issues/1204) -### Commit Details +### `hydroflow_macro` Commit Details @@ -137,7 +190,7 @@ Unchanged from previous release. - mark hydroflow_datalog, hydroflow_macro as unchanged for release -### Commit Statistics +### `hydroflow_macro` Commit Statistics @@ -146,7 +199,7 @@ Unchanged from previous release. - 1 commit was understood as [conventional](https://www.conventionalcommits.org). - 0 issues like '(#ID)' were seen in commit messages -### Commit Details +### `hydroflow_macro` Commit Details @@ -164,7 +217,7 @@ Unchanged from previous release. - implement state operator (#929) -### Commit Statistics +### `hydroflow_macro` Commit Statistics @@ -173,7 +226,7 @@ Unchanged from previous release. - 1 commit was understood as [conventional](https://www.conventionalcommits.org). - 1 unique issue was worked on: [#1041](https://github.com/hydro-project/hydroflow/issues/1041) -### Commit Details +### `hydroflow_macro` Commit Details @@ -212,8 +265,10 @@ Unchanged from previous release. -- - new implementation and Hydro Deploy setup -- + - new implementation and Hydro Deploy setup + -- -### Commit Statistics +### `hydroflow_macro` Commit Statistics @@ -222,7 +277,7 @@ Unchanged from previous release. - 2 commits were understood as [conventional](https://www.conventionalcommits.org). - 1 unique issue was worked on: [#909](https://github.com/hydro-project/hydroflow/issues/909) -### Commit Details +### `hydroflow_macro` Commit Details @@ -267,7 +322,7 @@ Unchanged from previous release. - Simplify graph printing code, add delta/cumul green edges, allow hiding of vars/subgraphs -### Commit Statistics +### `hydroflow_macro` Commit Statistics @@ -276,7 +331,7 @@ Unchanged from previous release. - 9 commits were understood as [conventional](https://www.conventionalcommits.org). - 6 unique issues were worked on: [#881](https://github.com/hydro-project/hydroflow/issues/881), [#882](https://github.com/hydro-project/hydroflow/issues/882), [#884](https://github.com/hydro-project/hydroflow/issues/884), [#898](https://github.com/hydro-project/hydroflow/issues/898), [#932](https://github.com/hydro-project/hydroflow/issues/932), [#938](https://github.com/hydro-project/hydroflow/issues/938) -### Commit Details +### `hydroflow_macro` Commit Details @@ -309,7 +364,7 @@ Unchanged from previous release. - Add `use` statements to hydroflow syntax And use in doc tests. -### Commit Statistics +### `hydroflow_macro` Commit Statistics @@ -318,7 +373,7 @@ Unchanged from previous release. - 1 commit was understood as [conventional](https://www.conventionalcommits.org). - 1 unique issue was worked on: [#845](https://github.com/hydro-project/hydroflow/issues/845) -### Commit Details +### `hydroflow_macro` Commit Details @@ -348,7 +403,7 @@ Unchanged from previous release. *latest = 2023-06-28 or something -### Commit Statistics +### `hydroflow_macro` Commit Statistics @@ -357,7 +412,7 @@ Unchanged from previous release. - 4 commits were understood as [conventional](https://www.conventionalcommits.org). - 3 unique issues were worked on: [#758](https://github.com/hydro-project/hydroflow/issues/758), [#780](https://github.com/hydro-project/hydroflow/issues/780), [#801](https://github.com/hydro-project/hydroflow/issues/801) -### Commit Details +### `hydroflow_macro` Commit Details @@ -395,7 +450,7 @@ Unchanged from previous release. - remove `hydroflow_internalmacro`, use `hydroflow_lang/build.rs` instead -### Commit Statistics +### `hydroflow_macro` Commit Statistics @@ -404,7 +459,7 @@ Unchanged from previous release. - 4 commits were understood as [conventional](https://www.conventionalcommits.org). - 2 unique issues were worked on: [#728](https://github.com/hydro-project/hydroflow/issues/728), [#730](https://github.com/hydro-project/hydroflow/issues/730) -### Commit Details +### `hydroflow_macro` Commit Details @@ -427,7 +482,7 @@ Unchanged from previous release. - Add initial Hydro Deploy docs Renamed from Hydro CLI because the CLI isn't really the main thing. Also moves the Hydroflow docs to a subdirectory and sets up a dropdown for multiple docs. -### Commit Statistics +### `hydroflow_macro` Commit Statistics @@ -436,7 +491,7 @@ Unchanged from previous release. - 1 commit was understood as [conventional](https://www.conventionalcommits.org). - 1 unique issue was worked on: [#686](https://github.com/hydro-project/hydroflow/issues/686) -### Commit Details +### `hydroflow_macro` Commit Details @@ -461,7 +516,7 @@ Unchanged from previous release. - Add rustdocs to hydroflow's proc macros -### Commit Statistics +### `hydroflow_macro` Commit Statistics @@ -470,7 +525,7 @@ Unchanged from previous release. - 2 commits were understood as [conventional](https://www.conventionalcommits.org). - 4 unique issues were worked on: [#661](https://github.com/hydro-project/hydroflow/issues/661), [#671](https://github.com/hydro-project/hydroflow/issues/671), [#677](https://github.com/hydro-project/hydroflow/issues/677), [#684](https://github.com/hydro-project/hydroflow/issues/684) -### Commit Details +### `hydroflow_macro` Commit Details @@ -496,7 +551,7 @@ Unchanged from previous release. - rustfmt group imports -### Commit Statistics +### `hydroflow_macro` Commit Statistics @@ -505,7 +560,7 @@ Unchanged from previous release. - 1 commit was understood as [conventional](https://www.conventionalcommits.org). - 2 unique issues were worked on: [#643](https://github.com/hydro-project/hydroflow/issues/643), [#660](https://github.com/hydro-project/hydroflow/issues/660) -### Commit Details +### `hydroflow_macro` Commit Details @@ -521,7 +576,7 @@ Unchanged from previous release. ## 0.0.0 (2023-04-25) -### Commit Statistics +### `hydroflow_macro` Commit Statistics @@ -529,7 +584,7 @@ Unchanged from previous release. - 0 commits were understood as [conventional](https://www.conventionalcommits.org). - 13 unique issues were worked on: [#162](https://github.com/hydro-project/hydroflow/issues/162), [#310](https://github.com/hydro-project/hydroflow/issues/310), [#311](https://github.com/hydro-project/hydroflow/issues/311), [#318](https://github.com/hydro-project/hydroflow/issues/318), [#329](https://github.com/hydro-project/hydroflow/issues/329), [#404](https://github.com/hydro-project/hydroflow/issues/404), [#419](https://github.com/hydro-project/hydroflow/issues/419), [#441 11/14](https://github.com/hydro-project/hydroflow/issues/441 11/14), [#441 14/14](https://github.com/hydro-project/hydroflow/issues/441 14/14), [#501](https://github.com/hydro-project/hydroflow/issues/501), [#603](https://github.com/hydro-project/hydroflow/issues/603), [#609](https://github.com/hydro-project/hydroflow/issues/609), [#617](https://github.com/hydro-project/hydroflow/issues/617) -### Commit Details +### `hydroflow_macro` Commit Details diff --git a/dfir_rs/CHANGELOG.md b/dfir_rs/CHANGELOG.md index 714c405bee85..2954f86104dd 100644 --- a/dfir_rs/CHANGELOG.md +++ b/dfir_rs/CHANGELOG.md @@ -5,8 +5,118 @@ All notable changes to this project will be documented in this file. The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.0.0/), and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html). +## Unreleased + +### Documentation + + - `getLines` support named section markers, fix #1320 + +### New Features + + - initial flo implementation + Basic first pass implementation, without changing the current scheduler, + no breaking behavior. Adds the flo syntax (`loop { ... }`), basic graph + structure checks, nested loop blocks, and two basic windowing operators + (`batch()` and `all_once()`). + + Next steps: + - Needs scheduler changes: + - Implement un-windowing operators + - Implement windowing operator `repeat_n()` - + - Pipeline flags: Needs checking of bounded vs unbounded for `batch()` + vs `all_once()` + - Needs checking that all inputs into a loop agree + - State type negotiations (for `batch()` and handoffs (?) - don't always + use `Vec`) + - Dag performance optimizations (lots of things) + +### Bug Fixes + + - avoid pulling in `clap` as a dependency + We only need it for examples, so this reduces compilation burden for + `trybuild`. + +### Other + + - fix stable tests + Sets `TRYBUILD=overwrite` on stable, as the output messages will often + not match `pinned-nightly` + + Adds env `HYDROFLOW_EXPECT_WARNINGS=ignore` to allow warning tests to + pass + +### Refactor + + - rename `two_pc` -> `two_pc_hf` for collision with hf+ `two_pc` example + - use `match_box` macro to compile on stable + - move rest of scheduler state into `Context` + +### New Features (BREAKING) + + - minimize dependencies pulled into `trybuild` builds + We don't need `hydroflow_lang` in the runtime builds, do some feature + flagging to avoid it. + - remove `import!`, fix #1110 + in prep for rust stable #1587 + + No good way to resolve the source file paths on stable + + No way to get good diagnostics on external files in general, at all + +### Refactor (BREAKING) + + - use `cfg(nightly)` instead of feature, remove `-Z` flag, use `Diagnostic::try_emit` + Previous PR (#1587) website build did not work because `panic = "abort"` + is set on wasm, leading to aborts for `proc_macro2::Span::unwrap()` + calls. + + All tests except trybuild seem to pass on stable, WIP #1587 next + +### `hydroflow` Commit Statistics + + + + - 10 commits contributed to the release. + - 38 days passed between releases. + - 10 commits were understood as [conventional](https://www.conventionalcommits.org). + - 10 unique issues were worked on: [#1555](https://github.com/hydro-project/hydroflow/issues/1555), [#1578](https://github.com/hydro-project/hydroflow/issues/1578), [#1585](https://github.com/hydro-project/hydroflow/issues/1585), [#1594](https://github.com/hydro-project/hydroflow/issues/1594), [#1597](https://github.com/hydro-project/hydroflow/issues/1597), [#1600](https://github.com/hydro-project/hydroflow/issues/1600), [#1601](https://github.com/hydro-project/hydroflow/issues/1601), [#1606](https://github.com/hydro-project/hydroflow/issues/1606), [#1608](https://github.com/hydro-project/hydroflow/issues/1608), [#1611](https://github.com/hydro-project/hydroflow/issues/1611) + +### `hydroflow` Commit Details + + + +
view details + + * **[#1555](https://github.com/hydro-project/hydroflow/issues/1555)** + - `getLines` support named section markers, fix #1320 ([`5c53e12`](https://github.com/hydro-project/hydroflow/commit/5c53e12d6845f60c95248164eea3e2c48a8ff270)) + * **[#1578](https://github.com/hydro-project/hydroflow/issues/1578)** + - Avoid pulling in `clap` as a dependency ([`64578f3`](https://github.com/hydro-project/hydroflow/commit/64578f39d6f87e3a96fe4c3a8599b92606e1510e)) + * **[#1585](https://github.com/hydro-project/hydroflow/issues/1585)** + - Initial flo implementation ([`b8acd84`](https://github.com/hydro-project/hydroflow/commit/b8acd843bdbcfb445bf942e697447f6bf58a10da)) + * **[#1594](https://github.com/hydro-project/hydroflow/issues/1594)** + - Move rest of scheduler state into `Context` ([`e440623`](https://github.com/hydro-project/hydroflow/commit/e440623a553f28c16ac0f87871c135aaefeec7fd)) + * **[#1597](https://github.com/hydro-project/hydroflow/issues/1597)** + - Use `match_box` macro to compile on stable ([`9ea5f06`](https://github.com/hydro-project/hydroflow/commit/9ea5f061ee0f116caf8fc4ea99b62a9c7691be2a)) + * **[#1600](https://github.com/hydro-project/hydroflow/issues/1600)** + - Remove `import!`, fix #1110 ([`f2a4bee`](https://github.com/hydro-project/hydroflow/commit/f2a4bee8cd6945937bed5bc22fd85efd8d0aef0a)) + * **[#1601](https://github.com/hydro-project/hydroflow/issues/1601)** + - Rename `two_pc` -> `two_pc_hf` for collision with hf+ `two_pc` example ([`33eed2f`](https://github.com/hydro-project/hydroflow/commit/33eed2f8edf56c2914afa536167ef895d107277d)) + * **[#1606](https://github.com/hydro-project/hydroflow/issues/1606)** + - Use `cfg(nightly)` instead of feature, remove `-Z` flag, use `Diagnostic::try_emit` ([`251b103`](https://github.com/hydro-project/hydroflow/commit/251b1039c71d45d3f86123dba1926026ded80824)) + * **[#1608](https://github.com/hydro-project/hydroflow/issues/1608)** + - Fix stable tests ([`7dea92b`](https://github.com/hydro-project/hydroflow/commit/7dea92b19e6b86566cc27babb457415896b6b608)) + * **[#1611](https://github.com/hydro-project/hydroflow/issues/1611)** + - Minimize dependencies pulled into `trybuild` builds ([`c65b4c4`](https://github.com/hydro-project/hydroflow/commit/c65b4c49f3a3f2dcc7c8c28d1871b88c6c954822)) +
+ ## 0.10.0 (2024-11-08) + + + + + + ### Chore - update `proc-macro2`, fixes span info, fix #729 @@ -15,22 +125,22 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0 ### New Features - generalized hash trie indexes for relational tuples - Generalized Hash Tries are part of the SIGMOD '23 FreeJoin - [paper](https://dl.acm.org/doi/abs/10.1145/3589295) by - Wang/Willsey/Suciu. They provide a compressed ("factorized") - representation of relations. By operating in the factorized domain, join - algorithms can defer cross-products and achieve asymptotically optimal - performance. - + Generalized Hash Tries are part of the SIGMOD '23 FreeJoin + [paper](https://dl.acm.org/doi/abs/10.1145/3589295) by + Wang/Willsey/Suciu. They provide a compressed ("factorized") + representation of relations. By operating in the factorized domain, join + algorithms can defer cross-products and achieve asymptotically optimal + performance. + --------- ### Bug Fixes - `cross_singleton()` forgot value if multiple runs in a tick, fix #1518 - Adds the minimal reproducer test from @shadaj - - Note this may have negative performance implications, as the singleton value now is stored in the state API (heap) instead of locally. If we use singleton syntax this duplicate allocation could probably be avoided. - + Adds the minimal reproducer test from @shadaj + + Note this may have negative performance implications, as the singleton value now is stored in the state API (heap) instead of locally. If we use singleton syntax this duplicate allocation could probably be avoided. + > Confirmed that this fixed the bugs in our Paxos implementation, no noticeable performance impact. @shadj - cleanup temp tcp networking code, fix race condition fix #1458 consolidate into one task to prevent races @@ -39,8 +149,8 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0 - fixes for latest nightly clippy - fixes for nightly clippy - a couple few spurious `too_many_arguments` and a spurious - `zombie_processes` still on current nightly (`clippy 0.1.84 (4392847410 + a couple few spurious `too_many_arguments` and a spurious + `zombie_processes` still on current nightly (`clippy 0.1.84 (4392847410 2024-10-21)`) ### Test @@ -50,21 +160,21 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0 ### Bug Fixes (BREAKING) - fix #1401 `lattice_bimorphism()` double-emit, add docs - Fixes the issue by combining the all values generated per subgraph - execution into one, which effectively de-duplicates the values. - + Fixes the issue by combining the all values generated per subgraph + execution into one, which effectively de-duplicates the values. + Adds basic docs. -### Commit Statistics +### `hydroflow` Commit Statistics - - 9 commits contributed to the release. + - 10 commits contributed to the release. - 69 days passed between releases. - 9 commits were understood as [conventional](https://www.conventionalcommits.org). - 8 unique issues were worked on: [#1444](https://github.com/hydro-project/hydroflow/issues/1444), [#1446](https://github.com/hydro-project/hydroflow/issues/1446), [#1497](https://github.com/hydro-project/hydroflow/issues/1497), [#1503](https://github.com/hydro-project/hydroflow/issues/1503), [#1505](https://github.com/hydro-project/hydroflow/issues/1505), [#1520](https://github.com/hydro-project/hydroflow/issues/1520), [#1522](https://github.com/hydro-project/hydroflow/issues/1522), [#1537](https://github.com/hydro-project/hydroflow/issues/1537) -### Commit Details +### `hydroflow` Commit Details @@ -87,6 +197,7 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0 * **[#1537](https://github.com/hydro-project/hydroflow/issues/1537)** - Fixes for latest nightly clippy ([`8442d1b`](https://github.com/hydro-project/hydroflow/commit/8442d1b524621a9f8b43372a9c25991efb33c25e)) * **Uncategorized** + - Release hydroflow_lang v0.10.0, hydroflow_datalog_core v0.10.0, hydroflow_datalog v0.10.0, hydroflow_deploy_integration v0.10.0, hydroflow_macro v0.10.0, lattices_macro v0.5.7, variadics v0.0.7, variadics_macro v0.5.5, lattices v0.5.8, multiplatform_test v0.3.0, pusherator v0.0.9, hydroflow v0.10.0, hydro_deploy v0.10.0, stageleft_macro v0.4.0, stageleft v0.5.0, stageleft_tool v0.4.0, hydroflow_plus v0.10.0, hydro_cli v0.10.0, safety bump 8 crates ([`dcd48fc`](https://github.com/hydro-project/hydroflow/commit/dcd48fc7ee805898d9b5ef0d082870e30615e95b)) - Ignore trybuild tests inconsistent on latest nightly ([`656ee32`](https://github.com/hydro-project/hydroflow/commit/656ee328c8710bce7370c851437a80ca3db46a5a)) @@ -103,8 +214,8 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0 - manually set versions for crates renamed in #1413 - lower min dependency versions where possible, update `Cargo.lock` - Moved from #1418 - + Moved from #1418 + --------- - drop unused dependencies @@ -118,9 +229,9 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0 - Added simulation framework. Will move it out of the blind gossip PR next. - add `cross_singleton` operator - This operator is necessary to eliminate performance bottlenecks in Paxos - where cross-products result in unnecessary cloning and lack of - short-circuit behavior results in values being cloned out of the + This operator is necessary to eliminate performance bottlenecks in Paxos + where cross-products result in unnecessary cloning and lack of + short-circuit behavior results in values being cloned out of the internal state of `reduce_keyed`. ### Bug Fixes @@ -132,18 +243,18 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0 - rename integration crates to drop CLI references - `Deployment.stop()` for graceful shutdown including updated `perf` profile downloading - * `perf` profile downloading moved from the `drop()` impl to `async fn - stop()` - * download perf data via stdout - * update async-ssh2-lite to 0.5 to cleanup tokio compat issues - + * `perf` profile downloading moved from the `drop()` impl to `async fn + stop()` + * download perf data via stdout + * update async-ssh2-lite to 0.5 to cleanup tokio compat issues + WIP for #1365 ### Style (BREAKING) - rename some `CLI`->`Deploy`, decapitalize acronym names -### Commit Statistics +### `hydroflow` Commit Statistics @@ -152,7 +263,7 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0 - 12 commits were understood as [conventional](https://www.conventionalcommits.org). - 11 unique issues were worked on: [#1370](https://github.com/hydro-project/hydroflow/issues/1370), [#1373](https://github.com/hydro-project/hydroflow/issues/1373), [#1399](https://github.com/hydro-project/hydroflow/issues/1399), [#1407](https://github.com/hydro-project/hydroflow/issues/1407), [#1409](https://github.com/hydro-project/hydroflow/issues/1409), [#1413](https://github.com/hydro-project/hydroflow/issues/1413), [#1416](https://github.com/hydro-project/hydroflow/issues/1416), [#1417](https://github.com/hydro-project/hydroflow/issues/1417), [#1420](https://github.com/hydro-project/hydroflow/issues/1420), [#1423](https://github.com/hydro-project/hydroflow/issues/1423), [#1428](https://github.com/hydro-project/hydroflow/issues/1428) -### Commit Details +### `hydroflow` Commit Details @@ -210,19 +321,19 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0 - allow `ensure_singleton_referencers_succeed_persist` to ignore `identity`/`tee`/`union` operators, helps #1290 - Adds tests for `ensure_singleton_referencers_succeed_persist` persist - insertion behavior - - Fixes some cases of #1290, when just identity operators - (union/tee/identity) are between the singleton referencer and its - preceding flow state. - + Adds tests for `ensure_singleton_referencers_succeed_persist` persist + insertion behavior + + Fixes some cases of #1290, when just identity operators + (union/tee/identity) are between the singleton referencer and its + preceding flow state. + TODO track per-operator instead: #1311 - improve spanning of singleton substitution code, add compile-fail test, fix #1294 Another small spanning improvement - add `add_state_tick` to state API, reset at end of each tick, fix #1298 - Option 2 of #1298 - + Option 2 of #1298 + * Main feature in the title is in `src/scheduled/{context,graph}.rs` * Codegen for some stateful operators (those which can be used as singletons, and some others) is changed to use the new API. * Add a test `test_cartesian_product_tick_state` for #1298 @@ -231,24 +342,24 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0 ### Refactor - improve diagnostics for missing generic params - Improves diagnostic messages a bit for when no generic params are - supplied but some are expected. Previously this would span to the entire + Improves diagnostic messages a bit for when no generic params are + supplied but some are expected. Previously this would span to the entire macro invocation. - improve diagnostics by re-spanning `#root` - Inspired by fixing the spans in `demux_enum` in #1271 - * re-span `#root` to `op_span` for better diagnostics - * use better span `func.inputs` in `demux` and `demux_enum` - * clippy fixups in `source_json`, `source_stdin` + Inspired by fixing the spans in `demux_enum` in #1271 + * re-span `#root` to `op_span` for better diagnostics + * use better span `func.inputs` in `demux` and `demux_enum` + * clippy fixups in `source_json`, `source_stdin` * fix #1201 (for the most part) ### Reverted - "feat(hydroflow): Added poll_futures and poll_futures_async operators.", fix #1183 - This reverts commit 997d90a76db9a4e05dbac35073a09548750ce342. - - We have been trying to figure out the semantics a bit, and want to give - it more thought before we commit to maintaining it - + This reverts commit 997d90a76db9a4e05dbac35073a09548750ce342. + + We have been trying to figure out the semantics a bit, and want to give + it more thought before we commit to maintaining it + Can un-revert and adjust the semantics later when we use it ### Test @@ -258,28 +369,28 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0 ### Bug Fixes (BREAKING) - remove singleton referencer `persist::<'static>()` insertion - Also enables singletons for `persist()` and ensures that only the - `'static` lifetime is used - - Singletons are supposed to act like `cross_join()`. I.e. if we have this - code: - ```rust - stream -> filter(|item| ... #y ...) -> ... - ``` - It should behave equivalently to - ```rust - stream -> cj[0]; - y -> cj[1]; - cj = cross_join() -> filter(|(item, y)| ...) -> ... - ``` - - This has a very unintuitive replaying behavior, if `y` receives multiple - updates: + Also enables singletons for `persist()` and ensures that only the + `'static` lifetime is used + + Singletons are supposed to act like `cross_join()`. I.e. if we have this + code: + ```rust + stream -> filter(|item| ... #y ...) -> ... + ``` + It should behave equivalently to + ```rust + stream -> cj[0]; + y -> cj[1]; + cj = cross_join() -> filter(|(item, y)| ...) -> ... + ``` + + This has a very unintuitive replaying behavior, if `y` receives multiple + updates: 1. `y` receives an item `10` 2. `stream` receives an item `20` 3. `(10, 20)` is emitted 4. `y` receives an item `11` -5. `(11, 20)` is emitted +5. `(11, 20)` is emitted In this case the item `20` gets emitted twice. ### Refactor (BREAKING) @@ -289,11 +400,11 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0 ### Style (BREAKING) - enable clippy `upper-case-acronyms-aggressive` - * rename `GCP` -> `Gcp`, `NodeID` -> `NodeId` - * update CI `cargo-generate` template testing to use PR's branch instead + * rename `GCP` -> `Gcp`, `NodeID` -> `NodeId` + * update CI `cargo-generate` template testing to use PR's branch instead of whatever `main` happens to be -### Commit Statistics +### `hydroflow` Commit Statistics @@ -302,7 +413,7 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0 - 16 commits were understood as [conventional](https://www.conventionalcommits.org). - 17 unique issues were worked on: [#1143](https://github.com/hydro-project/hydroflow/issues/1143), [#1216](https://github.com/hydro-project/hydroflow/issues/1216), [#1244](https://github.com/hydro-project/hydroflow/issues/1244), [#1260](https://github.com/hydro-project/hydroflow/issues/1260), [#1271](https://github.com/hydro-project/hydroflow/issues/1271), [#1273](https://github.com/hydro-project/hydroflow/issues/1273), [#1274](https://github.com/hydro-project/hydroflow/issues/1274), [#1280](https://github.com/hydro-project/hydroflow/issues/1280), [#1283](https://github.com/hydro-project/hydroflow/issues/1283), [#1295](https://github.com/hydro-project/hydroflow/issues/1295), [#1296](https://github.com/hydro-project/hydroflow/issues/1296), [#1297](https://github.com/hydro-project/hydroflow/issues/1297), [#1300](https://github.com/hydro-project/hydroflow/issues/1300), [#1309](https://github.com/hydro-project/hydroflow/issues/1309), [#1312](https://github.com/hydro-project/hydroflow/issues/1312), [#1332](https://github.com/hydro-project/hydroflow/issues/1332), [#1345](https://github.com/hydro-project/hydroflow/issues/1345) -### Commit Details +### `hydroflow` Commit Details @@ -367,7 +478,7 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0 - use workaround for `cargo smart-release` not properly ordering `dev-`/`build-dependencies` - expect custom config names to prevent warnings - See + See https://doc.rust-lang.org/nightly/cargo/reference/build-scripts.html#rustc-check-cfg - update pyo3, silence warnings in generated code @@ -423,7 +534,7 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0 - change `lattice_bimorphism` to take state via singleton arguments #969 #1058 - replace `state()` with `state_ref()` implementation -### Commit Statistics +### `hydroflow` Commit Statistics @@ -432,7 +543,7 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0 - 27 commits were understood as [conventional](https://www.conventionalcommits.org). - 23 unique issues were worked on: [#1120](https://github.com/hydro-project/hydroflow/issues/1120), [#1143](https://github.com/hydro-project/hydroflow/issues/1143), [#1152](https://github.com/hydro-project/hydroflow/issues/1152), [#1159](https://github.com/hydro-project/hydroflow/issues/1159), [#1164](https://github.com/hydro-project/hydroflow/issues/1164), [#1166](https://github.com/hydro-project/hydroflow/issues/1166), [#1167](https://github.com/hydro-project/hydroflow/issues/1167), [#1171](https://github.com/hydro-project/hydroflow/issues/1171), [#1176](https://github.com/hydro-project/hydroflow/issues/1176), [#1178](https://github.com/hydro-project/hydroflow/issues/1178), [#1182](https://github.com/hydro-project/hydroflow/issues/1182), [#1190](https://github.com/hydro-project/hydroflow/issues/1190), [#1191](https://github.com/hydro-project/hydroflow/issues/1191), [#1192](https://github.com/hydro-project/hydroflow/issues/1192), [#1193](https://github.com/hydro-project/hydroflow/issues/1193), [#1196](https://github.com/hydro-project/hydroflow/issues/1196), [#1197](https://github.com/hydro-project/hydroflow/issues/1197), [#1198](https://github.com/hydro-project/hydroflow/issues/1198), [#1199](https://github.com/hydro-project/hydroflow/issues/1199), [#1204](https://github.com/hydro-project/hydroflow/issues/1204), [#1232](https://github.com/hydro-project/hydroflow/issues/1232), [#1236](https://github.com/hydro-project/hydroflow/issues/1236), [#1238](https://github.com/hydro-project/hydroflow/issues/1238) -### Commit Details +### `hydroflow` Commit Details @@ -517,7 +628,7 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0 - add tests for #1147 (failing) -### Commit Statistics +### `hydroflow` Commit Statistics @@ -526,7 +637,7 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0 - 5 commits were understood as [conventional](https://www.conventionalcommits.org). - 3 unique issues were worked on: [#1134](https://github.com/hydro-project/hydroflow/issues/1134), [#1148](https://github.com/hydro-project/hydroflow/issues/1148), [#1150](https://github.com/hydro-project/hydroflow/issues/1150) -### Commit Details +### `hydroflow` Commit Details @@ -584,11 +695,11 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0 - include subgraph ID in pivot_run codegen - switch launch to be a macro to enable borrowing locals - add syntax for "splatting" columns - feat(hydroflow_datalog): add syntax for "splatting" columns - - The * operator can be used on the left-hand side of a rule to indicate - that each row should be duplicated into many rows with that column being - replaced by every value in iterable corresponding to the original + feat(hydroflow_datalog): add syntax for "splatting" columns + + The * operator can be used on the left-hand side of a rule to indicate + that each row should be duplicated into many rows with that column being + replaced by every value in iterable corresponding to the original expression. - allow dropping of tee handoff output to prevent memory leak - implement teeing handoff for scheduled layer @@ -616,7 +727,7 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0 - fix some `surface_codegen.rs` tests, actually test output, fix #1096 - add hydroflow scheduling spin test for #1050 #800 -### Commit Statistics +### `hydroflow` Commit Statistics @@ -625,7 +736,7 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0 - 27 commits were understood as [conventional](https://www.conventionalcommits.org). - 18 unique issues were worked on: [#1068](https://github.com/hydro-project/hydroflow/issues/1068), [#1086](https://github.com/hydro-project/hydroflow/issues/1086), [#1087](https://github.com/hydro-project/hydroflow/issues/1087), [#1089](https://github.com/hydro-project/hydroflow/issues/1089), [#1090](https://github.com/hydro-project/hydroflow/issues/1090), [#1091](https://github.com/hydro-project/hydroflow/issues/1091), [#1093](https://github.com/hydro-project/hydroflow/issues/1093), [#1094](https://github.com/hydro-project/hydroflow/issues/1094), [#1102](https://github.com/hydro-project/hydroflow/issues/1102), [#1113](https://github.com/hydro-project/hydroflow/issues/1113), [#1125](https://github.com/hydro-project/hydroflow/issues/1125), [#1128](https://github.com/hydro-project/hydroflow/issues/1128), [#1132](https://github.com/hydro-project/hydroflow/issues/1132), [#1133](https://github.com/hydro-project/hydroflow/issues/1133), [#1137](https://github.com/hydro-project/hydroflow/issues/1137), [#1140](https://github.com/hydro-project/hydroflow/issues/1140), [#1145](https://github.com/hydro-project/hydroflow/issues/1145), [#1146](https://github.com/hydro-project/hydroflow/issues/1146) -### Commit Details +### `hydroflow` Commit Details @@ -714,7 +825,7 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0 - fix imports for clippy - fix imports -### Commit Statistics +### `hydroflow` Commit Statistics @@ -723,7 +834,7 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0 - 9 commits were understood as [conventional](https://www.conventionalcommits.org). - 6 unique issues were worked on: [#1015](https://github.com/hydro-project/hydroflow/issues/1015), [#1057](https://github.com/hydro-project/hydroflow/issues/1057), [#1060](https://github.com/hydro-project/hydroflow/issues/1060), [#1061](https://github.com/hydro-project/hydroflow/issues/1061), [#1084](https://github.com/hydro-project/hydroflow/issues/1084), [#1085](https://github.com/hydro-project/hydroflow/issues/1085) -### Commit Details +### `hydroflow` Commit Details @@ -764,7 +875,7 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0 - ensure other graph methods handle edge types (#929) -### Commit Statistics +### `hydroflow` Commit Statistics @@ -773,7 +884,7 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0 - 4 commits were understood as [conventional](https://www.conventionalcommits.org). - 4 unique issues were worked on: [#1041](https://github.com/hydro-project/hydroflow/issues/1041), [#1051](https://github.com/hydro-project/hydroflow/issues/1051), [#1054](https://github.com/hydro-project/hydroflow/issues/1054), [#1055](https://github.com/hydro-project/hydroflow/issues/1055) -### Commit Details +### `hydroflow` Commit Details @@ -819,8 +930,8 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0 - split Rust core from Python bindings - support building graphs for symmetric clusters in Hydroflow+ - add initial test using Hydro CLI from Hydroflow+ - This also required a change to Hydroflow core to make it possible to run the dataflow itself on a single thread (using a LocalSet), even if the surrounding runtime is not single-threaded (required to work around deadlocks because we can't use async APIs inside Hydroflow+). This requires us to spawn any Hydroflow tasks (only for `dest_sink` at the moment) right next to when we run the dataflow rather than when the Hydroflow graph is initialized. From a conceptual perspective, this seems _more right_, since now creating a Hydroflow program will not result in any actual tasks running. - + This also required a change to Hydroflow core to make it possible to run the dataflow itself on a single thread (using a LocalSet), even if the surrounding runtime is not single-threaded (required to work around deadlocks because we can't use async APIs inside Hydroflow+). This requires us to spawn any Hydroflow tasks (only for `dest_sink` at the moment) right next to when we run the dataflow rather than when the Hydroflow graph is initialized. From a conceptual perspective, this seems _more right_, since now creating a Hydroflow program will not result in any actual tasks running. + In the third PR of this series, I aim to add a new Hydroflow+ operator that will automate the setup of a `dest_sink`/`source_stream` pair that span nodes. - new implementation and Hydro Deploy setup -- @@ -843,6 +954,8 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0 -- - new implementation and Hydro Deploy setup -- + - new implementation and Hydro Deploy setup + -- ### Bug Fixes @@ -866,9 +979,9 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0 - Update `surface_lattice_join` tests to use `DeepReveal` - Improvements prepping for release - - Adds the "spread"/"splat" `...` syntax to the three variadics macros. - - Adds `#[sealed]` traits. - - Adds testing of error messages. + - Adds the "spread"/"splat" `...` syntax to the three variadics macros. + - Adds `#[sealed]` traits. + - Adds testing of error messages. - Improves docs: `README.md` and Rust docs. - remove intrinsics from KVS bench, fixes warning @@ -876,7 +989,7 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0 - fold takes initial value by closure rather than by value -### Commit Statistics +### `hydroflow` Commit Statistics @@ -885,7 +998,7 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0 - 26 commits were understood as [conventional](https://www.conventionalcommits.org). - 23 unique issues were worked on: [#1003](https://github.com/hydro-project/hydroflow/issues/1003), [#1005](https://github.com/hydro-project/hydroflow/issues/1005), [#1024](https://github.com/hydro-project/hydroflow/issues/1024), [#1025](https://github.com/hydro-project/hydroflow/issues/1025), [#1026](https://github.com/hydro-project/hydroflow/issues/1026), [#1032](https://github.com/hydro-project/hydroflow/issues/1032), [#1036](https://github.com/hydro-project/hydroflow/issues/1036), [#899](https://github.com/hydro-project/hydroflow/issues/899), [#909](https://github.com/hydro-project/hydroflow/issues/909), [#942](https://github.com/hydro-project/hydroflow/issues/942), [#945](https://github.com/hydro-project/hydroflow/issues/945), [#948](https://github.com/hydro-project/hydroflow/issues/948), [#950](https://github.com/hydro-project/hydroflow/issues/950), [#959](https://github.com/hydro-project/hydroflow/issues/959), [#960](https://github.com/hydro-project/hydroflow/issues/960), [#967](https://github.com/hydro-project/hydroflow/issues/967), [#971](https://github.com/hydro-project/hydroflow/issues/971), [#974](https://github.com/hydro-project/hydroflow/issues/974), [#978](https://github.com/hydro-project/hydroflow/issues/978), [#979](https://github.com/hydro-project/hydroflow/issues/979), [#984](https://github.com/hydro-project/hydroflow/issues/984), [#986](https://github.com/hydro-project/hydroflow/issues/986), [#996](https://github.com/hydro-project/hydroflow/issues/996) -### Commit Details +### `hydroflow` Commit Details @@ -1004,7 +1117,7 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0 - clippy warning on multiline string in hydro_cli, py_udf - fix demux error messages and add tests - Clean up degenerate subgraph error message for consistency - Makes the pinned and latest nightly version have the same stderr output + Makes the pinned and latest nightly version have the same stderr output for consistent testing. - Make deadlock detector filter out spruious UDP `ConnectionReset`s on Windows - Ignore PermissionDenied on already-closed socket on Windows @@ -1032,7 +1145,7 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0 - Simplify graph printing code, add delta/cumul green edges, allow hiding of vars/subgraphs -### Commit Statistics +### `hydroflow` Commit Statistics @@ -1041,7 +1154,7 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0 - 43 commits were understood as [conventional](https://www.conventionalcommits.org). - 19 unique issues were worked on: [#882](https://github.com/hydro-project/hydroflow/issues/882), [#884](https://github.com/hydro-project/hydroflow/issues/884), [#885](https://github.com/hydro-project/hydroflow/issues/885), [#886](https://github.com/hydro-project/hydroflow/issues/886), [#887](https://github.com/hydro-project/hydroflow/issues/887), [#892](https://github.com/hydro-project/hydroflow/issues/892), [#893](https://github.com/hydro-project/hydroflow/issues/893), [#896](https://github.com/hydro-project/hydroflow/issues/896), [#897](https://github.com/hydro-project/hydroflow/issues/897), [#898](https://github.com/hydro-project/hydroflow/issues/898), [#902](https://github.com/hydro-project/hydroflow/issues/902), [#906](https://github.com/hydro-project/hydroflow/issues/906), [#918](https://github.com/hydro-project/hydroflow/issues/918), [#919](https://github.com/hydro-project/hydroflow/issues/919), [#923](https://github.com/hydro-project/hydroflow/issues/923), [#924](https://github.com/hydro-project/hydroflow/issues/924), [#926](https://github.com/hydro-project/hydroflow/issues/926), [#932](https://github.com/hydro-project/hydroflow/issues/932), [#935](https://github.com/hydro-project/hydroflow/issues/935) -### Commit Details +### `hydroflow` Commit Details @@ -1157,7 +1270,7 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0 shopping still erroring out for reasons unrelated to this PR - unify antijoin and difference with set and multiset semantics * fix: unify antijoin and difference with set and multiset semantics -* fix: replay semantics for antijoin and difference now work +* fix: replay semantics for antijoin and difference now work also added cross_join_multiset * fix: enforce sort for tests of anti_join and difference using assert_eq * fix: advance __borrow_ident beyond the current tick to prevent replay loops @@ -1172,8 +1285,8 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0 ### Other - join probe returns first item directly - * optimization: join probe returns first item directly - + * optimization: join probe returns first item directly + * update comments ### Test @@ -1188,7 +1301,7 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0 * address comments * fix clippy -### Commit Statistics +### `hydroflow` Commit Statistics @@ -1197,7 +1310,7 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0 - 22 commits were understood as [conventional](https://www.conventionalcommits.org). - 23 unique issues were worked on: [#820](https://github.com/hydro-project/hydroflow/issues/820), [#821](https://github.com/hydro-project/hydroflow/issues/821), [#822](https://github.com/hydro-project/hydroflow/issues/822), [#823](https://github.com/hydro-project/hydroflow/issues/823), [#833](https://github.com/hydro-project/hydroflow/issues/833), [#835](https://github.com/hydro-project/hydroflow/issues/835), [#837](https://github.com/hydro-project/hydroflow/issues/837), [#840](https://github.com/hydro-project/hydroflow/issues/840), [#842](https://github.com/hydro-project/hydroflow/issues/842), [#843](https://github.com/hydro-project/hydroflow/issues/843), [#844](https://github.com/hydro-project/hydroflow/issues/844), [#845](https://github.com/hydro-project/hydroflow/issues/845), [#846](https://github.com/hydro-project/hydroflow/issues/846), [#848](https://github.com/hydro-project/hydroflow/issues/848), [#851](https://github.com/hydro-project/hydroflow/issues/851), [#853](https://github.com/hydro-project/hydroflow/issues/853), [#857](https://github.com/hydro-project/hydroflow/issues/857), [#861](https://github.com/hydro-project/hydroflow/issues/861), [#870](https://github.com/hydro-project/hydroflow/issues/870), [#872](https://github.com/hydro-project/hydroflow/issues/872), [#874](https://github.com/hydro-project/hydroflow/issues/874), [#878](https://github.com/hydro-project/hydroflow/issues/878), [#880](https://github.com/hydro-project/hydroflow/issues/880) -### Commit Details +### `hydroflow` Commit Details @@ -1348,7 +1461,7 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0 - Rename `Bottom` -> `WithBot`, `Top` -> `WithTop`, constructors now take `Option`s 2/4 - Rename `Immut` -> `Point` lattice. -### Commit Statistics +### `hydroflow` Commit Statistics @@ -1357,7 +1470,7 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0 - 31 commits were understood as [conventional](https://www.conventionalcommits.org). - 25 unique issues were worked on: [#739](https://github.com/hydro-project/hydroflow/issues/739), [#743](https://github.com/hydro-project/hydroflow/issues/743), [#745](https://github.com/hydro-project/hydroflow/issues/745), [#748](https://github.com/hydro-project/hydroflow/issues/748), [#749](https://github.com/hydro-project/hydroflow/issues/749), [#755](https://github.com/hydro-project/hydroflow/issues/755), [#761](https://github.com/hydro-project/hydroflow/issues/761), [#763](https://github.com/hydro-project/hydroflow/issues/763), [#765](https://github.com/hydro-project/hydroflow/issues/765), [#772](https://github.com/hydro-project/hydroflow/issues/772), [#773](https://github.com/hydro-project/hydroflow/issues/773), [#774](https://github.com/hydro-project/hydroflow/issues/774), [#775](https://github.com/hydro-project/hydroflow/issues/775), [#778](https://github.com/hydro-project/hydroflow/issues/778), [#780](https://github.com/hydro-project/hydroflow/issues/780), [#784](https://github.com/hydro-project/hydroflow/issues/784), [#788](https://github.com/hydro-project/hydroflow/issues/788), [#789](https://github.com/hydro-project/hydroflow/issues/789), [#791](https://github.com/hydro-project/hydroflow/issues/791), [#792](https://github.com/hydro-project/hydroflow/issues/792), [#799](https://github.com/hydro-project/hydroflow/issues/799), [#801](https://github.com/hydro-project/hydroflow/issues/801), [#803](https://github.com/hydro-project/hydroflow/issues/803), [#804](https://github.com/hydro-project/hydroflow/issues/804), [#809](https://github.com/hydro-project/hydroflow/issues/809) -### Commit Details +### `hydroflow` Commit Details @@ -1442,7 +1555,7 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0 - rename `Fake` -> `Immut` -### Commit Statistics +### `hydroflow` Commit Statistics @@ -1451,7 +1564,7 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0 - 3 commits were understood as [conventional](https://www.conventionalcommits.org). - 0 issues like '(#ID)' were seen in commit messages -### Commit Details +### `hydroflow` Commit Details @@ -1519,7 +1632,7 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0 * remove group_by -### Commit Statistics +### `hydroflow` Commit Statistics @@ -1528,7 +1641,7 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0 - 17 commits were understood as [conventional](https://www.conventionalcommits.org). - 12 unique issues were worked on: [#686](https://github.com/hydro-project/hydroflow/issues/686), [#690](https://github.com/hydro-project/hydroflow/issues/690), [#692](https://github.com/hydro-project/hydroflow/issues/692), [#696](https://github.com/hydro-project/hydroflow/issues/696), [#697](https://github.com/hydro-project/hydroflow/issues/697), [#702](https://github.com/hydro-project/hydroflow/issues/702), [#706](https://github.com/hydro-project/hydroflow/issues/706), [#708](https://github.com/hydro-project/hydroflow/issues/708), [#714](https://github.com/hydro-project/hydroflow/issues/714), [#716](https://github.com/hydro-project/hydroflow/issues/716), [#719](https://github.com/hydro-project/hydroflow/issues/719), [#721](https://github.com/hydro-project/hydroflow/issues/721) -### Commit Details +### `hydroflow` Commit Details @@ -1588,7 +1701,7 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0 - remove `hydroflow::lang` module, move `Clear`, `MonotonicMap` to `hydroflow::util` instead - remove unused `PushHandoff` -### Commit Statistics +### `hydroflow` Commit Statistics @@ -1597,7 +1710,7 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0 - 5 commits were understood as [conventional](https://www.conventionalcommits.org). - 4 unique issues were worked on: [#661](https://github.com/hydro-project/hydroflow/issues/661), [#671](https://github.com/hydro-project/hydroflow/issues/671), [#677](https://github.com/hydro-project/hydroflow/issues/677), [#684](https://github.com/hydro-project/hydroflow/issues/684) -### Commit Details +### `hydroflow` Commit Details @@ -1637,7 +1750,7 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0 - rustfmt group imports - rustfmt prescribe flat-module `use` format -### Commit Statistics +### `hydroflow` Commit Statistics @@ -1646,7 +1759,7 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0 - 5 commits were understood as [conventional](https://www.conventionalcommits.org). - 14 unique issues were worked on: [#625](https://github.com/hydro-project/hydroflow/issues/625), [#638](https://github.com/hydro-project/hydroflow/issues/638), [#640](https://github.com/hydro-project/hydroflow/issues/640), [#641](https://github.com/hydro-project/hydroflow/issues/641), [#642](https://github.com/hydro-project/hydroflow/issues/642), [#644](https://github.com/hydro-project/hydroflow/issues/644), [#649](https://github.com/hydro-project/hydroflow/issues/649), [#650](https://github.com/hydro-project/hydroflow/issues/650), [#651](https://github.com/hydro-project/hydroflow/issues/651), [#654](https://github.com/hydro-project/hydroflow/issues/654), [#656](https://github.com/hydro-project/hydroflow/issues/656), [#657](https://github.com/hydro-project/hydroflow/issues/657), [#660](https://github.com/hydro-project/hydroflow/issues/660), [#667](https://github.com/hydro-project/hydroflow/issues/667) -### Commit Details +### `hydroflow` Commit Details @@ -1697,13 +1810,13 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0 - separate constructors into `new` and `new_from` - Add `NaiveCompare`, fix `Ord` & `SetUnion` bugs - - Fix `Min` compare reversed - - Impl compare for `Max` - - Fix `SetUnion` compare logical error (long standing old bug) - - Add `NaiveCompare` trait based on merge fn + - Fix `Min` compare reversed + - Impl compare for `Max` + - Fix `SetUnion` compare logical error (long standing old bug) + - Add `NaiveCompare` trait based on merge fn - Improve `SetUnion` tests -### Commit Statistics +### `hydroflow` Commit Statistics @@ -1712,7 +1825,7 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0 - 2 commits were understood as [conventional](https://www.conventionalcommits.org). - 6 unique issues were worked on: [#622](https://github.com/hydro-project/hydroflow/issues/622), [#629](https://github.com/hydro-project/hydroflow/issues/629), [#632](https://github.com/hydro-project/hydroflow/issues/632), [#633](https://github.com/hydro-project/hydroflow/issues/633), [#634](https://github.com/hydro-project/hydroflow/issues/634), [#635](https://github.com/hydro-project/hydroflow/issues/635) -### Commit Details +### `hydroflow` Commit Details @@ -1756,7 +1869,7 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0 Still pretty simple, but the beginnings of a configurable benchmarking harness for the KVS. -### Commit Statistics +### `hydroflow` Commit Statistics @@ -1764,7 +1877,7 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0 - 6 commits were understood as [conventional](https://www.conventionalcommits.org). - 191 unique issues were worked on: [#100](https://github.com/hydro-project/hydroflow/issues/100), [#101](https://github.com/hydro-project/hydroflow/issues/101), [#102](https://github.com/hydro-project/hydroflow/issues/102), [#111](https://github.com/hydro-project/hydroflow/issues/111), [#113](https://github.com/hydro-project/hydroflow/issues/113), [#116](https://github.com/hydro-project/hydroflow/issues/116), [#12](https://github.com/hydro-project/hydroflow/issues/12), [#120](https://github.com/hydro-project/hydroflow/issues/120), [#121](https://github.com/hydro-project/hydroflow/issues/121), [#122](https://github.com/hydro-project/hydroflow/issues/122), [#127](https://github.com/hydro-project/hydroflow/issues/127), [#137](https://github.com/hydro-project/hydroflow/issues/137), [#146](https://github.com/hydro-project/hydroflow/issues/146), [#147](https://github.com/hydro-project/hydroflow/issues/147), [#15](https://github.com/hydro-project/hydroflow/issues/15), [#155](https://github.com/hydro-project/hydroflow/issues/155), [#160](https://github.com/hydro-project/hydroflow/issues/160), [#162](https://github.com/hydro-project/hydroflow/issues/162), [#163](https://github.com/hydro-project/hydroflow/issues/163), [#164](https://github.com/hydro-project/hydroflow/issues/164), [#165](https://github.com/hydro-project/hydroflow/issues/165), [#18](https://github.com/hydro-project/hydroflow/issues/18), [#184](https://github.com/hydro-project/hydroflow/issues/184), [#187](https://github.com/hydro-project/hydroflow/issues/187), [#190](https://github.com/hydro-project/hydroflow/issues/190), [#207](https://github.com/hydro-project/hydroflow/issues/207), [#209](https://github.com/hydro-project/hydroflow/issues/209), [#211](https://github.com/hydro-project/hydroflow/issues/211), [#213](https://github.com/hydro-project/hydroflow/issues/213), [#222](https://github.com/hydro-project/hydroflow/issues/222), [#230](https://github.com/hydro-project/hydroflow/issues/230), [#231](https://github.com/hydro-project/hydroflow/issues/231), [#234](https://github.com/hydro-project/hydroflow/issues/234), [#235](https://github.com/hydro-project/hydroflow/issues/235), [#236](https://github.com/hydro-project/hydroflow/issues/236), [#237](https://github.com/hydro-project/hydroflow/issues/237), [#238](https://github.com/hydro-project/hydroflow/issues/238), [#239](https://github.com/hydro-project/hydroflow/issues/239), [#248](https://github.com/hydro-project/hydroflow/issues/248), [#249](https://github.com/hydro-project/hydroflow/issues/249), [#250](https://github.com/hydro-project/hydroflow/issues/250), [#254](https://github.com/hydro-project/hydroflow/issues/254), [#259](https://github.com/hydro-project/hydroflow/issues/259), [#261](https://github.com/hydro-project/hydroflow/issues/261), [#262](https://github.com/hydro-project/hydroflow/issues/262), [#268](https://github.com/hydro-project/hydroflow/issues/268), [#277](https://github.com/hydro-project/hydroflow/issues/277), [#278](https://github.com/hydro-project/hydroflow/issues/278), [#279](https://github.com/hydro-project/hydroflow/issues/279), [#28](https://github.com/hydro-project/hydroflow/issues/28), [#282](https://github.com/hydro-project/hydroflow/issues/282), [#284](https://github.com/hydro-project/hydroflow/issues/284), [#285](https://github.com/hydro-project/hydroflow/issues/285), [#288](https://github.com/hydro-project/hydroflow/issues/288), [#294](https://github.com/hydro-project/hydroflow/issues/294), [#295](https://github.com/hydro-project/hydroflow/issues/295), [#296](https://github.com/hydro-project/hydroflow/issues/296), [#298](https://github.com/hydro-project/hydroflow/issues/298), [#3](https://github.com/hydro-project/hydroflow/issues/3), [#30](https://github.com/hydro-project/hydroflow/issues/30), [#300](https://github.com/hydro-project/hydroflow/issues/300), [#301](https://github.com/hydro-project/hydroflow/issues/301), [#307](https://github.com/hydro-project/hydroflow/issues/307), [#309](https://github.com/hydro-project/hydroflow/issues/309), [#32](https://github.com/hydro-project/hydroflow/issues/32), [#321](https://github.com/hydro-project/hydroflow/issues/321), [#329](https://github.com/hydro-project/hydroflow/issues/329), [#33](https://github.com/hydro-project/hydroflow/issues/33), [#333](https://github.com/hydro-project/hydroflow/issues/333), [#34](https://github.com/hydro-project/hydroflow/issues/34), [#344](https://github.com/hydro-project/hydroflow/issues/344), [#350](https://github.com/hydro-project/hydroflow/issues/350), [#358](https://github.com/hydro-project/hydroflow/issues/358), [#363](https://github.com/hydro-project/hydroflow/issues/363), [#369](https://github.com/hydro-project/hydroflow/issues/369), [#37](https://github.com/hydro-project/hydroflow/issues/37), [#372](https://github.com/hydro-project/hydroflow/issues/372), [#374](https://github.com/hydro-project/hydroflow/issues/374), [#376](https://github.com/hydro-project/hydroflow/issues/376), [#38](https://github.com/hydro-project/hydroflow/issues/38), [#381](https://github.com/hydro-project/hydroflow/issues/381), [#382](https://github.com/hydro-project/hydroflow/issues/382), [#383](https://github.com/hydro-project/hydroflow/issues/383), [#388](https://github.com/hydro-project/hydroflow/issues/388), [#39](https://github.com/hydro-project/hydroflow/issues/39), [#397](https://github.com/hydro-project/hydroflow/issues/397), [#40](https://github.com/hydro-project/hydroflow/issues/40), [#403](https://github.com/hydro-project/hydroflow/issues/403), [#409](https://github.com/hydro-project/hydroflow/issues/409), [#411](https://github.com/hydro-project/hydroflow/issues/411), [#412](https://github.com/hydro-project/hydroflow/issues/412), [#413](https://github.com/hydro-project/hydroflow/issues/413), [#417](https://github.com/hydro-project/hydroflow/issues/417), [#420](https://github.com/hydro-project/hydroflow/issues/420), [#43](https://github.com/hydro-project/hydroflow/issues/43), [#431](https://github.com/hydro-project/hydroflow/issues/431), [#435](https://github.com/hydro-project/hydroflow/issues/435), [#437](https://github.com/hydro-project/hydroflow/issues/437), [#442](https://github.com/hydro-project/hydroflow/issues/442), [#443](https://github.com/hydro-project/hydroflow/issues/443), [#444](https://github.com/hydro-project/hydroflow/issues/444), [#445](https://github.com/hydro-project/hydroflow/issues/445), [#448 1/2](https://github.com/hydro-project/hydroflow/issues/448 1/2), [#448 2/2](https://github.com/hydro-project/hydroflow/issues/448 2/2), [#452](https://github.com/hydro-project/hydroflow/issues/452), [#459](https://github.com/hydro-project/hydroflow/issues/459), [#46](https://github.com/hydro-project/hydroflow/issues/46), [#460](https://github.com/hydro-project/hydroflow/issues/460), [#461](https://github.com/hydro-project/hydroflow/issues/461), [#465](https://github.com/hydro-project/hydroflow/issues/465), [#466](https://github.com/hydro-project/hydroflow/issues/466), [#468](https://github.com/hydro-project/hydroflow/issues/468), [#469](https://github.com/hydro-project/hydroflow/issues/469), [#470](https://github.com/hydro-project/hydroflow/issues/470), [#471](https://github.com/hydro-project/hydroflow/issues/471), [#472](https://github.com/hydro-project/hydroflow/issues/472), [#475](https://github.com/hydro-project/hydroflow/issues/475), [#477](https://github.com/hydro-project/hydroflow/issues/477), [#479](https://github.com/hydro-project/hydroflow/issues/479), [#48](https://github.com/hydro-project/hydroflow/issues/48), [#484](https://github.com/hydro-project/hydroflow/issues/484), [#487](https://github.com/hydro-project/hydroflow/issues/487), [#492](https://github.com/hydro-project/hydroflow/issues/492), [#493](https://github.com/hydro-project/hydroflow/issues/493), [#495](https://github.com/hydro-project/hydroflow/issues/495), [#497](https://github.com/hydro-project/hydroflow/issues/497), [#499](https://github.com/hydro-project/hydroflow/issues/499), [#500](https://github.com/hydro-project/hydroflow/issues/500), [#501](https://github.com/hydro-project/hydroflow/issues/501), [#502](https://github.com/hydro-project/hydroflow/issues/502), [#505](https://github.com/hydro-project/hydroflow/issues/505), [#509](https://github.com/hydro-project/hydroflow/issues/509), [#511](https://github.com/hydro-project/hydroflow/issues/511), [#512](https://github.com/hydro-project/hydroflow/issues/512), [#516](https://github.com/hydro-project/hydroflow/issues/516), [#518](https://github.com/hydro-project/hydroflow/issues/518), [#520](https://github.com/hydro-project/hydroflow/issues/520), [#521](https://github.com/hydro-project/hydroflow/issues/521), [#522](https://github.com/hydro-project/hydroflow/issues/522), [#523](https://github.com/hydro-project/hydroflow/issues/523), [#524](https://github.com/hydro-project/hydroflow/issues/524), [#526](https://github.com/hydro-project/hydroflow/issues/526), [#529](https://github.com/hydro-project/hydroflow/issues/529), [#530](https://github.com/hydro-project/hydroflow/issues/530), [#536](https://github.com/hydro-project/hydroflow/issues/536), [#538](https://github.com/hydro-project/hydroflow/issues/538), [#540](https://github.com/hydro-project/hydroflow/issues/540), [#541](https://github.com/hydro-project/hydroflow/issues/541), [#543](https://github.com/hydro-project/hydroflow/issues/543), [#547](https://github.com/hydro-project/hydroflow/issues/547), [#548 #550](https://github.com/hydro-project/hydroflow/issues/548 #550), [#549](https://github.com/hydro-project/hydroflow/issues/549), [#551](https://github.com/hydro-project/hydroflow/issues/551), [#555](https://github.com/hydro-project/hydroflow/issues/555), [#556](https://github.com/hydro-project/hydroflow/issues/556), [#558](https://github.com/hydro-project/hydroflow/issues/558), [#559](https://github.com/hydro-project/hydroflow/issues/559), [#56](https://github.com/hydro-project/hydroflow/issues/56), [#566](https://github.com/hydro-project/hydroflow/issues/566), [#567](https://github.com/hydro-project/hydroflow/issues/567), [#568](https://github.com/hydro-project/hydroflow/issues/568), [#571](https://github.com/hydro-project/hydroflow/issues/571), [#576](https://github.com/hydro-project/hydroflow/issues/576), [#578](https://github.com/hydro-project/hydroflow/issues/578), [#584](https://github.com/hydro-project/hydroflow/issues/584), [#586](https://github.com/hydro-project/hydroflow/issues/586), [#589](https://github.com/hydro-project/hydroflow/issues/589), [#590](https://github.com/hydro-project/hydroflow/issues/590), [#591](https://github.com/hydro-project/hydroflow/issues/591), [#593](https://github.com/hydro-project/hydroflow/issues/593), [#597](https://github.com/hydro-project/hydroflow/issues/597), [#598](https://github.com/hydro-project/hydroflow/issues/598), [#6](https://github.com/hydro-project/hydroflow/issues/6), [#60](https://github.com/hydro-project/hydroflow/issues/60), [#602](https://github.com/hydro-project/hydroflow/issues/602), [#605](https://github.com/hydro-project/hydroflow/issues/605), [#607](https://github.com/hydro-project/hydroflow/issues/607), [#608](https://github.com/hydro-project/hydroflow/issues/608), [#61](https://github.com/hydro-project/hydroflow/issues/61), [#617](https://github.com/hydro-project/hydroflow/issues/617), [#618](https://github.com/hydro-project/hydroflow/issues/618), [#63](https://github.com/hydro-project/hydroflow/issues/63), [#64](https://github.com/hydro-project/hydroflow/issues/64), [#8](https://github.com/hydro-project/hydroflow/issues/8), [#80](https://github.com/hydro-project/hydroflow/issues/80), [#84](https://github.com/hydro-project/hydroflow/issues/84), [#86](https://github.com/hydro-project/hydroflow/issues/86), [#89](https://github.com/hydro-project/hydroflow/issues/89), [#91](https://github.com/hydro-project/hydroflow/issues/91), [#95](https://github.com/hydro-project/hydroflow/issues/95), [#98](https://github.com/hydro-project/hydroflow/issues/98) -### Commit Details +### `hydroflow` Commit Details diff --git a/hydro_deploy/core/CHANGELOG.md b/hydro_deploy/core/CHANGELOG.md index f526879cdf38..c3b018adb6d9 100644 --- a/hydro_deploy/core/CHANGELOG.md +++ b/hydro_deploy/core/CHANGELOG.md @@ -5,8 +5,38 @@ All notable changes to this project will be documented in this file. The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.0.0/), and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html). +## Unreleased + +### New Features + + - use regular println when no tasks are active + Significantly improves the appearance of Hydroflow+ logs when the + terminal causes wrapping. + +### Commit Statistics + + + + - 1 commit contributed to the release. + - 38 days passed between releases. + - 1 commit was understood as [conventional](https://www.conventionalcommits.org). + - 1 unique issue was worked on: [#1577](https://github.com/hydro-project/hydroflow/issues/1577) + +### Commit Details + + + +
view details + + * **[#1577](https://github.com/hydro-project/hydroflow/issues/1577)** + - Use regular println when no tasks are active ([`8d550b9`](https://github.com/hydro-project/hydroflow/commit/8d550b94ae2c08486e1c2222d37e3ca8b5f018b7)) +
+ ## v0.10.0 (2024-11-08) + + + ### Chore - update pinned rust version, clippy lints, remove some dead code @@ -48,7 +78,7 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0 - - 4 commits contributed to the release. + - 5 commits contributed to the release. - 69 days passed between releases. - 4 commits were understood as [conventional](https://www.conventionalcommits.org). - 4 unique issues were worked on: [#1444](https://github.com/hydro-project/hydroflow/issues/1444), [#1449](https://github.com/hydro-project/hydroflow/issues/1449), [#1450](https://github.com/hydro-project/hydroflow/issues/1450), [#1537](https://github.com/hydro-project/hydroflow/issues/1537) @@ -67,6 +97,8 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0 - Add ability to have staged flows inside unit tests ([`afe78c3`](https://github.com/hydro-project/hydroflow/commit/afe78c343658472513b34d28658634b253148aee)) * **[#1537](https://github.com/hydro-project/hydroflow/issues/1537)** - Fixes for latest nightly clippy ([`8442d1b`](https://github.com/hydro-project/hydroflow/commit/8442d1b524621a9f8b43372a9c25991efb33c25e)) + * **Uncategorized** + - Release hydroflow_lang v0.10.0, hydroflow_datalog_core v0.10.0, hydroflow_datalog v0.10.0, hydroflow_deploy_integration v0.10.0, hydroflow_macro v0.10.0, lattices_macro v0.5.7, variadics v0.0.7, variadics_macro v0.5.5, lattices v0.5.8, multiplatform_test v0.3.0, pusherator v0.0.9, hydroflow v0.10.0, hydro_deploy v0.10.0, stageleft_macro v0.4.0, stageleft v0.5.0, stageleft_tool v0.4.0, hydroflow_plus v0.10.0, hydro_cli v0.10.0, safety bump 8 crates ([`dcd48fc`](https://github.com/hydro-project/hydroflow/commit/dcd48fc7ee805898d9b5ef0d082870e30615e95b)) ## v0.9.0 (2024-08-30) @@ -100,6 +132,26 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0 * #1395 * __->__ #1394 +### Refactor (BREAKING) + + - simplify process/cluster specs + --- + [//]: # (BEGIN SAPLING FOOTER) + Stack created with [Sapling](https://sapling-scm.com). Best reviewed + with + [ReviewStack](https://reviewstack.dev/hydro-project/hydroflow/pull/1394). + * #1395 + * __->__ #1394 + + simplify process/cluster specs + --- + [//]: # (BEGIN SAPLING FOOTER) + Stack created with [Sapling](https://sapling-scm.com). Best reviewed + with + [ReviewStack](https://reviewstack.dev/hydro-project/hydroflow/pull/1394). + * #1395 + * __->__ #1394 + ### Documentation - cleanup doc comments for clippy latest @@ -184,6 +236,26 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0 - use `buildstructor` to handle excessive `Deployment` method arguments, fix #1364 Adds new method `Deployment::AzureHost` + simplify process/cluster specs + --- + [//]: # (BEGIN SAPLING FOOTER) + Stack created with [Sapling](https://sapling-scm.com). Best reviewed + with + [ReviewStack](https://reviewstack.dev/hydro-project/hydroflow/pull/1394). + * #1395 + * __->__ #1394 + - end-to-end flamegraph generation, fix #1365 + Depends on #1370 + - `Deployment.stop()` for graceful shutdown including updated `perf` profile downloading + * `perf` profile downloading moved from the `drop()` impl to `async fn + stop()` + * download perf data via stdout + * update async-ssh2-lite to 0.5 to cleanup tokio compat issues + + WIP for #1365 + - use `buildstructor` to handle excessive `Deployment` method arguments, fix #1364 + Adds new method `Deployment::AzureHost` + ### Commit Statistics diff --git a/hydro_lang/CHANGELOG.md b/hydro_lang/CHANGELOG.md index 0556857807cb..4c5c4bf2fa4b 100644 --- a/hydro_lang/CHANGELOG.md +++ b/hydro_lang/CHANGELOG.md @@ -5,12 +5,191 @@ All notable changes to this project will be documented in this file. The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.0.0/), and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html). +## Unreleased + +### Chore + + - use same hashing library everywhere + +### New Features + + - add `round_robin` helpers for networking + Also fixes compiler crashes when using `.enumerate()` on an un-batched + stream. + +### Bug Fixes + + - drop nightly feature flag from trybuild codegen + - restrict lifetime parameters to be actually invariant + Our lifetimes were accidentally made covariant when the lifetime `'a` + was removed from the process/cluster tag type. This fixes that typing + hole, and also loosens some restrictions on the lifetime of deploy + environments. + +### Refactor + + - use `match_box` macro to compile on stable + - generalize quorum logic + +### New Features (BREAKING) + + - minimize dependencies pulled into `trybuild` builds + We don't need `hydroflow_lang` in the runtime builds, do some feature + flagging to avoid it. + - mark non-deterministic operators as unsafe and introduce timestamped streams + Big PR. + + First big change is we introduce a `Timestamped` location. This is a bit + of a hybrid between top-level locations and `Tick` locations. The idea + is that you choose where timestamps are generated, and then have a + guarantee that everything after that will be atomically computed (useful + for making sure we add payloads to the log before ack-ing). + + The contract is that an operator or module that takes a `Timestamped` + input must still be deterministic regardless of the stamps on messages + (which are hidden unless you `tick_batch`). But unlike a top-level + stream (which has the same constraints), you have the atomicity + guarantee. Right now the guarantee is trivial since we have one global + tick for everything. But in the future when we want to apply + @davidchuyaya's optimizations this will be helpful to know when there + are causal dependencies on when data can be sent to others. + + Second change is we mark every non-deterministic operator (modulo + explicit annotations such as `NoOrder`) with Rust's `unsafe` keyword. + This makes it super clear where non-determinism is taking place. + + I've used this to put `unsafe` blocks throughout our example code and + add `SAFETY` annotations that argue why the non-determinism is safe (or + point out that we've explicitly documented / expect non-determinism). I + also added `#![warn(unsafe_op_in_unsafe_fn)]` to the examples and the + template, since this forces good hygiene of annotating sources of + non-determinism even inside a module that is intentionally + non-deterministic. + + Paxos changes are mostly refactors, and I verified that the performance + is the same as before. + - allow runtime context to be referenced as a global constant + - allow cluster self ID to be referenced as a global constant + This eliminates the need to store `cluster.self_id()` in a local + variable first, instead you can directly reference `CLUSTER_SELF_ID`. + - introduce an unordered variant of streams to strengthen determinism guarantees + Previously, sending data from a `Cluster` would return a stream assumed + to have deterministic contents **and** ordering, which is false. This + introduces another type parameter for `Stream` which tracks whether + element ordering is expected to be deterministic, and restricts + operators such as `fold` and `reduce` to commutative aggregations + accordingly. + +### Bug Fixes (BREAKING) + + - rename `union` to `chain` and restrict LHS to be bounded + Returning a `Stream` from `union` on unbounded streams was unsound, + since the order of outputs is not deterministic. + +### Refactor (BREAKING) + + - use `cfg(nightly)` instead of feature, remove `-Z` flag, use `Diagnostic::try_emit` + Previous PR (#1587) website build did not work because `panic = "abort"` + is set on wasm, leading to aborts for `proc_macro2::Span::unwrap()` + calls. + + All tests except trybuild seem to pass on stable, WIP #1587 next + - further reduce namespace pollution + - don't re-export all of `hydroflow` + Reduces namespace pollution when wildcard-importing `hydroflow_plus`. + +### `hydroflow_plus` Commit Statistics + + + + - 15 commits contributed to the release. + - 38 days passed between releases. + - 15 commits were understood as [conventional](https://www.conventionalcommits.org). + - 15 unique issues were worked on: [#1559](https://github.com/hydro-project/hydroflow/issues/1559), [#1562](https://github.com/hydro-project/hydroflow/issues/1562), [#1565](https://github.com/hydro-project/hydroflow/issues/1565), [#1566](https://github.com/hydro-project/hydroflow/issues/1566), [#1568](https://github.com/hydro-project/hydroflow/issues/1568), [#1574](https://github.com/hydro-project/hydroflow/issues/1574), [#1575](https://github.com/hydro-project/hydroflow/issues/1575), [#1583](https://github.com/hydro-project/hydroflow/issues/1583), [#1584](https://github.com/hydro-project/hydroflow/issues/1584), [#1589](https://github.com/hydro-project/hydroflow/issues/1589), [#1590](https://github.com/hydro-project/hydroflow/issues/1590), [#1597](https://github.com/hydro-project/hydroflow/issues/1597), [#1598](https://github.com/hydro-project/hydroflow/issues/1598), [#1606](https://github.com/hydro-project/hydroflow/issues/1606), [#1611](https://github.com/hydro-project/hydroflow/issues/1611) + +### `hydroflow_plus` Commit Details + + + +
view details + + * **[#1559](https://github.com/hydro-project/hydroflow/issues/1559)** + - Restrict lifetime parameters to be actually invariant ([`f6989ba`](https://github.com/hydro-project/hydroflow/commit/f6989baf12631cf43a814123e274466740c2f159)) + * **[#1562](https://github.com/hydro-project/hydroflow/issues/1562)** + - Don't re-export all of `hydroflow` ([`9f3c8c4`](https://github.com/hydro-project/hydroflow/commit/9f3c8c468c58b7ec50d1c104fc24db0920d13c0d)) + * **[#1565](https://github.com/hydro-project/hydroflow/issues/1565)** + - Rename `union` to `chain` and restrict LHS to be bounded ([`eb1ad3a`](https://github.com/hydro-project/hydroflow/commit/eb1ad3a54705efb06ee3f0647deaa9a52731ae6e)) + * **[#1566](https://github.com/hydro-project/hydroflow/issues/1566)** + - Add `round_robin` helpers for networking ([`22de01f`](https://github.com/hydro-project/hydroflow/commit/22de01f5f566cd8cf7cb3bd31ae8bed99bf1e9ab)) + * **[#1568](https://github.com/hydro-project/hydroflow/issues/1568)** + - Introduce an unordered variant of streams to strengthen determinism guarantees ([`4c5ca31`](https://github.com/hydro-project/hydroflow/commit/4c5ca31486a9cfcbcba3af03aa30084a8b8dfcce)) + * **[#1574](https://github.com/hydro-project/hydroflow/issues/1574)** + - Allow cluster self ID to be referenced as a global constant ([`a93a5e5`](https://github.com/hydro-project/hydroflow/commit/a93a5e59e1681d325b3433193bb86254d23bdc77)) + * **[#1575](https://github.com/hydro-project/hydroflow/issues/1575)** + - Allow runtime context to be referenced as a global constant ([`f96676d`](https://github.com/hydro-project/hydroflow/commit/f96676d2d53d824c5c168e3db69722c2e9956fe2)) + * **[#1583](https://github.com/hydro-project/hydroflow/issues/1583)** + - Generalize quorum logic ([`ec55910`](https://github.com/hydro-project/hydroflow/commit/ec55910f5a41d4f08059b5feda4b96fbd058c959)) + * **[#1584](https://github.com/hydro-project/hydroflow/issues/1584)** + - Mark non-deterministic operators as unsafe and introduce timestamped streams ([`9393899`](https://github.com/hydro-project/hydroflow/commit/939389953875bf5f94ea84503a7a35efd7342282)) + * **[#1589](https://github.com/hydro-project/hydroflow/issues/1589)** + - Further reduce namespace pollution ([`78f6a32`](https://github.com/hydro-project/hydroflow/commit/78f6a3299fff822abaea50841800a07f0e2ae128)) + * **[#1590](https://github.com/hydro-project/hydroflow/issues/1590)** + - Use same hashing library everywhere ([`0dc709e`](https://github.com/hydro-project/hydroflow/commit/0dc709ed5a53c723f47fa1d10063e57bb50a63c8)) + * **[#1597](https://github.com/hydro-project/hydroflow/issues/1597)** + - Use `match_box` macro to compile on stable ([`9ea5f06`](https://github.com/hydro-project/hydroflow/commit/9ea5f061ee0f116caf8fc4ea99b62a9c7691be2a)) + * **[#1598](https://github.com/hydro-project/hydroflow/issues/1598)** + - Drop nightly feature flag from trybuild codegen ([`032cde6`](https://github.com/hydro-project/hydroflow/commit/032cde6492390733e3e82d7126cc84989bf31853)) + * **[#1606](https://github.com/hydro-project/hydroflow/issues/1606)** + - Use `cfg(nightly)` instead of feature, remove `-Z` flag, use `Diagnostic::try_emit` ([`251b103`](https://github.com/hydro-project/hydroflow/commit/251b1039c71d45d3f86123dba1926026ded80824)) + * **[#1611](https://github.com/hydro-project/hydroflow/issues/1611)** + - Minimize dependencies pulled into `trybuild` builds ([`c65b4c4`](https://github.com/hydro-project/hydroflow/commit/c65b4c49f3a3f2dcc7c8c28d1871b88c6c954822)) +
+ ## v0.10.0 (2024-11-08) + + + + + + + + + + + + + + + + + + + + + + + ### Chore - update pinned rust version, clippy lints, remove some dead code +### Refactor + + - start splitting out leader election into a separate module + --- + [//]: # (BEGIN SAPLING FOOTER) + Stack created with [Sapling](https://sapling-scm.com). Best reviewed + with + [ReviewStack](https://reviewstack.dev/hydro-project/hydroflow/pull/1485). + * #1493 + * #1492 + * #1489 + * #1488 + * #1487 + * #1486 + * __->__ #1485 + ### New Features - improve quickstart ergonomics @@ -18,30 +197,30 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0 - add decouple and simple test and two_pc - implement support for external network outputs - add ability to have staged flows inside unit tests - Whenever a Hydroflow+ program is compiled, it depends on a generated - `__staged` module, which contains the entire contents of the crate but - with every type / function made `pub` and exported, so that the compiled - UDFs can resolve local references appropriately. - - Previously, we would not do this for `#[cfg(test)]` modules, since they - may use `dev-dependencies` and therefore the generated module may fail - to compile when not in test mode. To solve this, when running a unit - test (marked with `hydroflow_plus::deploy::init_test()`) that uses - trybuild, we emit a version of the `__staged` module with `#[cfg(test)]` - modules included _into the generated trybuild sources_ because we can - guarantee via trybuild that the appropriate `dev-dependencies` are - available. - - This by itself allows crates depending on `hydroflow_plus` to have local - unit tests with Hydroflow+ logic inside them. But we also want to use - this support for unit tests inside `hydroflow_plus` itself. To enable - that, we eliminate the `hydroflow_plus_deploy` crate and move its - contents directly to `hydroflow_plus` itself so that we can access the - trybuild machinery without incurring a circular dependency. - + Whenever a Hydroflow+ program is compiled, it depends on a generated + `__staged` module, which contains the entire contents of the crate but + with every type / function made `pub` and exported, so that the compiled + UDFs can resolve local references appropriately. + + Previously, we would not do this for `#[cfg(test)]` modules, since they + may use `dev-dependencies` and therefore the generated module may fail + to compile when not in test mode. To solve this, when running a unit + test (marked with `hydroflow_plus::deploy::init_test()`) that uses + trybuild, we emit a version of the `__staged` module with `#[cfg(test)]` + modules included _into the generated trybuild sources_ because we can + guarantee via trybuild that the appropriate `dev-dependencies` are + available. + + This by itself allows crates depending on `hydroflow_plus` to have local + unit tests with Hydroflow+ logic inside them. But we also want to use + this support for unit tests inside `hydroflow_plus` itself. To enable + that, we eliminate the `hydroflow_plus_deploy` crate and move its + contents directly to `hydroflow_plus` itself so that we can access the + trybuild machinery without incurring a circular dependency. + Also fixes #1408 - add API for external network inputs - This is a key step towards being able to unit-test HF+ graphs, by being + This is a key step towards being able to unit-test HF+ graphs, by being able to have controlled inputs. Outputs next. - splice UDFs with type hints to avoid inference failures @@ -50,12 +229,12 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0 - properly handle `crate::` imports - be more careful about which parts of proposer and acceptor have to be maintained atomically - adjust default features to allow compilation to musl targets - Previously, the default `deploy` feature would pull in Hydro Deploy and - its transitive native dependencies. - + Previously, the default `deploy` feature would pull in Hydro Deploy and + its transitive native dependencies. + Also sets up `examples/paxos.rs` with CLI flags to deploy to GCP. - add missing `sample_every` for singletons - Discovered during a live-coding demo, we only had it for optionals + Discovered during a live-coding demo, we only had it for optionals before. ### Refactor @@ -72,78 +251,96 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0 - simplify latency calculations - complete split into leader election and sequencing phases - start splitting out leader election into a separate module - --- - [//]: # (BEGIN SAPLING FOOTER) - Stack created with [Sapling](https://sapling-scm.com). Best reviewed - with - [ReviewStack](https://reviewstack.dev/hydro-project/hydroflow/pull/1485). - * #1493 - * #1492 - * #1489 - * #1488 - * #1487 - * #1486 + --- + [//]: # (BEGIN SAPLING FOOTER) + Stack created with [Sapling](https://sapling-scm.com). Best reviewed + with + [ReviewStack](https://reviewstack.dev/hydro-project/hydroflow/pull/1485). + * #1493 + * #1492 + * #1489 + * #1488 + * #1487 + * #1486 + * __->__ #1485 + - simplify `persist_pullup` code + Instead of matching on `&mut` and juggling ownership, instead match on + the owned node and always replaced `*node = new_node` (sometimes itself) + - use max and min in Paxos and make client generic over ballots + + start splitting out leader election into a separate module + --- + [//]: # (BEGIN SAPLING FOOTER) + Stack created with [Sapling](https://sapling-scm.com). Best reviewed + with + [ReviewStack](https://reviewstack.dev/hydro-project/hydroflow/pull/1485). + * #1493 + * #1492 + * #1489 + * #1488 + * #1487 + * #1486 * __->__ #1485 - simplify `persist_pullup` code - Instead of matching on `&mut` and juggling ownership, instead match on + Instead of matching on `&mut` and juggling ownership, instead match on the owned node and always replaced `*node = new_node` (sometimes itself) - use max and min in Paxos and make client generic over ballots ### Style - fixes for nightly clippy - a couple few spurious `too_many_arguments` and a spurious - `zombie_processes` still on current nightly (`clippy 0.1.84 (4392847410 + a couple few spurious `too_many_arguments` and a spurious + `zombie_processes` still on current nightly (`clippy 0.1.84 (4392847410 2024-10-21)`) ### New Features (BREAKING) - implicitly apply default optimizations - This also changes the behavior of `with_default_optimize` to be - terminal, if users want to apply optimizations after these they should + This also changes the behavior of `with_default_optimize` to be + terminal, if users want to apply optimizations after these they should explicitly invoke the optimizations. - add an explicit API for creating tick contexts - Previously, each location had a (semantic) global clock that drives - ticks, and so all streams in a tick domain were all in the same atomic - block. For future optimizations, we'd like developers to be able to - place streams on the same location into different clocks to eliminate - synchronization between them, which in turn would allow the computations + Previously, each location had a (semantic) global clock that drives + ticks, and so all streams in a tick domain were all in the same atomic + block. For future optimizations, we'd like developers to be able to + place streams on the same location into different clocks to eliminate + synchronization between them, which in turn would allow the computations in those separate clocks to be potentially decoupled across machines. - strongly-typed runtime cluster IDs - Instead of `u32`s everywhere, we now have a `ClusterId` type that + Instead of `u32`s everywhere, we now have a `ClusterId` type that ensures that cluster IDs are not misused. - provide an API for creating cycles across tick iterations - Towards making it more clear which parts of a program depend on ticks + Towards making it more clear which parts of a program depend on ticks versus don't. ### Refactor (BREAKING) - eliminate remaining `Hf` name prefixes - location type parameter before boundedness - When looking at a prefix in an IDE, the location type argument is + When looking at a prefix in an IDE, the location type argument is generally more useful. - dedup signatures for `Singleton` and `Optional` - Also renames `cross_singleton` to `zip` when both sides are + Also renames `cross_singleton` to `zip` when both sides are singleton-like. - fold `Tick` vs `NoTick` into the location type parameter - Now, when the location is a top-level `Process` or `Cluster` that - corresponds to a `NoTick`, and for streams inside a tick we wrap the - location type (e.g. `Tick>`). This simplifies type + Now, when the location is a top-level `Process` or `Cluster` that + corresponds to a `NoTick`, and for streams inside a tick we wrap the + location type (e.g. `Tick>`). This simplifies type signatures for a lot of our example code. - simplify intervals and split Paxos-KV into separate module - move input APIs back to being on locations - move `self_id` and `members` to be APIs on cluster instead of builder -### Commit Statistics +### `hydroflow_plus` Commit Statistics - - 38 commits contributed to the release. + - 39 commits contributed to the release. - 69 days passed between releases. - 38 commits were understood as [conventional](https://www.conventionalcommits.org). - 38 unique issues were worked on: [#1434](https://github.com/hydro-project/hydroflow/issues/1434), [#1441](https://github.com/hydro-project/hydroflow/issues/1441), [#1443](https://github.com/hydro-project/hydroflow/issues/1443), [#1444](https://github.com/hydro-project/hydroflow/issues/1444), [#1449](https://github.com/hydro-project/hydroflow/issues/1449), [#1450](https://github.com/hydro-project/hydroflow/issues/1450), [#1451](https://github.com/hydro-project/hydroflow/issues/1451), [#1453](https://github.com/hydro-project/hydroflow/issues/1453), [#1455](https://github.com/hydro-project/hydroflow/issues/1455), [#1461](https://github.com/hydro-project/hydroflow/issues/1461), [#1464](https://github.com/hydro-project/hydroflow/issues/1464), [#1468](https://github.com/hydro-project/hydroflow/issues/1468), [#1471](https://github.com/hydro-project/hydroflow/issues/1471), [#1477](https://github.com/hydro-project/hydroflow/issues/1477), [#1485](https://github.com/hydro-project/hydroflow/issues/1485), [#1486](https://github.com/hydro-project/hydroflow/issues/1486), [#1488](https://github.com/hydro-project/hydroflow/issues/1488), [#1491](https://github.com/hydro-project/hydroflow/issues/1491), [#1505](https://github.com/hydro-project/hydroflow/issues/1505), [#1515](https://github.com/hydro-project/hydroflow/issues/1515), [#1516](https://github.com/hydro-project/hydroflow/issues/1516), [#1517](https://github.com/hydro-project/hydroflow/issues/1517), [#1519](https://github.com/hydro-project/hydroflow/issues/1519), [#1521](https://github.com/hydro-project/hydroflow/issues/1521), [#1523](https://github.com/hydro-project/hydroflow/issues/1523), [#1524](https://github.com/hydro-project/hydroflow/issues/1524), [#1525](https://github.com/hydro-project/hydroflow/issues/1525), [#1526](https://github.com/hydro-project/hydroflow/issues/1526), [#1527](https://github.com/hydro-project/hydroflow/issues/1527), [#1540](https://github.com/hydro-project/hydroflow/issues/1540), [#1541](https://github.com/hydro-project/hydroflow/issues/1541), [#1542](https://github.com/hydro-project/hydroflow/issues/1542), [#1543](https://github.com/hydro-project/hydroflow/issues/1543), [#1550](https://github.com/hydro-project/hydroflow/issues/1550), [#1551](https://github.com/hydro-project/hydroflow/issues/1551), [#1553](https://github.com/hydro-project/hydroflow/issues/1553), [#1554](https://github.com/hydro-project/hydroflow/issues/1554), [#1557](https://github.com/hydro-project/hydroflow/issues/1557) -### Commit Details +### `hydroflow_plus` Commit Details @@ -225,6 +422,8 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0 - Eliminate remaining `Hf` name prefixes ([`0bd3a2d`](https://github.com/hydro-project/hydroflow/commit/0bd3a2d2230cbef24210f71a3ea83d82d1cc7244)) * **[#1557](https://github.com/hydro-project/hydroflow/issues/1557)** - Implicitly apply default optimizations ([`8d8b4b2`](https://github.com/hydro-project/hydroflow/commit/8d8b4b2288746e0aa2a95329d91297820aee7586)) + * **Uncategorized** + - Release hydroflow_lang v0.10.0, hydroflow_datalog_core v0.10.0, hydroflow_datalog v0.10.0, hydroflow_deploy_integration v0.10.0, hydroflow_macro v0.10.0, lattices_macro v0.5.7, variadics v0.0.7, variadics_macro v0.5.5, lattices v0.5.8, multiplatform_test v0.3.0, pusherator v0.0.9, hydroflow v0.10.0, hydro_deploy v0.10.0, stageleft_macro v0.4.0, stageleft v0.5.0, stageleft_tool v0.4.0, hydroflow_plus v0.10.0, hydro_cli v0.10.0, safety bump 8 crates ([`dcd48fc`](https://github.com/hydro-project/hydroflow/commit/dcd48fc7ee805898d9b5ef0d082870e30615e95b)) ## v0.9.0 (2024-08-30) @@ -238,28 +437,76 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0 ### Chore - lower min dependency versions where possible, update `Cargo.lock` - Moved from #1418 - + Moved from #1418 + --------- ### Refactor (BREAKING) - simplify process/cluster specs - --- - [//]: # (BEGIN SAPLING FOOTER) - Stack created with [Sapling](https://sapling-scm.com). Best reviewed - with - [ReviewStack](https://reviewstack.dev/hydro-project/hydroflow/pull/1394). - * #1395 + --- + [//]: # (BEGIN SAPLING FOOTER) + Stack created with [Sapling](https://sapling-scm.com). Best reviewed + with + [ReviewStack](https://reviewstack.dev/hydro-project/hydroflow/pull/1394). + * #1395 * __->__ #1394 - defer network instantiation until after finalizing IR - --- - [//]: # (BEGIN SAPLING FOOTER) - Stack created with [Sapling](https://sapling-scm.com). Best reviewed - with - [ReviewStack](https://reviewstack.dev/hydro-project/hydroflow/pull/1377). - * #1395 - * #1394 + --- + [//]: # (BEGIN SAPLING FOOTER) + Stack created with [Sapling](https://sapling-scm.com). Best reviewed + with + [ReviewStack](https://reviewstack.dev/hydro-project/hydroflow/pull/1377). + * #1395 + * #1394 + * __->__ #1377 + +### Refactor (BREAKING) + + - simplify process/cluster specs + --- + [//]: # (BEGIN SAPLING FOOTER) + Stack created with [Sapling](https://sapling-scm.com). Best reviewed + with + [ReviewStack](https://reviewstack.dev/hydro-project/hydroflow/pull/1394). + * #1395 + * __->__ #1394 + - defer network instantiation until after finalizing IR + --- + [//]: # (BEGIN SAPLING FOOTER) + Stack created with [Sapling](https://sapling-scm.com). Best reviewed + with + [ReviewStack](https://reviewstack.dev/hydro-project/hydroflow/pull/1377). + * #1395 + * #1394 + * __->__ #1377 + + simplify process/cluster specs + --- + [//]: # (BEGIN SAPLING FOOTER) + Stack created with [Sapling](https://sapling-scm.com). Best reviewed + with + [ReviewStack](https://reviewstack.dev/hydro-project/hydroflow/pull/1394). + * #1395 + * __->__ #1394 + - defer network instantiation until after finalizing IR + --- + [//]: # (BEGIN SAPLING FOOTER) + Stack created with [Sapling](https://sapling-scm.com). Best reviewed + with + [ReviewStack](https://reviewstack.dev/hydro-project/hydroflow/pull/1377). + * #1395 + * #1394 + * __->__ #1377 + + defer network instantiation until after finalizing IR + --- + [//]: # (BEGIN SAPLING FOOTER) + Stack created with [Sapling](https://sapling-scm.com). Best reviewed + with + [ReviewStack](https://reviewstack.dev/hydro-project/hydroflow/pull/1377). + * #1395 + * #1394 * __->__ #1377 ### Documentation @@ -282,37 +529,52 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0 - remove `FlowProps` - rewrite IR in place to avoid stack overflow and disable cloning - Cloning was unsafe because values behind a `Rc>` in the - case of tee would be entangled with the old IR. - --- - [//]: # (BEGIN SAPLING FOOTER) - Stack created with [Sapling](https://sapling-scm.com). Best reviewed - with - [ReviewStack](https://reviewstack.dev/hydro-project/hydroflow/pull/1404). - * #1405 - * #1398 + Cloning was unsafe because values behind a `Rc>` in the + case of tee would be entangled with the old IR. + --- + [//]: # (BEGIN SAPLING FOOTER) + Stack created with [Sapling](https://sapling-scm.com). Best reviewed + with + [ReviewStack](https://reviewstack.dev/hydro-project/hydroflow/pull/1404). + * #1405 + * #1398 + * __->__ #1404 +* #1398 +* __->__ #1404 +* #1398 +* __->__ #1404 + - rewrite IR in place to avoid stack overflow and disable cloning + Cloning was unsafe because values behind a `Rc>` in the + case of tee would be entangled with the old IR. + --- + [//]: # (BEGIN SAPLING FOOTER) + Stack created with [Sapling](https://sapling-scm.com). Best reviewed + with + [ReviewStack](https://reviewstack.dev/hydro-project/hydroflow/pull/1404). + * #1405 + * #1398 * __->__ #1404 * #1398 * __->__ #1404 - rewrite IR in place to avoid stack overflow and disable cloning - Cloning was unsafe because values behind a `Rc>` in the - case of tee would be entangled with the old IR. - --- - [//]: # (BEGIN SAPLING FOOTER) - Stack created with [Sapling](https://sapling-scm.com). Best reviewed - with - [ReviewStack](https://reviewstack.dev/hydro-project/hydroflow/pull/1404). - * #1405 - * #1398 + Cloning was unsafe because values behind a `Rc>` in the + case of tee would be entangled with the old IR. + --- + [//]: # (BEGIN SAPLING FOOTER) + Stack created with [Sapling](https://sapling-scm.com). Best reviewed + with + [ReviewStack](https://reviewstack.dev/hydro-project/hydroflow/pull/1404). + * #1405 + * #1398 * __->__ #1404 ### New Features (BREAKING) - separate singletons into their own types - refactor API to have no-tick semantics by default - Now, by default streams exist at a "top-level" where there are no ticks - and operators run over the entire collection. To perform iterative - computations, developers must explicitly entire a tick domain (using + Now, by default streams exist at a "top-level" where there are no ticks + and operators run over the entire collection. To perform iterative + computations, developers must explicitly entire a tick domain (using `tick_batch`), and return to the outer domain (using `all_ticks`). ### Refactor (BREAKING) @@ -320,50 +582,89 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0 - rename integration crates to drop CLI references - disentangle instantiated nodes from locations - simplify process/cluster specs - --- - [//]: # (BEGIN SAPLING FOOTER) - Stack created with [Sapling](https://sapling-scm.com). Best reviewed - with - [ReviewStack](https://reviewstack.dev/hydro-project/hydroflow/pull/1394). - * #1395 + --- + [//]: # (BEGIN SAPLING FOOTER) + Stack created with [Sapling](https://sapling-scm.com). Best reviewed + with + [ReviewStack](https://reviewstack.dev/hydro-project/hydroflow/pull/1394). + * #1395 + * __->__ #1394 + - defer network instantiation until after finalizing IR + --- + [//]: # (BEGIN SAPLING FOOTER) + Stack created with [Sapling](https://sapling-scm.com). Best reviewed + with + [ReviewStack](https://reviewstack.dev/hydro-project/hydroflow/pull/1377). + * #1395 + * #1394 + * __->__ #1377 + - start rearranging stages of flow compilation to prepare for trybuild approach + + simplify process/cluster specs + --- + [//]: # (BEGIN SAPLING FOOTER) + Stack created with [Sapling](https://sapling-scm.com). Best reviewed + with + [ReviewStack](https://reviewstack.dev/hydro-project/hydroflow/pull/1394). + * #1395 * __->__ #1394 - defer network instantiation until after finalizing IR - --- - [//]: # (BEGIN SAPLING FOOTER) - Stack created with [Sapling](https://sapling-scm.com). Best reviewed - with - [ReviewStack](https://reviewstack.dev/hydro-project/hydroflow/pull/1377). - * #1395 - * #1394 + --- + [//]: # (BEGIN SAPLING FOOTER) + Stack created with [Sapling](https://sapling-scm.com). Best reviewed + with + [ReviewStack](https://reviewstack.dev/hydro-project/hydroflow/pull/1377). + * #1395 + * #1394 + * __->__ #1377 + + defer network instantiation until after finalizing IR + --- + [//]: # (BEGIN SAPLING FOOTER) + Stack created with [Sapling](https://sapling-scm.com). Best reviewed + with + [ReviewStack](https://reviewstack.dev/hydro-project/hydroflow/pull/1377). + * #1395 + * #1394 * __->__ #1377 - start rearranging stages of flow compilation to prepare for trybuild approach simplify process/cluster specs - --- - [//]: # (BEGIN SAPLING FOOTER) - Stack created with [Sapling](https://sapling-scm.com). Best reviewed - with - [ReviewStack](https://reviewstack.dev/hydro-project/hydroflow/pull/1394). - * #1395 + --- + [//]: # (BEGIN SAPLING FOOTER) + Stack created with [Sapling](https://sapling-scm.com). Best reviewed + with + [ReviewStack](https://reviewstack.dev/hydro-project/hydroflow/pull/1394). + * #1395 * __->__ #1394 - defer network instantiation until after finalizing IR - --- - [//]: # (BEGIN SAPLING FOOTER) - Stack created with [Sapling](https://sapling-scm.com). Best reviewed - with - [ReviewStack](https://reviewstack.dev/hydro-project/hydroflow/pull/1377). - * #1395 - * #1394 + --- + [//]: # (BEGIN SAPLING FOOTER) + Stack created with [Sapling](https://sapling-scm.com). Best reviewed + with + [ReviewStack](https://reviewstack.dev/hydro-project/hydroflow/pull/1377). + * #1395 + * #1394 * __->__ #1377 defer network instantiation until after finalizing IR - --- - [//]: # (BEGIN SAPLING FOOTER) - Stack created with [Sapling](https://sapling-scm.com). Best reviewed - with - [ReviewStack](https://reviewstack.dev/hydro-project/hydroflow/pull/1377). - * #1395 - * #1394 + --- + [//]: # (BEGIN SAPLING FOOTER) + Stack created with [Sapling](https://sapling-scm.com). Best reviewed + with + [ReviewStack](https://reviewstack.dev/hydro-project/hydroflow/pull/1377). + * #1395 + * #1394 + * __->__ #1377 + + defer network instantiation until after finalizing IR + --- + [//]: # (BEGIN SAPLING FOOTER) + Stack created with [Sapling](https://sapling-scm.com). Best reviewed + with + [ReviewStack](https://reviewstack.dev/hydro-project/hydroflow/pull/1377). + * #1395 + * #1394 * __->__ #1377 - start rearranging stages of flow compilation to prepare for trybuild approach @@ -371,7 +672,7 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0 - rename some `CLI`->`Deploy`, decapitalize acronym names -### Commit Statistics +### `hydroflow_plus` Commit Statistics @@ -380,7 +681,7 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0 - 20 commits were understood as [conventional](https://www.conventionalcommits.org). - 20 unique issues were worked on: [#1358](https://github.com/hydro-project/hydroflow/issues/1358), [#1368](https://github.com/hydro-project/hydroflow/issues/1368), [#1375](https://github.com/hydro-project/hydroflow/issues/1375), [#1376](https://github.com/hydro-project/hydroflow/issues/1376), [#1377](https://github.com/hydro-project/hydroflow/issues/1377), [#1394](https://github.com/hydro-project/hydroflow/issues/1394), [#1395](https://github.com/hydro-project/hydroflow/issues/1395), [#1398](https://github.com/hydro-project/hydroflow/issues/1398), [#1399](https://github.com/hydro-project/hydroflow/issues/1399), [#1404](https://github.com/hydro-project/hydroflow/issues/1404), [#1405](https://github.com/hydro-project/hydroflow/issues/1405), [#1410](https://github.com/hydro-project/hydroflow/issues/1410), [#1413](https://github.com/hydro-project/hydroflow/issues/1413), [#1420](https://github.com/hydro-project/hydroflow/issues/1420), [#1421](https://github.com/hydro-project/hydroflow/issues/1421), [#1423](https://github.com/hydro-project/hydroflow/issues/1423), [#1425](https://github.com/hydro-project/hydroflow/issues/1425), [#1427](https://github.com/hydro-project/hydroflow/issues/1427), [#1428](https://github.com/hydro-project/hydroflow/issues/1428), [#1430](https://github.com/hydro-project/hydroflow/issues/1430) -### Commit Details +### `hydroflow_plus` Commit Details @@ -430,9 +731,6 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0 - Release hydroflow_lang v0.9.0, hydroflow_datalog_core v0.9.0, hydroflow_datalog v0.9.0, hydroflow_deploy_integration v0.9.0, hydroflow_macro v0.9.0, lattices_macro v0.5.6, lattices v0.5.7, multiplatform_test v0.2.0, variadics v0.0.6, pusherator v0.0.8, hydroflow v0.9.0, stageleft_macro v0.3.0, stageleft v0.4.0, stageleft_tool v0.3.0, hydroflow_plus v0.9.0, hydro_deploy v0.9.0, hydro_cli v0.9.0, hydroflow_plus_deploy v0.9.0, safety bump 8 crates ([`0750117`](https://github.com/hydro-project/hydroflow/commit/0750117de7088c01a439b102adeb4c832889f171)) - - wrong stream type for source_interval add Clone bounds to cross_join and simplify broadcast logic overly restrictive input types for send_bincode_interleavedThe original types prevented usage in cluster-to-cluster communication. - ## v0.8.0 (2024-07-23) @@ -440,18 +738,18 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0 ### Reverted - "feat(hydroflow): Added poll_futures and poll_futures_async operators.", fix #1183 - This reverts commit 997d90a76db9a4e05dbac35073a09548750ce342. - - We have been trying to figure out the semantics a bit, and want to give - it more thought before we commit to maintaining it - + This reverts commit 997d90a76db9a4e05dbac35073a09548750ce342. + + We have been trying to figure out the semantics a bit, and want to give + it more thought before we commit to maintaining it + Can un-revert and adjust the semantics later when we use it ### Refactor (BREAKING) - require lifetime on `perist*()` operators -### Commit Statistics +### `hydroflow_plus` Commit Statistics @@ -460,7 +758,7 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0 - 2 commits were understood as [conventional](https://www.conventionalcommits.org). - 3 unique issues were worked on: [#1143](https://github.com/hydro-project/hydroflow/issues/1143), [#1216](https://github.com/hydro-project/hydroflow/issues/1216), [#1295](https://github.com/hydro-project/hydroflow/issues/1295) -### Commit Details +### `hydroflow_plus` Commit Details @@ -497,7 +795,7 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0 - handle send_bincode with local structs fix(hydroflow_plus): handle send_bincode with local structs -### Commit Statistics +### `hydroflow_plus` Commit Statistics @@ -506,7 +804,7 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0 - 6 commits were understood as [conventional](https://www.conventionalcommits.org). - 6 unique issues were worked on: [#1143](https://github.com/hydro-project/hydroflow/issues/1143), [#1151](https://github.com/hydro-project/hydroflow/issues/1151), [#1156](https://github.com/hydro-project/hydroflow/issues/1156), [#1157](https://github.com/hydro-project/hydroflow/issues/1157), [#1194](https://github.com/hydro-project/hydroflow/issues/1194), [#1238](https://github.com/hydro-project/hydroflow/issues/1238) -### Commit Details +### `hydroflow_plus` Commit Details @@ -541,14 +839,14 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0 - simplify Location trait to remove lifetimes - simplify lifetime bounds for processes and clusters - feat(hydroflow_plus): simplify lifetime bounds for processes and - clusters - - This allows `extract` to move the flow builder, which is a prerequisite - for having developers run the optimizer during deployment as well in + feat(hydroflow_plus): simplify lifetime bounds for processes and + clusters + + This allows `extract` to move the flow builder, which is a prerequisite + for having developers run the optimizer during deployment as well in case it changes the network topology. - move persist manipulation into a proper optimization - feat(hydroflow_plus): move persist manipulation into a proper + feat(hydroflow_plus): move persist manipulation into a proper optimization - add APIs for running optimization passes feat(hydroflow_plus): add APIs for running optimization passes @@ -557,7 +855,7 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0 - allow BuiltFlow to be cloned even if the deploy flavor can't -### Commit Statistics +### `hydroflow_plus` Commit Statistics @@ -566,7 +864,7 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0 - 6 commits were understood as [conventional](https://www.conventionalcommits.org). - 6 unique issues were worked on: [#1083](https://github.com/hydro-project/hydroflow/issues/1083), [#1098](https://github.com/hydro-project/hydroflow/issues/1098), [#1100](https://github.com/hydro-project/hydroflow/issues/1100), [#1101](https://github.com/hydro-project/hydroflow/issues/1101), [#1107](https://github.com/hydro-project/hydroflow/issues/1107), [#1140](https://github.com/hydro-project/hydroflow/issues/1140) -### Commit Details +### `hydroflow_plus` Commit Details @@ -605,7 +903,7 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0 - use an IR before lowering to Hydroflow Makes it possible to write custom optimization passes. -### Commit Statistics +### `hydroflow_plus` Commit Statistics @@ -614,7 +912,7 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0 - 3 commits were understood as [conventional](https://www.conventionalcommits.org). - 3 unique issues were worked on: [#1070](https://github.com/hydro-project/hydroflow/issues/1070), [#1080](https://github.com/hydro-project/hydroflow/issues/1080), [#1084](https://github.com/hydro-project/hydroflow/issues/1084) -### Commit Details +### `hydroflow_plus` Commit Details @@ -680,7 +978,7 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0 - disallow joining streams on different nodes - fix spelling of "propagate" -### Commit Statistics +### `hydroflow_plus` Commit Statistics @@ -688,7 +986,7 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0 - 23 commits were understood as [conventional](https://www.conventionalcommits.org). - 20 unique issues were worked on: [#1001](https://github.com/hydro-project/hydroflow/issues/1001), [#1003](https://github.com/hydro-project/hydroflow/issues/1003), [#1004](https://github.com/hydro-project/hydroflow/issues/1004), [#1006](https://github.com/hydro-project/hydroflow/issues/1006), [#1013](https://github.com/hydro-project/hydroflow/issues/1013), [#1021](https://github.com/hydro-project/hydroflow/issues/1021), [#1022](https://github.com/hydro-project/hydroflow/issues/1022), [#1023](https://github.com/hydro-project/hydroflow/issues/1023), [#1035](https://github.com/hydro-project/hydroflow/issues/1035), [#1036](https://github.com/hydro-project/hydroflow/issues/1036), [#899](https://github.com/hydro-project/hydroflow/issues/899), [#976](https://github.com/hydro-project/hydroflow/issues/976), [#978](https://github.com/hydro-project/hydroflow/issues/978), [#981](https://github.com/hydro-project/hydroflow/issues/981), [#982](https://github.com/hydro-project/hydroflow/issues/982), [#984](https://github.com/hydro-project/hydroflow/issues/984), [#989](https://github.com/hydro-project/hydroflow/issues/989), [#991](https://github.com/hydro-project/hydroflow/issues/991), [#993](https://github.com/hydro-project/hydroflow/issues/993), [#995](https://github.com/hydro-project/hydroflow/issues/995) -### Commit Details +### `hydroflow_plus` Commit Details diff --git a/stageleft/CHANGELOG.md b/stageleft/CHANGELOG.md index 466d60124526..c9667f94b3c6 100644 --- a/stageleft/CHANGELOG.md +++ b/stageleft/CHANGELOG.md @@ -5,8 +5,94 @@ All notable changes to this project will be documented in this file. The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.0.0/), and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html). +## Unreleased + +### New Features + + - extract initial Hydroflow+ utilities into a standard library + +### Bug Fixes + + - restrict lifetime parameters to be actually invariant + Our lifetimes were accidentally made covariant when the lifetime `'a` + was removed from the process/cluster tag type. This fixes that typing + hole, and also loosens some restrictions on the lifetime of deploy + environments. + +### Refactor + + - generalize quorum logic + +### New Features (BREAKING) + + - mark non-deterministic operators as unsafe and introduce timestamped streams + Big PR. + + First big change is we introduce a `Timestamped` location. This is a bit + of a hybrid between top-level locations and `Tick` locations. The idea + is that you choose where timestamps are generated, and then have a + guarantee that everything after that will be atomically computed (useful + for making sure we add payloads to the log before ack-ing). + + The contract is that an operator or module that takes a `Timestamped` + input must still be deterministic regardless of the stamps on messages + (which are hidden unless you `tick_batch`). But unlike a top-level + stream (which has the same constraints), you have the atomicity + guarantee. Right now the guarantee is trivial since we have one global + tick for everything. But in the future when we want to apply + @davidchuyaya's optimizations this will be helpful to know when there + are causal dependencies on when data can be sent to others. + + Second change is we mark every non-deterministic operator (modulo + explicit annotations such as `NoOrder`) with Rust's `unsafe` keyword. + This makes it super clear where non-determinism is taking place. + + I've used this to put `unsafe` blocks throughout our example code and + add `SAFETY` annotations that argue why the non-determinism is safe (or + point out that we've explicitly documented / expect non-determinism). I + also added `#![warn(unsafe_op_in_unsafe_fn)]` to the examples and the + template, since this forces good hygiene of annotating sources of + non-determinism even inside a module that is intentionally + non-deterministic. + + Paxos changes are mostly refactors, and I verified that the performance + is the same as before. + - allow cluster self ID to be referenced as a global constant + This eliminates the need to store `cluster.self_id()` in a local + variable first, instead you can directly reference `CLUSTER_SELF_ID`. + +### Commit Statistics + + + + - 5 commits contributed to the release. + - 38 days passed between releases. + - 5 commits were understood as [conventional](https://www.conventionalcommits.org). + - 5 unique issues were worked on: [#1559](https://github.com/hydro-project/hydroflow/issues/1559), [#1574](https://github.com/hydro-project/hydroflow/issues/1574), [#1583](https://github.com/hydro-project/hydroflow/issues/1583), [#1584](https://github.com/hydro-project/hydroflow/issues/1584), [#1591](https://github.com/hydro-project/hydroflow/issues/1591) + +### Commit Details + + + +
view details + + * **[#1559](https://github.com/hydro-project/hydroflow/issues/1559)** + - Restrict lifetime parameters to be actually invariant ([`f6989ba`](https://github.com/hydro-project/hydroflow/commit/f6989baf12631cf43a814123e274466740c2f159)) + * **[#1574](https://github.com/hydro-project/hydroflow/issues/1574)** + - Allow cluster self ID to be referenced as a global constant ([`a93a5e5`](https://github.com/hydro-project/hydroflow/commit/a93a5e59e1681d325b3433193bb86254d23bdc77)) + * **[#1583](https://github.com/hydro-project/hydroflow/issues/1583)** + - Generalize quorum logic ([`ec55910`](https://github.com/hydro-project/hydroflow/commit/ec55910f5a41d4f08059b5feda4b96fbd058c959)) + * **[#1584](https://github.com/hydro-project/hydroflow/issues/1584)** + - Mark non-deterministic operators as unsafe and introduce timestamped streams ([`9393899`](https://github.com/hydro-project/hydroflow/commit/939389953875bf5f94ea84503a7a35efd7342282)) + * **[#1591](https://github.com/hydro-project/hydroflow/issues/1591)** + - Extract initial Hydroflow+ utilities into a standard library ([`53da4c1`](https://github.com/hydro-project/hydroflow/commit/53da4c1c9b18562e7806adcaf3a3838f56b8ef1b)) +
+ ## v0.5.0 (2024-11-08) + + + ### Chore - update pinned rust version, clippy lints, remove some dead code @@ -29,7 +115,7 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0 - - 4 commits contributed to the release. + - 5 commits contributed to the release. - 69 days passed between releases. - 4 commits were understood as [conventional](https://www.conventionalcommits.org). - 4 unique issues were worked on: [#1434](https://github.com/hydro-project/hydroflow/issues/1434), [#1444](https://github.com/hydro-project/hydroflow/issues/1444), [#1449](https://github.com/hydro-project/hydroflow/issues/1449), [#1505](https://github.com/hydro-project/hydroflow/issues/1505) @@ -48,6 +134,8 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0 - Add API for external network inputs ([`8a80931`](https://github.com/hydro-project/hydroflow/commit/8a809315cd37929687fcabc34a12042db25d5767)) * **[#1505](https://github.com/hydro-project/hydroflow/issues/1505)** - Fixes for nightly clippy ([`47cb703`](https://github.com/hydro-project/hydroflow/commit/47cb703e771f7d1c451ceb9d185ada96410949da)) + * **Uncategorized** + - Release hydroflow_lang v0.10.0, hydroflow_datalog_core v0.10.0, hydroflow_datalog v0.10.0, hydroflow_deploy_integration v0.10.0, hydroflow_macro v0.10.0, lattices_macro v0.5.7, variadics v0.0.7, variadics_macro v0.5.5, lattices v0.5.8, multiplatform_test v0.3.0, pusherator v0.0.9, hydroflow v0.10.0, hydro_deploy v0.10.0, stageleft_macro v0.4.0, stageleft v0.5.0, stageleft_tool v0.4.0, hydroflow_plus v0.10.0, hydro_cli v0.10.0, safety bump 8 crates ([`dcd48fc`](https://github.com/hydro-project/hydroflow/commit/dcd48fc7ee805898d9b5ef0d082870e30615e95b)) ## v0.4.0 (2024-08-30) diff --git a/stageleft_macro/CHANGELOG.md b/stageleft_macro/CHANGELOG.md index ba0e9b7679f2..79da25d7341b 100644 --- a/stageleft_macro/CHANGELOG.md +++ b/stageleft_macro/CHANGELOG.md @@ -5,8 +5,43 @@ All notable changes to this project will be documented in this file. The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.0.0/), and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html). +## Unreleased + +### Chore + + - use same hashing library everywhere + +### New Features (BREAKING) + + - allow cluster self ID to be referenced as a global constant + This eliminates the need to store `cluster.self_id()` in a local + variable first, instead you can directly reference `CLUSTER_SELF_ID`. + +### Commit Statistics + + + + - 2 commits contributed to the release. + - 38 days passed between releases. + - 2 commits were understood as [conventional](https://www.conventionalcommits.org). + - 2 unique issues were worked on: [#1574](https://github.com/hydro-project/hydroflow/issues/1574), [#1590](https://github.com/hydro-project/hydroflow/issues/1590) + +### Commit Details + + + +
view details + + * **[#1574](https://github.com/hydro-project/hydroflow/issues/1574)** + - Allow cluster self ID to be referenced as a global constant ([`a93a5e5`](https://github.com/hydro-project/hydroflow/commit/a93a5e59e1681d325b3433193bb86254d23bdc77)) + * **[#1590](https://github.com/hydro-project/hydroflow/issues/1590)** + - Use same hashing library everywhere ([`0dc709e`](https://github.com/hydro-project/hydroflow/commit/0dc709ed5a53c723f47fa1d10063e57bb50a63c8)) +
+ ## v0.4.0 (2024-11-08) + + ### Chore - update pinned rust version, clippy lints, remove some dead code @@ -46,7 +81,7 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0 - - 4 commits contributed to the release. + - 5 commits contributed to the release. - 69 days passed between releases. - 4 commits were understood as [conventional](https://www.conventionalcommits.org). - 4 unique issues were worked on: [#1434](https://github.com/hydro-project/hydroflow/issues/1434), [#1444](https://github.com/hydro-project/hydroflow/issues/1444), [#1445](https://github.com/hydro-project/hydroflow/issues/1445), [#1450](https://github.com/hydro-project/hydroflow/issues/1450) @@ -65,6 +100,8 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0 - Support tuple patterns ([`486dfbe`](https://github.com/hydro-project/hydroflow/commit/486dfbe1fab51f1b7c7aa03a51e6e9e4e427b912)) * **[#1450](https://github.com/hydro-project/hydroflow/issues/1450)** - Add ability to have staged flows inside unit tests ([`afe78c3`](https://github.com/hydro-project/hydroflow/commit/afe78c343658472513b34d28658634b253148aee)) + * **Uncategorized** + - Release hydroflow_lang v0.10.0, hydroflow_datalog_core v0.10.0, hydroflow_datalog v0.10.0, hydroflow_deploy_integration v0.10.0, hydroflow_macro v0.10.0, lattices_macro v0.5.7, variadics v0.0.7, variadics_macro v0.5.5, lattices v0.5.8, multiplatform_test v0.3.0, pusherator v0.0.9, hydroflow v0.10.0, hydro_deploy v0.10.0, stageleft_macro v0.4.0, stageleft v0.5.0, stageleft_tool v0.4.0, hydroflow_plus v0.10.0, hydro_cli v0.10.0, safety bump 8 crates ([`dcd48fc`](https://github.com/hydro-project/hydroflow/commit/dcd48fc7ee805898d9b5ef0d082870e30615e95b)) ## v0.3.0 (2024-08-30) diff --git a/stageleft_tool/CHANGELOG.md b/stageleft_tool/CHANGELOG.md index 8614dcf47336..464f1905fdc8 100644 --- a/stageleft_tool/CHANGELOG.md +++ b/stageleft_tool/CHANGELOG.md @@ -1,7 +1,43 @@ +## Unreleased + +### Chore + + - use same hashing library everywhere + +### New Features (BREAKING) + + - allow cluster self ID to be referenced as a global constant + This eliminates the need to store `cluster.self_id()` in a local + variable first, instead you can directly reference `CLUSTER_SELF_ID`. + +### Commit Statistics + + + + - 2 commits contributed to the release. + - 38 days passed between releases. + - 2 commits were understood as [conventional](https://www.conventionalcommits.org). + - 2 unique issues were worked on: [#1574](https://github.com/hydro-project/hydroflow/issues/1574), [#1590](https://github.com/hydro-project/hydroflow/issues/1590) + +### Commit Details + + + +
view details + + * **[#1574](https://github.com/hydro-project/hydroflow/issues/1574)** + - Allow cluster self ID to be referenced as a global constant ([`a93a5e5`](https://github.com/hydro-project/hydroflow/commit/a93a5e59e1681d325b3433193bb86254d23bdc77)) + * **[#1590](https://github.com/hydro-project/hydroflow/issues/1590)** + - Use same hashing library everywhere ([`0dc709e`](https://github.com/hydro-project/hydroflow/commit/0dc709ed5a53c723f47fa1d10063e57bb50a63c8)) +
+ ## v0.4.0 (2024-11-08) + + + ### Chore - update pinned rust version, clippy lints, remove some dead code @@ -44,7 +80,7 @@ - - 4 commits contributed to the release. + - 5 commits contributed to the release. - 69 days passed between releases. - 4 commits were understood as [conventional](https://www.conventionalcommits.org). - 4 unique issues were worked on: [#1444](https://github.com/hydro-project/hydroflow/issues/1444), [#1450](https://github.com/hydro-project/hydroflow/issues/1450), [#1486](https://github.com/hydro-project/hydroflow/issues/1486), [#1527](https://github.com/hydro-project/hydroflow/issues/1527) @@ -63,6 +99,8 @@ - Complete split into leader election and sequencing phases ([`8b7b1c6`](https://github.com/hydro-project/hydroflow/commit/8b7b1c60fd33b78f9a4b0873bbbd150260ae2ad5)) * **[#1527](https://github.com/hydro-project/hydroflow/issues/1527)** - Properly handle `crate::` imports ([`2faffdb`](https://github.com/hydro-project/hydroflow/commit/2faffdbf2cc886da22e496df64f46aefa380766c)) + * **Uncategorized** + - Release hydroflow_lang v0.10.0, hydroflow_datalog_core v0.10.0, hydroflow_datalog v0.10.0, hydroflow_deploy_integration v0.10.0, hydroflow_macro v0.10.0, lattices_macro v0.5.7, variadics v0.0.7, variadics_macro v0.5.5, lattices v0.5.8, multiplatform_test v0.3.0, pusherator v0.0.9, hydroflow v0.10.0, hydro_deploy v0.10.0, stageleft_macro v0.4.0, stageleft v0.5.0, stageleft_tool v0.4.0, hydroflow_plus v0.10.0, hydro_cli v0.10.0, safety bump 8 crates ([`dcd48fc`](https://github.com/hydro-project/hydroflow/commit/dcd48fc7ee805898d9b5ef0d082870e30615e95b)) ## v0.3.0 (2024-08-30)