stageleft v0.1.0

Documentation

• actually committing empty CHANGELOG.md is required

New Features

• push down persists and implement Pi example
  Also fixes type inference issues with reduce the same way as we did for fold.
• support building graphs for symmetric clusters in Hydroflow+
• support crates that have no entrypoints
  Also includes various bugfixes needed for Hydroflow+.
• add a functional surface syntax using staging

Bug Fixes

• fix include! path separators on windows

Commit Statistics

• 6 commits contributed to the release over the course of 76 calendar days.
• 6 commits were understood as conventional.
• 5 unique issues were worked on: #1010, #1021, #899, #983, #984

Commit Details

view details

• #1010
  • Fix include! path separators on windows (8df66f8)
• #1021
  • Push down persists and implement Pi example (af6e3be)
• #899
  • Add a functional surface syntax using staging (8b63568)
• #983
  • Support crates that have no entrypoints (7108323)
• #984
  • Support building graphs for symmetric clusters in Hydroflow+ (174607d)
• Uncategorized
  • Actually committing empty CHANGELOG.md is required (3b36020)

pusherator v0.0.4

Chore

• manually set lockstep-versioned crates (and lattices) to version 0.5.1
  Setting manually since
  frewsxcv/rust-crates-index#159 is messing with
  smart-release

Commit Statistics

• 1 commit contributed to the release.
• 167 days passed between releases.
• 1 commit was understood as conventional.
• 0 issues like '(#ID)' were seen in commit messages

Commit Details

view details

• Uncategorized
  • Manually set lockstep-versioned crates (and lattices) to version 0.5.1 (1b555e5)

lattices v0.5.1

Chore

• manually set lockstep-versioned crates (and lattices) to version 0.5.1
  Setting manually since
  frewsxcv/rust-crates-index#159 is messing with
  smart-release
• fix clippy lints on latest nightly
• fix clippy::items_after_test_module, simplify rustdoc links

New Features

• Add DeepReveal trait
• add set_union_with_tombstones

Bug Fixes

• chat and two_pc no longer replay

Commit Statistics

• 7 commits contributed to the release over the course of 109 calendar days.
• 110 days passed between releases.
• 6 commits were understood as conventional.
• 4 unique issues were worked on: #1032, #942, #960, #967

Commit Details

view details

• #1032
  • Fixup! feat(lattices): Add DeepReveal trait (0bed8ca)
  • Add DeepReveal trait (e30602e)
• #942
  • Fix clippy::items_after_test_module, simplify rustdoc links (f6a7299)
• #960
  • Fix clippy lints on latest nightly (ba6afab)
• #967
  • Chat and two_pc no longer replay (0539e2a)
• Uncategorized
  • Manually set lockstep-versioned crates (and lattices) to version 0.5.1 (1b555e5)
  • Add set_union_with_tombstones (3f70199)

hydroflow_plus v0.5.1

Chore

• manually set lockstep-versioned crates (and lattices) to version 0.5.1
  Setting manually since
  frewsxcv/rust-crates-index#159 is messing with
  smart-release

Documentation

• actually committing empty CHANGELOG.md is required

New Features

• add core negation operators

• add APIs for declaring external ports on clusters

• implement keyed fold and reduce

• add interleaved shortcut when sending from a cluster

• push down persists and implement Pi example
  Also fixes type inference issues with reduce the same way as we did for fold.

• improve API naming and polish docs

• tweak naming of windowing operators

• provide simpler API for launching and minimize dependencies

• improve API naming and eliminate wire API for builders

• split API for building single-node graphs

• require explicit batching for aggregation operators

• support building graphs for symmetric clusters in Hydroflow+

• auto-configure Hydro Deploy based on Hydroflow+ plans

• add preliminary send_to operator for multi-node graphs

• add initial test using Hydro CLI from Hydroflow+
  This also required a change to Hydroflow core to make it possible to run the dataflow itself on a single thread (using a LocalSet), even if the surrounding runtime is not single-threaded (required to work around deadlocks because we can't use async APIs inside Hydroflow+). This requires us to spawn any Hydroflow tasks (only for dest_sink at the moment) right next to when we run the dataflow rather than when the Hydroflow graph is initialized. From a conceptual perspective, this seems more right, since now creating a Hydroflow program will not result in any actual tasks running.

  In the third PR of this series, I aim to add a new Hydroflow+ operator that will automate the setup of a dest_sink/source_stream pair that span nodes.

• allow Hydroflow+ programs to emit multiple graphs
  This PR adds support for tagging elements of Hydroflow+ graphs with a node ID, an integer which specifies which Hydroflow graph the computation should be emitted to. The generated code includes the Hydroflow graph for each node ID, so that the appropriate graph can be selected at runtime.

  At a larger scale, this is a precursor to adding network operators to Hydroflow+, which will allow distributed logic to be described in a single Hydroflow+ program by specifying points at which data is transferred between different graphs.

• add a functional surface syntax using staging

Bug Fixes

• persist cluster IDs for broadcast
  I'll follow this up with a unit test for this, but want to get this fixed ASAP first.
• rewrite uses of alloc crate in bincode operators
• disallow joining streams on different nodes
• fix spelling of "propagate"

Commit Statistics

• 24 commits contributed to the release over the course of 76 calendar days.
• 23 commits were understood as conventional.
• 20 unique issues were worked on: #1001, #1003, #1004, #1006, #1013, #1021, #1022, #1023, #1035, #1036, #899, #976, #978, #981, #982, #984, #989, #991, #993, #995

Commit Details

view details

• #1001
  • Disallow joining streams on different nodes (2addaed)
• #1003
  • Provide simpler API for launching and minimize dependencies (1edc5ae)
• #1004
  • Rewrite uses of alloc crate in bincode operators (bd2bf23)
• #1006
  • Tweak naming of windowing operators (44a308f)
• #1013
  • Improve API naming and polish docs (6eeb9be)
• #1021
  • Push down persists and implement Pi example (af6e3be)
• #1022
  • Add interleaved shortcut when sending from a cluster (5e6ebac)
• #1023
  • Implement keyed fold and reduce (73e9b68)
• #1035
  • Persist cluster IDs for broadcast (88a1796)
• #1036
  • Add core negation operators (5a03ed4)
• #899
  • Add a functional surface syntax using staging (8b63568)
• #976
  • Allow Hydroflow+ programs to emit multiple graphs (05fb135)
• #978
  • Add initial test using Hydro CLI from Hydroflow+ (e5bdd12)
• #981
  • Add preliminary send_to operator for multi-node graphs (27dabcf)
• #982
  • Auto-configure Hydro Deploy based on Hydroflow+ plans (9e27582)
• #984
  • Support building graphs for symmetric clusters in Hydroflow+ (174607d)
• #989
  • Fix spelling of "propagate" (38411ea)
• #991
  • Require explicit batching for aggregation operators (26f4d6f)
• #993
  • Split API for building single-node graphs (d288e51)
• #995
  • Improve API naming and eliminate wire API for builders (b7aafd3)
• Uncategorized
  • Release stageleft_macro v0.1.0, stageleft v0.1.0, hydroflow_plus v0.5.1 (1a48db5)
  • Actually committing empty CHANGELOG.md is required (3b36020)
  • Manually set lockstep-versioned crates (and lattices) to version 0.5.1 (1b555e5)
  • Add APIs for declaring external ports on clusters (7d930a2)

hydroflow_macro v0.5.1

Chore

• manually set lockstep-versioned crates (and lattices) to version 0.5.1
  Setting manually since
  frewsxcv/rust-crates-index#159 is messing with
  smart-release

New Features

• new implementation and Hydro Deploy setup

Commit Statistics

• 2 commits contributed to the release over the course of 58 calendar days.
• 110 days passed between releases.
• 2 commits were understood as conventional.
• 1 unique issue was worked on: #909

Commit Details

view details

• #909
  • New implementation and Hydro Deploy setup (6158a7a)
• Uncategorized
  • Manually set lockstep-versioned crates (and lattices) to version 0.5.1 (1b555e5)

hydroflow_lang v0.5.1

Chore

• manually set lockstep-versioned crates (and lattices) to version 0.5.1
  Setting manually since
  frewsxcv/rust-crates-index#159 is messing with
  smart-release
• fix clippy lints on latest nightly

New Features

• add core negation operators

• add checking of input edge types via OperatorConstraints::input_edgetype_fn

• add OperatorConstraints::input_edgetype_fn to validate input ref/val edges

• add operator edge type tracking into meta graph

• add OperatorConstraints::output_edgetype_fn to enable reference edges

• implement keyed fold and reduce

• push down persists and implement Pi example
  Also fixes type inference issues with reduce the same way as we did for fold.

• Add initial structure for by-reference edge types

• add initial test using Hydro CLI from Hydroflow+
  This also required a change to Hydroflow core to make it possible to run the dataflow itself on a single thread (using a LocalSet), even if the surrounding runtime is not single-threaded (required to work around deadlocks because we can't use async APIs inside Hydroflow+). This requires us to spawn any Hydroflow tasks (only for dest_sink at the moment) right next to when we run the dataflow rather than when the Hydroflow graph is initialized. From a conceptual perspective, this seems more right, since now creating a Hydroflow program will not result in any actual tasks running.

  In the third PR of this series, I aim to add a new Hydroflow+ operator that will automate the setup of a dest_sink/source_stream pair that span nodes.

• add a functional surface syntax using staging

• add defer() which is the same as defer_tick() except that it is lazy

Bug Fixes

• typo

• lattice ops are monotone and return stream of lattices

  • fix: lattice ops are monotone and return stream of lattices

  • fix: remove trailing whitespace

  • fix: remove warnings from unused tees in topolotree

  • fix: avoid error message in test

  • fix: set flow_prop_fn properly in _lattice_fold_batch.rs

  • fix: test_lattice_join_fused_join_map_union now checks assertions

• improve type inference for fold accumulators

• clippy lints on latest nightly

• fix spelling of "propagate"

• multiset_delta incorrect is_first_run_this_tick check, fixes #958
  Introduced in #906

  Also adds more multiset_delta tests.

• 2 nested module imports bugs
  The first bug is that when importing nested modules, when the flat
  graphs are merged together, they did not always attach to the correct
  input and output module boundaries.

  The second bug is that imports inside of modules were not relative to
  the module file, but they are now.

• avoid panic-ing on degen null()

• fix/improve rendering with --no-handoffs and double-labelled edges

Refactor

• unify node coloring code
• emit prologue code before all subgraph code
  Before, prologue code would be emitted before its subgraph, resulting in
  interleaving between subgraphs.
• remove old unused structured FlowProperties

Bug Fixes (BREAKING)

• fold takes initial value by closure rather than by value

Commit Statistics

• 27 commits contributed to the release over the course of 104 calendar days.
• 110 days passed between releases.
• 26 commits were understood as conventional.
• 20 unique issues were worked on: #1005, #1009, #1016, #1017, #1021, #1023, #1026, #1033, #1036, #1040, #899, #945, #947, #948, #949, #950, #959, #960, #978, #989

Commit Details

view details

• #1005
  • Improve type inference for fold accumulators (f0a0378)
• #1009
  • Clippy lints on latest nightly (d0b0a35)
• #1016
  • Add initial structure for by-reference edge types (a0af314)
• #1017
  • Fixup! feat(hydroflow_lang): add OperatorConstraints::input_edgetype_fn to validate input ref/val edges (f079b85)
  • Add checking of input edge types via OperatorConstraints::input_edgetype_fn (355bcd1)
  • Typo (a67f43f)
  • Add OperatorConstraints::input_edgetype_fn to validate input ref/val edges (e61d22d)
  • Add operator edge type tracking into meta graph (67c4195)
  • Add OperatorConstraints::output_edgetype_fn to enable reference edges (cdbc433)
• #1021
  • Push down persists and implement Pi example (af6e3be)
• #1023
  • Implement keyed fold and reduce (73e9b68)
• #1026
  • Lattice ops are monotone and return stream of lattices (5ed9be4)
• #1033
  • Emit prologue code before all subgraph code (1a80f1c)
• #1036
  • Add core negation operators (5a03ed4)
• #1040
  • Unify node coloring code (2b0a667)
• #899
  • Add a functional surface syntax using staging (8b63568)
• #945
  • Add defer() which is the same as defer_tick() except that it is lazy (7df0a0d)
• #947
  • Remove old unused structured FlowProperties (ff4bddd)
• #948
  • Fold takes initial value by closure rather than by value (3136e0f)
• #949
  • Fix/improve rendering with --no-handoffs and double-labelled edges (8ef14a3)
• #950
  • Avoid panic-ing on degen null() (35b1e9e)
• #959
  • multiset_delta incorrect is_first_run_this_tick check, fixes #958 (43280cb)
• #960
  • Fix clippy lints on latest nightly (ba6afab)
• #978
  • Add initial test using Hydro CLI from Hydroflow+ (e5bdd12)
• #989
  • Fix spelling of "propagate" (38411ea)
• Uncategorized
  • Manually set lockstep-versioned crates (and lattices) to version 0.5.1 (1b555e5)
  • 2 nested module imports bugs (f89d11a)

hydroflow_datalog_core v0.5.1

Chore

• manually set lockstep-versioned crates (and lattices) to version 0.5.1
  Setting manually since
  frewsxcv/rust-crates-index#159 is messing with
  smart-release

New Features

• implement keyed fold and reduce
• Add initial structure for by-reference edge types
• add defer() which is the same as defer_tick() except that it is lazy

Bug Fixes

• fix spelling of "propagate"

Refactor

• emit prologue code before all subgraph code
  Before, prologue code would be emitted before its subgraph, resulting in
  interleaving between subgraphs.

Commit Statistics

• 6 commits contributed to the release over the course of 103 calendar days.
• 110 days passed between releases.
• 6 commits were understood as conventional.
• 5 unique issues were worked on: #1016, #1023, #1033, #945, #989

Commit Details

view details

• #1016
  • Add initial structure for by-reference edge types (a0af314)
• #1023
  • Implement keyed fold and reduce (73e9b68)
• #1033
  • Emit prologue code before all subgraph code (1a80f1c)
• #945
  • Add defer() which is the same as defer_tick() except that it is lazy (7df0a0d)
• #989
  • Fix spelling of "propagate" (38411ea)
• Uncategorized
  • Manually set lockstep-versioned crates (and lattices) to version 0.5.1 (1b555e5)

hydroflow_datalog v0.5.1

Chore

• manually set lockstep-versioned crates (and lattices) to version 0.5.1
  Setting manually since
  frewsxcv/rust-crates-index#159 is messing with
  smart-release

Commit Statistics

• 1 commit contributed to the release.
• 110 days passed between releases.
• 1 commit was understood as conventional.
• 0 issues like '(#ID)' were seen in commit messages

Commit Details

view details

• Uncategorized
  • Manually set lockstep-versioned crates (and lattices) to version 0.5.1 (1b555e5)

hydroflow_cli_integration v0.5.1

New Features

• auto-configure Hydro Deploy based on Hydroflow+ plans

• add initial test using Hydro CLI from Hydroflow+
  This also required a change to Hydroflow core to make it possible to run the dataflow itself on a single thread (using a LocalSet), even if the surrounding runtime is not single-threaded (required to work around deadlocks because we can't use async APIs inside Hydroflow+). This requires us to spawn any Hydroflow tasks (only for dest_sink at the moment) right next to when we run the dataflow rather than when the Hydroflow graph is initialized. From a conceptual perspective, this seems more right, since now creating a Hydroflow program will not result in any actual tasks running.

  In the third PR of this series, I aim to add a new Hydroflow+ operator that will automate the setup of a dest_sink/source_stream pair that span nodes.

• split Rust core from Python bindings

Pre-Move Commit Statistics

• 2 commits contributed to the release over the course of 3 calendar days.
• 169 days passed between releases.
• 2 commits were understood as conventional.
• 2 unique issues were worked on: #978, #982

Chore

• manually set lockstep-versioned crates (and lattices) to version 0.5.1
  Setting manually since
  frewsxcv/rust-crates-index#159 is messing with
  smart-release
• generate pre-move changelogs for hydro_cli and hydroflow_cli_integration

Pre-Move Commit Details

view details

• #978
  • Add initial test using Hydro CLI from Hydroflow+ (e5bdd12)
• #982
  • Auto-configure Hydro Deploy based on Hydroflow+ plans (9e27582)

Commit Statistics

• 3 commits contributed to the release over the course of 39 calendar days.
• 209 days passed between releases.
• 3 commits were understood as conventional.
• 2 unique issues were worked on: #1046, #986

Commit Details

view details

• #1046
  • Generate pre-move changelogs for hydro_cli and hydroflow_cli_integration (69e0416)
• #986
  • Split Rust core from Python bindings (c50ca12)
• Uncategorized
  • Manually set lockstep-versioned crates (and lattices) to version 0.5.1 (1b555e5)

hydroflow v0.5.1

Chore

• manually set lockstep-versioned crates (and lattices) to version 0.5.1
  Setting manually since
  frewsxcv/rust-crates-index#159 is messing with
  smart-release
• fix clippy lints on latest nightly
• fix clippy::items_after_test_module, simplify rustdoc links
• update trybuild output for nightly bump

New Features

• add core negation operators

• provide simpler API for launching and minimize dependencies

• pass subgraph ID through deploy metadata

• split Rust core from Python bindings

• support building graphs for symmetric clusters in Hydroflow+

• add initial test using Hydro CLI from Hydroflow+
  This also required a change to Hydroflow core to make it possible to run the dataflow itself on a single thread (using a LocalSet), even if the surrounding runtime is not single-threaded (required to work around deadlocks because we can't use async APIs inside Hydroflow+). This requires us to spawn any Hydroflow tasks (only for dest_sink at the moment) right next to when we run the dataflow rather than when the Hydroflow graph is initialized. From a conceptual perspective, this seems more right, since now creating a Hydroflow program will not result in any actual tasks running.

  In the third PR of this series, I aim to add a new Hydroflow+ operator that will automate the setup of a dest_sink/source_stream pair that span nodes.

• new implementation and Hydro Deploy setup

• add a functional surface syntax using staging

• add left,right,outer join module examples

• add defer() which is the same as defer_tick() except that it is lazy

Bug Fixes

• lattice ops are monotone and return stream of lattices

  • fix: lattice ops are monotone and return stream of lattices

  • fix: remove trailing whitespace

  • fix: remove warnings from unused tees in topolotree

  • fix: avoid error message in test

  • fix: set flow_prop_fn properly in _lattice_fold_batch.rs

  • fix: test_lattice_join_fused_join_map_union now checks assertions

• re-export hydroflow_macro::DemuxEnum derive macro

• remove stale instructions about python driver for chat

• improve type inference for fold accumulators

• add 100ms wait to chat example to avoid dropped packet

• chat and two_pc no longer replay

• multiset_delta incorrect is_first_run_this_tick check, fixes #958
  Introduced in #906

  Also adds more multiset_delta tests.

• avoid panic-ing on degen null()

Refactor

• Update surface_lattice_join tests to use DeepReveal
• Improvements prepping for release
  • Adds the "spread"/"splat" ... syntax to the three variadics macros.
  • Adds #[sealed] traits.
  • Adds testing of error messages.
  • Improves docs: README.md and Rust docs.
• remove intrinsics from KVS bench, fixes warning

Bug Fixes (BREAKING)

• fold takes initial value by closure rather than by value

Commit Statistics

• 26 commits contributed to the release over the course of 109 calendar days.
• 110 days passed between releases.
• 26 commits were understood as conventional.
• 23 unique issues were worked on: #1003, #1005, #1024, #1025, #1026, #1032, #1036, #899, #909, #942, #945, #948, #950, #959, #960, #967, #971, #974, #978, #979, #984, #986, #996

Commit Details

view details

• #1003
  • Provide simpler API for launching and minimize dependencies (1edc5ae)
• #1005
  • Improve type inference for fold accumulators (f0a0378)
• #1024
  • Remove stale instructions about python driver for chat (01dec51)
• #1025
  • Re-export hydroflow_macro::DemuxEnum derive macro (45452ae)
• #1026
  • Lattice ops are monotone and return stream of lattices (5ed9be4)
• #1032
  • Update surface_lattice_join tests to use DeepReveal (26818be)
• #1036
  • Add core negation operators (5a03ed4)
• #899
  • Add a functional surface syntax using staging (8b63568)
• #909
  • New implementation and Hydro Deploy setup (6158a7a)
• #942
  • Fix clippy::items_after_test_module, simplify rustdoc links (f6a7299)
  • Update trybuild output for nightly bump (6bf846f)
• #945
  • Add defer() which is the same as defer_tick() except that it is lazy (7df0a0d)
• #948
  • Fold takes initial value by closure rather than by value (3136e0f)
• #950
  • Avoid panic-ing on degen null() (35b1e9e)
• #959
  • multiset_delta incorrect is_first_run_this_tick check, fixes #958 (43280cb)
• #960
  • Fix clippy lints on latest nightly (ba6afab)
• #967
  • Chat and two_pc no longer replay (0539e2a)
• #971
  • Add 100ms wait to chat example to avoid dropped packet (bc35a5a)
• #974
  • Improvements prepping for release (7e65a08)
• #978
  • Add initial test using Hydro CLI from Hydroflow+ (e5bdd12)
• #979
  • Remove intrinsics from KVS bench, fixes warning (7e1eae6)
• #984
  • Support building graphs for symmetric clusters in Hydroflow+ (174607d)
• #986
  • Split Rust core from Python bindings (c50ca12)
• #996
  • Pass subgraph ID through deploy metadata (46d87fa)
• Uncategorized
  • Manually set lockstep-versioned crates (and lattices) to version 0.5.1 (1b555e5)
  • Add left,right,outer join module examples (f327b02)