Multiple stable memory regions (and a new primitive type Region).

design/StableRegions-20230209.md

@@ -0,0 +1,222 @@
+# Stable Region Allocator Design
+
+- 20230209
+- Matthew Hammer
+
+## Purpose of this document
+
+This document gives a design for a new “stable region” allocator for Motoko’s runtime system, and accompanying `base` library.
+
+The role of this feature is summarized in the next section,
+followed by design considerations and details.
+
+
+## Role for "stable regions" in Motoko
+
+The current stable memory module in `base` has been "experimental" for a long time, and requires a more composable API to graduate from this status.
+
+Stable regions address the problem that today's `ExperimentalStableMemory` module only provides a single, monolithic memory that makes it unsuitable for directly building composable software parts.
+
+Stable regions permit a new API that supports composable use cases.
+
+Stable regions also bring Motoko closer to parity with Rust canister development support today, by giving a run-time-system-based analog of a special Rust library for stable data structures that allocates “pages” for them from stable memory in separate memory regions.
+
+
+## Design space
+
+The design space for the page allocator is defined by at least two
+tensions:
+
+ 1. fully-stable representation of allocator meta data **versus** fast load/store operations.
+
+ 2. Total scaling capacity **versus** minimum footprint for meta data.
+
+
+**Tension 1** is introduced because we want to avoid relying on the Motoko heap as the "ground truth" about the allocator's state.  If this heap is lost, as it is during an upgrade, then a developer may still want to recover all of the regions' data and meta data.
+
+On the other hand, during ordinary canister execution, ~we *do* want to rely on the heap (not stable memory) for meta data to avoid its higher access costs for load and store operations, and thus we need meta data in two places, both heap and stable memory.~
+
+Tension 1 is resolved by ~storing certain meta data twice, just as with the Rust implementation that serves as our basis.~ storing the data only in stable memory, and relying on those accesses becoming faster in the near future. (looking into the near-future roadmap for canister stable memory, it seems this is very likely.)
+
+Compared with the Rust version, we store enough extra meta data to permit:
+
+ - Regions whose page blocks are in arbitrary order, not
+   necessarily in order of smallest to highest address.
+
+ - 32767 Regions max (instead of 255 Regions max).
+
+We want to permit arbitrary page block orders to make a smooth
+transition to region reclamation and re-allocation in the near
+future, with potential integration into the Motoko GC.  The
+extra complexity is modest, and seems "worth" the cost.
+
+We change the maximum region limit because 255 may be too small in
+some extreme cases.
+
+We address the question of whether the new limit of 32k regions is
+"enough" in the Q&A section (it is, for all practical purposes)
+
+
+**Tension 2** is introduced because we want a design that will continue
+to work even when canisters can store more stable data than today (32GB).
+
+Tension 2 is resolved by making prudent representation choices.
+
+The representations we choose for regions and region identifiers
+permit a scaling to 256GB of stable data while still permitting meta
+data to be repeated in both stable and non-stable arenas.  These are
+the same limits imposed by the Rust implementation, for the same
+reasons.  See Q&A for more discussion.
+
+
+
+## Questions and answers
+
+### Q: What determines the 8MB non-empty region minimum?
+
+This size comes from wanting to grow a region by more than one page at
+a time (in terms of the way that the canister interacts with the
+system API, at least).  Rather than actually grow by a single page,
+the implementation grows by a "page block" (8MB) at a time.
+
+This choice means that there are 128 pages per page block, and that
+the maximum number of regions and region blocks are each relatively
+small (32k each).  Consequently, they can each be identified with a
+2-byte identifier, and we can pre-allocate tables to store certain
+relations about them, which is critical.
+
+### Q: Are 32767 regions enough?
+
+A: Permitting more than 32k regions may seem theoretically
+interesting, but is not practical given other parameters today that
+control the minimal footprint of a region (8MB) and dictate the
+maximum size of stable memory for a canister today (32GB).  With 32k
+regions at 8MB each, well over the maximum stable memory size is used
+(256GB compared to 32GB, the limit today)
+
+### Q: When is stable memory becoming less costly?
+
+TBA
+
+### Q: How does the cheaper stable memory access cost compare with ordinary heap memory access cost?
+
+TBA
+
+
+## Design details
+
+### API
+
+Internal region allocator operations:
+
+
+ - `initialize` -- called by the RTS, not by the Motoko developer.
+ - `rebuild` -- also called by the RTS, during an post-upgrade.
+
+
+User-facing region allocator operations:
+
+ - `region_new` -- create a dynamic region.
+ - `region_grow` -- grow region by a specified number of pages.
+ - `region_load` -- read some data from the region.
+ - `region_store` -- store some data into the region.
+
+And a special operation, for testing our design for future GC integration (bonus):
+
+- `region_release` -- release region and reuse region's page blocks.
+
+The `_release` operation is *not* part of the user-facing API nor part of the MVP,
+but supporting it is important because it means we can transition quickly to an integration
+with the ambient Motoko GC if we can support it.
+
+## Internal footprint
+
+The state of the allocator is stored in a combination of stable memory and heap memory.
+
+The stable memory state is sufficient to reconstitute the heap objects
+(`rebuild` operation, described in a subsection below).  That means
+that even if the heap is lost (like during an upgrade that fails) the
+stable memory state can fully describe the parts that should be
+rebuilt into the heap when the upgrade succeeds.
+
+### stable memory fields
+
+ - total allocated blocks, `Nat16`, max value is `32768`.
+ - total allocated regions, `Nat16`, max value is `32767` (one region is reserved for "no region").
+ - The `block-region` table (fixed size, about 131kb).
+ - The `region-blocks` table (fixed size, about 524kb).
+
+### representation of values of type `Region`
+
+ - We use the region ID, a `Nat16`, to represent the region as a value.
+ - To compute a load or store access, this requires consulting stable memory for (almost all) meta data.
+ - We rely on the cost of these stable memory accesses going down in the near future to support this design decision.
+
+### block-region table
+
+ - purpose:
+   - relate a block ID ("page block ID") to its region (if any) and its position in that region (see `rebuild`).
+
+   - NB: The organization of this table is not useful for efficient
+     access calculations of load or store (they require a linear
+     search that would be prohibitively slow). Rather, the
+     `region-blocks` table is designed for that purpose.  OTOH, this
+     table is suitable to do a "batch" rebuild of the dynamic heap-allocated vectors
+     in that table, which get lost during upgrades.
+
+ - 32768 entries (statically sized).
+ - 4 bytes per entry.
+ - entry type = `BlockRegion { region : Nat16, position : Nat16 }`
+ - the location of each entry gives its corresponding block ID.
+
+### region-blocks table
+
+ - purpose: 
+   - relate a region ID to its vector of block IDs, to compute an access efficiently (load or store).
+
+   - NB: The organization of this table is meant to support O(1) load
+     and store operations, but in so doing, it needs to use a
+     dynamically-sized vector for each region.  There is no a priori
+     way to know how to allocate these, and if we naively preallocate
+     them to each be the potential "maximal region" (with all blocks
+     allocated to it), the resulting preallocated table requires
+     gigabytes of space, and thus is prohibitively large.
+
+     At the same time, having dynamically-sized stable vectors is kind
+     of the point of regions (each is such a vector, in a sense), and
+     so requiring these first to implement regions' meta data seems
+     potentially circular in concept, if not also very complex.
+
+     So, we seem forced to use dynamically-sized heap vectors, and to
+     deal the compilations of integrating them with GC (how involved is
+     that?) and rebuilding them on upgrade.
+
+
+ - 32768 entries (statically sized, ignoring the `vec_ptr` objects referenced therein).
+ - 16 bytes per entry.
+ - entry type = `RegionBlocks { size_in_pages: Nat64; vec_capacity : Nat32; vec_ptr: Nat32 }`
+ - the location of each entry in the table gives its corresponding region ID.
+ - the `size_in_pages` field measures the size of the region in terms of _pages_, not blocks. 
+ - the `vec_capacity` and `vec_ptr` fields work with `size_in_pages`
+   to represent a growable vector that we call the region's **"access
+   vector"** (because "blocks vector" sounds a bit strange, and its
+   used to support O(1) access operations): 
+   - the access vector's address is held in `vec_ptr` and it has `vec_capacity` slots.
+   - the first `size_in_pages / 128` slots of `vec_ptr` contain a valid page block ID for the region.
+   - to support dynamic growth, _the access vector is held in the Motoko **heap** rather than **stable memory**._
+   - the access vector doubles when it grows.
+   - no region has more than 32k page blocks, so a `Nat16` suffices for `capacity`,
+   - but we use a `Nat32` for `capacity` to make all fields things word-aligned (does that matter?).
+   - during an upgrade, the access vectors are "rebuilt" in a batch (see `rebuild`).
+
+
+### Overview of `rebuild`
+
+When a canister upgrades, its heap is lost and the access vectors for each region are lost too (see `region-blocks` table section).
+
+We use the `block-region` and `region-blocks` tables in stable memory to rebuild these vectors in the latter table:
+
+ - The `region-blocks` table gives a size for each region, saying how large each vector for each region needs to be.
+ - The `block-region` table gives a relative position and region ID for each block ID.
+
+Once each regions' vectors have been allocated, the block-region table says how to fill them, one entry at a time.

design/StableRegions.md

@@ -0,0 +1,23 @@
+# Stable Region API
+
+See StableMemory.md for context of the current experimental API.
+
+This document aims to specify the API and memory representations for a generalization
+of this API that permits multiple _regions_ of stable memory, where each can be
+grown independently.
+
+The **region manager** is the state and logic to support this generalization.
+
+## Definitions and constants
+
+ - a **page** is 65536 bytes.
+ - a **page block** is a contiguous sequence of 128 pages (~8MB).
+ - a **page block index** is a 16 bit, index-based identifier for a page block.
+ - a **region** is a sequence of (generally non-contiguous) **page blocks**.
+ - the maximum number of page blocks is 32768.
+ - the maximum amount of stable memory for all regions is 256GB.
+
+
+
+
+###### TO DO -- finish this document.

rts/motoko-rts/src/lib.rs

@@ -34,6 +34,7 @@ mod tommath_bindings;
 pub mod types;
 pub mod utf8;
 mod visitor;
+pub mod region;
 
 use types::Bytes;
 

rts/motoko-rts/src/region.rs

@@ -0,0 +1,30 @@
+use crate::memory::{Memory};
+use crate::rts_trap_with;
+use crate::types::{Value};
+
+use motoko_rts_macros::ic_mem_fn;
+
+#[ic_mem_fn]
+pub unsafe fn region_new<M: Memory>(_mem: &mut M) -> Value {
+    rts_trap_with("TODO region_new");
+}
+
+#[ic_mem_fn]
+pub unsafe fn region_size<M: Memory>(_mem: &mut M, _r: Value) -> Value {
+    rts_trap_with("TODO region_grow");
+}
+
+#[ic_mem_fn]
+pub unsafe fn region_grow<M: Memory>(_mem: &mut M, _r: Value, _new_pages: Value) -> Value {
+    rts_trap_with("TODO region_grow");
+}
+
+#[ic_mem_fn]
+pub unsafe fn region_load_blob<M: Memory>(_mem: &mut M, _r: Value, _start: Value, _len: Value) -> Value {
+    rts_trap_with("TODO region_load_blob");
+}
+
+#[ic_mem_fn]
+pub unsafe fn region_store_blob<M: Memory>(_mem: &mut M, _r: Value, _start: Value, _blob: Value) {
+    rts_trap_with("TODO region_store_blob");
+}

rts/motoko-rts/src/types.rs

@@ -374,6 +374,8 @@ pub const TAG_CONCAT: Tag = 25;
 pub const TAG_NULL: Tag = 27;
 pub const TAG_ONE_WORD_FILLER: Tag = 29;
 pub const TAG_FREE_SPACE: Tag = 31;
+pub const TAG_REGION_OBJ: Tag = 33;
+pub const TAG_REGION_VEC: Tag = 35;
 
 // Special value to visit only a range of array fields.
 // This and all values above it are reserved and mean

src/codegen/compile.ml

@@ -933,6 +933,11 @@ module RTS = struct
     E.add_func_import env "rts" "text_singleton" [I32Type] [I32Type];
     E.add_func_import env "rts" "text_size" [I32Type] [I32Type];
     E.add_func_import env "rts" "text_to_buf" [I32Type; I32Type] [];
+    E.add_func_import env "rts" "region_new" [] [I32Type];
+    E.add_func_import env "rts" "region_size" [I32Type] [I32Type];
+    E.add_func_import env "rts" "region_grow" [I32Type; I32Type] [I32Type];
+    E.add_func_import env "rts" "region_load_blob" [I32Type; I32Type; I32Type] [I32Type];
+    E.add_func_import env "rts" "region_store_blob" [I32Type; I32Type; I32Type] [];
     E.add_func_import env "rts" "blob_of_principal" [I32Type] [I32Type];
     E.add_func_import env "rts" "principal_of_blob" [I32Type] [I32Type];
     E.add_func_import env "rts" "compute_crc32" [I32Type] [I32Type];
@@ -3374,6 +3379,20 @@ module Blob = struct
 
 end (* Blob *)
 
+module Region = struct
+  (*
+    See rts/motoko-rts/src/region.rs
+  *)
+  let new_ env =
+    E.call_import env "rts" "region_new"
+  let size env =
+    E.call_import env "rts" "region_size"
+  let grow env =
+    E.call_import env "rts" "region_grow"
+  let load_blob env = E.call_import env "rts" "region_load_blob"
+  let store_blob env = E.call_import env "rts" "region_store_blob"
+end
+
 module Text = struct
   (*
   Most of the heavy lifting around text values is in rts/motoko-rts/src/text.rs
@@ -4935,6 +4954,7 @@ module MakeSerialization (Strm : Stream) = struct
     | Any -> Some 16l
     | Non -> Some 17l
     | Prim Principal -> Some 24l
+    | Prim Region -> Some 25l
     | _ -> None
 
   (* some constants, also see rts/idl.c *)
@@ -9185,6 +9205,39 @@ and compile_prim_invocation (env : E.t) ae p es at =
     SR.Vanilla,
     GC.get_collector_instructions env ^^ BigNum.from_word64 env
 
+  (* Regions *)
+
+  | OtherPrim "regionNew", [] ->
+    SR.Vanilla,
+    Region.new_ env
+
+  | OtherPrim ("regionGrow"), [e0; e1] ->
+    SR.UnboxedWord64,
+    compile_exp_as env ae SR.Vanilla e0 ^^
+    compile_exp_as env ae SR.UnboxedWord64 e1 ^^
+    Region.grow env
+
+  | OtherPrim "regionSize", [e0] ->
+    SR.UnboxedWord32,
+    compile_exp_as env ae SR.Vanilla e0 ^^
+    Region.size env
+
+  | OtherPrim ("regionLoadBlob"), [e0; e1; e2] ->
+    SR.Vanilla,
+    compile_exp_as env ae SR.Vanilla e0 ^^
+    compile_exp_as env ae SR.UnboxedWord64 e1 ^^
+    compile_exp_as env ae SR.Vanilla e2 ^^
+    Blob.lit env "Blob size out of bounds" ^^
+    BigNum.to_word32_with env ^^
+    Region.load_blob env
+
+  | OtherPrim ("regionStoreBlob"), [e0; e1; e2] ->
+    SR.unit,
+    compile_exp_as env ae SR.Vanilla e0 ^^
+    compile_exp_as env ae SR.UnboxedWord64 e1 ^^
+    compile_exp_as env ae SR.Vanilla e2 ^^
+    Region.store_blob env
+
   (* Other prims, unary *)
 
   | OtherPrim ("global_timer_set"), [e] ->

src/mo_frontend/coverage.ml

@@ -94,6 +94,7 @@ let pick_val vs = function
   | T.Blob
   | T.Error
   | T.Principal
+  | T.Region
   | T.Float -> Any
 
 let rec expand_notval t n vs : desc list =

src/mo_idl/mo_to_idl.ml

@@ -71,6 +71,7 @@ let prim p =
   | Text -> I.PrimT I.Text
   | Blob -> I.BlobT
   | Principal -> I.PrincipalT
+  | Region -> assert false
   | Error -> assert false
 
 let rec typ t =

src/mo_types/arrange_type.ml

@@ -38,6 +38,7 @@ let prim = function
   | Blob -> Atom "Blob"
   | Error -> Atom "Error"
   | Principal -> Atom "Principal"
+  | Region -> Atom "Region"
 
 let con c = Atom (Type.string_of_con c)