Multiple stable memory regions (and a new primitive type Region).

design/StableRegions-20230209.md

@@ -0,0 +1,224 @@
+# Stable Region Allocator Design
+
+- 20230209
+- Matthew Hammer
+
+## Purpose of this document
+
+This document gives a design for a new “stable region” allocator for Motoko’s runtime system, and accompanying `base` library.
+
+The role of this feature is summarized in the next section,
+followed by design considerations and details.
+
+
+## Role for "stable regions" in Motoko
+
+The current stable memory module in `base` has been "experimental" for a long time, and requires a more composable API to graduate from this status.
+
+Stable regions address the problem that today's `ExperimentalStableMemory` module only provides a single, monolithic memory that makes it unsuitable for directly building composable software parts.
+
+Stable regions permit a new API that supports composable use cases.
+
+Stable regions also bring Motoko closer to parity with Rust canister development support today, by giving a run-time-system-based analog of a special Rust library for stable data structures that allocates “pages” for them from stable memory in separate memory regions.
+
+
+## Design space
+
+The design space for the page allocator is defined by at least two
+tensions:
+
+ 1. fully-stable representation of allocator meta data **versus** fast load/store operations.
+
+ 2. Total scaling capacity **versus** minimum footprint for meta data.
+
+
+**Tension 1** is introduced because we want to avoid relying on the Motoko heap as the "ground truth" about the allocator's state.  If this heap is lost, as it is during an upgrade, then a developer may still want to recover all of the regions' data and meta data.
+
+On the other hand, during ordinary canister execution, ~we *do* want to rely on the heap (not stable memory) for meta data to avoid its higher access costs for load and store operations, and thus we need meta data in two places, both heap and stable memory.~
+
+Tension 1 is resolved by ~storing certain meta data twice, just as with the Rust implementation that serves as our basis.~ storing the data only in stable memory, and relying on those accesses becoming faster in the near future. (looking into the near-future roadmap for canister stable memory, it seems this is very likely.)
+
+Compared with the Rust version, we store enough extra meta data to permit:
+
+ - Regions whose page blocks are in arbitrary order, not
+   necessarily in order of smallest to highest address.
+
+ - 32767 Regions max (instead of 255 Regions max).
+
+We want to permit arbitrary page block orders to make a smooth
+transition to region reclamation and re-allocation in the near
+future, with potential integration into the Motoko GC.  The
+extra complexity is modest, and seems "worth" the cost.
+
+We change the maximum region limit because 255 may be too small in
+some extreme cases.
+
+We address the question of whether the new limit of 32k regions is
+"enough" in the Q&A section (it is, for all practical purposes)
+
+
+**Tension 2** is introduced because we want a design that will continue
+to work even when canisters can store more stable data than today (32GB).
+
+Tension 2 is resolved by making prudent representation choices.
+
+The representations we choose for regions and region identifiers
+permit a scaling to 256GB of stable data while still permitting meta
+data to be repeated in both stable and non-stable arenas.  These are
+the same limits imposed by the Rust implementation, for the same
+reasons.  See Q&A for more discussion.
+
+
+
+## Questions and answers
+
+### Q: What determines the 8MB non-empty region minimum?
+
+This size comes from wanting to grow a region by more than one page at
+a time (in terms of the way that the canister interacts with the
+system API, at least).  Rather than actually grow by a single page,
+the implementation grows by a "page block" (8MB) at a time.
+
+This choice means that there are 128 pages per page block, and that
+the maximum number of regions and region blocks are each relatively
+small (32k each).  Consequently, they can each be identified with a
+2-byte identifier, and we can pre-allocate tables to store certain
+relations about them, which is critical.
+
+### Q: Are 32767 regions enough?
+
+A: Permitting more than 32k regions may seem theoretically
+interesting, but is not practical given other parameters today that
+control the minimal footprint of a region (8MB) and dictate the
+maximum size of stable memory for a canister today (32GB).  With 32k
+regions at 8MB each, well over the maximum stable memory size is used
+(256GB compared to 32GB, the limit today)
+
+### Q: When is stable memory becoming less costly?
+
+Spring 2023.
+
+### Q: How does the cheaper stable memory access cost compare with ordinary heap memory access cost?
+
+2-3x slower than ordinary heap memory.
+
+
+## Design details
+
+### API
+
+Internal region allocator operations:
+
+ - `initialize` -- called by the RTS, not by the Motoko developer.
+
+User-facing region allocator operations:
+
+ - `region_new` -- create a dynamic region.
+ - `region_grow` -- grow region by a specified number of pages.
+ - `region_load` -- read some data from the region.
+ - `region_store` -- store some data into the region.
+
+And a special operation, for testing our design for future GC integration (bonus):
+
+- `region_release` -- release region and reuse region's page blocks.
+
+The `_release` operation is *not* part of the user-facing API nor part of the MVP,
+but supporting it is important because it means we can transition quickly to an integration
+with the ambient Motoko GC if we can support it.
+
+Another special operation, for disaster recovery:
+
+ - `rebuild` -- not needed unless we need to recreate all Region objects from their stable-memory counterparts.
+
+
+## Internal footprint
+
+The state of the allocator is stored in a combination of:
+
+ - stable memory fields and tables and
+ - stable heap memory, in the form of objects of type `Region`.
+
+The stable memory state is sufficient to reconstitute the stable heap objects
+(`rebuild` operation, described in a subsection below).
+
+That means that even if the stable parts of the heap are lost, the
+stable memory state can fully describe the region objects that will be rebuilt when it succeeds.
+
+### stable memory fields
+
+ - total allocated blocks, `Nat16`, max value is `32768`.
+ - total allocated regions, `Nat16`, max value is `32767` (one region is reserved for "no region").
+ - The `block-region` table (fixed size, about 131kb).
+ - The `region` table (fixed size, about 262kb).
+
+### representation of values of type `Region`
+
+ - A singleton, heap-allocated object with mutable fields.
+ - While being heap-allocated, the object is also `stable` (can be stored in a `stable var`, etc).
+ - `RegionObject { size_in_pages: Nat64; id: Nat16; vec_capacity: Nat16; vec_ptr: Nat32 }`
+ - Field `size_in_pages` gives the number of pages allocated to the Region.
+ - Field `id` gives the Region's numerical id as an index into the `region` table.
+ - Fields `vec_capacity` and `vec_ptr` work with `size_in_pages`
+   to represent a growable vector that we call the region's **"access
+   vector"** (because "blocks vector" sounds a bit strange, and its
+   used to support O(1) access operations):
+   - the access vector's address is held in `vec_ptr` and it has `vec_capacity` slots.
+   - the first `size_in_pages / 128` slots of `vec_ptr` contain a valid page block ID for the region.
+   - the access vector doubles when it grows.
+   - no region has more than 32k page blocks, so a `Nat16` suffices for `capacity`,
+   - during an upgrade, the access vectors get serialized and deserialized as data `Blobs` (as if no pointers are inside each).
+
+
+### region-blocks relation
+
+The `region-blocks` relation is not materialized into a table in stable memory (doing so with a pre-allocated table would be prohibitively large).
+
+Instead, this relation is represented in two ways at the same time:
+ 1. by the set of heap-allocated region objects, and their access vectors.  The access vectors provide O(1) store and load support.
+ 2. by the `block-region` table and the `region` table, which together are sufficient to recreate all of the heap-allocated region objects.
+
+In ordinary operation, the second feature is not required.  In the event of an upgrade failure, however, it could be vital (See `rebuild`).
+
+### block-region table
+
+ - purpose:
+   - relate a block ID ("page block ID") to its region (if any) and its position in that region (see `rebuild`).
+
+   - NB: The organization of this table is not useful for efficient
+     access calculations of load or store (they require a linear
+     search that would be prohibitively slow).  OTOH, this
+     table is suitable to do a "batch" rebuild of the dynamic heap-allocated vectors
+     in that table, if need be (see `rebuild`).
+
+ - 32768 entries (statically sized).
+ - 4 bytes per entry.
+ - entry type = `BlockRegion { region : Nat16, position : Nat16 }`
+ - the location of each entry gives its corresponding block ID.
+
+### region table
+
+ - purpose:
+   - relate a region ID to its size in pages.
+
+   - NB: This table exists as a "stable backup" of the per-region fields
+     that exist in the set of region objects, except for the access vectors' elements (see `block-region` table for their data).
+
+ - 32768 entries (statically sized).
+ - 8 bytes per entry.
+ - entry type = `RegionBlocks { size_in_pages: Nat64; }`
+ - the location of each entry in the table gives its corresponding region ID.
+ - the `size_in_pages` field measures the size of the region in terms of _pages_, not blocks.
+
+
+### Overview of `rebuild`
+
+When upgrades work as expected, stable `Regions` are serialized and deserialized just like other stable data.
+
+For disaster recovery, we can **also** rebuild all of the region objects from data in stable memory.
+
+We use the `block-region` and `region` tables in stable memory to rebuild the regions' objects:
+
+ - The `region` table gives a size for each region, saying how large each vector for each region needs to be.
+ - The `block-region` table gives a relative position and region ID for each block ID.
+
+Once each regions' vectors have been allocated, the block-region table says how to fill them, one entry at a time.

design/StableRegions.md

@@ -0,0 +1,23 @@
+# Stable Region API
+
+See StableMemory.md for context of the current experimental API.
+
+This document aims to specify the API and memory representations for a generalization
+of this API that permits multiple _regions_ of stable memory, where each can be
+grown independently.
+
+The **region manager** is the state and logic to support this generalization.
+
+## Definitions and constants
+
+ - a **page** is 65536 bytes.
+ - a **page block** is a contiguous sequence of 128 pages (~8MB).
+ - a **page block index** is a 16 bit, index-based identifier for a page block.
+ - a **region** is a sequence of (generally non-contiguous) **page blocks**.
+ - the maximum number of page blocks is 32768.
+ - the maximum amount of stable memory for all regions is 256GB.
+
+
+
+
+###### TO DO -- finish this document.

rts/motoko-rts/src/lib.rs

@@ -34,6 +34,7 @@ mod tommath_bindings;
 pub mod types;
 pub mod utf8;
 mod visitor;
+pub mod region;
 
 use types::Bytes;
 

rts/motoko-rts/src/region.rs

@@ -0,0 +1,30 @@
+use crate::memory::{Memory};
+use crate::rts_trap_with;
+use crate::types::{Value};
+
+use motoko_rts_macros::ic_mem_fn;
+
+#[ic_mem_fn]
+pub unsafe fn region_new<M: Memory>(_mem: &mut M) -> Value {
+    rts_trap_with("TODO region_new");
+}
+
+#[ic_mem_fn]
+pub unsafe fn region_size<M: Memory>(_mem: &mut M, _r: Value) -> Value {
+    rts_trap_with("TODO region_grow");
+}
+
+#[ic_mem_fn]
+pub unsafe fn region_grow<M: Memory>(_mem: &mut M, _r: Value, _new_pages: Value) -> Value {
+    rts_trap_with("TODO region_grow");
+}
+
+#[ic_mem_fn]
+pub unsafe fn region_load_blob<M: Memory>(_mem: &mut M, _r: Value, _start: Value, _len: Value) -> Value {
+    rts_trap_with("TODO region_load_blob");
+}
+
+#[ic_mem_fn]
+pub unsafe fn region_store_blob<M: Memory>(_mem: &mut M, _r: Value, _start: Value, _blob: Value) {
+    rts_trap_with("TODO region_store_blob");
+}

rts/motoko-rts/src/types.rs

@@ -374,6 +374,8 @@ pub const TAG_CONCAT: Tag = 25;
 pub const TAG_NULL: Tag = 27;
 pub const TAG_ONE_WORD_FILLER: Tag = 29;
 pub const TAG_FREE_SPACE: Tag = 31;
+pub const TAG_REGION_OBJ: Tag = 33;
+pub const TAG_REGION_VEC: Tag = 35;
 
 // Special value to visit only a range of array fields.
 // This and all values above it are reserved and mean

src/codegen/compile.ml

@@ -933,6 +933,11 @@ module RTS = struct
     E.add_func_import env "rts" "text_singleton" [I32Type] [I32Type];
     E.add_func_import env "rts" "text_size" [I32Type] [I32Type];
     E.add_func_import env "rts" "text_to_buf" [I32Type; I32Type] [];
+    E.add_func_import env "rts" "region_new" [] [I32Type];
+    E.add_func_import env "rts" "region_size" [I32Type] [I32Type];
+    E.add_func_import env "rts" "region_grow" [I32Type; I32Type] [I32Type];
+    E.add_func_import env "rts" "region_load_blob" [I32Type; I32Type; I32Type] [I32Type];
+    E.add_func_import env "rts" "region_store_blob" [I32Type; I32Type; I32Type] [];
     E.add_func_import env "rts" "blob_of_principal" [I32Type] [I32Type];
     E.add_func_import env "rts" "principal_of_blob" [I32Type] [I32Type];
     E.add_func_import env "rts" "compute_crc32" [I32Type] [I32Type];
@@ -3374,6 +3379,20 @@ module Blob = struct
 
 end (* Blob *)
 
+module Region = struct
+  (*
+    See rts/motoko-rts/src/region.rs
+  *)
+  let new_ env =
+    E.call_import env "rts" "region_new"
+  let size env =
+    E.call_import env "rts" "region_size"
+  let grow env =
+    E.call_import env "rts" "region_grow"
+  let load_blob env = E.call_import env "rts" "region_load_blob"
+  let store_blob env = E.call_import env "rts" "region_store_blob"
+end
+
 module Text = struct
   (*
   Most of the heavy lifting around text values is in rts/motoko-rts/src/text.rs
@@ -4935,6 +4954,7 @@ module MakeSerialization (Strm : Stream) = struct
     | Any -> Some 16l
     | Non -> Some 17l
     | Prim Principal -> Some 24l
+    | Prim Region -> Some 25l
     | _ -> None
 
   (* some constants, also see rts/idl.c *)
@@ -9185,6 +9205,39 @@ and compile_prim_invocation (env : E.t) ae p es at =
     SR.Vanilla,
     GC.get_collector_instructions env ^^ BigNum.from_word64 env
 
+  (* Regions *)
+
+  | OtherPrim "regionNew", [] ->
+    SR.Vanilla,
+    Region.new_ env
+
+  | OtherPrim ("regionGrow"), [e0; e1] ->
+    SR.UnboxedWord64,
+    compile_exp_as env ae SR.Vanilla e0 ^^
+    compile_exp_as env ae SR.UnboxedWord64 e1 ^^
+    Region.grow env
+
+  | OtherPrim "regionSize", [e0] ->
+    SR.UnboxedWord32,
+    compile_exp_as env ae SR.Vanilla e0 ^^
+    Region.size env
+
+  | OtherPrim ("regionLoadBlob"), [e0; e1; e2] ->
+    SR.Vanilla,
+    compile_exp_as env ae SR.Vanilla e0 ^^
+    compile_exp_as env ae SR.UnboxedWord64 e1 ^^
+    compile_exp_as env ae SR.Vanilla e2 ^^
+    Blob.lit env "Blob size out of bounds" ^^
+    BigNum.to_word32_with env ^^
+    Region.load_blob env
+
+  | OtherPrim ("regionStoreBlob"), [e0; e1; e2] ->
+    SR.unit,
+    compile_exp_as env ae SR.Vanilla e0 ^^
+    compile_exp_as env ae SR.UnboxedWord64 e1 ^^
+    compile_exp_as env ae SR.Vanilla e2 ^^
+    Region.store_blob env
+
   (* Other prims, unary *)
 
   | OtherPrim ("global_timer_set"), [e] ->

src/mo_frontend/coverage.ml

@@ -94,6 +94,7 @@ let pick_val vs = function
   | T.Blob
   | T.Error
   | T.Principal
+  | T.Region
   | T.Float -> Any
 
 let rec expand_notval t n vs : desc list =

src/mo_idl/mo_to_idl.ml

@@ -71,6 +71,7 @@ let prim p =
   | Text -> I.PrimT I.Text
   | Blob -> I.BlobT
   | Principal -> I.PrincipalT
+  | Region -> assert false
   | Error -> assert false
 
 let rec typ t =