Reorganize and add to README

cuplv · Oct 3, 2021 · d96b8cb · d96b8cb
1 parent 840b5d3
commit d96b8cb
Show file tree

Hide file tree

Showing 29 changed files with 130 additions and 117 deletions.
diff --git a/.gitignore b/.gitignore
@@ -1 +1 @@
-dbtest/_build
+src/_build
diff --git a/README.md b/README.md
@@ -1,3 +1,22 @@
+This is an OCaml implementation of the SC Merge rules, using Cassandra/Scylla as a backend database.
+
+# Dependencies
+
+This implementation uses OCaml, and the `ocaml-scylla` library as an interface to scylla/cassandra.
+
+[ocaml-scylla](https://github.com/anmolsahoo25/ocaml-scylla)
+
+This has been packaged here as a Nix package (`./ocaml-scylla.nix`) for use in `nix-shell` (Nick's dev environment).
+Without `nix-shell`, the library should be installable with the following command (Nick has not tested it), according to the README for `ocaml-mrdt-v2` which also uses it.
+
+```
+$ opam pin add scylla [email protected]:anmolsahoo25/ocaml-scylla
+```
+
+[`ocaml-mrdt-v2`](https://github.com/anmolsahoo25/ocaml-mrdt-v2)
+
+# Demo
+
 First, get a scylla node running as a docker container.
 
 ```
@@ -13,35 +32,100 @@ $ docker logs some-scylla | tail
 
 [Official guide to running Scylla in Docker](https://docs.scylladb.com/operating-scylla/procedures/tips/best_practices_scylla_on_docker/)
 
-Next, run the `dbtest` program.
-This uses the ocaml-scylla library to query the scylla docker container
-The library is packaged for use in `nix-shell` by the `ocaml-scylla.nix` file.
+Next, run the demo program.
 
 [ocaml-scylla](https://github.com/anmolsahoo25/ocaml-scylla)
 
-Currently, the `dbtest` program creates a content-addressed store and branch-tracking store, assigns several value versions to the branches, and merges the changes between them using fast-forward and 3-way merge.
+First, the `demo` program creates a content-addressed store and branch-tracking store, assigns several value versions to the branches, and merges the changes between them using fast-forward and 3-way merge.
+
+Second, it recreates the blocked merge from the SC Merge notes using 6 branches.
+
+Third, it prints out information from the database tables.
 
 ```
 $ nix-shell
-(nix-shell) $ cd dbtest
+(nix-shell) $ cd src
 (nix-shell) $ dune build
-(nix-shell) $ _build/default/dbtest.exe
-Opening connection...
-Opened connection.
-Created store tables.
-Updated/created branch b1 with value "Hello".
-Forked new branch b2 off of b1.
-Updated/created branch b2 with value "Hello Earth".
-Updated/created branch b1 with value "Hello Moon".
-Pulled from b1 into b2 to get "Hello Moon Earth".
-Updated/created branch b1 with value "Hello Moon, Mars".
-Pulled from b2 into b1 to get "Hello Moon Earth, Mars".
-Pulled from b1 into b2 to get "Hello Moon Earth, Mars".
-Updated/created branch b1 with value "Hello Moon Earth, Mars, etc.".
-Pulled from b1 into b2 to get "Hello Moon Earth, Mars, etc.".
-```
-
-The bottom of the `dbtest.ml` file is the script of forks, updates, and pulls that produces the above behavior.
-This file also contains the simple string merge function used, `str_merge`.
+(nix-shell) $ _build/default/demo.exe
+...
+```
+
+(If not using nix-shell, skip the `nix-shell` call and make sure you have installed dune, ocaml, and ocaml-scylla.)
+
+At the end of the demo, the final pull from branch `cc2` into branch `cc1` fails because the two branches have concurrent LCAs with the third branch `cc3`.
+The branches `ca1`, `ca2`, `cc1`, `cc2`, and `cc3` represent the five branches in the blocked merge diagram from the SC Merge notes (with branch `c` being their common root branch).
+
+The bottom of the `demo.ml` file is the script of forks, updates, and pulls that produces the above behavior.
 
 [dbtest.ml](./dbtest/dbtest.ml)
+
+# Code
+
+The `demo.ml` file defines a CLI-friendly wrapper around the MergeDB module.
+MergeDB is the main entrypoint for development, providing implementations of the `commit`, `fork`, and `pull` (covering `fastforward` and `merge`) rules.
+
+[MergeDB interface](./src/mergeDB.mli)
+
+[MergeDB implementation](./src/mergeDB.ml)
+
+The MergeDB module in turn uses four database tables, through their associated modules.
+
+## `ContentStore`
+
+`ContentStore` is a content-addressed store for committed values.
+Values must implement the `Content.STORABLE` interface by providing hash and serialization functions to be stored in the `ContentStore`.
+The module `StringData` provides an implemenation for strings.
+
+```
+[Row schema] (key: blob, value blob)
+```
+
+## `VersionGraph`
+
+`VersionGraph` is an edgelist table storing parent-child relationships between versions.
+
+```
+[Row schema] (
+  child_branch blob,
+  child_version_num int,
+  child_content_id blob,
+  parent_branch blob,
+  parent_version_num int,
+  parent_content_id blob,
+)
+```
+
+Branch names could probably be Ascii strings, but I defined them as Blobs following the example from `ocaml-mrdt-v2` and have not changed it.
+This makes debugging the database using `cqlsh`, for example, inconvenient.
+However, `VersionGraph` and a few other modules have `debug_dump` functions that print the contents of the table to the console.
+
+## `HeadMap`
+
+`HeadMap` maintains the current head version of each branch.
+
+```
+[Row schema] (
+  branch blob,
+  version_num int,
+  content_id blob,
+)
+```
+
+The `branch` value is the primary key, and the three elements of the row make up the complete "version" value it points to.
+
+## `LcaMap`
+
+`LcaMap` maintains the LCAs of each pair of branches.
+
+```
+[Row schema] (
+  branch1 blob,
+  branch2 blob,
+  lca_branch blob,
+  lca_version_num int,
+  lca_content_id blob,
+)
+```
+
+The pair of branches is the primary key, and the other 3 elements make up the LCA version value.
+When inserting or selecting, the requested branch pair is always sorted so that only one ordering appears in the table.
diff --git a/dbtest.nix b/dbtest.nix
diff --git a/default.nix b/default.nix
@@ -1,15 +1,12 @@
 with import <nixpkgs> {};
 
 rec {
-  libcassandra = callPackage ./libcassandra.nix {};
+
+  # Anmol's Cassandra/Scylla interface
   ocaml-scylla = callPackage ./ocaml-scylla.nix {};
-  ocaml-mrdt = callPackage ./ocaml-mrdt.nix {
+
+  # Implementation of SC Merge rules
+  mergedb = callPackage ./mergedb.nix {
     inherit ocaml-scylla;
   };
-  ocaml-cassandra = callPackage ./ocaml-cassandra.nix {
-    inherit libcassandra;
-  };
-  dbtest = callPackage ./dbtest.nix {
-    inherit ocaml-scylla ocaml-mrdt;
-  };
 }
diff --git a/libcassandra.nix b/libcassandra.nix
diff --git a/mergedb.nix b/mergedb.nix
@@ -0,0 +1,15 @@
+{ fetchFromGitHub, ocamlPackages, ocaml-scylla }:
+
+with ocamlPackages;
+
+let pname = "mergedb";
+
+in
+
+buildDunePackage rec {
+  inherit pname;
+  version = "0";
+  src = ./src;
+  useDune2 = true;
+  buildInputs = [ ocaml-scylla ];
+}
diff --git a/ocaml-cassandra.nix b/ocaml-cassandra.nix
diff --git a/ocaml-mrdt.nix b/ocaml-mrdt.nix
diff --git a/shell.nix b/shell.nix
@@ -11,4 +11,4 @@ with import ./default.nix;
 #   ];
 # }
 
-dbtest
+mergedb
diff --git a/dbtest/content.ml → src/content.ml b/dbtest/content.ml → src/content.ml
diff --git a/dbtest/contentStore.ml → src/contentStore.ml b/dbtest/contentStore.ml → src/contentStore.ml
diff --git a/dbtest/dbtest.ml → src/demo.ml b/dbtest/dbtest.ml → src/demo.ml
diff --git a/dbtest/dune → src/dune b/dbtest/dune → src/dune
@@ -1,3 +1,3 @@
 (executable
-  (name dbtest)
+  (name demo)
   (libraries scylla))
diff --git a/dbtest/dune-project → src/dune-project b/dbtest/dune-project → src/dune-project
diff --git a/dbtest/headMap.ml → src/headMap.ml b/dbtest/headMap.ml → src/headMap.ml
diff --git a/dbtest/headMap.mli → src/headMap.mli b/dbtest/headMap.mli → src/headMap.mli
diff --git a/dbtest/keySpace.ml → src/keySpace.ml b/dbtest/keySpace.ml → src/keySpace.ml
diff --git a/dbtest/keySpace.mli → src/keySpace.mli b/dbtest/keySpace.mli → src/keySpace.mli
diff --git a/dbtest/lcaMap.ml → src/lcaMap.ml b/dbtest/lcaMap.ml → src/lcaMap.ml
diff --git a/dbtest/lcaMap.mli → src/lcaMap.mli b/dbtest/lcaMap.mli → src/lcaMap.mli
diff --git a/dbtest/mergeDB.ml → src/mergeDB.ml b/dbtest/mergeDB.ml → src/mergeDB.ml
diff --git a/dbtest/mergeDB.mli → src/mergeDB.mli b/dbtest/mergeDB.mli → src/mergeDB.mli
diff --git a/dbtest/stringData.ml → src/stringData.ml b/dbtest/stringData.ml → src/stringData.ml
diff --git a/dbtest/types.ml → src/types.ml b/dbtest/types.ml → src/types.ml
diff --git a/dbtest/util.ml → src/util.ml b/dbtest/util.ml → src/util.ml
diff --git a/dbtest/version.ml → src/version.ml b/dbtest/version.ml → src/version.ml
diff --git a/dbtest/version.mli → src/version.mli b/dbtest/version.mli → src/version.mli
diff --git a/dbtest/versionGraph.ml → src/versionGraph.ml b/dbtest/versionGraph.ml → src/versionGraph.ml
diff --git a/dbtest/versionGraph.mli → src/versionGraph.mli b/dbtest/versionGraph.mli → src/versionGraph.mli
-Original file line number
+Diff line change
@@ Expand Up / @@ -11,4 +11,4 @@ with import ./default.nix; @@
     #   ];
     # }
-    dbtest
+    mergedb