Merge pull request #95 from ethdebug/cleanup-docs

Organize pointers implementation guide a bit

packages/web/docs/implementation-guides/implementation-guides.mdx

@@ -0,0 +1,57 @@
+---
+sidebar_position: 4
+pagination_prev: null
+pagination_next: null
+---
+
+# Implementation guides
+
+This section of these docs serves to provide resources that guide readers who
+are looking to implement one or more components of **ethdebug/format**.
+Because of the distinct concerns involved in implementing this format on the
+compilation side vs. the concerns involved on the debugging side, this page
+lists and categorizes the available guides into the appropriate heading.
+
+
+## For debuggers
+
+<dl>
+<dt>**Guide: [Dereferencing pointers](/docs/implementation-guides/pointers)**</dt>
+<dd>
+This guide provides readers with a tour of the **@ethdebug/pointers**
+TypeScript reference implementation, showing example concrete logic for how a
+debugger might process **ethdebug/format** pointers.
+
+For an introduction to **ethdebug/format** pointers, please see
+the Pointer specification's [Overview](/spec/pointer/overview) and
+[Key&nbsp;concepts](/spec/pointer/concepts) pages.
+</dd>
+
+<dt>**Other guides**</dt>
+<dd>
+_Guides for other aspects of debugger-side **ethdebug/format** implementation
+are planned and still need to be written._
+</dd>
+
+</dl>
+
+## For compilers
+
+<dl>
+<dt>**No availble guides yet**</dt>
+<dd>
+_Guides for implementing **ethdebug/format** support inside a compiler are
+planned and still need to be written._
+</dd>
+</dl>
+
+:::tip[Work in progress]
+
+Sadly, things are looking a little scarce right now. Please stay tuned as work
+on this effort progresses.
+
+**Interested in helping out?** If you'd like to help with writing initial
+reference implementations for one or more schemas, please reach out in our
+[Matrix.chat](https://matrix.to/#/#ethdebug:matrix.org).
+
+:::

packages/web/docs/implementation-guides/pointers/dereference-logic/dereference-logic.mdx

@@ -1,20 +1,116 @@
 ---
 sidebar_position: 5
+sidebar_label: The dereference function
 ---
 
-# The dereference function
+import CodeListing from "@site/src/components/CodeListing";
 
-These next few pages cover how the components described thus far are combined
-to create the final `dereference(pointer: Pointer)` function.
+# The `dereference()` function
 
-- The [Summary](/docs/implementation-guides/pointers/dereference-logic/summary)
-  page broadly describes the control flow structure behind this function
-  implementation.
+## Summary
 
-- [Generating regions on the fly](/docs/implementation-guides/pointers/dereference-logic/generating-regions)
-  describes the process of recursively processing a pointer and reducing it to
-  a concrete list of fully-evaluated `Cursor.Region` objects.
+The pages in this section cover the internals of the `dereference()` function
+in the **@ethdebug/pointers** reference implementation.
 
-- [Making regions concrete](/docs/implementation-guides/pointers/dereference-logic/making-regions-concrete)
-  describes the process for converting a single `Pointer.Region` object into
-  its fully-evaluated `Cursor.Region` equivalent at runtime.
+The full signature of this function is as follows:
+
+<CodeListing
+  packageName="@ethdebug/pointers"
+  sourcePath="src/dereference/index.ts"
+  extract={
+    (sourceFile, project) => {
+      const definition = sourceFile.getFunction("dereference");
+      const tempSourceFile = project.createSourceFile(
+        "dereference-summary.ts",
+        "",
+        { overwrite: true }
+      );
+
+      for (const importDeclaration of sourceFile.getImportDeclarations()) {
+        tempSourceFile.addImportDeclaration(importDeclaration.getStructure());
+      }
+
+      const commentText = definition.getLeadingCommentRanges()
+        .map(range =>
+          sourceFile.getFullText()
+            .substring(range.getPos(), range.getEnd()))
+        .join("\n");
+
+      const declaration = tempSourceFile.addFunction({
+        name: definition.getName(),
+        parameters: definition.getParameters()
+          .map((param, index, array) => ({
+            name: param.getName(),
+            type: param.getType().getText(param),
+            hasQuestionToken: param.hasQuestionToken() || param.hasInitializer(),
+            leadingTrivia: "\n",
+            trailingTrivia: index < array.length - 1 ? undefined : "\n"
+          })),
+        returnType: definition.getReturnType().getText(definition),
+        hasDeclareKeyword: true,
+        isAsync: true,
+        leadingTrivia: `${commentText}\n`
+      });
+
+
+      return tempSourceFile.getFunction("dereference");
+    }
+  } />
+
+:::tip
+
+Remember from the
+[Cursors](/docs/implementation-guides/pointers/types/cursors) section that a `Cursor`
+provides a `view(state: Machine.State)` method, which returns an ordered
+collection of concrete `Cursor.Region` objects.
+
+:::
+
+### `DereferenceOptions`
+
+Note the optional `options: DereferenceOptions` argument. This argument
+allows for specifying additional information upfront that is necessary for
+viewing the cursor later. Currently, this is needed only for pointers that
+compose stack-located regions.
+
+<CodeListing
+  packageName="@ethdebug/pointers"
+  sourcePath="src/dereference/index.ts"
+  extract={sourceFile => sourceFile.getInterface("DereferenceOptions")}
+  />
+
+## Control flow architecture
+
+The `dereference()` function itself performs two tasks:
+1. Create a "simple cursor": a function that takes a machine state and
+   produces an asynchronous list of `Cursor.Region`s.
+2. Adapt this simple cursor to conform to the full `Cursor` interface
+
+Within the process of creating this simple cursor it gets more interesting:
+by leveraging JavaScript's
+[AsyncIterator](https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Global_Objects/AsyncIterator)s,
+the implementation can compute regions on the fly by recursively breaking down
+pointers into their nested child pointers.
+
+Since the desired end-result of `dereference()` is an object that can turn
+a pointer into its composite ordered list of concrete regions at a particular
+machine state, this implementation separates the concerns of generating this
+list from converting this list into the promised return interface.
+
+To generate this list asynchronously on the fly, the implementation uses a
+_stack of processing requests_ (which it calls "memos"), initially populated
+with a request to dereference the root pointer. Each memo represents a state
+or context change in some form: either a request to dereference a pointer or
+sub-pointer, a request to save a set of regions by their names, or a request
+to save the computed values of a set of variables by their identifiers.
+
+The other pages in this section proceed to go into more detail.
+
+<details>
+<summary>See the full `src/dereference/index.ts` module</summary>
+
+<CodeListing
+  packageName="@ethdebug/pointers"
+  sourcePath="src/dereference/index.ts"
+  />
+</details>

packages/web/docs/implementation-guides/pointers/dereference-logic/generating-regions.mdx

@@ -147,7 +147,7 @@ so that earlier memos in the list will be processed before later ones.
   />
 </details>
 
-### Processing a region
+## Processing a region
 
 The simplest kind of pointer is just a single region. (Remember that pointers
 are either regions or collections of other pointers.)
@@ -168,12 +168,12 @@ a request to save this region to process state by its name.
 
 This pointer evaluation process will be described later.
 
-### Processing collections
+## Processing collections
 
 The recursive cases are fairly straightforward following this architecture.
 
 
-#### Groups
+### Groups
 
 The simplest collection, a group of other pointers, yields no regions of its
 own, but instead pushes each of its child pointers for evaluation later:
@@ -188,7 +188,7 @@ It's essential that each of the child pointers get evaluated in the order
 they appear in the list, since later pointers may reference regions named
 earlier, etc.
 
-#### Lists
+### Lists
 
 List collections are more complex because they dynamically generate a number
 of composed pointers based on a runtime count value and introducing a
@@ -204,7 +204,7 @@ Note how, because each dynamic child pointer is evaluated based on the
 next incremented index value, the memos for updating this variable and
 evaluation the child pointer must be interspersed.
 
-#### Conditionals
+### Conditionals
 
 Conditional pointers evaluate to a child pointer given that some runtime
 condition evaluates to a nonzero value, optionally evaluating to a different
@@ -220,7 +220,7 @@ the `"else"` pointer if it is specified:
   extract={sourceFile => sourceFile.getFunction("processConditional")}
   />
 
-#### Scopes
+### Scopes
 
 Finally, the last kind of collection defined by this schema is for defining
 a scope of variables by identifier by specifying the expression values for

packages/web/docs/implementation-guides/pointers/overview.mdx → packages/web/docs/implementation-guides/pointers/pointers.mdx

@@ -1,11 +1,21 @@
 ---
 sidebar_position: 1
+sidebar_label: Dereferencing pointers
 ---
 
 import CodeBlock from "@theme/CodeBlock";
 import CodeListing from "@site/src/components/CodeListing";
 
-# Overview
+# Dereferencing pointers
+
+_An implementation guide for resolving **ethdebug/format/pointer**s
+into concrete regions and values in a running EVM._
+
+## Introduction
+
+This implementation guide describes the **@ethdebug/pointers** reference
+implementation, written in TypeScript and intended for distribution as an
+NPM package.
 
 Debuggers looking to support **ethdebug/format** must be prepared to
 dereference the data allocations used by compilers for all kinds of variables.
@@ -17,8 +27,6 @@ This format's [**ethdebug/format/pointer** schema](/spec/pointer/overview)
 provides such suitable expressiveness, but implementing the logic to read and
 evaluate data in this schema requires some careful consideration.
 
-This documentation section describes a reference implementation written in
-TypeScript and available as the **@ethdebug/pointers** NPM package.
 
 :::tip
 If you're reading this page without first having familiarized yourself with the