WIP: Runtime schema validation

[geoffreylitt]

Overview

This PR introduces some experimental utilities for defining runtime-checkable data schemas. It's a small step towards Cambria. It uses @effect/schema as a toolkit for defining schemas that can be runtime-checked, automatically turned into TS types, and transformed into one another.

The current rough plan is to try incubating these simple utilities within TEE and then perhaps pull them out into a reusable library if it seems useful.

The basic workflow: you have a doc URL, a schema (defined in Effect), and a component you want to use to display the document. Then you just pass them into a withDocument higher-order component:

import { withDocument } from "../../automerge-repo-schema-utils/LoadDocument"
import { TinyEssayEditor } from "../../tee/components/TinyEssayEditor"
import { Essay } from "../../tee/schemas/Essay"

export const MyComponent = ({ docUrl }) => {
  return <div>
    {withDocument(TinyEssayEditor, docUrl, Essay)}
  </div>
};

The outer wrapper component handles loading the document and checking whether its schema matches what's expected. It also renders simple UI: a loading screen, and an error+repair screen if the doc can't be parsed as the expected schema.

CleanShot.2024-01-04.at.17.13.40.mp4

The inner component (in this case, TinyEssayEditor) gets the following benefits: it's only rendered once the data is actually loaded and has been verified to fit the expected schema. This means no loading states or shotgun parsing in the inner component.

Ongoing work / open questions

• The schema for an essay should be more closely tied to other functions like semantic actions and export to different data formats.
• Explore schema transformations / upgrades further. Effect/schema has some utilities for this, which I've started to use a tiny bit, extracting titles from essays. We can probably make a simple one-way whole document transformation system that solves the most common linear upgrade migration paths without doing anything fancy involving bidirectional transformations or edit lenses.
• There are some unresolved questions around document and schema boundaries. It's often useful to define named schemas which are more granular than an entire document. Do we treat these schemas as all uniform, or do we assign some special properties to schemas which are intended to represent entire documents? My leaning initially is to try decoupling these because it should be straightforward to change document boundaries for sharing granularity purposes without blowing up all of your data schemas.
• Add better support for parsing Automerge URLs. It's definitely possible to define a custom schema for Automerge URLs that does the custom validation on parse. But there are some annoying wrinkles inter-operating with the existing branded type provided by the Automerge library.
• So far I've only defined schemas for essays. There's more work to do to define schemas for account documents and apply some of these new patterns to the Doc Explorer sidebar.

[geoffreylitt]

I've pushed some more updates which introduce the concept of a model. A model wraps a data schema and also provides things like initialization defaults, computed properties based on the contents of the document, and actions which can be performed on the data.

My goal is to keep this model layer somewhat separate from the schema layer so that you can use the lighter weight schema validation logic without opting into the entire model framework.

Things are still fairly rough here, continuing to work on finding the right abstractions.