New EXPERIMENTAL command to sync file state with codebase

[ChrisPenner]

Overview

We've run into a LOT of gotchas and unexpected fiddlyness in updates and deletes in UCM lately, in ways that make it quite difficult to deal with, and resulting in some non-ergonomic workflows (e.g. creating a new branch just to perform a delete).
As an experiment, this adds a new way of working with Unison codebases, more similar to traditional approaches.
This workflow introduces a new mode to UCM wherein your scratchfile is treated as the complete collection of code in your project (except libs). Anything removed from the scratch file will be treated as a delete from the codebase if you commit that version of the scratchfile.

I.e. it compiles your scratchfile just like it were a source file in any other programming language.

This allows you to do things like type-changing record field updates or deletes without UCM giving you grief when you try to update it, and also allows you to immediately parse & compile your whole project with a given set of changes without needing to iteratively propagate them into your codebase one bit at a time.

This is still very much just an experimental prototype, and likely to be a little broken in some ways, so tread lightly.

Known issues:

• LSP won't respect your current mode (yet)
• Typechecking is slow on large projects
• Messaging is poor and can definitely be improved.

Implementation notes

• Adds a new command experimental.commit (name not finalized, and clearly experimental).

• Adds new command experimental.commit.preview which shows what ☝🏼 WOULD do.

• Adds environment variable, you can set UNISON_LOAD_MODE=commit to switch UCM's default typecheck-on-save mode to match what the commit command does.
  The commit command is implemented by:

• parsing the scratchfile with ONLY libs in scope

• When committing, add all the definitions from the scratch-file to a branch containing only the libs of the current project.

• Display a namespace diff of the old branch to the new one to show what changed.

• This means no update propagations are needed, you handle them in the scratchfile while making the initial change.

Interesting/controversial decisions

Pretty sure this whole thing is interesting and/or controversial.

Test coverage

Added a transcript, but this probably needs a lot more testing in real-world situations.

Loose ends

This was a timeboxed experiment with a whole new paradigm, so the result is still a bit rough; things to add if we decide to invest more time:

• Fix edit.namespace to not include record field accessors.
• Speed up typechecking and leverage the codebase to skip duplicate work for unchanged definitions.
• Use Mitchell's synhash diff to detect propagated updates and things. We looked into using it, but the interface would require too much time to adapt to this use-case for what the timebox afforded. Esp. since it wasn't prepared to diff definitions which might not exist in the codebase yet.
• Fix up the LSP to respect modes, add jump-to-definition and other quality of life improvements

[aryairani]

I'm thinking this could be very useful in an "I know that the normal workflow is broken for what I'm trying to do", and then I'd want to toggle it on just temporarily. Would it be hard to be able to toggle the mode without restarting ucm?

[ChrisPenner]

@aryairani you don't need to swap the mode in order to use it, you can just call experimental.commit or experimental.commit.preview.

If we do want a command to switch the mode that's not hard either, but does add to the complexity of UCM a little bit.