JasperFx Ecosystem Dependency Reorganization

[jeremydmiller]

This is in Wolverine just because it needs to go somewhere, and because Wolverine is kind of at the top.

"Problems"

• There are some shared framework-y elements like the application assembly determination that are duplicated between Marten & Wolverine today, with Ermine coming soon-ish
• I think it would be hugely advantageous to have some sharing of the OptimizeArtifictWorkflow between Marten & Wolverine, just so there's less repetitive junk going on in bootstrapping -- especially when there are becoming some many different things that should or could be different in Development vs Production mode
• Wolverine 3.0 is coming soon, so Wolverine can accommodate breaking changes, but there's no other justification for a Marten 8 in the immediate future. At least until Ermine later this year
• Lamar is already on the chopping block. Questioning whether Oakton should be as well

Today

Some of the current dependencies between the greater JasperFx / Critter Stack world:

I've omitted Weasel, but it's of course there and would be impacted by this too.

Proposed Future

Okay, so this is going to require a full point Marten 8 and a new Wolverine as well. Maybe this doesn't exactly happen until Wolverine 4. Might help to start with some temporary [Obsolete] tags in especially Marten

Here's the big points, I think:

• The basic goal is to simplify the dependency structure and just make it look like things are more cohesive. Should also move in the direction of having fewer knobs and dials for users to have to do to go with the Critter Stack in terms of how many Nugets have to be added or how many different settings need to be used
• Lamar just disappears, but will still be supported for usage within Wolverine. That work is close to done already
• We introduce a brand new, shared library called JasperFx.Framework that will hold some newer stuff related to shared configuration helpers and logic that is very specific to Marten/Wolverine/future critter stack tools, but not generic enough to live in JasperFx.Core that's just utilities and more or less the "junk drawer"
• Combine JasperFx.CodeGeneration into JasperFx.Framework
• LEAVE JasperFx.Compiler alone, other than to change its dependency to JasperFx.Framework
• Oakton goes away, with basically everything it does moving to the new shared JasperFx.Framework, including both the command line parsing and execution, as well as the IEnvironmentCheck and IStatefulResource models
• Fold the commands in Marten.CommandLine and really Weasel.CommandLine into Weasel.Core and/or Marten. Trying to get to fewer assemblies and less stuff to add
• Here's a hurtful one, move as much of the projection support in Marten including the async daemon code to a more generic JasperFx.Projections library. This is 100% in preparation for tackling "Ermine", the intended Sql Server backed event store lib

Finer Grained Thoughts

• Centralize the configuration of the code generation GenerationRules
• Centralize the application assembly determination that both Wolverine & Marten & Ermine in the future need
• Centralize the "optimize artifact workflow" logic that all the tools use
• Set us up for some streamlined bootstrapping alternatives. Anything that reduces the friction of starting a new project can only help

[jeremydmiller]

Wolverine & Marten Compliance Tests

To help build more Event Sourcing + CQRS products with Wolverine as a baseline, let's start to pull a standard set of compliance tests for Wolverine extensions and possible future Marten alternatives. Some of this already exists, or "almost" exists, but put it in a publicly published Nuget so it's usable outside of the main Wolverine or Marten repositories:

• The existing TransportCompliance work
• The smoke tests on stateful resource compliance that today is copy/pasted
• Scheduled test integration
• Envelope persistence -- this is a bit bigger because of the API surface, but it would be super helpful for the upcoming RavenDb integration
• Inbox/outbox integration tests ?
• Persisted saga tests -- already kind of exists, but it's mostly copy/pasted. See if that can be generalized
• Leadership assignment and node failover -- ugh, this will stink
• Scheduled job execution
• Event sourcing conformance testing -- much bigger. Might require some changes to Marten first, but do in preparation for Ermine. Kind of assumes that there will be an interface for event capture that is more or less standardized between different critter stack versions of event sourcing

[jeremydmiller]

Maybe a different take on this. What if Oakton 7.0 becomes what I wanted JasperFx.Framework to be? It's goofy naming, but in this case Oakton essentially becomes the foundational piece for JasperFx tools. It already has some things like the IStatefulResource model that shouldn't technically be in Oakton anyway

[jeremydmiller]

Latest Thinking (September 21st)

JasperFx Repository

• Build a new library named JasperFx that will subsume what is now JasperFx.Core, JasperFx.CodeGeneration, and Oakton. Keep the same namespace for the core and code generation elements
• Kind of orphan Oakton as is where its types mostly inherit from the same types in the new JasperFx lib? Or don't worry about it
• Retain the dependency on Spectre.Console because it's really light in terms of its dependency tree
• Subsume JasperFx.CodeGeneration.Commands into the new JasperFx library
• JasperFx.RuntimeCompiler just moves to the new JasperFx repository
• Actually write documentation for all of JasperFx including the code generation stuff

Streamlined "Stateful Resource" Model

Have a single place where you can set all the "should I build this on demand at runtime, or not" settings across
both Marten & Wolverine & whatever other critters exist.

By standardizing on this model across the board, some one off things in Marten & Wolverine should go away:

• Deprecate Marten specific options to do database set up on app start that was introduced in Marten 5 (I think)
• Delete one off AutoProvision() / AutoPurgeOnStartup() options on Wolverine transports
• Eliminate the one off boolean flag in Wolverine for disabling automatic envelope storage setup

In all cases, use the AutoCreate enum -- currently in Weasel, but moving to JasperFx -- to specify the setup rules

// Helper for overriding in tests. If `generatedCodePath` isn't specified, but it under `Internal/Generated` of the designated 
// assembly, assuming it's in a project named the same as the assembly
builder.Services.OverrideApplicationAssembly(assembly, string? generatedCodePath = null);

// Syntax for configuring the stateful resource model
builder.Services.ConfigureJasperFx(opts => {
    // Shouldn't be needed very often, but *sometimes* people just want to do things differently
    // instead of using idiomatic "Development" / "Testing" / "Production"
    opts.DevelopmentEnvironmentNames = ["Development", "Testing"];

    // This would be the default of either
    opts.Development.AutoCreate = AutoCreate.CreateOrUpdate;

    // Do *once* for all of the critters
    opts.Development.TypeLoadMode = TypeLoadMode.Dynamic;

    // Maybe make this auto. Probably make this a prop instead
    opts.Development.ResourceSetupOnStartup();

    // Default is false. Would apply to both AutoPurgeOnStartup() in Wolverine and Marten.ResetState()
    // Development time helper!!!
    // Could do stuff like delete all outstanding inbox/outbox messages, reset Marten state, purge queues
    opts.Development.ResetAllStateOnStartup = true;

    opts.Production.AutoCreate = AutoCreate.CreateOrUpdate;

    // Maybe make this auto. Probably make this a prop instead
    opts.Production.ResourceSetupOnStartup();

    // Do *once* for all of the critters
    opts.Production.TypeLoadMode = TypeLoadMode.Static;
});

Also, keep the current extension methods on IServiceCollection that refer to the stateful resource model

Other Proposed Streamlining

• Weasel.Core takes a dependency on JasperFx, then subsume Weasel.CommandLine into Weasel.Core
• Consider thinning down the Weasel command line offerings anyway since it all overlaps with the lower level JasperFx command line (was Oakton resources command)
• Merge Marten.CommandLine directly into Marten because why not

JasperFx.EventStore

MORE HERE SOON

[jeremydmiller]

Update

What's shown as "JasperFx.Framework" and "JasperFx.Core" are being subsumed into a new library called just "JasperFx" as well as "Oakton". To avoid disturbing existing Oakton users too much, Oakton will be somewhat rebuilt as a shim over the newer "JasperFx". All this shuffling around hopefully helps us reduce the total number of Nugets floating around in our ecosystem and also make there be less wiring, configuration, and general overhead to set up projects.