A workflow for installation of flopy and DFN code file generation

[deltamarnix]

After a discussion with @mjreno and @wpbonelli.

Problem Description

flopy should be able to load in any kind of MODFLOW6 input model, disregarding of which version of MODFLOW6 it was made in. @jdhughes indicated to sometimes receive MODFLOW6 models from external partners for debugging purposes.

The definition (dfn) files are used to generate source code in both MODFLOW6 and flopy, but only on release. That means that the version of flopy is hard linked to the version of MODFLOW6. Incompatibility issues will arise when trying to load a MODFLOW6 model with the wrong version of flopy, as some variable or package may not be available while loading the model.

As a downside of this tied release cycle, the user needs to downgrade their flopy version when they want to use an older version of MODFLOW. Bug fixes on flopy are therefore also reverted, and the latest features as well. If development wants to fix a bug in flopy, it should theoretically do that for all flopy versions that are out there, making it hard for maintenance.

Proposal

The release cycle of flopy and MODFLOW should be split, so that it is possible to release new MODFLOW6 versions without releasing a new flopy, and fixing flopy issues without having to patch them on all earlier released flopy versions.

We can achieve this concept by changing the installation process for users and making some minor changes in already existing functions. Instead of generating the source code packages from the dfn files during development time and storing the dfn files in both MODFLOW and flopy, the MODFLOW repository should be the leading truth of the dfn files.

The proposal is to distribute the dfn files along with MODFLOW6, so that the get_modflow() function also downloads the dfn files. When the dfn files are downloaded, the code generation can kick off to install those extra python classes in the local environment of the user.

sequenceDiagram
actor User
User ->> pip: pip install flopy
User ->> flopy: flopy.get_modflow("6.1.3")
flopy ->> modflow_server: download modflow 6.1.3
flopy ->> flopy: flopy.code_generation()

Loading

I believe that this proposed solution can already be performed within the current FloPy code base, as the get_modflow() functionality already exists. The only thing that would change for the user, is that they need to call this functionality, or otherwise the mf6 package will not yet be available in their python environment.

Advantages

The proposed solution makes it possible to separate releases of MODFLOW from flopy, giving the opportunity to perform flopy patches and add features without having to release a new version of MODFLOW. And to update MODFLOW without having to release a new version of flopy.

It also gives the opportunity for users to switch between MODFLOW versions without having to install a different version of flopy, they would only have to call get_modflow() again in order to set up their flopy environment correctly.

There would only be a single source of truth for the dfn files, so no dfn duplication in multiple repositories.

Disadvantages

I believe that MODFLOW input files do not contain a version number in their configuration or metadata. That makes it hard for the flopy code to validate if it is possible to load the input files or not. Adding such information would be a great addition, as we could then return error messages when the flopy generated code does not match the model to load from disk.

[deltamarnix]

A similar proposal was done by @wpbonelli here: modflowpy/flopy#1970 (comment), but the effort was stalled I believe.