Skip to content

Commit

Permalink
refactor(createpackages): use jinja for mf6 module code generation
Browse files Browse the repository at this point in the history
  • Loading branch information
wpbonelli committed Oct 10, 2024
1 parent 3818033 commit a7f2644
Show file tree
Hide file tree
Showing 25 changed files with 2,107 additions and 1,234 deletions.
5 changes: 4 additions & 1 deletion .github/workflows/commit.yml
Original file line number Diff line number Diff line change
Expand Up @@ -181,10 +181,13 @@ jobs:
working-directory: autotest
run: |
pytest -v -m="not example" -n=auto --cov=flopy --cov-append --cov-report=xml --durations=0 --keep-failed=.failed --dist loadfile
coverage report
env:
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}

- name: Report coverage
working-directory: autotest
run: coverage report

- name: Upload failed test outputs
uses: actions/upload-artifact@v4
if: failure()
Expand Down
73 changes: 73 additions & 0 deletions autotest/test_codegen.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,73 @@
import pytest

from autotest.conftest import get_project_root_path
from flopy.mf6.utils.codegen.context import get_context_names
from flopy.mf6.utils.codegen.dfn import Dfn
from flopy.mf6.utils.codegen.make import (
DfnName,
make_all,
make_context,
make_targets,
)

PROJ_ROOT = get_project_root_path()
MF6_PATH = PROJ_ROOT / "flopy" / "mf6"
TGT_PATH = MF6_PATH / "modflow"
DFN_PATH = MF6_PATH / "data" / "dfn"
DFN_NAMES = [
dfn.stem
for dfn in DFN_PATH.glob("*.dfn")
if dfn.stem not in ["common", "flopy"]
]


@pytest.mark.parametrize("dfn_name", DFN_NAMES)
def test_dfn_load(dfn_name):
dfn_path = DFN_PATH / f"{dfn_name}.dfn"
with open(dfn_path, "r") as f:
dfn = Dfn.load(f, name=DfnName(*dfn_name.split("-")))
if dfn_name in ["sln-ems", "exg-gwfprt", "exg-gwfgwe", "exg-gwfgwt"]:
assert not any(dfn)
else:
assert any(dfn)


@pytest.mark.parametrize(
"dfn, n_vars, n_flat, n_meta",
[("gwf-ic", 2, 2, 0), ("prt-prp", 18, 40, 1)],
)
def test_make_context(dfn, n_vars, n_flat, n_meta):
with open(DFN_PATH / "common.dfn") as f:
commonvars = Dfn.load(f)

with open(DFN_PATH / f"{dfn}.dfn") as f:
dfn = DfnName(*dfn.split("-"))
definition = Dfn.load(f, name=dfn)

context_names = get_context_names(dfn)
context_name = context_names[0]
context = make_context(context_name, definition, commonvars)
assert len(context_names) == 1
assert len(context.variables) == n_vars
assert len(context.definition) == n_flat
assert len(context.definition.metadata) == n_meta


@pytest.mark.parametrize("dfn_name", DFN_NAMES)
def test_make_targets(dfn_name, function_tmpdir):
with open(DFN_PATH / "common.dfn") as f:
common = Dfn.load(f)

with open(DFN_PATH / f"{dfn_name}.dfn", "r") as f:
dfn_name = DfnName(*dfn_name.split("-"))
dfn = Dfn.load(f, name=dfn_name)

make_targets(dfn, function_tmpdir, commonvars=common)
for ctx_name in get_context_names(dfn_name):
source_path = function_tmpdir / ctx_name.target
assert source_path.is_file()


def test_make_all(function_tmpdir):
make_all(DFN_PATH, function_tmpdir, verbose=True)
assert any(function_tmpdir.glob("*.py"))
158 changes: 0 additions & 158 deletions autotest/test_generate_classes.py

This file was deleted.

52 changes: 29 additions & 23 deletions docs/mf6_dev_guide.md
Original file line number Diff line number Diff line change
Expand Up @@ -10,38 +10,44 @@ FPMF6 uses meta-data files located in flopy/mf6/data/dfn to define the model and

All meta-data can be accessed from the flopy.mf6.data.mfstructure.MFStructure class. This is a singleton class, meaning only one instance of this class can be created. The class contains a sim_struct attribute (which is a flopy.mf6.data.mfstructure.MFSimulationStructure object) which contains all of the meta-data for all package files. Meta-data is stored in a structured format. MFSimulationStructure contains MFModelStructure and MFInputFileStructure objects, which contain the meta-data for each model type and each "simulation-level" package (tdis, ims, ...). MFModelStructure contains model specific meta-data and a MFInputFileStructure object for each package in that model. MFInputFileStructure contains package specific meta-data and a MFBlockStructure object for each block contained in the package file. MFBlockStructure contains block specific meta-data and a MFDataStructure object for each data structure defined in the block, and MFDataStructure contains data structure specific meta-data and a MFDataItemStructure object for each data item contained in the data structure. Data structures define the structure of data that is naturally grouped together, for example, the data in a numpy recarray. Data item structures define the structure of specific pieces of data, for example, a single column of a numpy recarray. The meta-data defined in these classes provides all the information FloPy needs to read and write MODFLOW 6 package and name files, create the Flopy interface, and check the data for various constraints.


***
MFStructure --+ MFSimulationStructure --+ MFModelStructure --+ MFInputFileStructure --+ MFBlockStructure --+ MFDataStructure --+ MFDataItemStructure

Figure 1: FPMF6 generic data structure classes. Lines connecting classes show a relationship defined between the two connected classes. A "*" next to the class means that the class is a sub-class of the connected class. A "+" next to the class means that the class is contained within the connected class.
***
```mermaid
classDiagram
MFStructure --* "1" MFSimulationStructure : has
MFSimulationStructure --* "1+" MFModelStructure : has
MFModelStructure --* "1" MFInputFileStructure : has
MFInputFileStructure --* "1+" MFBlockStructure : has
MFBlockStructure --* "1+" MFDataStructure : has
MFDataStructure --* "1+" MFDataItemStructure : has
```

Figure 1: Generic data structure hierarchy. Connections show composition relationships.

Package and Data Base Classes
-----------------------------------------------

The package and data classes are related as shown below in figure 2. On the top of the figure 2 is the MFPackage class, which is the base class for all packages. MFPackage contains generic methods for building data objects and reading and writing the package to a file. MFPackage contains a MFInputFileStructure object that defines how the data is structured in the package file. MFPackage also contains a dictionary of blocks (MFBlock). The MFBlock class is a generic class used to represent a block within a package. MFBlock contains a MFBlockStructure object that defines how the data in the block is structured. MFBlock also contains a dictionary of data objects (subclasses of MFData) contained in the block and a list of block headers (MFBlockHeader) for that block. Block headers contain the block's name and optionally data items (eg. iprn).


***
MFPackage --+ MFBlock --+ MFData

MFPackage --+ MFInputFileStructure

MFBlock --+ MFBlockStructure

MFData --+ MFDataStructure

MFData --* MFArray --* MFTransientArray

MFData --* MFList --* MFTransientList

MFData --* MFScalar --* MFTransientScalar

MFTransientData --* MFTransientArray, MFTransientList, MFTransientScalar
```mermaid
classDiagram
MFPackage --* "1+" MFBlock : has
MFBlock --* "1+" MFData : has
MFPackage --* "1" MFInputFileStructure : has
MFBlock --* "1" MFBlockStructure : has
MFData --* "1" MFDataStructure : has
MFData --|> MFArray
MFArray --|> MFTransientArray
MFData --|> MFList
MFList --|> MFTransientList
MFData --|> MFScalar
MFScalar --|> MFTransientScalar
MFTransientData --|> MFTransientArray
MFTransientData --|> MFTransientList
MFTransientData --|> MFTransientScalar
```
Figure 2: FPMF6 package and data classes. Lines connecting classes show a relationship defined between the two connected classes. A "*" next to the class means that the class is a sub-class of the connected class. A "+" next to the class means that the class is contained within the connected class.
***

There are three main types of data, MFList, MFArray, and MFScalar data. All three of these data types are derived from the MFData abstract base class. MFList data is the type of data stored in a spreadsheet with different column headings. For example, the data describing a flow barrier are of type MFList. MFList data is stored in numpy recarrays. MFArray data is data of a single type (eg. all integer values). For example, the model's HK values are of type MFArray. MFArrays are stored in numpy ndarrays. MFScalar data is a single data item. Most MFScalar data are options. All MFData subclasses contain an MFDataStructure object that defines the expected structure and types of the data.

Expand Down
6 changes: 6 additions & 0 deletions flopy/mf6/data/dfn/utl-tas.dfn
Original file line number Diff line number Diff line change
Expand Up @@ -19,6 +19,7 @@ type keyword
shape
reader urword
optional false
in_record true
longname
description xxx

Expand All @@ -29,6 +30,7 @@ shape any1d
tagged false
reader urword
optional false
in_record true
longname
description Name by which a package references a particular time-array series. The name must be unique among all time-array series used in a package.

Expand All @@ -48,6 +50,7 @@ type keyword
shape
reader urword
optional false
in_record true
longname
description xxx

Expand All @@ -59,6 +62,7 @@ shape
tagged false
reader urword
optional false
in_record true
longname
description Interpolation method, which is either STEPWISE or LINEAR.

Expand All @@ -78,6 +82,7 @@ type keyword
shape
reader urword
optional false
in_record true
longname
description xxx

Expand All @@ -88,6 +93,7 @@ shape time_series_name
tagged false
reader urword
optional false
in_record true
longname
description Scale factor, which will multiply all array values in time series. SFAC is an optional attribute; if omitted, SFAC = 1.0.

Expand Down
Loading

0 comments on commit a7f2644

Please sign in to comment.