Issues with migrating data_catalog to v1

[veenstrajelmer]

HydroMT version checks

• I have checked that this issue has not already been reported.
• I have checked that this bug exists on the latest version of HydroMT.

Reproducible Example

In the process of updating hydromt_delft3dfm to support hydromt v1 (Deltares/hydromt_delft3dfm#162), I am running into numerous problems. The most reproducible one is with reading a model from a yml file with a local data_catalog yml:

    model = DFlowFMModel(
        root=join(EXAMPLEDIR, "dflowfm_local"), 
        mode="r", 
        data_libs=[join(TESTDATADIR, "test_data.yaml")]
    )

The yml files are available at https://github.com/Deltares/hydromt_delft3dfm/tree/main/tests/data

Minimal hydromt code to reproduce:

from hydromt.data_catalog import DataCatalog
# from https://github.com/Deltares/hydromt_delft3dfm/blob/8862432e235ff5740c83dd322d76d22f6b22271b/tests/data/test_data.yaml
dcat = DataCatalog(data_libs=[r"c:\DATA\hydromt_delft3dfm\tests\data\test_data.yaml"])

Current behaviour

I got several validation errors:

ValidationError: 5 validation errors for GeoDataFrameSource
uri
  Field required [type=missing, input_value={'driver': 'vector', 'pat...\\data', 'metadata': {}}, input_type=dict]
    For further information visit https://errors.pydantic.dev/2.9/v/missing
driver
  Value error, Unknown 'name': 'vector' [type=value_error, input_value='vector', input_type=str]
    For further information visit https://errors.pydantic.dev/2.9/v/value_error
path
  Extra inputs are not permitted [type=extra_forbidden, input_value='local_data/1D_rivers.geojson', input_type=str]
    For further information visit https://errors.pydantic.dev/2.9/v/extra_forbidden
crs
  Extra inputs are not permitted [type=extra_forbidden, input_value='32647', input_type=str]
    For further information visit https://errors.pydantic.dev/2.9/v/extra_forbidden
rename
  Extra inputs are not permitted [type=extra_forbidden, input_value={'BRANCH_ID': 'branchid', 'BR_TYPE': 'branchtype'}, input_type=dict]
    For further information visit https://errors.pydantic.dev/2.9/v/extra_forbidden

I could work around all these, but have some suggestions to improve this migration for others.

Desired behaviour

After some digging, I read on https://github.com/Deltares/hydromt/blob/main/docs/guides/plugin_dev/migrating_to_v1.rst, that I convert the data catalog with https://github.com/Deltares/hydromt/blob/main/scripts/migrate_catalog_to_v1.py. Useful setup, but:

• It would be more useful to raise an error that an old datacatalog is supplied which is not supported
• It would be very helpful if the script would be integrated into hydromt, so one can just call it as a function. Numerous people will have to do this.
• If both the above things are implemented, you could also even consider supporting old data catalogs.
• I had to set “driver=raster” with every RasterDataset, hydromt<1 did not require this so it would be convenient if it would be resolved automatically in the migration function/script.
• Would be great to get the name of the dataset that is imported before the list of validationerrrors are given, makes it easier to find what is wrong. This was quite some trial and error now.
• “meta: null” was added as first line, resulting in AttributeError: 'NoneType' object has no attribute 'get'