Update EEZ data source to enable direct download in the workflow

[brynpickering]

Would aid #56 and simplify everything by removing the remaining manual download requirement for the workflow. It has the option to be enhanced by specifying only the countries of interest in the API request (https://geo.vliz.be/geoserver/MarineRegions/wfs?service=WFS&version=1.0.0&request=DescribeFeatureType&typeNames=MarineRegions:eez&cql_filter=territory1=...&outputFormat=SHAPE-ZIP) but perhaps it is unnecessary as the filesize is relatively small.

[timtroendle]

That's great. How come this is possible suddenly? Was this always possible?

My main concern is the dataset seems to be not exactly the same. What do we know about the differences? That info should be in the changelog.

[brynpickering]

So this cannot be merged until solar-and-wind-potentials is also updated (see calliope-project/solar-and-wind-potentials#22), since there is now a mismatch in IDs.

[timtroendle]

There are two alternatives, so that we don't have to delay that. One would be to copy the code into this repository. Not perfect of course, but pragmatic. The code to determine shared coasts is neither long nor complicated. We could use that chance to rethink the algorithm. The old takes a long time to compute, and maybe we can improve that here.

The other would be to provide a map between the two ids here.

[brynpickering]

If updating this shared coast code for speed, I would do something like the following:

intersections = {}
areas = {}
# This can be multithreaded
for geom_idx, geom_data in units.iterrows():
    for eez_idx, eez_data in eez_new.iterrows():
        # Speed up the process by only considering EEZs where the country is involved in some way
        if geom_data.country_code.isin(eez_data[["iso_ter1", "iso_ter2", "iso_ter3"]]):
            # The EEZ buffer ensures that intersections are definitely covered, even when the two shapefiles don't perfectly align.
            # The exact buffer could/should be configurable
            if geom_data.geometry.intersects(eez_data.geometry.buffer(0.1)):
                # Intersection length and buffered area are both possible valid measures, I'd only pick one in the end
                # The buffer here is more restricted.
                intersections[(geom_data.id, eez_data.mrgid)] = geom_data.geometry.intersection(eez_data.geometry.buffer(0.01)).length
                area[(geom_data.id, eez_data.mrgid)] = geom_data.geometry.intersection(eez_data.geometry.buffer(0.01)).area
intersections_series = pd.Series(intersections)
intersections_normalised = intersections_series.div(intersections_series.sum(level=1))

[timtroendle]

@brynpickering, I'd like to merge this soon instead of waiting for custom shapes. What do you think of fixing the ID in solar-and-wind-potentials and release that data on Zenodo? It's basically a tiny fix in https://zenodo.org/record/3533038.

[brynpickering]

@timtroendle see calliope-project/solar-and-wind-potentials#23

[brynpickering]

@timtroendle would you be able to update zenodo so that this PR can be updated to point to the correct EEZ IDs?

[timtroendle]

Done. The new data are available at 10.5281/zenodo.5112963.

[brynpickering]

@timtroendle I've tested this with the minimal workflow locally without issue. This should be good to go.

[timtroendle]

@brynpickering, did you run the entire minimal workflow including tests? I let them run just now to analyse the difference this makes to the capacity factors and the tests fail. In fact, all offshore wind capacity factors are 0. Is that not the case for you?

[timtroendle]

Forget what I said, I forgot to force a re-download of the data.

[timtroendle]

Great work removing the final manual download!