Post-process results for visualisation

[andrewphilipsmith]

This PR covers some aspects of visualising the results from MapReader. So far, it mainly focuses on the post-processing results into a format suitable for the visualisations. Only geographical applications of MapReader have been considered so far.

This PR is currently a draft. See below for the outstanding tasks before this PR can be merged.

Data Processing:

The post-processing steps implemented in the results-post-processing.ipynb notebook are listed below. The results of each of the intermediate stages are saved:

• Converts the CSV results into Geopackages and transforms them into Web Mercator (i.e. the same as the map sheets)
• Generates the patches as polygons. Where there are overlapping map sheets, this results in overlapping patches.
• Merges (dissolves) the polygons based on pred value (eg rail/building/both classification).
• (partially implemented) Generates vector tiles from the merged polygons.
• (partially implemented) Generates a centre line for the rail space.
• (Not yet implemented) Interpolation of a raster image of the rail space.

Coordinate Transformations:

I have opted for reprojecting the results to match the map sheets (rather than reprojecting the map sheets to match the results).

• Visually this looks better than warping the map sheets.
• Better for calculating the patches. However, because it appears that the patch centroids have been transformed to WGS84 and then reprojected back to Web Mercator, there is a risk of gaps and slithers when the patch polygons are generated. If so, a better solution, long term, would be for MapReader to save the results (including the patches) in that CRS that they were initially generated in.

Other Notes:

• I have included a QGIS project file for now. In the short term I find it a useful way to explore some of the options for visualising the results (if others do too that's a bonus). There is an outside chance that we might consider that this is a helpful way to share some common visualisations with end-users.
• I've not included any of the generated data files in the repo. They should go somewhere more suitable such as SharePoint.

Remaining tasks before merging this PR:

• At present, there are a few global parameters at the top of the notebook, which are primarily parameters that (presumably) would have been inputs to the relevant MapReader run. There are two options (a) add some instructions for users to add appropriate values here or (b) find some way to find these from MapReader itself programmatically.
• It is indented that the processing below would be generalisable to any result set from MapReader. This has not been tested. No doubt, some refactoring is required to ensure this is possible.
• Most of this was written before I found the notebooks in the examples directory in the repo. There would be value in spending some effort in bringing this notebook in line with the structure and coding standards of the existing notebooks.

[andrewphilipsmith]

This PR implements some of the ideas discussed in #20.

[review-notebook-app]

Check out this pull request on 

See visual diffs & provide feedback on Jupyter Notebooks.

---

Powered by ReviewNB

[andrewphilipsmith]

I think this PR can be closed unmerged. There are two interesting conceptual threads to this PR, but both have better implementations or explanations elsewhere.

Exporting railspace polygons

There is an unsatisfactory attempt at exporting the railspace patches, dissolved into polygons.

Later, I wrote a better process, which took a more sensible approach to creating many individual polygons from adjacent patches, rather than one massive multi-polygon covering the whole country. This later approach was used to create the results that were included in the Felt maps.

I believe that this latter approach was merged into MapReader, though I cannot quickly find it now.

Comparision with StopsGB

The file examples/postproc/compare_with_stopsgb.ipynb is an initial attempt to compare the MapReader outputs with the StopsGB data. However, it turned out there were too many problems with the StopsGB data to make this useful.

There is a more detailed and self-contained exploration of the problems with the StopsGB data here: https://github.com/alan-turing-institute/Living-with-Machines/issues/2696#issuecomment-1131571469. I never followed up on how to fix those geocoding problems. If there is a chance to revise the StopsGB data, it would be interesting to try again and compare it with the MapReader outputs.