Merge pull request #165 from Dewberry/documentation/v050

Documentation/v0.5.0

docs/reference.rst → docs/api_reference.rst

@@ -4,7 +4,14 @@ API Reference / Postman Collection
 
 For reference and documentation of the API, please open the postman collection for the version of ripple1d 
 
-`v0.4.1: <https://github.com/Dewberry/ripple1d/blob/v0.4.1/ripple1d/api/postman_collection.json> `_ This beta version contains the endpoints included in the first production testing release. Note that the following variables should be set in the postman environment
+
+`v0.5.0: <https://github.com/Dewberry/ripple1d/blob/3c90acc3fa212fde9c9b361dc3b907beaca17919/ripple1d/api/postman_collection.json>`_ This beta version contains new endpoints: 
+  - `geom_to_gpkg`: Extract the data from a model source dirctory to a gepoackage.
+  - `conflate`: Conflate all reaches from the NWM network corresponding to the source model.
+  - `conflation_metrics`: Apply conflation metrics for a conflated source model.
+
+
+`v0.4.1-v0.4.2: <https://github.com/Dewberry/ripple1d/blob/666190451620e033e8783241c020d2cde21660c9/ripple1d/api/postman_collection.json>`_ This beta version contains the endpoints included in the first production testing release. Note that the following variables should be set in the postman environment
 
 
 .. code-block:: YAML

docs/index.rst

@@ -15,7 +15,8 @@ on the NOAA National Water Model network.
    :caption: For Users
 
    for_users.rst
-   reference.rst
+   api_reference.rst
+   tech_reference.rst
    users_change_log.rst
 
 

docs/users_change_log.rst

@@ -3,10 +3,72 @@ Change Log for Users
 
 Go to the `Releases <https://github.com/Dewberry/ripple1d/releases.html>`_  page for a list of all releases.
 
-Bugfix Release 0.4.1
+
+Feature Release 0.5.0
 ~~~~~~~~~~~~~~~~~~~~~
 Users Changelog
 ----------------
+
+This release of `ripple1d` incorporates geometry extraction, conflation, and conflation metrics into the API, and fixes several bugs.
+
+
+Features Added
+----------------
+**Conflation improvements**
+
+- The source HEC-RAS river centerline is now clipped to the most upstream and most downstream cross sections prior to starting conflation. This helps prevent identifying network reaches that are far away from the cross sections and improves the accuracy of the conflation.  
+- Overlapped reaches are now tracked and documented in the conflation json file.
+- A bbox column has been added to the network parquet file for faster reading. This was especially needed for the new conflation endpoint since each request needs to load the parquet file. Load times without the bbox column were between 5-20 seconds; this is reduced to 1-2 seconds with the bbox column. 
+- The conflation function now reads locally instead of from s3.
+- The conflation function no longer creates a STAC item.
+- RAS metadata is now added to the conflation json. 
+- The source network's metadata is now added to the conflation json.
+- Length and flow units are now documented in the conflation json file.
+
+**Conflation Metrics**
+
+Three metrics are computed to asses the qualitiy of the conflation:
+
+- `Coverage`: The the start and end location of the reach coverage is computed as a ratio of the length of the network reach.
+- `Lengths`: The lengths between the most upstream cross section and most downstream cross section along the network reach and source HEC-RAS Model's centerline is computed. The ratio of the two lengths is also provided.
+- `XS`: The distance between where the network reach and HEC-RAS Model's centerline intersects the cross sections is computed. A similar comparison is performed using the cross section's thalweg location and the network reaches intersection location with the cross sections. The mean, min, max, std, and quartiles are provided as a summary for both comparisons.  
+
+
+**Geometry Extraction improvements**
+
+- A new function to verify .prj file is a HEC-RAS file has been added.
+- The extracted geopackage now contians a non-spatial metadata table for the souce HEC-RAS model. 
+- Tests have been added for extracting geopackage from HEC-RAS model.
+- Additional attributes are added to the source model gpkg for downstream use. 
+- Units are extracted from the source RAS model and added to metadata.
+
+**API**
+
+- An endpoint was added for extracting geometry and relevant metdata for the soure HEC-RAS models and storing it in in a geopackage. 
+- An endpoint to compute conflation metric was added.
+- An endpoint for conflation (which includes metrics calculations) was added.
+- Tests were added for the conflation, conflation metrics, and geopackage endpoints.
+
+
+Bug Fixes
+----------
+
+- Reaches whose conflation results indicate upstream and downstream cross sections are the same are now considered a failed conflation. 
+- The function to create a concave hull for the cross sections has been improved when junctions are present. 
+- Eclipsed reaches are now better identified and are documented in the conflation json with a boolean. 
+- A check is now performed to ensure cross sections intersect the source HEC-RAS model's river centerline. If cross sections do not intersect the centerline they are dropped. 
+- A conflation json is no longer written for source HEC-RAS models that fail to conflate. 
+- Handling has been added to subset gpkg endpoint for river stationings of interpolated. These river stations contain an "*" to indicate interpolated cross section.
+- Several issues with the automated API tests were identified and fixed. 
+- API tests no longer re-run gpkg_from_ras and conflate_model for every reach; just once per source test model. 
+- When API tests pass the resulting files are now removed automatically. Resulting files for tests that fail are not removed so that the tester can better trouble shoot.
+
+
+
+Bugfix Release 0.4.1-0.4.2
+~~~~~~~~~~~~~~~~~~~~~~~~~~~
+Users Changelog
+----------------
 This release of `ripple1d` fixes several bugs identified during testing.
 
 Features Added
@@ -21,9 +83,7 @@ Bug Fixes
 - A bug due to a hard coded terrain path name causing an error on  `create_fim_lib` has been resolved.
 - A bug associated with the `ripple_version` parameter has been resolved by removing the parameter from the body of requests (see note in API above).
 - An issue with including lateral structures (not yet implemented) in the ras geometry files causing hang ups  has been resolved. This fix resolved another issue where stationing was mis-applied in the newly created ras geometry files.
-
-
------------
+- A bug which caused a failure when calling subset_gpkg in cases where the model geometries are simple (no structures / no junctions).
 
 
 Feature Release 0.4.0
@@ -59,11 +119,13 @@ Features Added
 
 
 **API**
- - `ripple_version` is no longer a required argument for any endpoint.
+
+- `ripple_version` is no longer a required argument for any endpoint.
 
 
 Bug Fixes
 ----------
 Numerous small bug fixes were made to enable the support of hydraulic structures. Other notable bugs include:
- - HEC-RAS stations with length > 8 characters are now supported.
- - Mangled profile names resulting from negative elevations producing FIM libraries has been fixed.
+
+- HEC-RAS stations with length > 8 characters are now supported.
+- Mangled profile names resulting from negative elevations producing FIM libraries has been fixed.

pyproject.toml

@@ -4,7 +4,7 @@ build-backend = "setuptools.build_meta"
 
 [project]
 name = "ripple1d"
-version = "0.4.2"
+version = "0.5.0"
 description = "HEC-RAS model automation"
 readme = "README.md"
 maintainers = [

ripple1d/api/postman_collection.json

@@ -1,6 +1,6 @@
 {
 	"info": {
-		"_postman_id": "c4afca18-6b93-4c83-aa82-30306d98dcb2",
+		"_postman_id": "a4de6034-22c7-4a2b-b91c-12494de19040",
 		"name": "ripple1d",
 		"description": "Collection for processing existing HEC-RAS models for use in the production of Flood Inundation Maps (FIMs) and rating curves for use in near-real time flood forecasting on the NOAA National Water Model",
 		"schema": "https://schema.getpostman.com/json/collection/v2.1.0/collection.json",
@@ -75,7 +75,7 @@
 						"header": [],
 						"body": {
 							"mode": "raw",
-							"raw": "{\r\n    \"src_gpkg_path\": \"{{source_model_directory}}\\\\Baxter.gpkg\",\r\n    \"conflation_json\": \"{{source_model_directory}}\\\\Baxter.conflation.json\",\r\n    \"nwm_pq_path\": \"{{nwm_data_directory}}\\\\flows.parquet\"\r\n    \r\n}",
+							"raw": "{\r\n    \"source_model_directory\": \"{{source_model_directory}}\",\r\n    \"source_network\": {\"file_name\":\"{{nwm_data_directory}}\\\\flows.parquet\",\r\n    \"version\":\"2.1\", // optional\r\n    \"file_hash\": \"a6786ba02073f0233b9a69afc49d92bf-178\", // optional\r\n    \"type\":\"nwm_hydrofabric\"}  \r\n}",
 							"options": {
 								"raw": {
 									"language": "json"
@@ -93,7 +93,7 @@
 								"execution"
 							]
 						},
-						"description": "Create a new model for a given nwm_id (NWM reach) from an existing HEC-RAS model."
+						"description": "Compute conflation metrics for a given gpkg/conflation json."
 					},
 					"response": []
 				},
@@ -104,7 +104,7 @@
 						"header": [],
 						"body": {
 							"mode": "raw",
-							"raw": "{\r\n    \"source_model_directory\": \"{{source_model_directory}}\",\r\n    \"source_network\": {\"file_name\":\"{{nwm_data_directory}}\\\\flows.parquet\",\r\n                        \"version\":\"2.1\",\r\n                        \"type\":\"nwm_hydrofabric\"}\r\n    \r\n}",
+							"raw": "{\r\n    \"source_model_directory\": \"{{source_model_directory}}\",\r\n    \"source_network\": {\"file_name\":\"{{nwm_data_directory}}\\\\flows.parquet\",\r\n                        \"version\":\"2.1\",  // optional\r\n                        \"type\":\"nwm_hydrofabric\"}\r\n    \r\n}",
 							"options": {
 								"raw": {
 									"language": "json"
@@ -122,7 +122,7 @@
 								"execution"
 							]
 						},
-						"description": "Create a new model for a given nwm_id (NWM reach) from an existing HEC-RAS model."
+						"description": "Conflate a source HEC-RAS model with the NWM network."
 					},
 					"response": []
 				},
@@ -162,21 +162,21 @@
 						"header": [],
 						"body": {
 							"mode": "raw",
-							"raw": "{\r\n    \"source_model_directory\": \"{{source_model_directory}}\",\r\n    \"crs\": 2227,\r\n    \"metadata\": {\"stac_item\":\"sldakj;ga\"}\r\n}\r\n",
+							"raw": "{\r\n    \"source_model_directory\": \"{{source_model_directory}}\",\r\n    \"crs\": 2227,\r\n    \"metadata\": {\"stac_api\":\"https://stac2.dewberryanalytics.com\",  // optional\r\n                \"stac_collection_id\":\"ebfe-12090301_LowerColoradoCummins\",  // optional\r\n                \"stac_item_id\":\"137a9667-e5cf-4cea-b6ec-2e882a42fdc8\"}  // optional\r\n}\r\n",
 							"options": {
 								"raw": {
 									"language": "json"
 								}
 							}
 						},
 						"url": {
-							"raw": "{{url}}/processes/extract_gpkg/execution",
+							"raw": "{{url}}/processes/gpkg_from_ras/execution",
 							"host": [
 								"{{url}}"
 							],
 							"path": [
 								"processes",
-								"extract_gpkg",
+								"gpkg_from_ras",
 								"execution"
 							]
 						},
@@ -238,7 +238,7 @@
 								"execution"
 							]
 						},
-						"description": "Run a range of flows for Normal Depth on a HEC-RAS model. The results will be used to inform flows required to develop rating curves at a provided increment."
+						"description": "Run a range of flows for dormal depth on a HEC-RAS model. The results will be used to inform flows required to develop rating curves at a provided increment."
 					},
 					"response": []
 				},
@@ -296,7 +296,7 @@
 								"execution"
 							]
 						},
-						"description": "Run a series of HEC-RAS simulations using Known Water Surface Elevations."
+						"description": "Run a HEC-RAS simulation using Known Water Surface Elevations."
 					},
 					"response": []
 				},
@@ -353,7 +353,8 @@
 								"nwm_reach_model_stac",
 								"execution"
 							]
-						}
+						},
+						"description": "Create a STAC item for a HEC-RAS model representing a NWM reach."
 					},
 					"response": []
 				},
@@ -381,7 +382,8 @@
 								"fim_lib_stac",
 								"execution"
 							]
-						}
+						},
+						"description": "Create a STAC item for a FIM library for a NWM reah."
 					},
 					"response": []
 				}

ripple1d/version.py

@@ -1,3 +1,3 @@
 """ripple1d version."""
 
-__version__ = "0.4.2"
+__version__ = "0.5.0"