Merge pull request #6 from IFRCGo/emdat-transformer

EM-DAT initial implementation

.gitignore

@@ -161,3 +161,4 @@ cython_debug/
 # asv environments
 .asv
 tests/data-files/temp/
+tests/gaul_gpkg_and_license.zip

README.md

@@ -56,8 +56,16 @@ uv sync
 
 To run the tests:
 
+1. Make the test with actual calls to http and write them in the cassette files:
+
+```sh
+uv run pytest -v -s --record-mode rewrite
+```
+
+2. Run the tests with the recorded calls:
+
 ```sh
-pytest
+uv run pytest -v -s
 ```
 
 ## Contributing
@@ -67,7 +75,6 @@ Contributions are welcome! Please open an issue or submit a pull request.
 ## License
 
 This project is licensed under the Apache License, Version 2.0. See the [LICENSE](LICENSE) file for more details.
-```
 
 ### Links
 

docs/docs/api/sources/emdat.md

@@ -1,25 +1,88 @@
 # EMDAT
 
-The EMDAT (Emergency Events Database) module provides functionality for working with EM-DAT data.
+The EMDAT (Emergency Events Database) module provides functionality for working with [EM-DAT](https://public.emdat.be) data.
+Full Specification of the transformation can be found [here](https://github.com/IFRCGo/monty-stac-extension/tree/main/model/sources/EMDAT).
 
 ```python
-from pystac_monty.sources.emdat import EMDATTransformer
+from pystac_monty.sources.emdat import EMDATTransformer, EMDATDataSource
+```
+
+## EMDATDataSource
+
+The `EMDATDataSource` class handles EM-DAT data input in either Excel format or as a pandas DataFrame.
+
+```python
+class EMDATDataSource(MontyDataSource):
+    """
+    EM-DAT data source that can handle both Excel files and pandas DataFrames.
+    
+    Args:
+        source_url: URL of the data source
+        data: Either Excel content as string or pandas DataFrame containing EM-DAT data
+    """
 ```
 
 ## EMDATTransformer
 
-Transforms EM-DAT data into STAC Items.
+The `EMDATTransformer` class transforms EM-DAT data into STAC Items with the Monty extension.
+
+```python
+class EMDATTransformer:
+    """
+    Transforms EM-DAT data into STAC Items with Monty extension.
+    
+    Args:
+        data: EMDATDataSource containing the EM-DAT data
+        geocoder: Optional GAULGeocoder instance for enhanced location handling
+    """
+```
+
+A geocoder is needed to handle location data in the EM-DAT data. The `GAULGeocoder` class is provided for this purpose. It is designed to work with the Global Administrative Unit Layers (GAUL) dataset as described in the EMDAT tutorial: [Making Maps](https://doc.emdat.be/docs/additional-resources-and-tutorials/tutorials/python_tutorial_2/)
+
+```python
+from pystac_monty.geocoding import GAULGeocoder
+
+geocoder = GAULGeocoder("path/to/gaul.gpkg")
+```
+
+### Output STAC Items
+
+The transformer creates three types of STAC Items:
+
+1. Event Items (`emdat-event-*`)
+   - Basic event information including location and dates
+   - Monty extension with hazard codes and country codes
+
+2. Hazard Items (`emdat-hazard-*`)
+   - Derived from event items
+   - Additional hazard details including severity and classification
+
+3. Impact Items (`emdat-impact-*`)
+   - Created for each impact metric
+   - Includes detailed impact information with type, value, and units
+
+### Example Usage
 
 ```python
-class EMDATTransformer(BaseTransformer):
-    """Transforms EM-DAT data into STAC Items with Monty extension."""
-    def transform(self, emdat_data: dict) -> pystac.Item:
-        """
-        Transform EM-DAT data into a STAC Item.
-        
-        Args:
-            emdat_data: Dictionary containing EM-DAT event data
-            
-        Returns:
-            pystac.Item: A STAC Item with Monty extension
-        """
+from pystac_monty.sources.emdat import EMDATTransformer, EMDATDataSource
+from pystac_monty.geocoding import GAULGeocoder
+
+# Create data source from Excel file
+data = EMDATDataSource(
+    source_url="https://public.emdat.be",
+    data="path/to/emdat_data.xlsx"
+)
+
+# Initialize geocoder (optional)
+geocoder = GAULGeocoder("path/to/gaul.gpkg")
+
+# Create transformer
+transformer = EMDATTransformer(data, geocoder)
+
+# Generate STAC items
+items = transformer.make_items()
+
+# Or generate specific types of items
+event_items = transformer.make_source_event_items()
+hazard_items = transformer.make_hazard_event_items()
+impact_items = transformer.make_impact_items()

pyproject.toml

@@ -112,10 +112,14 @@ dev-dependencies = [
     "codespell<2.3",
     "coverage>=7.6.2",
     "doc8>=1.1.2",
+    "fiona>=1.10.0",
+    "geopandas>=1.0.1",
+    "geopy>=2.4.0",
     "html5lib>=1.1",
     "jinja2>=3.1.4",
     "jsonschema>=4.23.0",
     "mypy>=1.11.2",
+    "openpyxl>=3.1.0",
     "orjson>=3.10.7",
     "packaging>=24.1",
     "parameterized",
@@ -126,6 +130,7 @@ dev-dependencies = [
     "pytest-recording>=0.13.2",
     "requests>=2.32.3",
     "ruff>=0.6.9",
+    "shapely>=2.0.0",
     "types-html5lib>=1.1.11.20240806",
     "types-jsonschema>=4.23.0.20240813",
     "types-orjson>=3.6.2",

pystac_monty/extension.py

@@ -263,7 +263,7 @@ class HazardDetail(ABC):
     <monty#montyhazard_detail>` docs for details.
     """
 
-    properties: dict[str, Any] = {}
+    properties: dict[str, Any]
 
     def __init__(
         self,
@@ -273,6 +273,7 @@ def __init__(
         severity_label: str | None = None,
         estimate_type: MontyEstimateType | None = None,
     ) -> None:
+        self.properties = {}
         self.cluster = cluster
         if severity_value:
             self.severity_value = severity_value
@@ -348,7 +349,7 @@ class ImpactDetail(ABC):
     <monty#montyimpact_detail>` docs for details.
     """
 
-    properties: dict[str, Any] = {}
+    properties: dict[str, Any]
 
     def __init__(
         self,
@@ -358,6 +359,7 @@ def __init__(
         unit: str | None = None,
         estimate_type: MontyEstimateType = None,
     ) -> None:
+        self.properties = {}
         self.category = category
         self.type = type
         self.value = value