From 567ef85ae1c8b4ed8e2a0fdd04e6fbe626cacb29 Mon Sep 17 00:00:00 2001 From: Damian Borowiecki Date: Mon, 1 Jul 2024 16:36:41 +0200 Subject: [PATCH] Updated documentation --- .../docs/IntegrationOfFacilities.md | 230 +++++++++++++----- 1 file changed, 174 insertions(+), 56 deletions(-) diff --git a/sigeca_data_import_microservice/docs/IntegrationOfFacilities.md b/sigeca_data_import_microservice/docs/IntegrationOfFacilities.md index 3fbc571..6baab9e 100644 --- a/sigeca_data_import_microservice/docs/IntegrationOfFacilities.md +++ b/sigeca_data_import_microservice/docs/IntegrationOfFacilities.md @@ -1,36 +1,37 @@ # Integration of Facilities from SIGECA To eLMIS ## Background -Goal behind the integartion is to have facilities stored in SIGECA Central integrated into the OpenLMIS platform. This will allow users to use predefined resource and have single point of truth for the resource. +Goal behind the integartion is to have facilities stored in SIGECA Central integrated into the OpenLMIS platform. This will allow users to use predefined resource and have single point of truth for the facilities. ## External System Payload Example Payload comming from the SIGECA Central (based on Mapa Sanitario reosurce): ```json5 -{ - "units": [ - { - "name": "HEALTH CENTER BELA VISTA", - "code": "470010", - "abbreviation": "HC", - "category": "Health Center", - "ownership": "Public - National Health Service", - "management": "Public", - "municipality": "Ambriz", - "province": "Bengo", - "operational": true, - "latitude": "7.81807", - "longitude": "1380299", - "services_offered": [ - { - "service_offered_id": 2, - "name": "Internal Medicine" - }, - [...] - ] - }, - [...] - ] -} +[ + { + "id": "0003101f-158a-4d51-80c9-8232256ffe63", + "name": "HEALTH CENTER BELA VISTA", + "is_deleted": true, + "code": "470010", + "acronym": "HC", + "category": "Health Center", + "ownership": "Public - National Health Service", + "management": "Public", + "municipality": "Ambriz", + "province": "Bengo", + "is_operational": true, + "latitude": "7.81807", + "longitude": "1380299", + "services_offered": [ + { + "service_offered_id": 2, + "name": "Internal Medicine" + }, + [...] + ] + }, + [...] +] + ``` ## Mapping of the data @@ -105,7 +106,7 @@ Here is a table mapping the fields from the source JSON schema to the target JSO | `units[].operational` | `active` | Direct mapping | | `units[].latitude` | `latitude` | Direct mapping | | `units[].longitude` | `longitude` | Direct mapping | -| `units[].services_offered[].service_offered_id` | Not Applicable | Not possible to map with internal id due to format. | +| `units[].services_offered[].code` | `supportedPrograms[].code` | Direct | | `units[].services_offered[].name` | `supportedPrograms[].name` | Direct mapping | | Not applicable | `id` | Generated or obtained from source | | Not applicable | `description` | Not provided | @@ -132,8 +133,8 @@ No matchings will rely on the actual Foreign Keys. Facilities will be identified by their `code`. The integration process will follow these steps: 1. **Add New Facility**: If a facility with the given `code` does not exist in the target system, a new facility will be created. -2. **Update Existing Facility**: If a facility with the given `code` exists but its details have changed, the existing facility will be updated with the new details. -3. **Delete Facility**: If a facility present in the target system is not included in the payload, it will be deleted. +2. **Update Existing Facility**: If a facility with the given `code` exists but its details have changed, the existing facility will be updated with the new details except for the supported programs. Supported programs will be added to existing ones, not fully integrated. +3. **Delete Facility**: If a facility present in the target system is not included in the payload, it will be disabled. It will remain in the database but flag for active and enabled will be set. ### Foreign Relations @@ -141,18 +142,21 @@ Facilities will be identified by their `code`. The integration process will foll - **Mapping**: Facilities will be assigned to geographic zones based on the `municipality` and `province` fields. - **Drilldown Logic**: - If `municipality` is provided, the facility will be added at the municipality level. - - If `municipality` is not provided but `province` is provided, the facility will be added at the province level. - - If neither `municipality` nor `province` are provided, new geographic zones will be created as needed. + - If `municipality` or `province` are provided, but don't exist in the system then new geographic zones will be created as needed. + - If `municipality` or `province` is not provided then record is skipped. 2. **Facility Operator** - - **Mapping**: The `ownership` field in the payload will be matched with the `name` field in facility operators in the target system. + - **Mapping**: Ownership is skipped for the time being. 3. **Type of Facility** - **Mapping**: The `category` field in the payload will be matched with the `name` field in the target system's type of facility. 4. **Services Offered** - - **Current Status**: The matching logic for services offered is still under development by another team. - - **Expected Logic**: It is anticipated that the provision names will match information stored in the supported facilities products. They will be matched based on the name. + - **Mapping**: Programs are assigned based on the `code` fields. + - **Drillwodn Logic** + - If product with coresponding `code` is not found in the system then it's added in the most basic form. + - If facility already exists and is updated then new products that are not yet assigned to the facility are added. + - Products are never removed from facilities based on the sigeca content. ### Example Mapping Logic @@ -168,7 +172,7 @@ Facilities will be identified by their `code`. The integration process will foll - Target System: - Check if "Ambriz" exists in the geographic zones. If yes, map the facility to this municipality. - If "Ambriz" does not exist, check for "Bengo". If "Bengo" exists, map the facility to this province. - - If neither exist, create new geographic zones as necessary. + - If neither exist, create new geographic zones on both levels. #### Type of Facility Example - Payload: @@ -219,29 +223,16 @@ Facilities will be identified by their `code`. The integration process will foll - As the synchronization will always start from the full facility list in the external system, persistent logs in the database are not required. ### Log Details -- **Transaction Logs**: Record each facility synchronization, including the timestamp, facility details, and the result of the synchronization (e.g., added, updated, deleted). -- **Error Logs**: Capture errors such as missing mandatory fields and detail the facility and the missing field. +- **Transaction Logs**: Record synchronization processes, including the type and result of the synchronization (e.g., added, updated, deleted). +- **Error Logs**: Capture errors such as missing mandatory fields and detail the facility and the missing field. Each failed facility synchronization should be saved. - **Warning Logs**: Record warnings such as malformed data, including details of the facility and the nature of the malformed data. - **Discrepancy Logs**: Document any discrepancies between the payload and the target system, detailing the discrepancy, the affected facility, and relevant information. -### Example Log Entry -```json -{ - "timestamp": "2024-06-12T12:34:56Z", - "facility_code": "470010", - "action": "update", - "result": "success", - "details": { - "error": "message" - } -} -``` - ### Products integration -- The integration of services offered by facilities is outside the current scope. If the product is not stored in the database it will be skipped. -- If a service is not present in the target system, it cannot be created solely based on the name provided in the payload. -- Manual intervention will be required to handle the integration of products. +- The full integration of services beyond their definition is outside the current scope. Mapa Sanitario doens't provide data to handle requisitions and user assignments. +- It is recommended to configure programs for MS codes in the OpenLMIS ahead of time. +- Manual intervention will be required to handle the integration of products that is beyond simple listing. ### Special Cases and Exceptions - **API Failures**: @@ -266,13 +257,14 @@ The synchronization process will be handled by a dedicated microservice. This mi - Validation checks will be performed to ensure data integrity, including: - Verifying the presence of mandatory fields. - Checking for malformed data (e.g., invalid latitude/longitude formats). + - Providing missing referenced entities including: facility types, programs and geo zones. 3. **Determining Relevant Data for Update** - The transformed data will be compared with the existing data in the target system. - SQL queries will be used to identify: - New facilities to be added. - Existing facilities to be updated. - - Facilities to be deleted (if not present in the payload). + - Facilities to be deleted (soft delete in form of data updating the facility with enabled=false flag). 4. **Triggering Application API** - For identified changes, the microservice will trigger the target system's application API to perform the necessary operations (add/update/delete). @@ -281,7 +273,7 @@ The synchronization process will be handled by a dedicated microservice. This mi ### Detailed Process Flow 1. **Task Scheduling** - - The microservice uses a scheduler (cron job) to run synchronization tasks at regular intervals (e.g., hourly, daily). + - The microservice uses a scheduler (Python APScheduler) to run synchronization tasks at regular intervals (e.g., hourly, daily). 2. **Data Pulling** - REST API Request: @@ -323,6 +315,21 @@ The synchronization process will be handled by a dedicated microservice. This mi ``` 6. **API Trigger for Data Changes** + ###### Facility + | Field | Type | Mandatory| + |---|---|---| + |id|String|Generated| + |code|String|Mandatory| + |name|String|Mandatory| + |geographicZone|Object|Mandatory| + |type|Object|Mandatory| + |active|Boolean|Default true| + |enabled|Boolean|Default true| + |openLmisAccessible|Boolean|Default true| + |supportedPrograms|Array|Optional| + |latitude|Numeric|Optional| + |longitude|Numberic|Optional| + - Add new facility: ```http POST /api/facilities @@ -337,14 +344,125 @@ The synchronization process will be handled by a dedicated microservice. This mi - Update existing facility: ```http PUT /api/facilities/{id} + { + "id": "{id}", + "code": "470010", # Mandatory + "name": "HEALTH CENTER BELA VISTA",# Mandatory + "geographicZone": { # Mandatory + "id": "{id}" + }, + "type": { # Mandatory + "id": "{id}" + }, + "active": true, # Default true + "enabled": true, # Default true + "openLmisAccessible": true, # Default true + "supportedPrograms": [ # Optional + { + "id": "f7419e56-227b-4008-a6e0-f6cf1b0fb5bd" // New + } + ] + } + ``` + - Delete facility: + ```http + UPDATE /api/facilities/{id} + ``` + + - Add new facility: + ```http + POST /api/facilities { "name": "HEALTH CENTER BELA VISTA", + "code": "470010", "abbreviation": "HC", "category": "Health Center", ... } ``` + - Update existing facility: + ```http + PUT /api/facilities/{id} + { + "id": "{id}", + "code": "470010", + "name": "HEALTH CENTER BELA VISTA", + "geographicZone": { + "id": "{id}" + }, + "type": { + "id": "{id}" + }, + "active": true, + "enabled": true, + "openLmisAccessible": true, + "supportedPrograms": [ + { + "id": "{id}", + "supportActive": true, + "supportLocallyFulfilled": false, + "supportStartDate": "2024-07-01" + }, + { + "id": "f7419e56-227b-4008-a6e0-f6cf1b0fb5bd" // New + } + ] + } + ``` - Delete facility: ```http - DELETE /api/facilities/{id} + UPDATE /api/facilities/{id} ``` + + ###### Facility Type + | Field | Type | Mandatory| + |---|---|---| + |code|String|Generated From Name| + |name|String|Mandatory| + |displayOrder|String|Generated (last available order+1)| + * Add Type + ```http + POST /api/facilityTypes + { + "code": "hp/hg", + "name": "HP/HG", + "displayOrder": "328" + } + ``` + + ###### Geographical Zone + | Field | Type | Mandatory| + |---|---|---| + |code|String|Generated From Name| + |name|String|Mandatory| + |displayOrder|String|Generated (last available order+1)| + |level|String|Always Level 3 Ref| + |parent|String|Either Province ID From Municiaplity or First available Level 1 Region for Probince| + * Add Type + ```http + POST /api/geographicZones + { + "code": "gz-huambo", + "name": "Huambo", + "level": {"id": "1c430656-ea48-427c-bf05-67fb390e2d8f"}, + "parent": "1072ad09-e38e-4067-b17f-429bbc5cafe3" + } + ``` + ###### Programs + | Field | Type | Mandatory| + |---|---|---| + |code|String|Mandatory| + |name|String|Mandatory| + |description|Same as Name| + * Add Type + ```http + POST /api/geographicZones + { + "code": "7", + "name": "Cardiologia", + "description": "Cardiologia" + } + ``` + + + \ No newline at end of file