Skip to content

Commit

Permalink
Updated documentation
Browse files Browse the repository at this point in the history
  • Loading branch information
@dborowiecki
dborowiecki committed Jul 1, 2024
1 parent 672e1ac commit 567ef85
Showing 1 changed file with 174 additions and 56 deletions.
230 changes: 174 additions & 56 deletions sigeca_data_import_microservice/docs/IntegrationOfFacilities.md
View file
Original file line number Diff line number Diff line change
@@ -1,36 +1,37 @@
# Integration of Facilities from SIGECA To eLMIS

## Background
Goal behind the integartion is to have facilities stored in SIGECA Central integrated into the OpenLMIS platform. This will allow users to use predefined resource and have single point of truth for the resource.
Goal behind the integartion is to have facilities stored in SIGECA Central integrated into the OpenLMIS platform. This will allow users to use predefined resource and have single point of truth for the facilities.

## External System Payload Example
Payload comming from the SIGECA Central (based on Mapa Sanitario reosurce):
```json5
{
"units": [
{
"name": "HEALTH CENTER BELA VISTA",
"code": "470010",
"abbreviation": "HC",
"category": "Health Center",
"ownership": "Public - National Health Service",
"management": "Public",
"municipality": "Ambriz",
"province": "Bengo",
"operational": true,
"latitude": "7.81807",
"longitude": "1380299",
"services_offered": [
{
"service_offered_id": 2,
"name": "Internal Medicine"
},
[...]
]
},
[...]
]
}
[
{
"id": "0003101f-158a-4d51-80c9-8232256ffe63",
"name": "HEALTH CENTER BELA VISTA",
"is_deleted": true,
"code": "470010",
"acronym": "HC",
"category": "Health Center",
"ownership": "Public - National Health Service",
"management": "Public",
"municipality": "Ambriz",
"province": "Bengo",
"is_operational": true,
"latitude": "7.81807",
"longitude": "1380299",
"services_offered": [
{
"service_offered_id": 2,
"name": "Internal Medicine"
},
[...]
]
},
[...]
]

```

## Mapping of the data
Expand Down Expand Up @@ -105,7 +106,7 @@ Here is a table mapping the fields from the source JSON schema to the target JSO
| `units[].operational` | `active` | Direct mapping |
| `units[].latitude` | `latitude` | Direct mapping |
| `units[].longitude` | `longitude` | Direct mapping |
| `units[].services_offered[].service_offered_id` | Not Applicable | Not possible to map with internal id due to format. |
| `units[].services_offered[].code` | `supportedPrograms[].code` | Direct |
| `units[].services_offered[].name` | `supportedPrograms[].name` | Direct mapping |
| Not applicable | `id` | Generated or obtained from source |
| Not applicable | `description` | Not provided |
Expand All @@ -132,27 +133,30 @@ No matchings will rely on the actual Foreign Keys.
Facilities will be identified by their `code`. The integration process will follow these steps:

1. **Add New Facility**: If a facility with the given `code` does not exist in the target system, a new facility will be created.
2. **Update Existing Facility**: If a facility with the given `code` exists but its details have changed, the existing facility will be updated with the new details.
3. **Delete Facility**: If a facility present in the target system is not included in the payload, it will be deleted.
2. **Update Existing Facility**: If a facility with the given `code` exists but its details have changed, the existing facility will be updated with the new details except for the supported programs. Supported programs will be added to existing ones, not fully integrated.
3. **Delete Facility**: If a facility present in the target system is not included in the payload, it will be disabled. It will remain in the database but flag for active and enabled will be set.

### Foreign Relations

1. **Geographic Zone**
- **Mapping**: Facilities will be assigned to geographic zones based on the `municipality` and `province` fields.
- **Drilldown Logic**:
- If `municipality` is provided, the facility will be added at the municipality level.
- If `municipality` is not provided but `province` is provided, the facility will be added at the province level.
- If neither `municipality` nor `province` are provided, new geographic zones will be created as needed.
- If `municipality` or `province` are provided, but don't exist in the system then new geographic zones will be created as needed.
- If `municipality` or `province` is not provided then record is skipped.

2. **Facility Operator**
- **Mapping**: The `ownership` field in the payload will be matched with the `name` field in facility operators in the target system.
- **Mapping**: Ownership is skipped for the time being.

3. **Type of Facility**
- **Mapping**: The `category` field in the payload will be matched with the `name` field in the target system's type of facility.

4. **Services Offered**
- **Current Status**: The matching logic for services offered is still under development by another team.
- **Expected Logic**: It is anticipated that the provision names will match information stored in the supported facilities products. They will be matched based on the name.
- **Mapping**: Programs are assigned based on the `code` fields.
- **Drillwodn Logic**
- If product with coresponding `code` is not found in the system then it's added in the most basic form.
- If facility already exists and is updated then new products that are not yet assigned to the facility are added.
- Products are never removed from facilities based on the sigeca content.


### Example Mapping Logic
Expand All @@ -168,7 +172,7 @@ Facilities will be identified by their `code`. The integration process will foll
- Target System:
- Check if "Ambriz" exists in the geographic zones. If yes, map the facility to this municipality.
- If "Ambriz" does not exist, check for "Bengo". If "Bengo" exists, map the facility to this province.
- If neither exist, create new geographic zones as necessary.
- If neither exist, create new geographic zones on both levels.

#### Type of Facility Example
- Payload:
Expand Down Expand Up @@ -219,29 +223,16 @@ Facilities will be identified by their `code`. The integration process will foll
- As the synchronization will always start from the full facility list in the external system, persistent logs in the database are not required.

### Log Details
- **Transaction Logs**: Record each facility synchronization, including the timestamp, facility details, and the result of the synchronization (e.g., added, updated, deleted).
- **Error Logs**: Capture errors such as missing mandatory fields and detail the facility and the missing field.
- **Transaction Logs**: Record synchronization processes, including the type and result of the synchronization (e.g., added, updated, deleted).
- **Error Logs**: Capture errors such as missing mandatory fields and detail the facility and the missing field. Each failed facility synchronization should be saved.
- **Warning Logs**: Record warnings such as malformed data, including details of the facility and the nature of the malformed data.
- **Discrepancy Logs**: Document any discrepancies between the payload and the target system, detailing the discrepancy, the affected facility, and relevant information.

### Example Log Entry
```json
{
"timestamp": "2024-06-12T12:34:56Z",
"facility_code": "470010",
"action": "update",
"result": "success",
"details": {
"error": "message"
}
}
```


### Products integration
- The integration of services offered by facilities is outside the current scope. If the product is not stored in the database it will be skipped.
- If a service is not present in the target system, it cannot be created solely based on the name provided in the payload.
- Manual intervention will be required to handle the integration of products.
- The full integration of services beyond their definition is outside the current scope. Mapa Sanitario doens't provide data to handle requisitions and user assignments.
- It is recommended to configure programs for MS codes in the OpenLMIS ahead of time.
- Manual intervention will be required to handle the integration of products that is beyond simple listing.

### Special Cases and Exceptions
- **API Failures**:
Expand All @@ -266,13 +257,14 @@ The synchronization process will be handled by a dedicated microservice. This mi
- Validation checks will be performed to ensure data integrity, including:
- Verifying the presence of mandatory fields.
- Checking for malformed data (e.g., invalid latitude/longitude formats).
- Providing missing referenced entities including: facility types, programs and geo zones.

3. **Determining Relevant Data for Update**
- The transformed data will be compared with the existing data in the target system.
- SQL queries will be used to identify:
- New facilities to be added.
- Existing facilities to be updated.
- Facilities to be deleted (if not present in the payload).
- Facilities to be deleted (soft delete in form of data updating the facility with enabled=false flag).

4. **Triggering Application API**
- For identified changes, the microservice will trigger the target system's application API to perform the necessary operations (add/update/delete).
Expand All @@ -281,7 +273,7 @@ The synchronization process will be handled by a dedicated microservice. This mi
### Detailed Process Flow

1. **Task Scheduling**
- The microservice uses a scheduler (cron job) to run synchronization tasks at regular intervals (e.g., hourly, daily).
- The microservice uses a scheduler (Python APScheduler) to run synchronization tasks at regular intervals (e.g., hourly, daily).

2. **Data Pulling**
- REST API Request:
Expand Down Expand Up @@ -323,6 +315,21 @@ The synchronization process will be handled by a dedicated microservice. This mi
```

6. **API Trigger for Data Changes**
###### Facility
| Field | Type | Mandatory|
|---|---|---|
|id|String|Generated|
|code|String|Mandatory|
|name|String|Mandatory|
|geographicZone|Object|Mandatory|
|type|Object|Mandatory|
|active|Boolean|Default true|
|enabled|Boolean|Default true|
|openLmisAccessible|Boolean|Default true|
|supportedPrograms|Array|Optional|
|latitude|Numeric|Optional|
|longitude|Numberic|Optional|

- Add new facility:
```http
POST /api/facilities
Expand All @@ -337,14 +344,125 @@ The synchronization process will be handled by a dedicated microservice. This mi
- Update existing facility:
```http
PUT /api/facilities/{id}
{
"id": "{id}",
"code": "470010", # Mandatory
"name": "HEALTH CENTER BELA VISTA",# Mandatory
"geographicZone": { # Mandatory
"id": "{id}"
},
"type": { # Mandatory
"id": "{id}"
},
"active": true, # Default true
"enabled": true, # Default true
"openLmisAccessible": true, # Default true
"supportedPrograms": [ # Optional
{
"id": "f7419e56-227b-4008-a6e0-f6cf1b0fb5bd" // New
}
]
}
```
- Delete facility:
```http
UPDATE /api/facilities/{id}
```

- Add new facility:
```http
POST /api/facilities
{
"name": "HEALTH CENTER BELA VISTA",
"code": "470010",
"abbreviation": "HC",
"category": "Health Center",
...
}
```
- Update existing facility:
```http
PUT /api/facilities/{id}
{
"id": "{id}",
"code": "470010",
"name": "HEALTH CENTER BELA VISTA",
"geographicZone": {
"id": "{id}"
},
"type": {
"id": "{id}"
},
"active": true,
"enabled": true,
"openLmisAccessible": true,
"supportedPrograms": [
{
"id": "{id}",
"supportActive": true,
"supportLocallyFulfilled": false,
"supportStartDate": "2024-07-01"
},
{
"id": "f7419e56-227b-4008-a6e0-f6cf1b0fb5bd" // New
}
]
}
```
- Delete facility:
```http
DELETE /api/facilities/{id}
UPDATE /api/facilities/{id}
```

###### Facility Type
| Field | Type | Mandatory|
|---|---|---|
|code|String|Generated From Name|
|name|String|Mandatory|
|displayOrder|String|Generated (last available order+1)|
* Add Type
```http
POST /api/facilityTypes
{
"code": "hp/hg",
"name": "HP/HG",
"displayOrder": "328"
}
```

###### Geographical Zone
| Field | Type | Mandatory|
|---|---|---|
|code|String|Generated From Name|
|name|String|Mandatory|
|displayOrder|String|Generated (last available order+1)|
|level|String|Always Level 3 Ref|
|parent|String|Either Province ID From Municiaplity or First available Level 1 Region for Probince|
* Add Type
```http
POST /api/geographicZones
{
"code": "gz-huambo",
"name": "Huambo",
"level": {"id": "1c430656-ea48-427c-bf05-67fb390e2d8f"},
"parent": "1072ad09-e38e-4067-b17f-429bbc5cafe3"
}
```
###### Programs
| Field | Type | Mandatory|
|---|---|---|
|code|String|Mandatory|
|name|String|Mandatory|
|description|Same as Name|
* Add Type
```http
POST /api/geographicZones
{
"code": "7",
"name": "Cardiologia",
"description": "Cardiologia"
}
```



0 comments on commit 567ef85

Please sign in to comment.