Visualising Projects Using ODK OData Endpoints

[spwoodcock]

Why We Use OData

• FMTM takes the same stance as ODK for data visualisation: we can't possibly anticipate all user requirements, so instead provide open standard endpoints do the data can be consumed by other tools.
• PowerBI is one tool that can read OData spec endpoints.
• Users can then generate tables / graphs to their exact specification with the given data

How To Connect

• Related issues with some documentation
  • Add OData spec endpoints for project export and visualisation #1440
  • Document the process of using OData endpoints for data visualisation #1778
• Official ODK docs page: https://docs.getodk.org/tutorial-mapping-households/

Entity Data

OData endpoint docs: https://docs.getodk.org/central-api-odata-endpoints/#id3

• This data relates to the features we are mapping, e.g. buildings.
• The three useful fields we need here are:
  • status
    • 0=pending
    • 1=in progress
    • 2=mapped
    • 6=bad (there was an issue mapping the feature)
  • task_id shows which task area the feature falls within (FMTM features are subdivided into task areas)
  • submission_id gives the UUID of any submission that has been made for this feature.
    • We need this field to link up the Entity data, to the submission data, described below.

Submission Data

OData endpoint: https://docs.getodk.org/central-api-odata-endpoints/#odata-dataset-service

• This data is related to submission made from users.
• Generally there should only be one submission per entity.
• There is probably a lot of project specific data we can't use here! The key fields are likely:
  • submission ID (instanceId field I think?)
  • username (who submitted)
  • validation fields, giving an indication on if the building is digitised correctly.
• The submission UUID should map to the entity submission_id field.
• Given this, we can create a composite / calculated dataset combining the two endpoints.

Two Required Dashboards

Ideally these dashboards could have most of their structure configured for display in Power BI desktop, with a little bit of dynamic tweaking upon download (hopefully this is possible somehow?):

• Downloading for a project would inject the ODK credentials for connection to the data.
• Downloading for an organisation would inject both credentials and the project ids to visualise on the ODK server (those mapped within the organisation).

Project Dashboard

Stats for a specific project.

1. Simple counts of the following:

• Number of features mapped out of total.
• Number of entirely new features added (from new_feature submission field).
• From entity status=6, we then determine what the issue was from the submission data:
  • Number of features that were marked as not existing
  • Number of features that were digitised incorrectly.

2. Time series of mapping progress over time (rate of mapping)

3. Mapper leaderboard for total number of features mapped per user

Organisation Dashboard

The organisation dashboard should have an indication of mapping progress, across multiple projects.

1. Total mapping completion (percent mapped), across all projects.

2. Total number of complete projects out of total (projects where no entity has status=0 or status=1)

3. Other stats?

[ozz-gis]

Hi Sam,
I created a quick Power BI template. Please see your email. I could only attached a pdf version here. If you have issues to connect, you can follow the steps below:

Open the .pbix File in Power BI Desktop.

Go to Data Source Settings:

From the Home tab → Transform Data → Data Source Settings.
Locate the ODK Data Source:

Select the ODK API URL.
Edit Credentials:

Choose the correct Authentication Method (likely Basic Authentication).
Enter your username and password.
Apply Changes:

Click OK and Refresh the data.

Please let me know if you have any questions. It is probably multiple and repeating info but I wanted to bring it in so we can eliminate or brainstorm on it.
Thanks,
Ozgur

Projects_ODK.pdf

[spwoodcock]

Thank you @ozz-gis !

Do you know if it could be possible to configure the project credentials via code?

I have never looked into this - no worries if you aren't sure 👍

The ideal would be that we can download the pbix preconfigured for the project, from the project page

[manjitapandey]

This seems so cool but can we also do comparision between the status of submission of ward 2 and 3 on same chart?? May be on stacked bar chart??

[spwoodcock]

By comparison do you mean combined statistics from both projects?

@ozz-gis is looking into creating a summary page to aggregate the stats from multiple projects 👍