You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Currently, users are required to manually input the period for data retrieval, which can lead to inconsistencies or errors if an outdated or unavailable period is selected. We need to enhance the functionality to automatically select and use the most recent available period for all countries to ensure data consistency and streamline the workflow.
The problem with this task is two-fold:
Past and Future Dates with No Data:
Some future dates (e.g., September 2025, 2026) appear in dropdowns but have no data because they are in the future. Simply selecting the most forward period (e.g., 2026) is incorrect as no data is available. Additionally, if a recent period (e.g., September 2025) has no data yet, pulling this period will result in an error, and we would need to default to an earlier period.
Different Release Schedules for Countries:
Not all countries have the same data release schedule. For example, some countries have data released every September, while Mozambique releases data in December for future years. This inconsistency means that an automated solution must be aware of these country-specific release schedules.
Currently, the output data includes a period column, which indicates which period the data corresponds to. We could consider adding a warning to notify users if the selected period is incorrect or missing data.
Proposed Tasks
Discuss preferences with users
Decide on the approach.
Option 1: Semi-automated Version: Use the Most Recent Yearly Period Based on Current Date for all Countries with Manual Updates
Example: If it's August 2025, use September 2024 data for all countries. If it is October 2025, use September 2025.
Challenge: This option may require additional warnings or error handling for missing data, and we would need to make country-by-country adjustments (e.g., Mozambique’s December release schedule). This solution will require ongoing maintenance to ensure it accommodates changes in release schedules.
Option 2: Automated Version: Run a Script to Scrape the Website and Find the Most Recent Data for Each Country
This would involve running a script before pull_naomi to check the website and identify the most recent available data for each country.
Challenge: If a country has data released for only one indicator during a specific period, we must decide whether to pull data for that period or find the most recent data across all indicators for that country. This would be a large undertaking.
Option 3: Continue Manual Input for Periods
Approach: Keep the current setup where users input the periods they want.
Consequence: No "most recent" parameter option, leaving the decision entirely to the user. This will not address the issue of inconsistencies and errors caused by incorrect period selection.
Option 4: Manual Update by Maintainer
Approach: The package maintainer manually updates the "most recent" parameter input whenever new data is released.
This would be a labor-intensive option, but it ensures the most recent data is always used.
Document decision, reasoning, and path forward
Implement any changes as necessary
The text was updated successfully, but these errors were encountered:
Description
Currently, users are required to manually input the period for data retrieval, which can lead to inconsistencies or errors if an outdated or unavailable period is selected. We need to enhance the functionality to automatically select and use the most recent available period for all countries to ensure data consistency and streamline the workflow.
The problem with this task is two-fold:
Past and Future Dates with No Data:
Some future dates (e.g., September 2025, 2026) appear in dropdowns but have no data because they are in the future. Simply selecting the most forward period (e.g., 2026) is incorrect as no data is available. Additionally, if a recent period (e.g., September 2025) has no data yet, pulling this period will result in an error, and we would need to default to an earlier period.
Different Release Schedules for Countries:
Not all countries have the same data release schedule. For example, some countries have data released every September, while Mozambique releases data in December for future years. This inconsistency means that an automated solution must be aware of these country-specific release schedules.
Currently, the output data includes a
period column
, which indicates which period the data corresponds to. We could consider adding a warning to notify users if the selected period is incorrect or missing data.Proposed Tasks
Option 1: Semi-automated Version: Use the Most Recent Yearly Period Based on Current Date for all Countries with Manual Updates
Option 2: Automated Version: Run a Script to Scrape the Website and Find the Most Recent Data for Each Country
pull_naomi
to check the website and identify the most recent available data for each country.Option 3: Continue Manual Input for Periods
Option 4: Manual Update by Maintainer
The text was updated successfully, but these errors were encountered: