When aggregating, recalculate last record if partial #525
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Until now, when aggregating, it ignored and did not store the last record of the aggregated time series if it contained the MISS flag. This was because more source time series data might become available later. However it also meant that if more source time series data never became available, the last aggregated record would never be calculated. In addition, we sometimes want to get an estimation of the current daily value a bit before midnight. Now the last record is calculated alright given enough data (i.e. if the missing data are up to max_missing), and it is recalculated when more source data become available. This commit also upgrades pthelma to 2.1, and, correspondingly, changes the missing flag from "MISS" to "MISSING3", where, in this example, 3 is the number of missing values in the source data. This will not migrate existing records. This lack of migration should not cause any problems with aggregation.
Codecov ReportAll modified and coverable lines are covered by tests ✅
Additional details and impacted files@@ Coverage Diff @@
## master #525 +/- ##
=======================================
Coverage 97.30% 97.30%
=======================================
Files 51 51
Lines 3234 3234
=======================================
Hits 3147 3147
Misses 87 87 ☔ View full report in Codecov by Sentry. |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
No description provided.