-
Notifications
You must be signed in to change notification settings - Fork 16
TimeSeries
The section covers aspects of timeseries data used by CWMS.
A timeseries is simply a collection of values over time for a specific item of interest. For CWMS, as we are dealing with water, this could be something like Stage (height of water in a river channel) at a particular site or the flow of water out of a dam.
CWMS Contains multiple times of
This is the simplest timeseries that most people think of; a measurement is made on regular intervals, hourly, every 15minutes, etc and recorded. The majority of CWMS data falls into this category. You will know if data is missing as there will be a row returned to you with a time but no data element.
Example Request: https://cwms-public.usace.army.mil/cwms-data/timeseries?office=SWT&name=TENK.Flow-Res%20Out.Ave.1Hour.1Hour.Rev-Regi-Flowgroup&begin=PT-6h&format=xml
Example Result:
<?xml version="1.0" encoding="windows-1252"?>
<time-series>
<quality-codes>
<!-- The following quality codes are used in this dataset-->
<code value="0" meaning="Unscreened"/>
</quality-codes>
<time-series>
<office>SWT</office>
<name>TENK.Flow-Res Out.Ave.1Hour.1Hour.Rev-Regi-Flowgroup</name>
<alternate-names>
<name>07197500.Flow-Res Out.Ave.1Hour.1Hour.Rev-Regi-Flowgroup</name>
<name>CE61840E.Flow-Res Out.Ave.1Hour.1Hour.Rev-Regi-Flowgroup</name>
<name>OK10311.Flow-Res Out.Ave.1Hour.1Hour.Rev-Regi-Flowgroup</name><name>TENO2.Flow-Res Out.Ave.1Hour.1Hour.Rev-Regi-Flowgroup</name>
</alternate-names>
<regular-interval-values interval="PT1H" unit="cfs" segment-count="1">
<segment position="1" first-time="2022-01-13T10:00:00Z" last-time="2022-01-13T15:00:00Z" value-count="6"><!-- value quality-code -->
32.14286 0
32.14286 0
32.14286 0
32.14286 0
32.14286 0
32.14286 0
</segment>
</regular-interval-values>
</time-series>
</time-series>Some data does not come in at fixed interval. This could be something like release changes that happen only when a change is required. Such as for changes in flood control conditions or water delivery requirements.
Some data often comes in at fix intervals but there are conditions where the interval might change. The Los Angeles district does this for their operations as the environment is prone to flash flooding. Shorting the data interval provides for greater operational awareness, but keeping in long in normal times helps save storage.
example query curl -X 'GET' \ 'https://cwms-public.usace.army.mil/cwms-data/timeseries?name=5th%20St-SAR%20in%20Santa%20Ana.Stage.Inst.~15Minutes.0.GOES-rev&office=SPL&unit=ft&begin=PT-4h' \ -H 'accept: application/json;version=2'
Not here we had to use curl, the control of output format is being moved to the accept header instead of a query parameter.
example data
{
"begin": "2022-01-13T16:05:45.147Z",
"end": "2022-01-13T16:05:45.147Z",
"interval": {
"seconds": 0,
"nano": 0,
"negative": true,
"zero": true,
"units": [
{
"duration": {
"seconds": 0,
"nano": 0,
"negative": true,
"zero": true
},
"durationEstimated": true,
"dateBased": true,
"timeBased": true
}
]
},
"name": "string",
"nextPage": "string",
"officeId": "string",
"page": "string",
"pageSize": 0,
"total": 0,
"units": "string",
"value-columns": [
{
"name": "string",
"ordinal": 0,
"datatype": "string"
}
],
"values": {
"date-time": 0,
"value": 0,
"quality-code": 0
},
"verticalDatumInfo": {
"office": "string",
"unit": "string",
"location": "string",
"native-datum": "string",
"elevation": 0,
"offsets": [
{
"estimate": true,
"toDatum": "string",
"value": 0
}
]
}
}Covered in more detail in a later section, midnight is not the same in all portions of the Nation. In additional midnight isn't always the same UTC time within a single area of the nation. This concept of timeseries exists for when the intervals of a timeseries can change but are predictable AND expected. While still irregular data, for most purposes such as data alignment in reports, it can be treated as regular interval data.
However if you are doing mathematic operations IT IS NOT REGULAR INTERVAL DATA and any changes due to daylight savings MUST be accounted for.
Example: Not yet available from RADAR API.
For time series values that are composites of data over time, like an average or sum, the duration is the window time from the source data that was used to generate a given value.
Sometimes data must be editted, sometimes something like a new forecast comes in. Either way it is prefered not to loose the orginal contents. This system allows us to specify a group of values within a timeseries so that it can be later retrieved, either to know what it was or to recreate computed data as it would have been at a certain time.
The concept of versioning can be applies to any of the above timeseries, and the timeseries catalog entry for a given timeseries will list the available version.
Example: Not yet available from the RADAR API.
The US Army Corps of Engineers operations within all 50 states, coordinating with local and international operators to determine how best to provide flood control to the nation. Given the long history of doing this many of our operations are still based on daily time steps. Usually (but not always) at Midnight Local Time.
As such when looking for daily data you will often see ~1Day (and in the future 1DayLocal) in as the interval to the time series. If this is the case be prepared for the UTC time of the dataset to change.
Example: Midnight data for The Sacramento District (Pacific Local Time)
Midnight will be 0700 UTC during the daylight savings portions of the year and 0800 during the non-daylight savings portions of the year.