Dates/times and other structures in R-Instat

[rdstern]

We already load the openair package - I think because it has a wind rose command. In addition it has a single data set that will become very useful for our climatic work. It is called mydata and is hourly data for 7.5 years, so that is 65,000 observations. It includes wind speed and wind direction. All potentially very useful! (And it links to the openair project run from Kings College, London, so we may soon get in touch.)

It opens easily into R-Instat. And it has a single date/time column. But as it is hourly it isn't yet recognised by our date functions. So there are two points to this issue.

1. How easy/possible would it be to start adding date/time types of data to R-Instat, so the hourly data is recognised as a date type? It may be a good time to consider this aspect, because a) we will be insisting on a single date column for the climatic work, and b) we will have more examples (particularly with data from automatic stations) of sub-daily data. Could this be added to Danny's work next week without affecting the other important tasks too much?

2. This looks to be a great example of data to use as we practice using R-Instat. My list is growing as follows:
   a) Our usual little survey (36 observations)
   b) The diamonds data (50,000 observations) from ggplot2
   c) Now this one - called mydata with 55,000 observations of hourly data.

We need a few more, but I quite like the idea of using relatively few, but interesting, data sets for most of our examples - and videos. I particularly need some good examples of daily climatic data, and we have plenty, but not yet one that is ideal. I have written to different countries to see what I can do. Currently the best is from Dodoma, Tanzania. We should also have at least one really interesting data set on the corruption work. Perhaps that could be the World Bank data set?

Roger

[rdstern]

WKenya_StationInfo.xlsx

Here are the rainfall data from Western Kenya that we can use as an example for the climatic work - and it can be shared freely. There are 2 files, one wit about 600,000 rows of rainfall data from 52 stations (plus one to get rid of).

The second gives the station information. I assume - as both have come from ClimSoft - that the names match, so we can link them. And then add to the station information or make an annual file, etc.

[dannyparsons]

Good to test with this data again and also reconsider variables containing time

[dannyparsons]

A nice very simple package for date independent times is hms https://cran.r-project.org/web/packages/hms/index.html.
It has very little, but if we don't need anything more complicated than in here then I think we should implement this since it's simple and also part of the tidyverse.

[rdstern]

Looks good. We do need to extend our facilities so we can analyse within-day data. This is partly for the hourly synoptic data, but also for data from automatic stations.
From a quick look, this seems to be a great find.

[dannyparsons]

This is to deal with columns only containing a time. I think it was a gap in R as there are lots of ways to deal with date and time in a single column not nothing for time separately.
We will need to handle both cases presumably so might need to combine this with other tools for date/time columns.

[rdstern]

@dannyparsons can we consider how to accommodate more structures in R-Instat - perhaps in the forthcoming version 7. We have dates and could perhaps (here? have times and possibly also recognise date times.
I would like to start being able to cope with hourly data in some way, particularly as we have the era5 data as well as synoptic data now from West Africa.

In addition with have the new structured menu. There we are able to define circular data and also now survival data. Circular data is recognised in the column metadata, but not yet as a type in the data frame? Survival isn't recognised in either.

So I suggest dt for date time and t for time - as important. And s for structured (usually numeric) variables.