Add fraction of evaporation to the start of the rains dialog

[rdstern]

@lilyclements and @MeSophie this is the latest request from CIMH of something we had in the original Instat that is not yet in R-Instat. Here is the dialog in the old Instat:

@volloholic suggests it would be (very?) easy to add to R-Instat.

If we do, then I assume we will have an evaporation variable in the data with no missing values. We don't give the option of a constant. Instead, if constant, we just make a variable with that value. If there are missing, then (before using the dialog), we suggest using replace to insert a sensible value, e.g. 5mm per day. If we only have the values for a single year, then we use Merge to stretch it out for every year.

Then the calculation gets the moving total for the rainfall exactly as it does now. It does the same for the fractional moving total of the evaporation. Then the start is that first day (each year) that the moving rainfall total exceeds the moving evaporation total.

Now here is the current start of the rains dialog:

I'm not clear just how to add this extra component. This is also the only dialog that is so complicated. So I wonder if this is our occasion to change the structure of the by adding another sub-dialog button. Could we perhaps keep the first line - with all possibilities - on this dialog and then have a Further Conditions button for the other 3 conditions? Maybe after the Further Conditions button - if one or more is specified, then the word Specified appears on the main dialog after the button?

Then for Evaporation option (on the main dialog, we have a Fraction up-down control, with a default of 0.5 and steps of ,01 with lower limit of 0.01 and upper limit of 5. Then a single Evaporation: receiver for the evaporation variable.

[lilyclements]

@rdstern I don't mind or have any strong opinions on the layout.

In terms of R code to do what you are asking, I've got a draft below on what I am understanding we want:

1. Take our evaporation column and multiply it by the value in the NUD to get the fraction of evaporation
2. We get the rolling sum over N days for rainfall, as normal
3. We get the rolling sum over N days for fraction of evaporation
4. Filter to where the rolling sum of the rainfall total exceeds the moving evaporation total

I'm sharing some R Code below which does this if you wanted to try it out:

# Create variable called "evap" through calculations dialog:
dodoma <- data_book$get_data_frame(data_name="dodoma", use_current_filter=FALSE)
attach(what=dodoma)
scalars <- data_book$get_scalars(data_name="dodoma")
attach(what=scalars)
evap <- 5
data_book$add_columns_to_data(data_name="dodoma", col_name="evap", col_data=evap, before=FALSE)
detach(name=dodoma, unload=TRUE)
detach(name=scalars, unload=TRUE)
data_book$append_to_variables_metadata(data_name="dodoma", col_names="evap", property="labels", new_val="")
rm(list=c("evap", "dodoma", "scalars"))

# Dialog: Start of Rains
roll_sum_rain <- instat_calculation$new(type="calculation", function_exp="RcppRoll::roll_sumr(x=rain, n=2, fill=NA, na.rm=FALSE)", result_name="roll_sum_rain", calculated_from=list("dodoma"="rain"))

# we multiply our evaporation column by the value in the fraction NUD (0.05 here)
fraction_evap <- instat_calculation$new(type="calculation", function_exp="evap*0.05", result_name="fraction_evap", calculated_from=list("dodoma"="evap"))

# We take our evaporation fraction variable and run a rolling sum on it
roll_sum_evap <- instat_calculation$new(type="calculation", function_exp="RcppRoll::roll_sumr(x=fraction_evap, n=2, fill=NA, na.rm=FALSE)", result_name="roll_sum_evap", sub_calculations = list(fraction_evap))

# Then we filter to where the rolling sum of the rainfall total exceeds the moving evaporation total
conditions_filter <- instat_calculation$new(type="filter", function_exp="((rain >= 0.85) & roll_sum_rain > roll_sum_evap) | is.na(x=rain) | is.na(x=roll_sum_rain)", sub_calculations=list(roll_sum_rain, roll_sum_evap))

# then continue as normal:
grouping_by_year <- instat_calculation$new(type="by", calculated_from=list("dodoma"="year"))
doy_filter <- instat_calculation$new(type="filter", function_exp="doy_366 >= 1 & doy_366 <= 366", calculated_from=calc_from_convert(x=list(dodoma="doy_366")))
start_of_rains_doy <- instat_calculation$new(type="summary", function_exp="ifelse(test=is.na(x=dplyr::first(x=rain)) | is.na(x=dplyr::first(x=roll_sum_rain)), yes=NA, no=dplyr::first(x=doy_366, default=NA))", result_name="start_rain", save=2)
start_rain_date <- instat_calculation$new(type="summary", function_exp="dplyr::if_else(condition=is.na(x=dplyr::first(x=rain)) | is.na(x=dplyr::first(x=roll_sum_rain)), true=as.Date(NA), false=dplyr::first(date, default=NA))", result_name="start_rain_date", save=2)
start_of_rains_status <- instat_calculation$new(type="summary", function_exp="n() > 0", result_name="start_rain_status", save=2)
start_of_rains_combined <- instat_calculation$new(type="combination", manipulations=list(conditions_filter, grouping_by_year, doy_filter), sub_calculation=list(start_of_rains_doy, start_rain_date, start_of_rains_status))
data_book$run_instat_calculation(calc=start_of_rains_combined, display=FALSE, param_list=list(drop=FALSE))

rm(list=c("start_of_rains_combined", "conditions_filter", "roll_sum_rain", "grouping_by_year", "doy_filter", "start_of_rains_doy", "start_rain_date", "start_of_rains_status"))

Here, we have added two new lines, and altered one. The two new lines are:

fraction_evap <- instat_calculation$new(type="calculation", function_exp="evap*0.05", result_name="fraction_evap", calculated_from=list("dodoma"="evap"))
roll_sum_evap <- instat_calculation$new(type="calculation", function_exp="RcppRoll::roll_sumr(x=fraction_evap, n=2, fill=NA, na.rm=FALSE)", result_name="roll_sum_evap", sub_calculations = list(fraction_evap))

Then we altered our conditions filter to now be where roll_sum_rain > roll_sum_evap rather than roll_sum_rain > some_value (and added roll_sum_evap into the sub_calculations):

conditions_filter <- instat_calculation$new(type="filter", function_exp="((rain >= 0.85) & roll_sum_rain > roll_sum_evap) | is.na(x=rain) | is.na(x=roll_sum_rain)", sub_calculations=list(roll_sum_rain, roll_sum_evap))

[rdstern]

@MeSophie I suggest this is an exciting improvement in 3 ways.
a) We add a useful new feature to R-Instat - and one that has been requested - by CIMH, see @lilyclements post above
b) It is one of the last items in the original Instat still to be added to R-Instat
c) We are using it to change the dialog into one that is more "normal" for R-Instat, i.e. simple main dialog with sub-dialog when the user needs more. The current structure is an oddity in R-Instat.
I hope the changes will be easy too!

The sub-dialog will be of the 3 rows of controls above that start Number of Rain Days, Dry Spell, Dry Period.
In the main dialog therefore:
a) In the Display group, change Day of Year to just Day. Move the controls over a bit to the left, so the whole dialog will become slightly narrower.
b) The Total Rainfall and over Days remains as now. Then replace Calculate Rainfall Value By, with just Value: This is now below the T of Total.
c) There are then 3 radio buttons, vertically. The first is Amount as now, including the up-down. The third is Percentile with the up-down, as now.
d) The second radio button is new. It is labelled Evaporation. It then has a single receiver for an evaporation variable. Then there is a label Fraction: This has a new up-down with default 0.5. It goes from 0.01 in steps of 0.01 to 10.
e) Then there is a new checkbox, with label Additional. The default is unchecked. If checked, then a radio button with label Add, becomes visible and clicking on that opens the new sub-dialog.
f) I'd like something else after the Add button to indicate that an additional (or more than one) condition has been added. I wonder how that should be done simply and well? If you have an idea, please include it, otherwise leave that for now.
g) Now the sub-dialog. It has label Additional Conditions at the top.
h) Then just those 3 check-boxes. I suggest orientate them down, rather than across. So the top row has the 3 checkboxes. They are labelled now (just) Rain Days, Dry Spell, Dry Period. Default is unchecked and the other controls, as now, are only visible when one of them is checked.
i) Then they are added just as now, and there is the usual 3 bottom controls.