Streamlining e-PICSA

[rdstern]

We have just concluded 4 days of work with ZMD to fulfil the GIZ instructions to have summaries from the 40 main stations in Zambia in Google buckets by the end of the contract on 31 August. @jkmusyoka now goes to Malawi and is reasonably confident of being able to do the same for the 21 main stations in Malawi next week.
However, we have, in hindsight, been extremely lucky. I am also delighted that the GIZ extension provides an opprtunity for us to make this into a smooth process. I therefore list here some of the "rocky bits", together with suggestions for possibly improvements that I hope we can make.

1. Installation was very tricky. We had problems like never before on the installation. These were eventually resolved by James on all except one machine. James may give more details, but (on the problem installations) most were solved by uninstalling and trying again as administrator. Of course you need administrator level (I assume?) to install, but why have to state this explicitely?
2. Almost all the machines already had some version of R installed. I thought that the R-Instat installation was designed to be totally separate? We found it to be the opposite, in that R-Instat appears to look first in the R installation for any packages needed. If so, that's a very risky proposition, because each installation is then dependent on the (sometimes very old) versions of some packages. We have written an issue on this (Serious issue on where R-Instat looks for R-packages? #9109). I assume this is a @ChrisMarsh82 task to delegate or investigate?
3. The R packages issue above is maybe a reason that the new export to Google buckets could not be made to work on any of the participants computers. It works smoothly on James (and I think Lily's) machines. After efforts it now works on my machine. Note the complication James had to overcome as a consequence. The uploading had to be from his machine. So the data for the 40 stations had to be provided to him, to undertake the uploading. @lilyclements for reasons we still don't clearly understand, the export dialog needs the daily data data frame. But Zambia Met made it clear we could not have a copy of the daily data for other than Eastern Province. So how did James manage the export?
4. Please can we avoid this need for the daily data dataframe in our smooth version of the export dialog.
5. @Patowhiz we had your improved Import from Climsoft dialog. However, we couldn't use it. For reasons we don't understand Celestine was able to connect to climsoft directly, but not via your dialog. (we had IT people present who contend this is probably a networking, rather than an R-Instat problem. We were keen to try on another machine, which is the used by Boniface to supply climsoft data normally, but unfortunately that remains the one machine on which we consistently get the dreaded exception error immediately it loads R-Instat. This is not urgent to resolve in Zambia now, but would be good to sort out once the extension starts. However, they were excited by the idea, and I suggest testing this aspect should be done routinely and would therefore be a task for you on your first (Friday) in Eswatini.
6. As an aside, these installation and running issues would have made this training very tricky indeed with just one person. I note this in particular in relation to the forthcoming Eswatini training where there will be 20 participants. We had only 10 this week!
7. To be continued!

[Patowhiz]

@rdstern thanks for the useful updates.

@rdstern considering @jkmusyoka is going to Malawi to do the same next week. Are there emergency fixes that you would like addressed and a new release made before Monday. @ChrisMarsh82 and @N-thony it may be worth reproducing the described environments to iron out the installation issues. @rdstern I understand the challenges that the 2 of you may have gone through.

@rdstern did you try testing any of the participants machine with our online e-picsa database? If the machine can connect to the online database through the dialog then it should be able to connect to their database. Like you said, the network needs to be checked and appropriate connection settings should be used as well.

Looking forward to understanding the 'wizardry' that @jkmusyoka used in regards to exporting data that he was not given :) .

[rdstern]

7. I assume the definitions information is further objects when the appropriate dialogs are used? Could they have a checkbox so they are only produced when doing e-picsa? And could they be managed like the other objects? And could there be a general object attached to the summary sheet so we don't need the daily data sheet in the export dialog? It is a very sensitive issue!

[rdstern]

8. The export dialog is much better now. But the filling of the fields in the sub-dialog makes the controls go like a mad thing. Could the code be examined?
9. Could there be an e-picsa preview dialog once the data are in the bucket?
10. And maybe e-picsa could help with data rescue;!! This is potentially big and will be explained separately. Perhaps there could be an additional tab on the data.

[lilyclements]

@rdstern thanks for this.

4. We do not need to read in the "data" anymore, this can be removed from the dialog. Apologies if I didn't make that clear enough when we last updated the dialog/check it when we last reviewed.
   How about just putting a placeholder data frame in there for now? Or I can quickly remove it from the dialog VB code if we plan to do another R-Instat update.