dump_release script: additional dependencies

[lgeistlinger]

Hi @jwokaty @lwaldron

I am currently working on refactoring bugsigdbr::importBugSigDB to pull from BugSigDBExports as outlined here.

As a result bugsigdbr's dump_release.R script has gained additional dependencies such as rvest for scraping. These new dependencies are not supposed to become dependencies of bugsigdbr and are causing BugSigDBExports GHA to break.

@jwokaty can you adapt BugSigDBExport's GHA by installing rvest in the container? Just let me know if you have any questions. Many thanks!

I also think that the dump_release.R script should now actually be moved from bugsigdbr to BugSigDBExports.

[jwokaty]

Should I go ahead and move dump_release.R and https://github.com/waldronlab/bugsigdbr/issues/12 to this repository?

[lgeistlinger]

Yes please go ahead. Thanks!

[lgeistlinger]

Looks great, only thing missing is to adapt the GHA itself to run the dump_release.R script from BugSigDBExports instead of bugsigdbr.

BugSigDBExports/.github/workflows/export-bugsigdb.yml

Line 41 in 0cc1b28

Rscript $GITHUB_WORKSPACE/bugsigdbr/inst/scripts/dump_release.R $(date +'%F') $GITHUB_WORKSPACE/BugSigDBExports

[jwokaty]

Thank you for catching that!

[jwokaty]

(These are notes for myself since I need to stop here for the day.) There's some unexpected behavior happening in the runner, which is causing it to timeout during the export and I believe might be described at https://stackoverflow.com/questions/33295686/rvest-error-in-open-connectionx-rb-timeout-was-reached. The first solution worked for one run of the github action, but not subsequent runs. (Why?) I might have to add timeout-minutes to the step. (I've been testing in my local repo: https://github.com/jwokaty/BugSigDBExports/actions.)

[lgeistlinger]

Not sure whether it's at all related but we've lately struggled with timeouts at bugsigdb.org which might also cause timeouts in exporting content: waldronlab/BugSigDB#97

[lgeistlinger]

It looks like this is still failing with

Fatal error: cannot open file '/__w/BugSigDBExports/BugSigDBExports/bugsigdbr/inst/scripts/dump_release.R': No such file or directory

Can we trigger a GHA once fixed so that files get exported on the repo. Would be great as I would need that for testing some more of bugsigdbr pulling the files from BugSigDBExports. Thanks!

[lgeistlinger]

https://github.com/waldronlab/BugSigDBExports/runs/3514111578?check_suite_focus=true

[jwokaty]

What's run there isn't the current the yaml file where the path is corrected. This is the current file: https://github.com/waldronlab/BugSigDBExports/blob/main/.github/workflows/export-bugsigdb.yml. I was confused when I saw the notification earlier. My only explanation here is that the same failed run was rerun (in which case the yaml isn't updated; it is whatever it was at the time it was originally run). I want to wait until tomorrow when it should run to see if it is correct. If it is still not correct at that time, I will address it. (I was able to successfully run it in my repo, but I was still perplexed when I saw the notification!)

[jwokaty]

Do you want me to add code so that you can manually trigger an export?

[lgeistlinger]

Ah I see - rerunning the job takes the repo at the state when the job was carried out first. If there is an export tomorrow, then that’s perfectly fine for my purposes. Many thanks!