Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

change docker setup to install released versions of hubverse packages #30

Open
matthewcornell opened this issue Feb 6, 2025 · 3 comments

Comments

@matthewcornell
Copy link
Member

From @zkamvar 's review in PR [clean up renv usage to fix incompatibility between stored CRAN repositories and renv version. update README.md files. regenerate renv.lock files #29]:


There are a couple of issues that still need to be addressed after this PR:

  1. The installations of the GitHub packages from reichlab negates the effort to install released versions of hubverse packages because they always update to the latest version (as the Remotes: field always points to the main branch). This cannot be solved by using renv::install(). The code in the development are considered stable, but it does not match what the models will be validated against.
  2. The workflow for building and deploying the docker containers is a manual one and can be automated, but this would require tests to confirm that it works.

Both 1 and 2 are important in the event that the containers need to be rebuilt or updated (e.g. a severe bug in any of the software used or a change to the hub requires newer versions of hubverse packages).


@matthewcornell
Copy link
Member Author

from @zkamvar in slack:

install2.r --skipinstalled --error --repo getOption remotes # install remotes for the github installer

# install hubverse packages and all dependencies (includes cran zoltr, but we will update that later)
install2.r --skipinstalled --error \
  --repos https://hubverse-org.r-universe.dev/ \
  --repos getOption \
  hubData \
  hubVis \
  hubEnsembles

# install up-to-date reichlab packages
installGithub.r \
  --update FALSE \
  --repos getOption \
  reichlab/zoltr \
  reichlab/covidData \
  reichlab/idforecastutils

# remove detritus
rm -rf /tmp/downloaded_packages \
    && strip /usr/local/lib/R/site-library/*/libs/*.so

@matthewcornell
Copy link
Member Author

matthewcornell commented Feb 6, 2025

Additional thoughts:

  • The above could be run via a model-specific bash script. This would guarantee that we get the released versions of github-based repos. This is the recommended next step, per @zkamvar .
  • Alternatively, we could add a per-model DESCRIPTION file and use something like https://desc.r-lib.org load it. Additional_repositories: https://hubverse-org.r-universe.org . This would eliminate step 6 in the README.md
  • Finally, the reichlab repos could be moved to their own r-universe repo. This would allow the DESCRIPTION file method to work with Additional_repositories .

@zkamvar
Copy link
Member

zkamvar commented Feb 6, 2025

To put a finer point on the third point:

Finally, the reichlab repos could be moved to their own r-universe repo. This would allow the DESCRIPTION file method to work with Additional_repositories.

This also benefits the script I presented in #30 (comment) as it would reduce the script to this

# install hubverse packages and all dependencies (includes cran zoltr, but we will update that later)
install2.r --skipinstalled --error \
  --repos https://hubverse-org.r-universe.dev/ \
  --repos https://reichlab.r-universe.dev/ \
  --repos getOption \
  hubData \
  hubVis \
  hubEnsembles \
  zoltr \
  covidData \
  idforecastutils

# remove detritus
rm -rf /tmp/downloaded_packages \
    && strip /usr/local/lib/R/site-library/*/libs/*.so

An example DESCRIPTION file would look something like this:

Package: flu_ar2
Title: Flusight AR2 (not a package)
Version: 1.0.0
Authors@R: c(person(,"Infectious Disease Modeling Consortium", c("cph")))
Maintainer: Matt <[email protected]>
Description: This is not a package. It contains dependency requirements for the flu_ar2 model
License: MIT
Imports:
    hubData
    hubVis
    hubEnsembles
    zoltr
    covidData
    idforecastutils
Additional_repositories: https://hubverse-org.r-universe.dev, https://reichlab.r-universe.dev
Encoding: UTF-8

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants