Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

renv::restore() does not restore lockfile package versions #1953

Open
klmr opened this issue Jul 30, 2024 · 10 comments
Open

renv::restore() does not restore lockfile package versions #1953

klmr opened this issue Jul 30, 2024 · 10 comments

Comments

@klmr
Copy link
Contributor

klmr commented Jul 30, 2024

Inside an renv project I installed a new version of a package (and numerous updated dependency versions) via renv::install(the_pkg). Unfortunately this caused some issues, so now I want to revert to a known-good version.

So I ran renv::restore(clean = TRUE). The function runs for a while before finishing without creating any output. Unfortunately apparently nothing was done: renv::status() shows me that the lockfile versions of the packages were not restored (see output below). Running renv::restore() repeatedly (with or without clean = TRUE) and restarting the R session does not change this situation.

Am I misunderstanding how renv::restore() is supposed to behave in this situation?

R console output
> renv::restore(clean = TRUE)
> renv::status()
The following package(s) are in an inconsistent state:

 package installed recorded used
 httr    y         y        n   

The following package(s) are out of sync [lockfile != library]:

# RSPM ---------------------------------------------------------------------
--
- askpass       [repo: CRAN != RSPM; ver: 1.1 != 1.2.0]
- brio          [repo: CRAN != RSPM; ver: 1.1.3 != 1.1.5]
- bslib         [repo: CRAN != RSPM; ver: 0.5.1 != 0.7.0]
- cachem        [repo: CRAN != RSPM; ver: 1.0.8 != 1.1.0]
- callr         [repo: CRAN != RSPM; ver: 3.7.3 != 3.7.6]
- cli           [repo: CRAN != RSPM; ver: 3.6.1 != 3.6.3]
- cpp11         [repo: CRAN != RSPM; ver: 0.4.6 != 0.4.7]
- curl          [repo: CRAN != RSPM; ver: 5.0.2 != 5.2.1]
- desc          [repo: CRAN != RSPM; ver: 1.4.2 != 1.4.3]
- digest        [repo: CRAN != RSPM; ver: 0.6.33 != 0.6.36]
- downlit       [repo: CRAN != RSPM; ver: 0.4.3 != 0.4.4]
- evaluate      [repo: CRAN != RSPM; ver: 0.22 != 0.24.0]
- fansi         [repo: CRAN != RSPM; ver: 1.0.4 != 1.0.6]
- fastmap       [repo: CRAN != RSPM; ver: 1.1.1 != 1.2.0]
- fs            [repo: CRAN != RSPM; ver: 1.6.3 != 1.6.4]
- glue          [repo: CRAN != RSPM; ver: 1.6.2 != 1.7.0]
- highr         [repo: CRAN != RSPM; ver: 0.10 != 0.11]
- htmltools     [repo: CRAN != RSPM; ver: 0.5.6 != 0.5.8.1]
- httr2         [repo: CRAN != RSPM; ver: 0.2.3 != 1.0.2]
- jsonlite      [repo: CRAN != RSPM; ver: 1.8.7 != 1.8.8]
- knitr         [repo: CRAN != RSPM; ver: 1.44 != 1.48]
- lifecycle     [repo: CRAN != RSPM; ver: 1.0.3 != 1.0.4]
- openssl       [repo: CRAN != RSPM; ver: 2.0.6 != 2.2.0]
- pkgdown       [repo: CRAN != RSPM; ver: 2.0.7 != 2.1.0]
- processx      [repo: CRAN != RSPM; ver: 3.8.2 != 3.8.4]
- ps            [repo: CRAN != RSPM; ver: 1.7.5 != 1.7.7]
- ragg          [repo: CRAN != RSPM; ver: 1.2.5 != 1.3.2]
- rmarkdown     [repo: CRAN != RSPM; ver: 2.25 != 2.27]
- sass          [repo: CRAN != RSPM; ver: 0.4.7 != 0.4.9]
- systemfonts   [repo: CRAN != RSPM; ver: 1.0.5 != 1.1.0]
- textshaping   [repo: CRAN != RSPM; ver: 0.3.7 != 0.4.0]
- tinytex       [repo: CRAN != RSPM; ver: 0.47 != 0.52]
- utf8          [repo: CRAN != RSPM; ver: 1.2.3 != 1.2.4]
- withr         [repo: CRAN != RSPM; ver: 2.5.0 != 3.0.0]
- xfun          [repo: CRAN != RSPM; ver: 0.40 != 0.46]
- yaml          [repo: CRAN != RSPM; ver: 2.3.7 != 2.3.10]

See ?renv::status() for advice on resolving these issues.
sessionInfo() output
R version 4.3.1 (2023-06-16)
Platform: aarch64-apple-darwin20 (64-bit)
Running under: macOS Sonoma 14.3.1

Matrix products: default
BLAS:   /Library/Frameworks/R.framework/Versions/4.3-arm64/Resources/lib/libRblas.0.dylib
LAPACK: /Library/Frameworks/R.framework/Versions/4.3-arm64/Resources/lib/libRlapack.dylib;  LAPACK version 3.11.0

locale:
[1] en_GB.UTF-8/UTF-8/en_GB.UTF-8/C/en_GB.UTF-8/en_GB.UTF-8

time zone: Europe/Zurich
tzcode source: internal

attached base packages:
[1] stats     graphics  grDevices datasets  utils     methods   base

loaded via a namespace (and not attached):
[1] compiler_4.3.1 tools_4.3.1    yaml_2.3.10    renv_1.0.7     pak_0.7.2
@kevinushey
Copy link
Collaborator

This all seems quite unexpected -- renv::restore() should definitely at least be printing a summary of the changes it wants to make, and give you an opportunity to respond.

If you run with the R profiler enabled, e.g. using:

Rprof("renv.Rprof")
renv::restore(clean = TRUE)
Rprof(NULL)
summaryRprof("renv.Rprof")

does the output give you any hints as to what renv is busy doing? (Alternatively, are you able to debug(renv::restore) and step through to see how far you get before the function exits?)

@klmr
Copy link
Contributor Author

klmr commented Jul 31, 2024

Hm. I now get some output from renv::restore(clean = TRUE), even though nothing in this repository was changed:

✔ Loading metadata database ... done

ℹ No downloads are needed
✔ 1 pkg:  [2.7s]

However, the behaviour is otherwise the same.

Here’s the Rprof output:

summaryRprof("renv.rprof")
$by.self
                            self.time self.pct total.time total.pct
"unlink"                         0.52    33.33       0.52     33.33
"lazyLoadDBfetch"                0.14     8.97       0.24     15.38
"normalizePath"                  0.08     5.13       0.08      5.13
"dir.create"                     0.06     3.85       0.06      3.85
"file.exists"                    0.06     3.85       0.06      3.85
"see_if"                         0.06     3.85       0.06      3.85
"rs_run_with_output"             0.04     2.56       0.24     15.38
"rs__get_result_and_output"      0.04     2.56       0.08      5.13
"base::unique"                   0.04     2.56       0.04      2.56
"dynGet"                         0.04     2.56       0.04      2.56
"renv_json_read_remap"           0.04     2.56       0.04      2.56
"rs__write_for_sure"             0.04     2.56       0.04      2.56
"system2"                        0.04     2.56       0.04      2.56
"tryCatch"                       0.02     1.28       1.02     65.38
"<Anonymous>"                    0.02     1.28       0.72     46.15
"rs_call"                        0.02     1.28       0.08      5.13
"find.package"                   0.02     1.28       0.06      3.85
"file.info"                      0.02     1.28       0.04      2.56
".POSIXct"                       0.02     1.28       0.02      1.28
"any"                            0.02     1.28       0.02      1.28
"get_result"                     0.02     1.28       0.02      1.28
"make_vanilla_script_expr"       0.02     1.28       0.02      1.28
"mode"                           0.02     1.28       0.02      1.28
"processx::poll"                 0.02     1.28       0.02      1.28
"read_all"                       0.02     1.28       0.02      1.28
"renv_filebacked_get"            0.02     1.28       0.02      1.28
"rs__handle_condition"           0.02     1.28       0.02      1.28
"serialize"                      0.02     1.28       0.02      1.28
"socketConnection"               0.02     1.28       0.02      1.28
"strsplit"                       0.02     1.28       0.02      1.28
"substring"                      0.02     1.28       0.02      1.28

$by.total
                                            total.time total.pct self.time
"renv::restore"                                   1.56    100.00      0.00
"renv_pak_restore"                                1.48     94.87      0.00
"tryCatch"                                        1.02     65.38      0.02
"doTryCatch"                                      1.00     64.10      0.00
"tryCatchList"                                    1.00     64.10      0.00
"tryCatchOne"                                     1.00     64.10      0.00
"FUN"                                             0.96     61.54      0.00
"map_lgl"                                         0.92     58.97      0.00
"vapply"                                          0.92     58.97      0.00
"renv_cache_synchronize_impl"                     0.86     55.13      0.00
"<Anonymous>"                                     0.72     46.15      0.02
"withCallingHandlers"                             0.60     38.46      0.00
"eval"                                            0.58     37.18      0.00
"pak$pkg_install"                                 0.56     35.90      0.00
"remote"                                          0.56     35.90      0.00
"unlink"                                          0.52     33.33      0.52
"restore"                                         0.52     33.33      0.00
"suppressWarnings"                                0.34     21.79      0.00
"new_remote_session"                              0.30     19.23      0.00
"restart_remote_if_needed"                        0.30     19.23      0.00
"lazyLoadDBfetch"                                 0.24     15.38      0.14
"rs_run_with_output"                              0.24     15.38      0.04
"load_private_package"                            0.24     15.38      0.00
"load_private_packages"                           0.24     15.38      0.00
"set_function_envs"                               0.24     15.38      0.00
"rs$run_with_output"                              0.16     10.26      0.00
"load_private_cli"                                0.12      7.69      0.00
"..getNamespace"                                  0.10      6.41      0.00
"force"                                           0.10      6.41      0.00
"loadNamespace"                                   0.10      6.41      0.00
"renv_cache_copy"                                 0.10      6.41      0.00
"renv_file_same"                                  0.10      6.41      0.00
"normalizePath"                                   0.08      5.13      0.08
"rs__get_result_and_output"                       0.08      5.13      0.04
"rs_call"                                         0.08      5.13      0.02
"%||%"                                            0.08      5.13      0.00
"catchall"                                        0.08      5.13      0.00
"private$get_result_and_output"                   0.08      5.13      0.00
"private$parse_msg"                               0.08      5.13      0.00
"renv_file_copy_dir"                              0.08      5.13      0.00
"renv_file_copy"                                  0.08      5.13      0.00
"renv_file_move"                                  0.08      5.13      0.00
"renv_lock_acquire"                               0.08      5.13      0.00
"renv_scope_lock"                                 0.08      5.13      0.00
"rs__parse_msg_funcs[[code]]"                     0.08      5.13      0.00
"rs__parse_msg"                                   0.08      5.13      0.00
"rs_read"                                         0.08      5.13      0.00
"rs_run"                                          0.08      5.13      0.00
"rs$run"                                          0.08      5.13      0.00
"self$call"                                       0.08      5.13      0.00
"self$read"                                       0.08      5.13      0.00
"structure"                                       0.08      5.13      0.00
"dir.create"                                      0.06      3.85      0.06
"file.exists"                                     0.06      3.85      0.06
"see_if"                                          0.06      3.85      0.06
"find.package"                                    0.06      3.85      0.02
"assert_that"                                     0.06      3.85      0.00
"catch"                                           0.06      3.85      0.00
"conn_create_pipepair"                            0.06      3.85      0.00
"do.call"                                         0.06      3.85      0.00
"initialize"                                      0.06      3.85      0.00
"lapply"                                          0.06      3.85      0.00
"method"                                          0.06      3.85      0.00
"process_initialize"                              0.06      3.85      0.00
"renv_lock_acquire_impl"                          0.06      3.85      0.00
"renv_path_normalize"                             0.06      3.85      0.00
"rs_init"                                         0.06      3.85      0.00
"with_envvar"                                     0.06      3.85      0.00
"base::unique"                                    0.04      2.56      0.04
"dynGet"                                          0.04      2.56      0.04
"renv_json_read_remap"                            0.04      2.56      0.04
"rs__write_for_sure"                              0.04      2.56      0.04
"system2"                                         0.04      2.56      0.04
"file.info"                                       0.04      2.56      0.02
"available_packages"                              0.04      2.56      0.00
"dynamic"                                         0.04      2.56      0.00
"index"                                           0.04      2.56      0.00
"map"                                             0.04      2.56      0.00
"paste"                                           0.04      2.56      0.00
"private$write_for_sure"                          0.04      2.56      0.00
"renv_available_packages_impl"                    0.04      2.56      0.00
"renv_available_packages_latest_repos_impl"       0.04      2.56      0.00
"renv_available_packages_latest"                  0.04      2.56      0.00
"renv_available_packages_query"                   0.04      2.56      0.00
"renv_bootstrap_platform_prefix"                  0.04      2.56      0.00
"renv_cache_find"                                 0.04      2.56      0.00
"renv_file_backup"                                0.04      2.56      0.00
"renv_file_copy_dir_cp"                           0.04      2.56      0.00
"renv_file_copy_dir_impl"                         0.04      2.56      0.00
"renv_file_info"                                  0.04      2.56      0.00
"renv_index_set"                                  0.04      2.56      0.00
"renv_json_read_default"                          0.04      2.56      0.00
"renv_json_read"                                  0.04      2.56      0.00
"renv_lockfile_read"                              0.04      2.56      0.00
"renv_pak_init"                                   0.04      2.56      0.00
"renv_pak_stream"                                 0.04      2.56      0.00
"renv_paths_cache"                                0.04      2.56      0.00
"renv_platform_prefix"                            0.04      2.56      0.00
"renv_system_exec"                                0.04      2.56      0.00
"renv_watchdog_notify_impl"                       0.04      2.56      0.00
"renv_watchdog_notify"                            0.04      2.56      0.00
"unique"                                          0.04      2.56      0.00
"with.default"                                    0.04      2.56      0.00
"with"                                            0.04      2.56      0.00
".POSIXct"                                        0.02      1.28      0.02
"any"                                             0.02      1.28      0.02
"get_result"                                      0.02      1.28      0.02
"make_vanilla_script_expr"                        0.02      1.28      0.02
"mode"                                            0.02      1.28      0.02
"processx::poll"                                  0.02      1.28      0.02
"read_all"                                        0.02      1.28      0.02
"renv_filebacked_get"                             0.02      1.28      0.02
"rs__handle_condition"                            0.02      1.28      0.02
"serialize"                                       0.02      1.28      0.02
"socketConnection"                                0.02      1.28      0.02
"strsplit"                                        0.02      1.28      0.02
"substring"                                       0.02      1.28      0.02
".make_numeric_version"                           0.02      1.28      0.00
"[.numeric_version"                               0.02      1.28      0.00
"["                                               0.02      1.28      0.00
"base::numeric_version"                           0.02      1.28      0.00
"download"                                        0.02      1.28      0.00
"downloader"                                      0.02      1.28      0.00
"empty"                                           0.02      1.28      0.00
"ensure_directory"                                0.02      1.28      0.00
"ensure_parent_directory"                         0.02      1.28      0.00
"filebacked"                                      0.02      1.28      0.00
"format.default"                                  0.02      1.28      0.00
"numeric_version"                                 0.02      1.28      0.00
"renv_activate_version"                           0.02      1.28      0.00
"renv_available_packages_filter_version"          0.02      1.28      0.00
"renv_available_packages_filter"                  0.02      1.28      0.00
"renv_available_packages_success"                 0.02      1.28      0.00
"renv_defer_get"                                  0.02      1.28      0.00
"renv_defer_id"                                   0.02      1.28      0.00
"renv_description_read"                           0.02      1.28      0.00
"renv_download_auth_type"                         0.02      1.28      0.00
"renv_download_auth"                              0.02      1.28      0.00
"renv_download_impl"                              0.02      1.28      0.00
"renv_infrastructure_write_activate"              0.02      1.28      0.00
"renv_lock_release_impl"                          0.02      1.28      0.00
"renv_lock_release"                               0.02      1.28      0.00
"renv_lockfile_load"                              0.02      1.28      0.00
"renv_restore_successful"                         0.02      1.28      0.00
"renv_scope_tempfile"                             0.02      1.28      0.00
"renv_socket_connect"                             0.02      1.28      0.00
"startswith"                                      0.02      1.28      0.00
"tolower"                                         0.02      1.28      0.00
                                            self.pct
"renv::restore"                                 0.00
"renv_pak_restore"                              0.00
"tryCatch"                                      1.28
"doTryCatch"                                    0.00
"tryCatchList"                                  0.00
"tryCatchOne"                                   0.00
"FUN"                                           0.00
"map_lgl"                                       0.00
"vapply"                                        0.00
"renv_cache_synchronize_impl"                   0.00
"<Anonymous>"                                   1.28
"withCallingHandlers"                           0.00
"eval"                                          0.00
"pak$pkg_install"                               0.00
"remote"                                        0.00
"unlink"                                       33.33
"restore"                                       0.00
"suppressWarnings"                              0.00
"new_remote_session"                            0.00
"restart_remote_if_needed"                      0.00
"lazyLoadDBfetch"                               8.97
"rs_run_with_output"                            2.56
"load_private_package"                          0.00
"load_private_packages"                         0.00
"set_function_envs"                             0.00
"rs$run_with_output"                            0.00
"load_private_cli"                              0.00
"..getNamespace"                                0.00
"force"                                         0.00
"loadNamespace"                                 0.00
"renv_cache_copy"                               0.00
"renv_file_same"                                0.00
"normalizePath"                                 5.13
"rs__get_result_and_output"                     2.56
"rs_call"                                       1.28
"%||%"                                          0.00
"catchall"                                      0.00
"private$get_result_and_output"                 0.00
"private$parse_msg"                             0.00
"renv_file_copy_dir"                            0.00
"renv_file_copy"                                0.00
"renv_file_move"                                0.00
"renv_lock_acquire"                             0.00
"renv_scope_lock"                               0.00
"rs__parse_msg_funcs[[code]]"                   0.00
"rs__parse_msg"                                 0.00
"rs_read"                                       0.00
"rs_run"                                        0.00
"rs$run"                                        0.00
"self$call"                                     0.00
"self$read"                                     0.00
"structure"                                     0.00
"dir.create"                                    3.85
"file.exists"                                   3.85
"see_if"                                        3.85
"find.package"                                  1.28
"assert_that"                                   0.00
"catch"                                         0.00
"conn_create_pipepair"                          0.00
"do.call"                                       0.00
"initialize"                                    0.00
"lapply"                                        0.00
"method"                                        0.00
"process_initialize"                            0.00
"renv_lock_acquire_impl"                        0.00
"renv_path_normalize"                           0.00
"rs_init"                                       0.00
"with_envvar"                                   0.00
"base::unique"                                  2.56
"dynGet"                                        2.56
"renv_json_read_remap"                          2.56
"rs__write_for_sure"                            2.56
"system2"                                       2.56
"file.info"                                     1.28
"available_packages"                            0.00
"dynamic"                                       0.00
"index"                                         0.00
"map"                                           0.00
"paste"                                         0.00
"private$write_for_sure"                        0.00
"renv_available_packages_impl"                  0.00
"renv_available_packages_latest_repos_impl"     0.00
"renv_available_packages_latest"                0.00
"renv_available_packages_query"                 0.00
"renv_bootstrap_platform_prefix"                0.00
"renv_cache_find"                               0.00
"renv_file_backup"                              0.00
"renv_file_copy_dir_cp"                         0.00
"renv_file_copy_dir_impl"                       0.00
"renv_file_info"                                0.00
"renv_index_set"                                0.00
"renv_json_read_default"                        0.00
"renv_json_read"                                0.00
"renv_lockfile_read"                            0.00
"renv_pak_init"                                 0.00
"renv_pak_stream"                               0.00
"renv_paths_cache"                              0.00
"renv_platform_prefix"                          0.00
"renv_system_exec"                              0.00
"renv_watchdog_notify_impl"                     0.00
"renv_watchdog_notify"                          0.00
"unique"                                        0.00
"with.default"                                  0.00
"with"                                          0.00
".POSIXct"                                      1.28
"any"                                           1.28
"get_result"                                    1.28
"make_vanilla_script_expr"                      1.28
"mode"                                          1.28
"processx::poll"                                1.28
"read_all"                                      1.28
"renv_filebacked_get"                           1.28
"rs__handle_condition"                          1.28
"serialize"                                     1.28
"socketConnection"                              1.28
"strsplit"                                      1.28
"substring"                                     1.28
".make_numeric_version"                         0.00
"[.numeric_version"                             0.00
"["                                             0.00
"base::numeric_version"                         0.00
"download"                                      0.00
"downloader"                                    0.00
"empty"                                         0.00
"ensure_directory"                              0.00
"ensure_parent_directory"                       0.00
"filebacked"                                    0.00
"format.default"                                0.00
"numeric_version"                               0.00
"renv_activate_version"                         0.00
"renv_available_packages_filter_version"        0.00
"renv_available_packages_filter"                0.00
"renv_available_packages_success"               0.00
"renv_defer_get"                                0.00
"renv_defer_id"                                 0.00
"renv_description_read"                         0.00
"renv_download_auth_type"                       0.00
"renv_download_auth"                            0.00
"renv_download_impl"                            0.00
"renv_infrastructure_write_activate"            0.00
"renv_lock_release_impl"                        0.00
"renv_lock_release"                             0.00
"renv_lockfile_load"                            0.00
"renv_restore_successful"                       0.00
"renv_scope_tempfile"                           0.00
"renv_socket_connect"                           0.00
"startswith"                                    0.00
"tolower"                                       0.00

$sample.interval
[1] 0.02

$sampling.time
[1] 1.56

@kevinushey
Copy link
Collaborator

Based on that output, I suspect the issue here lies with how pak is being invoked by renv... do you have better luck with options(renv.config.pak.enabled = FALSE)?

It's also possible this is the same as https://github.com/rstudio/renv/blob/main/NEWS.md?plain=1#L47-L49 -- if that's the case, then this is a bug with how renv is invoking pak, and should be resolved in the development version of renv.

@klmr
Copy link
Contributor Author

klmr commented Aug 2, 2024

Okay, disabling ‘pak’ did change the behaviour, it asks to restore a bunch of packages, and does so after prompting. However, afterwards the library is still not synchronised, as shown by renv::status():

> renv::restore(clean = TRUE)
The following package(s) will be updated:

# CRAN -----------------------------------------------------------------------
- askpass       [repo: RSPM -> CRAN; ver: 1.2.0 -> 1.1]
- brio          [repo: RSPM -> CRAN; ver: 1.1.5 -> 1.1.3]
- bslib         [repo: RSPM -> CRAN; ver: 0.7.0 -> 0.5.1]
- cachem        [repo: RSPM -> CRAN; ver: 1.1.0 -> 1.0.8]
- callr         [repo: RSPM -> CRAN; ver: 3.7.6 -> 3.7.3]
- cli           [repo: RSPM -> CRAN; ver: 3.6.3 -> 3.6.1]
- cpp11         [repo: RSPM -> CRAN; ver: 0.4.7 -> 0.4.6]
- curl          [repo: RSPM -> CRAN; ver: 5.2.1 -> 5.0.2]
- desc          [repo: RSPM -> CRAN; ver: 1.4.3 -> 1.4.2]
- digest        [repo: RSPM -> CRAN; ver: 0.6.36 -> 0.6.33]
- downlit       [repo: RSPM -> CRAN; ver: 0.4.4 -> 0.4.3]
- evaluate      [repo: RSPM -> CRAN; ver: 0.24.0 -> 0.22]
- fansi         [repo: RSPM -> CRAN; ver: 1.0.6 -> 1.0.4]
- fastmap       [repo: RSPM -> CRAN; ver: 1.2.0 -> 1.1.1]
- fs            [repo: RSPM -> CRAN; ver: 1.6.4 -> 1.6.3]
- glue          [repo: RSPM -> CRAN; ver: 1.7.0 -> 1.6.2]
- highr         [repo: RSPM -> CRAN; ver: 0.11 -> 0.10]
- htmltools     [repo: RSPM -> CRAN; ver: 0.5.8.1 -> 0.5.6]
- httr2         [repo: RSPM -> CRAN; ver: 1.0.2 -> 0.2.3]
- jsonlite      [repo: RSPM -> CRAN; ver: 1.8.8 -> 1.8.7]
- knitr         [repo: RSPM -> CRAN; ver: 1.48 -> 1.44]
- lifecycle     [repo: RSPM -> CRAN; ver: 1.0.4 -> 1.0.3]
- openssl       [repo: RSPM -> CRAN; ver: 2.2.0 -> 2.0.6]
- pak           [0.7.2 -> *]
- pkgdown       [repo: RSPM -> CRAN; ver: 2.1.0 -> 2.0.7]
- processx      [repo: RSPM -> CRAN; ver: 3.8.4 -> 3.8.2]
- ps            [repo: RSPM -> CRAN; ver: 1.7.7 -> 1.7.5]
- ragg          [repo: RSPM -> CRAN; ver: 1.3.2 -> 1.2.5]
- rmarkdown     [repo: RSPM -> CRAN; ver: 2.27 -> 2.25]
- sass          [repo: RSPM -> CRAN; ver: 0.4.9 -> 0.4.7]
- systemfonts   [repo: RSPM -> CRAN; ver: 1.1.0 -> 1.0.5]
- textshaping   [repo: RSPM -> CRAN; ver: 0.4.0 -> 0.3.7]
- tinytex       [repo: RSPM -> CRAN; ver: 0.52 -> 0.47]
- utf8          [repo: RSPM -> CRAN; ver: 1.2.4 -> 1.2.3]
- withr         [repo: RSPM -> CRAN; ver: 3.0.0 -> 2.5.0]
- xfun          [repo: RSPM -> CRAN; ver: 0.46 -> 0.40]
- yaml          [repo: RSPM -> CRAN; ver: 2.3.10 -> 2.3.7]

Do you want to proceed? [Y/n]: y

- Removing pak [0.7.2] ... OK [removed from library]
Installation of 'curl 5.0.2' was requested, but the following constraints are not met:
- httr2 (requires curl >= 5.1.0)
renv will try to install 'curl 5.2.1' instead.

Installation of 'htmltools 0.5.6' was requested, but the following constraints are not met:
- bslib (requires htmltools >= 0.5.8)
renv will try to install 'htmltools 0.5.8.1' instead.

Installation of 'sass 0.4.7' was requested, but the following constraints are not met:
- bslib (requires sass >= 0.4.9)
renv will try to install 'sass 0.4.9' instead.

# Downloading packages -------------------------------------------------------
- Downloading sass from CRAN ...                OK [2.9 Mb in 0.66s]
Installation of 'highr 0.10' was requested, but the following constraints are not met:
- knitr (requires highr >= 0.11)
renv will try to install 'highr 0.11' instead.

Installation of 'xfun 0.40' was requested, but the following constraints are not met:
- knitr (requires xfun >= 0.44)
renv will try to install 'xfun 0.46' instead.

- Downloading xfun from CRAN ...                OK [155.5 Kb in 0.19s]
Successfully downloaded 2 packages in 1.1 seconds.

# Installing packages --------------------------------------------------------
- Installing cli ...                            OK [copied local binary and cached]
- Installing curl ...                           OK [linked from cache]
- Installing glue ...                           OK [copied local binary and cached]
- Installing lifecycle ...                      OK [copied local binary and cached]
- Installing askpass ...                        OK [copied local binary and cached]
- Installing openssl ...                        OK [copied local binary and cached]
- Installing withr ...                          OK [copied local binary and cached]
- Installing httr2 ...                          OK [copied local binary and cached]
- Installing cpp11 ...                          OK [copied local binary and cached]
- Installing systemfonts ...                    OK [copied local binary and cached]
- Installing ps ...                             OK [copied local binary and cached]
- Installing processx ...                       OK [copied local binary and cached]
- Installing callr ...                          OK [copied local binary and cached]
- Installing fastmap ...                        OK [copied local binary and cached]
- Installing utf8 ...                           OK [copied local binary and cached]
- Installing cachem ...                         OK [copied local binary and cached]
- Installing digest ...                         OK [copied local binary and cached]
- Installing htmltools ...                      OK [linked from cache]
- Installing jsonlite ...                       OK [copied local binary and cached]
- Installing fs ...                             OK [copied local binary and cached]
- Installing sass ...                           OK [built from source and cached in 22s]
- Installing bslib ...                          OK [copied local binary and cached in 0.34s]
- Installing evaluate ...                       OK [copied local binary and cached]
- Installing xfun ...                           OK [built from source and cached in 2.4s]
- Installing highr ...                          OK [linked from cache]
- Installing yaml ...                           OK [copied local binary and cached]
- Installing knitr ...                          OK [copied local binary and cached]
- Installing tinytex ...                        OK [copied local binary and cached]
- Installing rmarkdown ...                      OK [copied local binary and cached in 0.15s]
- Installing textshaping ...                    OK [copied local binary and cached]
- Installing ragg ...                           OK [copied local binary and cached]
- Installing brio ...                           OK [copied local binary and cached in 0.16s]
- Installing desc ...                           OK [copied local binary and cached in 0.44s]
- Installing fansi ...                          OK [copied local binary and cached]
- Installing downlit ...                        OK [copied local binary and cached]
- Installing pkgdown ...                        OK [copied local binary and cached in 0.45s]
The dependency tree was repaired during package installation:
- askpass     [1.2.0]
- brio        [1.1.5]
- bslib       [0.7.0]
- cachem      [1.1.0]
- callr       [3.7.6]
- cli         [3.6.3]
- cpp11       [0.4.7]
- curl        [5.2.1]
- desc        [1.4.3]
- digest      [0.6.36]
- downlit     [0.4.4]
- evaluate    [0.24.0]
- fansi       [1.0.6]
- fastmap     [1.2.0]
- fs          [1.6.4]
- glue        [1.7.0]
- highr       [0.11]
- htmltools   [0.5.8.1]
- httr2       [1.0.2]
- jsonlite    [1.8.8]
- knitr       [1.48]
- lifecycle   [1.0.4]
- openssl     [2.2.0]
- pkgdown     [2.1.0]
- processx    [3.8.4]
- ps          [1.7.7]
- ragg        [1.3.2]
- rmarkdown   [2.27]
- sass        [0.4.9]
- systemfonts [1.1.0]
- textshaping [0.4.0]
- tinytex     [0.52]
- utf8        [1.2.4]
- withr       [3.0.0]
- xfun        [0.46]
- yaml        [2.3.10]
Call `renv::snapshot()` to capture these dependencies in the lockfile.

> renv::status()
The following package(s) are in an inconsistent state:

 package installed recorded used
 httr    y         y        n

The following package(s) are out of sync [lockfile != library]:

# CRAN -----------------------------------------------------------------------
- sass          [0.4.7 != 0.4.9]
- xfun          [0.40 != 0.46]

# RSPM -----------------------------------------------------------------------
- askpass       [repo: CRAN != RSPM; ver: 1.1 != 1.2.0]
- brio          [repo: CRAN != RSPM; ver: 1.1.3 != 1.1.5]
- bslib         [repo: CRAN != RSPM; ver: 0.5.1 != 0.7.0]
- cachem        [repo: CRAN != RSPM; ver: 1.0.8 != 1.1.0]
- callr         [repo: CRAN != RSPM; ver: 3.7.3 != 3.7.6]
- cli           [repo: CRAN != RSPM; ver: 3.6.1 != 3.6.3]
- cpp11         [repo: CRAN != RSPM; ver: 0.4.6 != 0.4.7]
- curl          [repo: CRAN != RSPM; ver: 5.0.2 != 5.2.1]
- desc          [repo: CRAN != RSPM; ver: 1.4.2 != 1.4.3]
- digest        [repo: CRAN != RSPM; ver: 0.6.33 != 0.6.36]
- downlit       [repo: CRAN != RSPM; ver: 0.4.3 != 0.4.4]
- evaluate      [repo: CRAN != RSPM; ver: 0.22 != 0.24.0]
- fansi         [repo: CRAN != RSPM; ver: 1.0.4 != 1.0.6]
- fastmap       [repo: CRAN != RSPM; ver: 1.1.1 != 1.2.0]
- fs            [repo: CRAN != RSPM; ver: 1.6.3 != 1.6.4]
- glue          [repo: CRAN != RSPM; ver: 1.6.2 != 1.7.0]
- highr         [repo: CRAN != RSPM; ver: 0.10 != 0.11]
- htmltools     [repo: CRAN != RSPM; ver: 0.5.6 != 0.5.8.1]
- httr2         [repo: CRAN != RSPM; ver: 0.2.3 != 1.0.2]
- jsonlite      [repo: CRAN != RSPM; ver: 1.8.7 != 1.8.8]
- knitr         [repo: CRAN != RSPM; ver: 1.44 != 1.48]
- lifecycle     [repo: CRAN != RSPM; ver: 1.0.3 != 1.0.4]
- openssl       [repo: CRAN != RSPM; ver: 2.0.6 != 2.2.0]
- pkgdown       [repo: CRAN != RSPM; ver: 2.0.7 != 2.1.0]
- processx      [repo: CRAN != RSPM; ver: 3.8.2 != 3.8.4]
- ps            [repo: CRAN != RSPM; ver: 1.7.5 != 1.7.7]
- ragg          [repo: CRAN != RSPM; ver: 1.2.5 != 1.3.2]
- rmarkdown     [repo: CRAN != RSPM; ver: 2.25 != 2.27]
- systemfonts   [repo: CRAN != RSPM; ver: 1.0.5 != 1.1.0]
- textshaping   [repo: CRAN != RSPM; ver: 0.3.7 != 0.4.0]
- tinytex       [repo: CRAN != RSPM; ver: 0.47 != 0.52]
- utf8          [repo: CRAN != RSPM; ver: 1.2.3 != 1.2.4]
- withr         [repo: CRAN != RSPM; ver: 2.5.0 != 3.0.0]
- yaml          [repo: CRAN != RSPM; ver: 2.3.7 != 2.3.10]

See ?renv::status() for advice on resolving these issues.

I guess the reason for this is explained by the messages during installation (“Installation of 'PKG VERSION' was requested, but the following constraints are not met: […] renv will try to install 'PKG OTHE_VERSION' instead.”) but it’s not clear why this happens, since the previously snapshotted state of the library was internally consistent.

@klmr
Copy link
Contributor Author

klmr commented Aug 2, 2024

Hm. Actually from the output of renv::status() it looks like renv::restore() didn’t actually do… anything?!

I tried the same using the dev version of ‘renv’, with the same result.

@kevinushey
Copy link
Collaborator

This sounds similar to a recent issue where renv's dependency repair seemed to malfunction when the restore would act upon a library path that already contained differing versions of packages...

For diagnostics, does the issue still occur if you restore into a separate library path, e.g. something like:

renv::restore(library = "/tmp/renv-library")

Or alternatively, attempt the restore only after cleaning up the library paths?

(This might be suitable as a workaround, but even so it seems like there's definitely a bug in renv that needs to be fixed.)

@kevinushey
Copy link
Collaborator

Could you also share a couple of files?

  • The lockfile you're using for renv::restore();
  • The current state of your library paths, as via e.g. renv::snapshot(lockfile = "renv-current.lock")

This should let me try and reproduce from a similar state as from what you're working with.

@klmr
Copy link
Contributor Author

klmr commented Aug 2, 2024

Okay, I tried restoring into a separate library path, and I originally got the same behaviour as when disabling ‘pak’ above: it restores everything, but afterwards renv::status() still showed the mismatching versions — identical output as above. But then I remembered that .libPaths() wasn’t adjusted, and once I added the temporary library path to .libPaths(), renv::status() showed me a consistent state.

Maybe that’s a hint at what’s going wrong? I.e. that renv::restore() might somehow be using an incorrect library path? (I did check the library by stepping through renv::restore, and it corresponds to the value of .libPaths()[1], as expected.)

(At this point I should also note that renv::status() is in fact showing the correct status: I have manually verified that the package versions in the library and in the lockfile were as shown; so the error isn’t in renv::status().)

I’ve attached the requested files. Unfortunately the project is relatively complex. I can try to simplify it if absolutely necessary. I’ll also list my ‘renv’ config below. I’ve censored some company specifics. All censored CRAN-like repo URIs lead to valid PPM instances (I tested them just now to be sure).

‘renv’ config
> lapply(renv::config, \(f) f()) |> structure(class = 'simple.list')
                      _
activate.prompt       TRUE
autoloader.enabled    TRUE
auto.snapshot         FALSE
bitbucket.host        api.bitbucket.org/2.0
copy.method           auto
connect.timeout       20
connect.retry         3
cache.enabled         TRUE
cache.symlinks        TRUE
dependency.errors     reported
dependencies.limit    1000
exported.functions    *
external.libraries    /Users/USER/Library/Application Support/R/nvimcom/%p-library/%v
filebacked.cache      TRUE
github.host           api.github.com
gitlab.host           gitlab.com
install.build         FALSE
install.remotes       TRUE
install.shortcuts     TRUE
install.staged        TRUE
install.transactional TRUE
install.verbose       FALSE
locking.enabled       FALSE
mran.enabled          FALSE
pak.enabled           TRUE
ppm.enabled           TRUE
ppm.default           TRUE
ppm.url               https://packagemanager.posit.co/cran/latest
rspm.enabled          TRUE
sandbox.enabled       TRUE
shims.enabled         TRUE
snapshot.inference    TRUE
snapshot.validate     TRUE
synchronized.check    TRUE
updates.check         FALSE
updates.parallel      2
user.environ          TRUE
user.library          FALSE
user.profile          FALSE

And the paths:

> lapply(renv::config, \(f) f()) |> structure(class = 'simple.list')
         _
root     /Users/USER/Library/Caches/R/renv
library  /Users/USER/Library/Application Support/R/renv/bigbrother-1cca52a8/R-4.3/aarch64-apple-darwin20
lockfile /Users/USER/Code/bigbrother/renv.lock
settings renv/settings.json
cache    /Users/USER/Library/Caches/R/renv/cache/v5/R-4.3/aarch64-apple-darwin20
sandbox  /Users/USER/Library/Caches/R/renv/sandbox/R-4.3/aarch64-apple-darwin20/ac5c2659

🗜️ renv-lockfiles.zip

@kevinushey
Copy link
Collaborator

Darn -- even trying to replicate a restore using the same set of packages as in your case, I'm not able to reproduce. 😞

Based on the error, my suspicion is that something is going wrong when computing and adding the requirements for each package during restore here:

renv/R/retrieve.R

Lines 1052 to 1064 in 6d80457

# figure out the dependency fields to use -- if the user explicitly requested
# this package be installed, but also provided a 'dependencies' argument in
# the call to 'install()', then we want to use those
fields <- if (package %in% state$packages) the$install_dependency_fields else "strong"
deps <- renv_dependencies_discover_description(path, subdir = subdir, fields = fields)
if (length(deps$Source))
deps$Source <- record$Package
rowapply(deps, function(dep) {
package <- dep[["Package"]]
requirements[[package]] <- requirements[[package]] %||% stack()
requirements[[package]]$push(dep)
})

Perhaps renv_dependencies_discover_description() is returning the wrong value here for some reason; e.g. using the version of a package currently-loaded or available in the library paths, as opposed to the version that was just downloaded?

A couple other options you could try setting, before attempting restore:

# disable internal caching / memoization of computed values
options(renv.dynamic.enabled = FALSE)

# disable caching of values computed from files on disk
options(renv.config.filebacked.cache = FALSE)

If either of these help, it could help narrow down the issue as well.

@Fuco1
Copy link

Fuco1 commented Sep 22, 2024

I'm using version 1.0.8 and with pak I also get incorrect versions installed. All the versions in the library are newer which leads me to believe pack simply grabs whatever is the latest version at the moment.

I tried it now with pak disable and with the two options you provided and the result is the same.

# CRAN -----------------------------------------------------------------------
- broom            [1.0.5 != 1.0.6]
- colorspace       [2.1-0 != 2.1-1]
- data.table       [1.15.4 != 1.16.0]
- DBI              [1.2.2 != 1.2.3]
- distributional   [0.4.0 != 0.5.0]
- fabletools       [0.4.2 != 0.5.0]
- feasts           [0.3.2 != 0.4.0]
- filelock         [r-lib/filelock != 1.0.3]
- ggnewscale       [0.4.10 != 0.5.0]
- jsonlite         [1.8.8 != 1.8.9]
- odbc             [1.4.2 != 1.5.0]
- openssl          [2.1.2 != 2.2.2]
- rlang            [1.1.3 != 1.1.4]
- tsibble          [1.1.4 != 1.1.5]
- urca             [1.3-3 != 1.3-4]
- yaml             [2.3.8 != 2.3.10]

I guess the theory that it is trying to bump versions to get a resolvable set might be correct. But I don't know which package might be causing the problem and also... that should rather produce an error.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

3 participants