"Global state has changed" warning

[rempsyc]

We rely on brms in the report package, and we have a workflow checking for warnings in tests. The workflow is failing due to a warning caused by brms (see easystats/report#448),

Global state has changed

Essentially, the warning seems to arise from a test simply calling the brms::brms() function as below:

model <- suppressMessages(suppressWarnings(brms::brm(
    mpg ~ qsec + wt,data = mtcars, refresh = 0, iter = 1, seed = 333)))

Generates that "Global state has changed" warning. The detailed error from the workflow is available e.g., here: https://github.com/easystats/report/actions/runs/9797434007/job/27054023826?pr=447#step:5:196

── Warning (test-report.brmsfit.R:3:1): report.brms ────────────────────────────
Global state has changed:
     names(before[[4]])                | names(after[[4]])                      
[94] "PIPX_HOME"                       | "PIPX_HOME"                       [94] 
[95] "PKGCACHE_HTTP_VERSION"           | "PKGCACHE_HTTP_VERSION"           [95] 
[96] "PKGLOAD_PARENT_TEMPDIR"          | "PKGLOAD_PARENT_TEMPDIR"          [96] 
                                       - "PKG_CPPFLAGS"                    [97] 
                                       - "PKG_LIBS"                        [98] 
[97] "POWERSHELL_DISTRIBUTION_CHANNEL" | "POWERSHELL_DISTRIBUTION_CHANNEL" [99] 
[98] "PWD"                             | "PWD"                             [100]
[99] "RENV_CONFIG_REPOS_OVERRIDE"      | "RENV_CONFIG_REPOS_OVERRIDE"      [101]                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                         
      names(before[[4]])        | names(after[[4]])              
[153] "TESTTHAT"                | "TESTTHAT"                [155]
[154] "TZ"                      | "TZ"                      [156]
[155] "USER"                    | "USER"                    [157]
                                - "USE_CXX17"               [158]
[156] "VCPKG_INSTALLATION_ROOT" | "VCPKG_INSTALLATION_ROOT" [159]
[157] "XDG_CONFIG_HOME"         | "XDG_CONFIG_HOME"         [160]
[158] "XDG_RUNTIME_DIR"         | "XDG_RUNTIME_DIR"         [161]                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                   
before[[4]][94:99] vs after[[4]][94:101]
  PIPX_BIN_DIR"/opt/pipx_bin"
  PIPX_HOME"/opt/pipx"
  PKGCACHE_HTTP_VERSION"2"
  PKGLOAD_PARENT_TEMPDIR"/tmp/RtmpQHAqXH"
+ PKG_CPPFLAGS"  -I\"/home/runner/work/_temp/Library/Rcpp/include/\"  -I\"/home/runner/work/_temp/Library/RcppEigen/include/\"  -I\"/home/runner/work/_temp/Library/RcppEigen/include/unsupported\"  -I\"/home/runner/work/_temp/Library/BH/include\" -I\"/home/runner/work/_temp/Library/StanHeaders/include/src/\"  -I\"/home/runner/work/_temp/Library/StanHeaders/include/\"  -I\"/home/runner/work/_temp/Library/RcppParallel/include/\"  -I\"/home/runner/work/_temp/Library/rstan/include\" -DEIGEN_NO_DEBUG  -DBOOST_DISABLE_ASSERTS  -DBOOST_PENDING_INTEGER_LOG2_HPP  -DSTAN_THREADS  -DUSE_STANC3 -DSTRICT_R_HEADERS  -DBOOST_PHOENIX_NO_VARIADIC_EXPRESSION  -D_HAS_AUTO_PTR_ETC=0  -include '/home/runner/work/_temp/Library/StanHeaders/include/stan/math/prim/fun/Eigen.hpp'  -D_REENTRANT -DRCPP_PARALLEL_USE_TBB=1 "
+ PKG_LIBS" '/home/runner/work/_temp/Library/rstan/lib//libStanServices.a' -L'/home/runner/work/_temp/Library/StanHeaders/lib/' -lStanHeaders -L'/home/runner/work/_temp/Library/RcppParallel/lib/' -ltbb "
  POWERSHELL_DISTRIBUTION_CHANNEL"GitHub-Actions-ubuntu22"
  PWD"/home/runner/work/report/report"
  RENV_CONFIG_REPOS_OVERRIDE"https://packagemanager.posit.co/cran/__linux__/jammy/latest"
      before[[4]]              | after[[4]]                    
[153] "true"                   | "true"                   [155]
[154] "UTC"                    | "UTC"                    [156]
[155] "runner"                 | "runner"                 [157]
                               - "1"                      [158]
[156] "/usr/local/share/vcpkg" | "/usr/local/share/vcpkg" [159]
[157] "/home/runner/.config"   | "/home/runner/.config"   [160]
[158] "/run/user/1001"         | "/run/user/1001"         [161]                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                   

Would you have any idea as to how we can fix this on our end, or is this something that needs fixing in brms or in some of its dependencies?

[paul-buerkner]

I think these are things that rstan does. You might want to ask on Stan discourse about it.

[rempsyc]

For reference, in the corresponding Discourse topic, Bob Carpenter commented:

I looked up this error and it seems to arise when a test from testthat finds that global state has changed. I agree that it’s bad form to change global state in a test. Tracking this down is going to require finding where RStan or brms or something they call changes the global state.

In particular, this appears to arise from setting a few new flags (PKG_CPPFLAGS, PKG_LIBS and USE_CXX17). One way to deal with this would probably be to set them ahead of time. The other way to deal with it would be to write something that wraps the tests in something that removes warning messages.

Overall, testing for generic state changes like this is probably going to be brittle going forward if it’s not something that RStan or brms care about.

Another alternative might be to switch to the cmdstanr back end. Otherwise, someone needs to track through the code, figure out where those three flags get set, then wrap them safely so that they are guaranteed to be unset (I don’t know if R has exceptions and can support a try/catch/finally pattern).

I suppose it is unlikely brms will change its backend to cmdstanr... What next steps of actions would you recommend here?

[paul-buerkner]

brms supports the cmdstanr backend already via the backend = "cmdstanr" option. Is this what you mean?

[rempsyc]

Thanks for pointing this out—I've tested with backend = "cmdstanr" and the "Global state has changed" warning still occurs, suggesting the backend isn't the issue. I will report on the other thread about this finding.