Display error traceback when vignettes fail in R CMD build #2390

hadley · 2025-01-22T19:52:07Z

Recording some work so that I can come back to it later.

R/output.R

hadley · 2025-01-22T19:53:03Z

R/output.R

    res[i] = xfun:::handle_error(
      withCallingHandlers(
        if (tangle) process_tangle(group) else process_group(group),
        error = function(e) {
          if (progress && is.function(pb$interrupt)) pb$interrupt()
-          if (xfun::pkg_available('rlang', '1.0.0')) rlang::entrace(e)
+
+          if (xfun::pkg_available('rlang', '1.0.0')) {


For vignettes, this implies that your package needs to depend on rlang. I think that's ok, but should be documented somewhere.

depend on rlang

Adding as Suggest would be enough ?

Yeah, sorry, I meant depend in the loose sense.

hadley · 2025-01-22T19:54:24Z

R/output.R

 # return a string to point out the current location in the doc
 get_loc = function(label = '') {
-  paste0(current_lines(), label, sprintf(' (%s)', knit_concord$get('infile')))
+  paste0(knit_concord$get('infile'), ":", current_lines(), label)


This uses a more standard format for file + line numbers that VS Code/Positron will autolink (when you hold down cmd in the terminal). cli hyperlinks don't work in R CMD build, because even if they did, it would open in the file in the check directory, not the current project.

This should help solve this issue then

Feature request: Use hyperlinks when quitting (errors in qmd / Rmd) #2153

I can see it works in Terminal in Positron, but not in R console unfortunately;

Yeah, although using cli would be even more useful there (although not helpful for vignettes)

hadley · 2025-02-17T15:00:09Z

I think this is ready for review. When I install this version of knitr and build https://github.com/hadley/testvignette, I see this:

E  creating vignettes (1.1s)
   --- re-building ‘fail.Rmd’ using rmarkdown
   
   Quitting from fail.Rmd:16-18 [unnamed-chunk-1]
   ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
   <error/rlang_error>
   Error in `h()`:
   ! This is a problem
   ---
   Backtrace:
       ▆
    1. └─global f()
    2.   └─global g()
    3.     └─global h()
   ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
   
   Error: processing vignette 'fail.Rmd' failed with diagnostics:
   This is a problem
   --- failed re-building ‘fail.Rmd’
   
   SUMMARY: processing the following file failed:
     ‘fail.Rmd’
   
   Error: Vignette re-building failed.
   Execution halted

There's quite a lot of duplication in the output, but I don't see an obvious way to resolve this, because it's generated by R CMD build, not knitr or rmarkdown.

bastistician · 2025-02-17T17:35:42Z

Thanks for looking into this. Inspecting knitr-based vignette errors when mass-checking R packages has been a bit cumbersome in the past, mostly because the call was missing (so I have sometimes used a patched R to get more output). So I was happy to find this PR and hope that diagnostics will be provided for both vignette build and check errors in the future.

Both R CMD build and check use tools::buildVignettes() to build ("weave") each package vignette using the engine it declares. When the engine's weave() function errors, this is reported with

Error: processing vignette '<file>' failed with diagnostics:
<conditionMessage>

For Sweave vignettes, the message includes the call (from evaluating chunk expressions using try()) as well as the chunk number/label (from RweaveTryStop()). Given that this PR is specifically about building package vignettes, I wonder if it would make more sense to amend the weave() functions in knitr's vignette engines rather than process_file()? Could knitr:::vweave() etc. catch errors and collect all useful information in the message?

Alternatively, I have experimented with a minor change of the buildVignettes output from <conditionMessage> to <as.character(e)> (the patch mentioned above), which would include both the call and the message from the error condition, also for "rlang_error"s (but not their backtrace it seems), so something like:

Error: processing vignette 'fail.Rmd' failed with diagnostics:
Error in `h()`:
! This is a problem

Printing the condition object might be another option.

gaborcsardi · 2025-02-19T15:35:41Z

AFAICT you already get a traceback for R CMD build, with line numbers and chunk names:

❯ R CMD build testvignette
* checking for file ‘testvignette/DESCRIPTION’ ... OK
* preparing ‘testvignette’:
* checking DESCRIPTION meta-information ... OK
* installing the package to build vignettes
* creating vignettes ... ERROR
--- re-building ‘fail.Rmd’ using rmarkdown

Quitting from fail.Rmd:16-18 [unnamed-chunk-1]
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
<error/rlang_error>
Error in `h()`:
! This is a problem
---
Backtrace:
    ▆
 1. └─global f()
 2.   └─global g()
 3.     └─global h()
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

Error: processing vignette 'fail.Rmd' failed with diagnostics:
This is a problem
--- failed re-building ‘fail.Rmd’

SUMMARY: processing the following file failed:
  ‘fail.Rmd’

Error: Vignette re-building failed.
Execution halted

hadley · 2025-02-19T18:48:35Z

@bastistician those all sounds like reasonable alternative implementations, but I can't quite tell if you think there's something missing in this PR?

bastistician · 2025-02-19T21:20:23Z

Yes, I was especially hoping for improved error reporting for knitr-based vignettes also during R CMD check (e.g., when an updated dependency breaks a vignette or downstream vignettes are broken in a revdep check). AFAICS this PR adds special instrumentation only for R CMD build so wouldn't improve the check output:

* checking re-building of vignette outputs ... ERROR
Error(s) in re-building vignettes:
  ...
--- re-building ‘fail.Rmd’ using rmarkdown

Quitting from fail.Rmd:16-18 [unnamed-chunk-1]
Error: processing vignette 'fail.Rmd' failed with diagnostics:
This is a problem
--- failed re-building ‘fail.Rmd’

In my comment above I was reflecting whether the error handling should instead go to knitr's vignette engines so that it affects tools::buildVignettes() regardless of being called during build or check.

OTOH, I can also see some value in a more general change of what tools::buildVignettes() outputs in case it catches a "weave" error, not just the conditionMessage(). I think that would need a report on R's Bugzilla.

hadley · 2025-02-19T22:20:45Z

@bastistician oh gotcha, we could easily add a || is_R_CMD_check() to make this PR to work then too.

cderv

Looks good.

could easily add a || is_R_CMD_check() to make this PR to work then too.

Doing this seems to work also

* using log directory 'C:/Users/chris/AppData/Local/Temp/testvignette/testvignette.Rcheck'
* using R version 4.4.2 (2024-10-31 ucrt)
* using platform: x86_64-w64-mingw32
* R was compiled by
    gcc.exe (GCC) 13.3.0
    GNU Fortran (GCC) 13.3.0
* running under: Windows 11 x64 (build 26100)
* using session charset: UTF-8
* checking for file 'testvignette/DESCRIPTION' ... OK
* this is package 'testvignette' version '0.0.0.9000'
* package encoding: UTF-8
* checking package namespace information ... OK
* checking package dependencies ... OK
* checking if this is a source package ... OK
* checking if there is a namespace ... OK
* checking for executable files ... OK
* checking for hidden files and directories ... OK
* checking for portable file names ... OK
* checking whether package 'testvignette' can be installed ... OK
* checking installed package size ... OK
* checking package directory ... OK
* checking DESCRIPTION meta-information ... OK
* checking top-level files ... OK
* checking for left-over files ... OK
* checking index information ... OK
* checking package subdirectories ... OK
* checking whether the package can be loaded ... OK
* checking whether the package can be loaded with stated dependencies ... OK
* checking whether the package can be unloaded cleanly ... OK
* checking whether the namespace can be loaded with stated dependencies ... OK
* checking whether the namespace can be unloaded cleanly ... OK
* checking loading without being on the library search path ... OK
* checking files in 'vignettes' ... WARNING
Files in the 'vignettes' directory but no files in 'inst/doc':
  'fail.Rmd'
* checking examples ... NONE
* checking for unstated dependencies in vignettes ... OK
* checking package vignettes ... WARNING
Directory 'inst/doc' does not exist.
Package vignette without corresponding single PDF/HTML:
  'fail.Rmd'
* checking re-building of vignette outputs ... ERROR
Error(s) in re-building vignettes:
  ...
--- re-building 'fail.Rmd' using rmarkdown

Quitting from fail.Rmd:16-18 [unnamed-chunk-1]
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
<error/rlang_error>
Error in `h()`:
! This is a problem
---
Backtrace:
    ▆
 1. └─global f()
 2.   └─global g()
 3.     └─global h()
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

Error: processing vignette 'fail.Rmd' failed with diagnostics:
This is a problem
--- failed re-building 'fail.Rmd'

SUMMARY: processing the following file failed:
  'fail.Rmd'

Error: Vignette re-building failed.
Execution halted

* checking PDF version of manual ... OK
* DONE
Status: 1 ERROR, 2 WARNINGs

cderv · 2025-02-20T12:21:29Z

R/output.R

    res[i] = xfun:::handle_error(
      withCallingHandlers(
        if (tangle) process_tangle(group) else process_group(group),
        error = function(e) {
          if (progress && is.function(pb$interrupt)) pb$interrupt()
-          if (xfun::pkg_available('rlang', '1.0.0')) rlang::entrace(e)
+
+          if (xfun::pkg_available('rlang', '1.0.0')) {


depend on rlang

Adding as Suggest would be enough ?

cderv · 2025-02-20T12:35:23Z

R/output.R

 # return a string to point out the current location in the doc
 get_loc = function(label = '') {
-  paste0(current_lines(), label, sprintf(' (%s)', knit_concord$get('infile')))
+  paste0(knit_concord$get('infile'), ":", current_lines(), label)


This should help solve this issue then

Feature request: Use hyperlinks when quitting (errors in qmd / Rmd) #2153

I can see it works in Terminal in Positron, but not in R console unfortunately;

yihui

we could easily add a || is_R_CMD_check() to make this PR to work

I'll merge the PR after that is added. Thanks!

hadley · 2025-02-27T20:51:47Z

@yihui done!

hadley · 2025-02-28T19:30:48Z

Thanks @yihui!

WIP: Display error tracebacks when vignettes fail

675932c

hadley commented Jan 22, 2025

View reviewed changes

hadley added 2 commits January 23, 2025 07:56

Run during build; not check. Use simpler rlang technique

36df7f0

Polishing

4ce865f

hadley changed the title ~~WIP: Display error tracebacks when vignettes fail~~ Display error traceback when vignettes fail in R CMD build Feb 17, 2025

hadley marked this pull request as ready for review February 17, 2025 15:00

Add comment

27e107c

cderv approved these changes Feb 20, 2025

View reviewed changes

yihui approved these changes Feb 21, 2025

View reviewed changes

Add is_R_CMD_check()

a7b8733

yihui added 2 commits February 28, 2025 10:42

cosmetic changes, and add news

295a88d

one more cosmetic change [ci skip]

b0f6a55

yihui merged commit c47d34f into yihui:master Feb 28, 2025
1 check passed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Display error traceback when vignettes fail in R CMD build #2390

Display error traceback when vignettes fail in R CMD build #2390

hadley commented Jan 22, 2025 •

edited

Loading

hadley Jan 22, 2025

cderv Feb 20, 2025

hadley Feb 20, 2025

hadley Jan 22, 2025 •

edited

Loading

cderv Feb 20, 2025

hadley Feb 20, 2025

hadley commented Feb 17, 2025

bastistician commented Feb 17, 2025

gaborcsardi commented Feb 19, 2025

hadley commented Feb 19, 2025

bastistician commented Feb 19, 2025

hadley commented Feb 19, 2025

cderv left a comment •

edited

Loading

cderv Feb 20, 2025

cderv Feb 20, 2025

yihui left a comment

hadley commented Feb 27, 2025

hadley commented Feb 28, 2025

Display error traceback when vignettes fail in R CMD build #2390

Display error traceback when vignettes fail in R CMD build #2390

Conversation

hadley commented Jan 22, 2025 • edited Loading

hadley Jan 22, 2025

Choose a reason for hiding this comment

cderv Feb 20, 2025

Choose a reason for hiding this comment

hadley Feb 20, 2025

Choose a reason for hiding this comment

hadley Jan 22, 2025 • edited Loading

Choose a reason for hiding this comment

cderv Feb 20, 2025

Choose a reason for hiding this comment

hadley Feb 20, 2025

Choose a reason for hiding this comment

hadley commented Feb 17, 2025

bastistician commented Feb 17, 2025

gaborcsardi commented Feb 19, 2025

hadley commented Feb 19, 2025

bastistician commented Feb 19, 2025

hadley commented Feb 19, 2025

cderv left a comment • edited Loading

Choose a reason for hiding this comment

cderv Feb 20, 2025

Choose a reason for hiding this comment

cderv Feb 20, 2025

Choose a reason for hiding this comment

yihui left a comment

Choose a reason for hiding this comment

hadley commented Feb 27, 2025

hadley commented Feb 28, 2025

hadley commented Jan 22, 2025 •

edited

Loading

hadley Jan 22, 2025 •

edited

Loading

cderv left a comment •

edited

Loading