tar_load() in branched Rmd

[awalsh17]

Here is my situation. I am using static branching to run a pipeline of targets across a set of starting parameters. The end of the pipeline is a Rmd report. Everything worked groovy for a non-branched version of my _targets.R. Now I am having an issue.

Here is an example, say final_data is a target that is a list. In the branched workflow, I actually have multiple of these:
final_data_var1, final_data_var2, final_data_var3

Here is something similar to what I have in my Rmd:

tar_load(final_data)

print(final_data$item)

How do I tell targets that I want the branched version in these calls in the Rmd?

Here is a small example:
_targets.R is

library(targets)
library(tarchetypes)

branch_values <- tibble::tibble(
  numbers = c(1,2,3),
  letters = c("a","b","c")
)

new_targets <- tar_map(
  values = branch_values,
  names = "letters",
  tar_target(final_data, 
             paste(numbers, letters)),
  tar_render(final_report,
             "myreport.Rmd")
)
list(new_targets)

myreport.Rmd contains:

---

title: "Untitled"
date: "10/20/2021"
output: html_document

knitr::opts_chunk$set(echo = TRUE)

tar_read(final_data)

When tar_make() I get:
• start target final_data_a
• built target final_data_a
• start target final_data_b
• built target final_data_b
• start target final_data_c
• built target final_data_c
• start target final_report_a
Quitting from lines 13-14 (myreport.Rmd)
x error target final_report_a
• end pipeline
Error : target final_data not found
Error: callr subprocess failed: target final_data not found

The only workaround (that seems cursed) is this for the Rmd where you have a param that is the branch name (var1, var2, var3)

do.call(tar_read, list(paste0("final_data_",params$name)))

[wlandau]

As a workaround, if you want to load all the data in the report, you can call tar_load(starts_with("final_data")). Functions like tar_load() understand tidyselect syntax. From then on though, you would have to reference each dataset by branch name. Alternatively, to create a target with all the data together in one object, you could have a look at tarchetypes::tar_combine().

[awalsh17]

Thanks! I will look into tar_combine() more.
This will just force me to re-think how I was planning to report out results from each branch. I did not want one report that combines the results from all the branches, but rather a unique report for each.

[wlandau]

I did not want one report that combines the results from all the branches, but rather a unique report for each.

For one report per dataset, you could extend the static branching to tar_render() and use the params argument to supply the dataset as an R Markdown parameter. (Would need the report to support parameterized R Markdown.) That would render the report once per dataset. But because of how tar_render() gets resolved, you will probably need tar_eval() instead of tar_map().

[awalsh17]

Thanks again @wlandau - I enjoy the targets package and I appreciate the work you put in to make it great!

This has been a good exercise for me to better think about how I should organize the whole project. Just in case anyone else find this (or future me), this minimal example works like you suggested:

_targets.R:

library(targets)
library(tarchetypes)

branch_values <- tibble::tibble(
  numbers = c(1,2,3),
  letters = c("a","b","c")
)

map_targets <- tar_map(
  values = branch_values,
  names = "letters",
  tar_target(final_data, 
             paste(numbers, letters)),
  tar_render(final_report,
             params = list(data_input = final_data),
             "myreport.Rmd",
             output_file = paste0(letters, ".html"))
  
)

list(map_targets)

myreport.Rmd:

yaml has this:

title: "Untitled"
date: "10/20/2021"
output: html_document
params:
data_input: "final_data_a"

code chunk has this:

params$data_input

[simonrolph]

Thanks @awalsh17 for providing your code solution as I encountered the same issue of wanting to tar_render within static branching (with a report per branch). Your solution worked great. Since 2021 has anyone got a better solution? @wlandau

[wlandau]

@awalsh17's solution from #666 (reply in thread) is really good, as good as it gets I think.

[dshemetov]

I'm running into an issue with @awalsh17's solution, after slightly modifying their example (to remove the letters name collision) I get this error (make an empty myreport.Rmd first):

library(targets)
library(tarchetypes)

branch_values <- tibble::tibble(
  x = c(1, 2, 3),
  y = c("a", "b", "c")
)

list(
  tar_map(
    values = branch_values,
    names = "y",
    tar_target(
      final_data,
      paste(x, y)
    ),
    tar_render(
      final_report,
      "myreport.Rmd",
      output_file = paste0(y, ".html"),
      params = list(data_input = final_data)
    )
  )
)

Error: object 'y' not found

Which I think is coming from these lines in tar_render(). I tried commenting out those type checks in tar_render() and in tar_render_raw(), but had no luck resolving the issue. Am I right in thinking that for some reason output_file can't see the y variable? If you comment out that argument in tar_render() above, the code works fine.

[wlandau]

There's a lot going on in tar_render(), and tar_map() isn't generally compatible. That's why I mentioned tar_eval() before, which branches over expressions instead of realized tar_target() output. An example is at https://books.ropensci.org/targets/static.html#metaprogramming.

[dshemetov]

Thanks @wlandau, appreciate your help.