eval_select()-inspired target dependency

[jdtrat]

Hiya! Question/idea that is semi-related to #82.

I'm wondering if there's a way to use something like eval_select() to make a target depend on a subset of previous targets. In my head, this would be something like tar_target(name, command, cue = tar_cue_depends(on = starts_with("file_")). I found a way to do this using tar_combine() (see below) but am wondering if there's an easier way to achieve this? If not, do you think it's something possible/worth doing? I'd be happy to work on a PR!

---

Consider the following pipeline where I map over the year, subset data, and write it to a file. I then have a function 'upload' that I want to depend on all file_* targets.

library(targets)

tar_dir({
  tar_script({
    library(tarchetypes)
    library(palmerpenguins)
    library(tidyselect)
    library(dplyr)
    
    map_vals <- data.frame(year = c("2007", "2008"))
    
    #' Upload function
    #' @description Example function called for side effects (like uploading
    #'   documents)
    #' @param dependencies A list of upstream targets that will trigger an
    #'   upload
    #' @param upload_arg Additional arg for this repres, here printed out as a
    #'   side effect instead of being used to upload something
    upload <- function(dependencies, upload_arg) {
      print(upload_arg)
      return(invisible(dependencies))
    }
    
    mapped <- tar_map(
      tar_target(
        first,
        penguins %>%
          dplyr::filter(year == year) %>%
          dplyr::slice_head(n = 6)
      ),
      tar_target(
        last,
        penguins %>%
          dplyr::filter(year == year) %>%
          dplyr::slice_tail(n = 6)
      ),
      values = map_vals,
      unlist = TRUE # Flattened list of targets with informative names.
    )
    
    combined <- purrr::map(
      map_vals$year, 
      function(year) {
        tar_map(
          unlist = TRUE,
          tar_combine(
            df_yearly,
            mapped[eval_select(ends_with(year), mapped)],
            command = dplyr::bind_rows(!!!.x, .id = "name")
          ),
          tar_target(
            file_yearly,
            write.csv(df_yearly, 
                      file = paste0("df_", year, ".csv")
                      ),
            format = "file"
          ),
          values = data.frame(suffix = year),
          names = "suffix"
          )
        }
      ) %>%
      purrr::flatten()
    
    endgame <- tar_combine(
      uploaded,
      combined[eval_select(starts_with("file"), combined)],
      command = upload(
        dependencies = list(!!!.x),
        upload_arg = "Uploading!"
      )
    )
    
    list(mapped, combined, endgame)
    
  })
  tar_visnetwork(targets_only = TRUE)
})

^{Created on 2022-02-14 by the reprex package (v2.0.1)}

[wlandau]

Great use of eval_select() in tar_combine(), the use case in static branching is compelling. In the general case, you can manually set the dependencies of a target with the deps argument of tar_target_raw(). However, I feel the vast majority of use cases do not require this because the dependencies you need are implicitly declared when you write the R command for the target.

[jdtrat]

Thanks! That's a trick I got from you in #82 😄 This type of workflow comes up a lot for how I use static branching . As shown below (full reprex since I realized the above didn't quite work 😅), replacing the 'endgame' target made with tar_combine with tar_target_raw works as desired.

It makes use of the same eval_select() trick, but I'm wondering if there's a way to access the target names at runtime? Something like that I could pass into the deps argument of tar_target_raw(). Interactively, I would do tar_manifest(starts_with("file"))$name, but that won't work in the _targets.R file because of the infinite recursion risk (thanks for that error message!)

library(targets)

tar_dir({
  tar_script({
    library(tarchetypes)
    library(palmerpenguins)
    library(tidyselect)
    library(dplyr)

    map_vals <- data.frame(year = c("2007", "2008"))

    #' Upload function
    #' @description Example function called for side effects (like uploading
    #'   documents)
    #' @param dependencies A list of upstream targets that will trigger an
    #'   upload
    #' @param upload_arg Additional arg for this repres, here printed out as a
    #'   side effect instead of being used to upload something
    upload <- function(dependencies, upload_arg) {
      print(upload_arg)
      return(invisible(dependencies))
    }

    mapped <- tar_map(
      tar_target(
        first,
        penguins %>%
          dplyr::filter(year == year) %>%
          dplyr::slice_head(n = 6)
      ),
      tar_target(
        last,
        penguins %>%
          dplyr::filter(year == year) %>%
          dplyr::slice_tail(n = 6)
      ),
      values = map_vals,
      unlist = TRUE # Flattened list of targets with informative names.
    )

    combined <- purrr::map(
      map_vals$year,
      function(year) {
        tar_map(
          unlist = TRUE,
          tar_combine(
            df_yearly,
            mapped[eval_select(ends_with(year), mapped)],
            command = dplyr::bind_rows(!!!.x, .id = "name")
          ),
          tar_target(
            file_yearly,
            {
              write.csv(df_yearly,
                file = paste0("df_", year, ".csv")
              )
              paste0("df_", year, ".csv")
            },
            format = "file"
          ),
          values = data.frame(year = year),
          names = "year"
        )
      }
    ) %>%
      purrr::flatten()

    endgame <- tar_target_raw(
      "uploaded",
      expression(
        upload(
          dependencies = NULL,
          upload_arg = "Uploading with `tar_target_raw()`!"
        )
      ),
      deps = names(combined[eval_select(starts_with("file"), combined)])
    )

    list(mapped, combined, endgame)
  })
  tar_visnetwork(targets_only = TRUE)
})

^{Created on 2022-02-15 by the reprex package (v2.0.1)}

[wlandau]

You can access the name of a target object with target_object$settings$name, and functions like tar_map() include named lists in the targets they return.