[help] Dynamically branch over pairs of file paths

[adrianadepalma]

Help

• I understand and agree to https://books.ropensci.org/targets/help.html.

Description

Here's a simplistic example of what I need to do. I first create two sets of files, by mapping over years.

list(
  targets::tar_target(year, c(2000, 2020)),
  targets::tar_target(
    first_files, 
    glue::glue("firstfile_{year}.tif"),
    pattern = map(year),
    format = "file"
  ),
  targets::tar_target(
    second_files,
    glue::glue("secondfile_{year}.tif"),
    pattern = map(year),
    format = "file"
  )
)

Fine so far. But now I need to send each pair of files (i.e. by year) into another function. e.g.

myfun <- function(first_file, second_file) {
  result <- terra::rast(first_file) * terra::rast(second_file)
  return(result)
}

targets::tar_target(
  final_files,
  myfun(first_files, second_files),
  pattern = map(first_files, second_files)
) # doesn't work

I know that I somehow need to create the branches for those files in advance of creating the final target, but I need one branch per pair of files, not one branch per file, so the examples in the help and tar_files doesn't seem to do the trick. Any help would be much appreciated on how to specify the target for the pair of files. I have a feeling it is something simple that I've missed...

Thank you

[wlandau]

The approach you sketched should work. What error are you seeing? The example below works for me.

# _targets.R file:
library(targets)

writeLines("first_2000", "first_2000.txt")
writeLines("first_2020", "first_2020.txt")
writeLines("second_2000", "second_2000.txt")
writeLines("second_2020", "second_2020.txt")

list(
  tar_target(year, c(2000, 2020)),
  tar_target(
    first_files, 
    glue::glue("first_{year}.txt"),
    pattern = map(year),
    format = "file"
  ),
  tar_target(
    second_files,
    glue::glue("second_{year}.txt"),
    pattern = map(year),
    format = "file"
  ),
  tar_target(
    bind_files,
    paste(readLines(first_files), readLines(second_files)),
    pattern = map(first_files, second_files)
  )
)

# R console:
> tar_make()
▶ start target year
● built target year [0 seconds]
▶ start branch second_files_d64f49b5
● built branch second_files_d64f49b5 [0.001 seconds]
▶ start branch second_files_60e6d5f1
● built branch second_files_60e6d5f1 [0 seconds]
● built pattern second_files
▶ start branch first_files_d64f49b5
● built branch first_files_d64f49b5 [0 seconds]
▶ start branch first_files_60e6d5f1
● built branch first_files_60e6d5f1 [0 seconds]
● built pattern first_files
▶ start branch bind_files_a0febd05
● built branch bind_files_a0febd05 [0.001 seconds]
▶ start branch bind_files_53a15535
● built branch bind_files_53a15535 [0.001 seconds]
● built pattern bind_files
▶ end pipeline [0.104 seconds]
 
> tar_read(bind_files)
     bind_files_a0febd05      bind_files_53a15535 
"first_2000 second_2000" "first_2020 second_2020"

[adrianadepalma]

I was getting errors about not being able to read the file as a SpatRaster (rast cannot open this file as a SpatRaster CUsersusernamefoldername...). I think I was getting things wrong by trying to group by year as well. This works now! Thank you very much!