Tracking tar_map values object

[kevin-m-kent]

Hello!

One more question for me today (thanks again for your help with the first). I am using tar_map with a values tibble object that I might add rows to from one day to the next. Is it possible to track this tibble object as a target and only run the new branches or rows that have been altered from the tibble object?

I've tried adding it as a file target and then reading it before passing to tar_map but I'm getting a weird error message about not having a _targets folder even though I'm using tar_make. My other thought would be to make each row a separate file and then read them all in each time. I also tried using a target directly in that values argument for tar_map but it doesn't seem to evaluate it as a target.

Thank you,

Kevin

[wlandau]

In tar_map(), values exists prior to running (or even defining) the pipeline, which means it cannot be a target. But in dynamic branching, you can easily map over the rows of a tibble target as long as iteration = "vector" in tar_target() for the tibble (default).

Is it possible to track this tibble object as a target and only run the new branches or rows that have been altered from the tibble object?

When you add rows, new targets should automatically spawn and old ones should remain up to date, which is the desired behavior.

[kevin-m-kent]

ok thank you, I'll give that process a try. I appreciate your help/thoughts about this

[kevin-m-kent]

Is there a way to anticipate what the dynamically generated branches will be named? For instance, using the name of the target that is being iterated over? I remember seeing in the documentation for tar_target that this is done deterministically for an actual pipeline but I didn't catch how that actually happens. This solution solves the issue with updating for new rows but makes the part of picking out particular targets generated dynamically harder. Basically, I'd like to always take the first n and the last n targets generated from a particular target step (that is using dynamic branching, last and first defined relative to the tibble that I'm branching over). I'm thinking I might need to split the code a bit separate the two types of targets I'm producing.

I also tried to use head and tail on a dynamically generated branch for that purpose, but judging by how long it was taking and not making progress I'm assuming that it doesn't really work in that way.

[wlandau]

Is there a way to anticipate what the dynamically generated branches will be named? For instance, using the name of the target that is being iterated over? I remember seeing in the documentation for tar_target that this is done deterministically for an actual pipeline but I didn't catch how that actually happens.

tar_branches() produces accurate branch names some of the time, but they are not useful here. Dependency detection can register entire patterns as dependencies but not individual dynamic branches.

Basically, I'd like to always take the first n and the last n targets generated from a particular target step (that is using dynamic branching, last and first defined relative to the tibble that I'm branching over). I'm thinking I might need to split the code a bit separate the two types of targets I'm producing.

head() and tail() in pattern can do this. Sketch:

# _targets.R
list(
  tar_target(l, letters),
  tar_target(all_branches, pattern = map(l)),
  tar_target(selected_branches, pattern = head(all_branches)),
  tar_target(aggregated_selected_branches, selected_branches) # aggregation happens automatically
)

I also tried to use head and tail on a dynamically generated branch for that purpose, but judging by how long it was taking and not making progress I'm assuming that it doesn't really work in that way.

Right, it is not possible to control or preselect individual branch names, or pick individual branch names to depend on. The hashes suffixes at the end of branch names are not useful to the user. I describe some of the reasons for this at ropensci/software-review#401 (comment)

[kevin-m-kent]

ah this is golden. I was doing the head and tail operations in the second argument, not the pattern. But this works! I had to write the name of the target that I'm taking the head or tail of in the second argument position or else it threw me an error about a missing expression. But also maybe you left it out as a sketch. Either way, got it working! Thanks a million.