How would you share targets objects with a team of analysts? [general]

[daranzolin]

Help

• I understand and agree to https://books.ropensci.org/targets/help.html.

Description

I have a series of targets pipelines. The intermediary targets objects (data frames, lists of parameters, etc.), are useful for ad-hoc analyses for my team, and I'm wondering how to best make them available.

I could: (1) load the intermediary targets objects into a database that everyone can connect to; or (2) create functions that directly access the _targets store, e.g.:

get_object_x <- function() {
    targets::tar_read(object_x, store = "PATH_TO_STORE_ON_SHARED_DRIVE")
}

Is there a best practice here, or does it really not matter? I'm curious what other people do.

[noamross]

We store targets on AWS S3 bucket (https://books.ropensci.org/targets/cloud-storage.html).

[daranzolin]

I thought about the cloud, and while sleeker I think it's also superfluous when: (1) the targets themselves are not large; and (2) it's free to store them locally.

[wlandau]

You could identify a small number of key outputs, save them as format = "file" targets, and then send them manually. In situations I have encountered, it is usually the case that only a small subset of components ever need to be shared.

https://github.com/databio/unitar is slightly related.

[snowpong]

You can have your own local cloud. Setup your own AWS compatible object storage with min.io. It's free. @noamross wrote a nice tutorial for how to do it https://github.com/noamross/targets-minio-versioning which I used as a starting point for our setup.