tar_download: no ETag or Last-Modified for url

[jimjam-slam]

I'd like to use tar_download to download and store source data for my analysis. But when I try to run the pipeline, I get:

Error:
! Error running targets::tar_make()
  Target errors: targets::tar_meta(fields = error, complete_only = TRUE)
  Tips: https://books.ropensci.org/targets/debugging.html
  Last error: _paths_ no ETag or Last-Modified for url: [URL REDACTED]. Please choose a storage format other than format = "url" and pursue different workarounds to rerun the target when the remote data changes.

I understand that this function is designed to watch a remote source, but is there a way to disable that and just have it invalidate when the downloaded files are missing (other than writing a target manually)?

I was a bit confused because the error (and the documentation for tar_download) implies that the file argument can be set, but it also sounds like that is fixed when using tar_download (since it returns two target objects - one with format = "url" and one with format = "file").

Thanks!

[wlandau]

You could try cue = tar_cue(file = FALSE) in tar_download(), or you could use an ordinary tar_target(..., command = tarchetypes::tar_download_run(), format = "file").

tarchetypes/R/tar_download.R

Lines 158 to 180 in 4f75eff

	tar_download_run <- function(
	urls,
	paths,
	method,
	quiet,
	mode,
	cacheOK,
	extra,
	headers
	) {
	for (index in seq_along(urls)) {
	utils::download.file(
	url = urls[index],
	destfile = paths[index],
	method = method,
	quiet = quiet,
	mode = mode,
	extra = extra,
	headers = headers
	)
	}
	paths
	}