Fix reverse dependency checks

targets 0.13.1

• Fix reverse dependency checks.

{paws} options and tar_source()

targets 0.13.0

Bug fixes

• Do not bootstrap the junction of a stem unless the target is branched over (#858, @dipterix).
• For non-"file" AWS targets, immediately delete the scratch file after the target is uploaded (#889, @stuvet).

New features

• Allow extra arguments to paws functions via ... in tar_resources_aws() (#855, @michkam89).
• Add tar_source() to conveniently source R scripts (e.g. in _targets.R).

Enhancements

• Color ordinary targets messages the default theme color, and color warnings and errors red (#856, @gorkang).
• Automatically supply job names in the scripts generated by use_targets().
• Inherit resources one-by-one in nested fashion from tar_option_get("resources") (#892). See the revised "Resources" section of the tar_resources() help file for details.

Improvements to use_targets(), tar_mermaid(), and error printing

targets 0.12.1

New features

• Add arguments legend and color to further configure tar_mermaid() (#848, @noamross).
• For HPC schedulers like SLURM and SGE, use_targets() now creates a job.sh script to run the pipeline as a cluster job (#839).

Enhancements

• Use lapply() to source scripts in use_targets(). Avoids defining a global variable for the file.
• Recursively find scripts to source in the use_targets() _targets.R file.
• Refactor error printing.

A better use_targets(), tar_load_everything(), and bug fixes

targets 0.12.0

Bug fixes

• Fix tar_mermaid() graph ordering.
• Hash the node names and quote the label names of tar_mermaid() graphs to avoid JavaScript keywords.
• Remove superfluous line breaks in the node labels of graph visuals.
• Fix metadata migration to version >= 0.10.0 (#812, @tjmahr).
• data.table::fread() with encoding equal to getOption("encoding") if available (#814, @svraka). Only works with UTF-8 and latin1 because that is what data.table supports.
• Force add files in GitHub Actions workflow job (#815, @tarensanders).

New features

• use_targets() now writes a _targets.R file tailored to the project in the current working directory (#639, @noamross).
• Move the old use_targets() to use_targets_rmd().

Enhancements

• Load packages when loading data for downstream targets in the pipeline (#713).
• Handle edge case when getOption("OutDec") is not "." to prevent time stamps from being corrupted (#433, @jarauh).
• Added helper function tar_load_everything() to quickly load all targets (#823, @malcolmbarrett)

Google Cloud Storage, mermaid.js static graphs, and error = "null"

targets 0.11.0

Bug fixes

• Print out the relevant target names if targets have conflicting names.
• Catch all the target warnings instead of just reporting the last one.
• Allow 200 group URL status codes instead of just 200 (#797, @petrbouchal).

New features

• Add Google Cloud Storage via tar_target(..., repository = "gcp") (#720, @MarkEdmondson1234). Special thanks to @MarkEdmondson1234 for the cloud storage utilities in R/utils_gcp.R
• mermaid.js static graphs with tar_mermaid() (#775, @yonicd).
• Implement tar_target(..., error = "null")to allow errored targets to return NULL and continue (#807, @zoews). Errors are still registered, those targets are not up to date, and downstream targets have an easier time continuing on.
• Implement tar_assert_finite().
• tar_destroy(), tar_delete(), and tar_prune() now attempt to delete cloud data for the appropriate targets (#799). In addition, tar_exist_objects() and tar_objects() now report about target data in the cloud when applicable. Add a new cloud argument to each function to optionally suppress this new behavior.
• Add a zoom_speed argument to tar_visnetwork() and tar_glimpse() (#749, @dipterix).
• Report the total runtime of the pipeline in the "verbose", "verbose_positives", "timestamp", and "timesamp_positives" reporters.

Enhancements

• Allow target name character strings to have attributes (#758, @psanker).
• Sort metadata rows when the pipeline finishes so that version-controlling the metadata is easier (#766, @jameelalsalam).

Deprecations

• Deprecate the "aws_*" storage format values in favor of a new repository argument (#803). In other words, tar_target(..., format = "aws_qs") is now tar_target(..., format = "qs", repository = "aws"). And internally, storage classes with multiple inheritance are created dynamically as opposed to having hard-coded source files. All this paves the way to add new cloud storage platforms without combinatorial chaos.

Custom data storage formats

targets 0.10.0

Bug fixes

• Add class "tar_nonexportable" to format = "aws_keras" and format = "aws_torch" stores.
• Export S3 methods of generic tar_make_interactive_load_target().

New features

• Allow entirely custom storage formats through tar_target(format = tar_format(...)) (#736).
• Add a new function tar_call() to return the targets function currently running (from _targets.R or a target).
• Add a new function tar_active() to tell whether the pipeline is currently running. Detects if it is called from tar_make() or similar function.

Enhancements

• Add Sys.getenv("TAR_PROJECT") to the output of tar_envvars().
• Set the store field of tar_runtime prior to sourcing _targets.R so tar_store() works in target scripts.
• Explicitly export all the environment variables from tar_envvars() to targets run on parallel workers.
• Allow format = "file" targets to return character(0) (#728, @programLyrique).
• Automatically remove non-targets from the target list and improve target list error messages (#731, @billdenney).
• Link to resources on deploying to RStudio Connect (#745, @ian-flores).

Data version control in the cloud

targets 0.9.0

Highlights

• Track the version ID of AWS S3-backed targets if the bucket is version-enabled (#711). If you put your targets in AWS and the metadata and code under version control, you can git checkout a different branch of your code and all you targets will stay up to date.
• Refactor the AWS path format internally. It now consists of arbitrarily extensible key-value pairs so more AWS S3 functionality may be added more seamlessly going forward (#711).
• Switch the AWS S3 backend to paws (#711).

New features

• Add a region argument to tar_resources_aws() to allow the user to explicitly declare a region for each AWS S3 buckets (@caewok, #681). Different buckets can now have different regions. This feature required modifying the metadata path for AWS storage formats. Before, the first element of the path was simply the bucket name. Now, it is internally formatted like "bucket=BUCKET:region=REGION", where BUCKET is the user-supplied bucket name and REGION is the user-supplied region name. The new targets is back-compatible with the old metadata format, but if you run the pipeline with targets >= 0.8.1.9000 and then downgrade to targets <= 0.8.1, any AWS targets will break.
• Add new reporters timestamp_positives" and "verbose_positives" that omit messages for skipped targets (@psanker, #683).
• Implement tar_assert_file().
• Implement tar_reprex() for creating easier reproducible examples of pipelines.
• Implement tar_store() to get the path to the store of the currently running pipeline (#714, @MilesMcBain).
• Automatically write a _targets/user/ folder to encourage gittargets users to put custom files there for data version control.

Bug fixes

• Make sure tar_path() uses the current store path of the currently running pipeline instead of tar_config_get("store") (#714, @MilesMcBain).

Enhancements

• Refactor the automatic .gitignore file inside the data store to allow the metadata to be committed to version control more easily (#685, #711).
• Document target name requirements in tar_target() and tar_target_raw() (@tjmahr, #679).
• Catch and relay any the error if a target cannot be checked in target_should_run.tar_builder(). These kinds of errors sometimes come up with AWS storage.
• Fix the documentation of the reporters.
• Only write _targets/.gitignore for new data stores so the user can delete the .gitignore file without it mysteriously reappearing (#685).

Documentation patch

targets 0.8.1

New features

• Add arguments strict and silent to allow tar_load() and tar_load_raw() to bypass targets that cannot be loaded.

Enhancements

• Improve tidyselect docs in tar_make() (#640, @dewoller).
• Use namespaced call to tar_dir() in tar_test() (#642, @billdenney).
• Improve tar_assert_target_list() error message (@kkami1115, #654).
• Throw an informative error if a target name starts with a dot (@dipterix, #662).
• Improve help files of tar_destroy() and related cleanup functions (@billdenney, #675).

Multi-project config

targets 0.8.0

Bug fixes

• Hash the correct files in tar_target(target_name, ..., format = "aws_file"). Previously, _targets/objects/target_name was also hashed if it existed.

New features

• Implement a new tar_config_unset() function to delete one or more configuration settings from the YAML configuration file.
• Implement the TAR_CONFIG environment variable to set the default file path of the YAML configuration file with project settings (#622, @yyzeng, @atusy, @nsheff, @wdkrnls). If TAR_CONFIG is not set, the file path is still _targets.yaml.
• Restructure the YAML configuration file format to handle configuration information for multiple projects (using the config package) and support the TAR_PROJECT environment variable to select the current active project for a given R session. The old single-project format is gracefully deprecated (#622, @yyzeng, @atusy, @nsheff, @wdkrnls).
• Implement retrieval = "none" and storage = "none" to anticipate loading/saving targets from other languages, e.g. Julia (@MilesMcBain).
• Add a new tar_definition() function to get the target definition object of the current target while that target is running in a pipeline.
• If called inside an AWS target, tar_path() now returns the path to the staging file instead of _targets/objects/target_name. This ensures you can still write to tar_path() in storage = "none" targets and the package will automatically hash the right file and upload it to the cloud. (This behavior does not apply to formats "file" and "aws_file", where it is never necessary to set storage = "none".)

Enhancements

• Use eval(parse(text = ...), envir = tar_option_set("envir") instead of source() in the _targets.R file for Target Markdown.
• Allow feather and parquet formats to accept objects of class RecordBatch and Table (@MilesMcBain).
• Let knitr load the Target Markdown engine (#469, @nviets, @yihui). Minimum knitr version is now 1.34.
• In the tar_resources_future() help file, encourage the use of plan to specify resources.

New helpers, better relaying in Target Markdown, and better exception handling

targets 0.7.0

Bug fixes

• Ensure error = "continue" does not cause errored targets to have NULL values.
• Relay output and messages in Target Markdown interactive mode (using the R/default knitr engine).

New features

• Expose the poll_connection, stdout, and stderr arguments of callr::r_bg() in tar_watch() (@mpadge).
• Add new helper functions to list targets in each progress category: tar_started(), tar_skipped(), tar_built(), tar_canceled(), and tar_errored().
• Add new helper functions tar_interactive(), tar_noninteractive(), and tar_toggle() to differentially suppress code in non-interactive and interactive mode in Target Markdown (#607, @33Vito).

Enhancements

• Handle future errors within targets (#570, @stuvet).
• Handle storage errors within targets (#571, @stuvet).
• In Target Markdown in non-interactive mode, suppress messages if the message knitr chunk option is FALSE (#574, @jmbuhr).
• In Target Markdown, if tar_interactive is not set, choose interactive vs non-interactive mode based on isTRUE(getOption("knitr.in.progress")) instead of interactive().
• Convert errors loading dependencies into errors running targets (@stuvet).