Skip to content

Commit

Permalink
Add basic pipeline for data fetching
Browse files Browse the repository at this point in the history
Signed-off-by: Liang Zhang <[email protected]>
  • Loading branch information
psychelzh committed Aug 29, 2023
1 parent 7096605 commit e26d255
Show file tree
Hide file tree
Showing 4 changed files with 87 additions and 0 deletions.
1 change: 1 addition & 0 deletions NAMESPACE
Original file line number Diff line number Diff line change
Expand Up @@ -4,6 +4,7 @@ export(fetch_data)
export(fetch_parameterized)
export(prepare_fetch_data)
export(preproc_data)
export(use_targets)
export(wrangle_data)
import(rlang)
import(tidyselect)
31 changes: 31 additions & 0 deletions R/use_targets.R
Original file line number Diff line number Diff line change
@@ -0,0 +1,31 @@
#' Create standard data fetching targets pipeline script
#'
#' This function creates a standard data fetching targets pipeline script
#' for you to fill in.
#'
#' @return NULL (invisible). This function is called for its side effects.
#' @export
use_targets <- function() {
script <- "_targets.R"
if (file.exists(script)) {
cli::cli_alert_info(
sprintf("File {.file %s} exists. Stash and retry.", script)

Check warning on line 12 in R/use_targets.R

View check run for this annotation

Codecov / codecov/patch

R/use_targets.R#L9-L12

Added lines #L9 - L12 were not covered by tests
)
return(invisible())

Check warning on line 14 in R/use_targets.R

View check run for this annotation

Codecov / codecov/patch

R/use_targets.R#L14

Added line #L14 was not covered by tests
}
copy_success <- file.copy(
system.file(
"pipelines", "use_targets.R",
package = "tarflow.iquizoo"

Check warning on line 19 in R/use_targets.R

View check run for this annotation

Codecov / codecov/patch

R/use_targets.R#L16-L19

Added lines #L16 - L19 were not covered by tests
),
script

Check warning on line 21 in R/use_targets.R

View check run for this annotation

Codecov / codecov/patch

R/use_targets.R#L21

Added line #L21 was not covered by tests
)
if (!copy_success) {
cli::cli_alert_danger("Sorry, copy template failed.")
return(invisible())

Check warning on line 25 in R/use_targets.R

View check run for this annotation

Codecov / codecov/patch

R/use_targets.R#L23-L25

Added lines #L23 - L25 were not covered by tests
}
cli::cli_alert_success(
sprintf("File {.file %s} crated successfully.", script)

Check warning on line 28 in R/use_targets.R

View check run for this annotation

Codecov / codecov/patch

R/use_targets.R#L27-L28

Added lines #L27 - L28 were not covered by tests
)
return(invisible())

Check warning on line 30 in R/use_targets.R

View check run for this annotation

Codecov / codecov/patch

R/use_targets.R#L30

Added line #L30 was not covered by tests
}
40 changes: 40 additions & 0 deletions inst/pipelines/use_targets.R
Original file line number Diff line number Diff line change
@@ -0,0 +1,40 @@
# Created by tarflow.iquizoo::use_targets().
# Follow the comments below to fill in this target script.
# Then follow the manual to check and run the pipeline:
# https://books.ropensci.org/targets/walkthrough.html#inspect-the-pipeline

# Load packages required to define the pipeline:
library(targets)

Check notice on line 7 in inst/pipelines/use_targets.R

View check run for this annotation

codefactor.io / CodeFactor

inst/pipelines/use_targets.R#L7

Function "library" is undesirable. As an alternative, use roxygen2's @importFrom statement in packages and `::` in scripts, instead of modifying the global search path. (undesirable_function_linter)
# library(tarchetypes) # Load other packages as needed.

Check warning on line 8 in inst/pipelines/use_targets.R

View workflow job for this annotation

GitHub Actions / lint

file=inst/pipelines/use_targets.R,line=8,col=3,[commented_code_linter] Commented code should be removed.

Check warning on line 8 in inst/pipelines/use_targets.R

View workflow job for this annotation

GitHub Actions / lint

file=inst/pipelines/use_targets.R,line=8,col=3,[commented_code_linter] Commented code should be removed.

# Set target options:
tar_option_set(
packages = c("tarflow.iquizoo", "preproc.iquizoo") # packages that your targets need to run

Check warning on line 12 in inst/pipelines/use_targets.R

View workflow job for this annotation

GitHub Actions / lint

file=inst/pipelines/use_targets.R,line=12,col=81,[line_length_linter] Lines should not be more than 80 characters.

Check warning on line 12 in inst/pipelines/use_targets.R

View workflow job for this annotation

GitHub Actions / lint

file=inst/pipelines/use_targets.R,line=12,col=81,[line_length_linter] Lines should not be more than 80 characters.
# format = "qs", # Optionally set the default storage format. qs is fast.
#
# For distributed computing in tar_make(), supply a {crew} controller
# as discussed at https://books.ropensci.org/targets/crew.html.
# Choose a controller that suits your needs. For example, the following
# sets a controller with 2 workers which will run as local R processes:
#
# controller = crew::crew_controller_local(workers = 2)

Check warning on line 20 in inst/pipelines/use_targets.R

View workflow job for this annotation

GitHub Actions / lint

file=inst/pipelines/use_targets.R,line=20,col=7,[commented_code_linter] Commented code should be removed.

Check warning on line 20 in inst/pipelines/use_targets.R

View workflow job for this annotation

GitHub Actions / lint

file=inst/pipelines/use_targets.R,line=20,col=7,[commented_code_linter] Commented code should be removed.
#
# Set other options as needed.
)

# Run the R scripts in the R/ folder with your custom functions:
tar_source()
# source("other_functions.R") # Source other scripts as needed.

Check warning on line 27 in inst/pipelines/use_targets.R

View workflow job for this annotation

GitHub Actions / lint

file=inst/pipelines/use_targets.R,line=27,col=3,[commented_code_linter] Commented code should be removed.

Check warning on line 27 in inst/pipelines/use_targets.R

View workflow job for this annotation

GitHub Actions / lint

file=inst/pipelines/use_targets.R,line=27,col=3,[commented_code_linter] Commented code should be removed.

tbl_params <- tibble::tribble(
~course_name, ~course_period,
# replace course name and course period with your own
"# COURSE NAME", "# COURSE PERIOD"
)

# Replace the target list below with your own:
list(
# change what to scores or raw_data if you want to
tarflow.iquizoo::prepare_fetch_data(tbl_params, what = "all")
# more targets goes here
)
15 changes: 15 additions & 0 deletions man/use_targets.Rd

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

0 comments on commit e26d255

Please sign in to comment.