worker resource logging

R/crew_client.R

@@ -394,6 +394,19 @@ crew_class_client <- R6::R6Class(
         complete = as.integer(daemons[, "complete"]),
         socket = as.character(rownames(daemons))
       )
+    },
+    #' @description Get the process IDs of the local process and the
+    #'   `mirai` dispatcher (if started).
+    #' @return An integer vector of process IDs of the local process and the
+    #'   `mirai` dispatcher (if started).
+    pids = function() {
+      out <- c(local = Sys.getpid())
+      dispatcher <- private$.dispatcher
+      if (!is.null(dispatcher)) {
+        out <- c(out, ps::ps_pid(dispatcher))
+        names(out)[2L] <- paste0("dispatcher-", private$.name)
+      }
+      out
     }
   )
 )

R/crew_controller.R

@@ -1386,6 +1386,15 @@ crew_class_controller <- R6::R6Class(
       mirai::stop_mirai(tasks)
       invisible()
     },
+    #' @description Get the process IDs of the local process and the
+    #'   `mirai` dispatcher (if started).
+    #' @return An integer vector of process IDs of the local process and the
+    #'   `mirai` dispatcher (if started).
+    #' @param controllers Not used. Included to ensure the signature is
+    #'   compatible with the analogous method of controller groups.
+    pids = function(controllers = NULL) {
+      private$.client$pids()
+    },
     #' @description Terminate the workers and the `mirai` client.
     #' @return `NULL` (invisibly).
     #' @param controllers Not used. Included to ensure the signature is

R/crew_controller_group.R

@@ -956,6 +956,18 @@ crew_class_controller_group <- R6::R6Class(
       }
       tibble::as_tibble(do.call(what = rbind, args = out))
     },
+    #' @description Get the process IDs of the local process and the
+    #'   `mirai` dispatchers (if started).
+    #' @return An integer vector of process IDs of the local process and the
+    #'   `mirai` dispatcher (if started).
+    #' @param controllers Character vector of controller names.
+    #'   Set to `NULL` to select all controllers.
+    pids = function(controllers = NULL) {
+      control <- private$.select_controllers(controllers)
+      out <- map(control, ~.x$pids())
+      out <- map(unname(out), ~.x[names(.x) != "local"])
+      c(local = Sys.getpid(), unlist(out))
+    },
     #' @description Terminate the workers and disconnect the client
     #'   for one or more controllers.
     #' @return `NULL` (invisibly).

R/crew_options_local.R

@@ -16,7 +16,7 @@
 #'   will go to different log files with informative suffixes.
 #' @examples
 #'   crew_options_local()
-crew_options_local = function(
+crew_options_local <- function(
   log_directory = NULL,
   log_join = TRUE
 ) {

R/crew_options_metrics.R

@@ -35,7 +35,7 @@
 #'   log entries written to `path`.
 #' @examples
 #'   crew_options_metrics(path = "/dev/stdout")
-crew_options_metrics = function(path = NULL, seconds_interval = 5) {
+crew_options_metrics <- function(path = NULL, seconds_interval = 5) {
   out <- structure(
     list(path = path, seconds_interval = seconds_interval),
     class = c("crew_options_metrics", "crew_options")

tests/testthat/test-crew_controller_group.R

@@ -39,12 +39,14 @@ crew_test("crew_controller_group()", {
     expect_null(x$controllers[[index]]$client$started)
   }
   expect_false(x$started())
+  expect_equal(length(x$pids()), 1L)
   x$start()
   expect_true(x$started())
   expect_true(x$empty())
   expect_false(x$saturated())
   expect_true(x$empty(controllers = "a"))
   expect_true(x$empty(controllers = "b"))
+  expect_equal(length(x$pids()), 3L)
   for (index in seq_len(2)) {
     expect_true(x$controllers[[index]]$client$started)
   }

tests/testthat/test-crew_controller_local.R

@@ -15,6 +15,7 @@ crew_test("crew_controller_local()", {
   expect_false(x$started())
   expect_null(x$summary())
   expect_null(x$autoscaling)
+  expect_equal(length(x$pids()), 1L)
   x$start()
   expect_true(x$empty())
   expect_false(x$saturated())
@@ -45,6 +46,7 @@ crew_test("crew_controller_local()", {
   expect_equal(s$tasks, 0L)
   expect_true(x$client$started)
   expect_true(x$started())
+  expect_equal(length(x$pids()), 2L)
   # first task
   expect_equal(x$pushed, 0L)
   expect_equal(x$popped, 0L)

vignettes/introduction.Rmd

@@ -199,7 +199,7 @@ controller$client$summary()
 
 # Resources
 
-`crew` utilizes background processes, and it may consume a lot of memory for big data workloads. A common cause of crashes is running out of computer memory. If you are running `crew` in a [`targets`](https://docs.ropensci.org/targets) pipeline (as explained [here in the `targets` user manual](https://books.ropensci.org/targets/crew.html)), consider setting `storage = "worker"` and `retrieval = "worker` in `tar_option_set()` to minimize memory consumption of the local processes (see also the [performance chapter](https://books.ropensci.org/targets/performance.html)).
+`crew` utilizes background processes, and it may consume a lot of memory for big data workloads. A common cause of crashes is running out of computer memory. If you are running `crew` in a [`targets`](https://docs.ropensci.org/targets/) pipeline (as explained [here in the `targets` user manual](https://books.ropensci.org/targets/crew.html)), consider setting `storage = "worker"` and `retrieval = "worker` in `tar_option_set()` to minimize memory consumption of the local processes (see also the [performance chapter](https://books.ropensci.org/targets/performance.html)).
 
 As of `crew` version 0.9.5.9007, you monitor resources by supplying a log file path to the `log_resources` argument of the controller. That way, normal controller methods write to the log as a side effect every `seconds_interval` seconds. The output file is a comma-separated values (CSV) file which can be read into R with `readr::read_csv()`.
 

vignettes/risks.Rmd

@@ -25,7 +25,7 @@ In the event of a poorly-timed crash or network error, these processes may not t
 
 ## Crashes
 
-The local process or [`mirai`](https://github.com/shikokuchuo/mirai) dispatcher process could crash. A common cause of crashes is running out of computer memory. The "Resources" section of the [introduction](https://wlandau.github.io/crew/articles/introduction.html) explains how to monitor memory usage. If you are running `crew` in a [`targets`](https://docs.ropensci.org/targets) pipeline (as explained [here in the `targets` user manual](https://books.ropensci.org/targets/crew.html)), consider setting `storage = "worker"` and `retrieval = "worker` in `tar_option_set()` to minimize memory consumption of the local processes (see also the [performance chapter](https://books.ropensci.org/targets/performance.html)).
+The local process or [`mirai`](https://github.com/shikokuchuo/mirai) dispatcher process could crash. A common cause of crashes is running out of computer memory. The "Resources" section of the [introduction](https://wlandau.github.io/crew/articles/introduction.html) explains how to monitor memory usage. If you are running `crew` in a [`targets`](https://docs.ropensci.org/targets/) pipeline (as explained [here in the `targets` user manual](https://books.ropensci.org/targets/crew.html)), consider setting `storage = "worker"` and `retrieval = "worker` in `tar_option_set()` to minimize memory consumption of the local processes (see also the [performance chapter](https://books.ropensci.org/targets/performance.html)).
 
 In addition, `crew` worker processes may crash silently at runtime, or they may fail to launch or connect at all. The reasons may be platform-specific, but here are some common possibilities: