You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
I've started with @wlandau {drake} instead of {targets} because I haven't dived yet into the latter.
But this is more about {future.clustermq} than about drake vs. targets and if this one works at some point, porting it to drake might not be that hard hopefully.
Note: It could be that the semantics for the workers object at the linked reference are totally wrong - I do not have deep knowledge of the {clustermq} internals nor the {future} internals.
I was just playing around with what was already there.
In essence this is just a start to push this initiative a bit and maybe we can all do this together.
# remotes::install_github("pat-s/future.clustermq@test)
library(drake)
# Create the template file. You may have to modify it.# drake_hpc_template_file("slurm_clustermq.tmpl")# Configure clustermq.# options(clustermq.scheduler = "slurm", template = "slurm_clustermq_future.tmpl")
options(clustermq.scheduler="multiprocess")
suppressPackageStartupMessages(library(future.clustermq))
# FIXME: is this even correct?future::plan("clustermq")
load_mtcars_example()
make(my_plan, parallelism="future", jobs=1)
The text was updated successfully, but these errors were encountered:
I was playing around today with {future.clustermq} and wanted to share my experience. Maybe @mschubert has some thoughts here to push this initiative by @HenrikBengtsson.
This relates to futureverse/future#204.
I've started with @wlandau {drake} instead of {targets} because I haven't dived yet into the latter.
But this is more about {future.clustermq} than about drake vs. targets and if this one works at some point, porting it to drake might not be that hard hopefully.
I got stuck at the step sending the
future
to the worker for evaluation.The following snippet should get you to https://github.com/pat-s/future.clustermq/blob/e0a25dc4cedd12663aca1a561cff9a72e00ec3b4/R/ClusterMQFuture-class.R#L240 where I am trying to evaluate the expression.
Note: It could be that the semantics for the
workers
object at the linked reference are totally wrong - I do not have deep knowledge of the {clustermq} internals nor the {future} internals.I was just playing around with what was already there.
In essence this is just a start to push this initiative a bit and maybe we can all do this together.
The text was updated successfully, but these errors were encountered: