Open
Description
Users may inadvertently send hundreds of gigabytes of data via the network by using closures. Consider the following:
do_my_stuff = function() {
huge_object = runif(1e9)
my_parallel = function(i) {
}
Q(my_parallel, i=1:1000, n_jobs=1000)
}
This will send 7.6 Gb to 1000 workers (= 7.6 Tb total), without any of the workers requiring it.
It should not be that easy to make this mistake.
Related: #200, likely re-introduced with globals
package changes close to 9f01bbc