我想使用parLapply,我正在设置我的代码,就像这里介绍的那样:http://www.win-vector.com/blog/2016/01/parallel-computing-in-r/
前几次运行得很好。但是,在我当前的parLapply
调用中,我得到了错误Error in checkForRemoteErrors(val) : 3 nodes produced errors; first error: could not find function "bindToEnv"
。
这里有一个简短的例子:
#' Copy arguments into env and re-bind any function's lexical scope to bindTargetEnv .
#'
#' See http://winvector.github.io/Parallel/PExample.html for example use.
#'
#'
#' Used to send data along with a function in situations such as parallel execution
#' (when the global environment would not be available). Typically called within
#' a function that constructs the worker function to pass to the parallel processes
#' (so we have a nice lexical closure to work with).
#'
#' @param bindTargetEnv environment to bind to
#' @param objNames additional names to lookup in parent environment and bind
#' @param names of functions to NOT rebind the lexical environments of
bindToEnv <- function(bindTargetEnv=parent.frame(),objNames,doNotRebind=c()) {
# Bind the values into environment
# and switch any functions to this environment!
for(var in objNames) {
val <- get(var,envir=parent.frame())
if(is.function(val) && (!(var %in% doNotRebind))) {
# replace function's lexical environment with our target (DANGEROUS)
environment(val) <- bindTargetEnv
}
# assign object to target environment, only after any possible alteration
assign(var,val,envir=bindTargetEnv)
}
}
ccc <- 1
# Parallel
cl <- parallel::makeCluster(getOption("cl.cores", 3))
junk <- parallel::clusterEvalQ(cl, c(library(data.table)))
f <- function(x) {
bindToEnv(objNames = 'ccc')
return(x+x)
}
b <- do.call(rbind, parallel::parLapply(cl, 1:10, f))
如果我不添加bindToEnv
一切正常.我做错了什么?谢谢!
1条答案
按热度按时间rdlzhqv91#
在创建集群之前,需要使用
clusterExport()
导出所定义的函数和对象。