本文整理了Java中org.apache.spark.broadcast.Broadcast.destroy()
方法的一些代码示例,展示了Broadcast.destroy()
的具体用法。这些代码示例主要来源于Github
/Stackoverflow
/Maven
等平台,是从一些精选项目中提取出来的代码,具有较强的参考意义,能在一定程度帮忙到你。Broadcast.destroy()
方法的具体详情如下:
包路径:org.apache.spark.broadcast.Broadcast
类名称:Broadcast
方法名:destroy
暂无
代码示例来源:origin: apache/tinkerpop
protected void broadcastMemory(final JavaSparkContext sparkContext) {
this.broadcast.destroy(true); // do we need to block?
final Map<String, Object> toBroadcast = new HashMap<>();
this.sparkMemory.forEach((key, object) -> {
if (!object.value().isEmpty() && this.memoryComputeKeys.get(key).isBroadcast())
toBroadcast.put(key, object.value());
});
this.broadcast = sparkContext.broadcast(toBroadcast);
}
代码示例来源:origin: ai.grakn/grakn-kb
protected void broadcastMemory(final JavaSparkContext sparkContext) {
this.broadcast.destroy(true); // do we need to block?
final Map<String, Object> toBroadcast = new HashMap<>();
this.sparkMemory.forEach((key, object) -> {
if (!object.value().isEmpty() && this.memoryComputeKeys.get(key).isBroadcast()) {
toBroadcast.put(key, object.value());
}
});
this.broadcast = sparkContext.broadcast(toBroadcast);
}
代码示例来源:origin: org.apache.tinkerpop/spark-gremlin
protected void broadcastMemory(final JavaSparkContext sparkContext) {
this.broadcast.destroy(true); // do we need to block?
final Map<String, Object> toBroadcast = new HashMap<>();
this.sparkMemory.forEach((key, object) -> {
if (!object.value().isEmpty() && this.memoryComputeKeys.get(key).isBroadcast())
toBroadcast.put(key, object.value());
});
this.broadcast = sparkContext.broadcast(toBroadcast);
}
代码示例来源:origin: cerner/bunsen
/**
* Pops a BroadcastableValueSets from the user-defined function stack.
*
* @param spark the spark session
* @return true if there is still a registered in_valuset UDF, false otherwise
*/
public static synchronized boolean popUdf(SparkSession spark) {
if (valueSetStack.isEmpty()) {
return false;
} else {
// Cleanup the previously broadcast valuesets
Broadcast<BroadcastableValueSets> old = valueSetStack.pop();
old.destroy();
if (valueSetStack.isEmpty()) {
return false;
} else {
// Re-apply the previous function.
Broadcast<BroadcastableValueSets> current = valueSetStack.peek();
spark.udf()
.register("in_valueset",
new InValuesetUdf(current),
DataTypes.BooleanType);
return true;
}
}
}
代码示例来源:origin: com.cerner.bunsen/bunsen-core
/**
* Pops a BroadcastableValueSets from the user-defined function stack.
*
* @param spark the spark session
* @return true if there is still a registered in_valuset UDF, false otherwise
*/
public static synchronized boolean popUdf(SparkSession spark) {
if (valueSetStack.isEmpty()) {
return false;
} else {
// Cleanup the previously broadcast valuesets
Broadcast<BroadcastableValueSets> old = valueSetStack.pop();
old.destroy();
if (valueSetStack.isEmpty()) {
return false;
} else {
// Re-apply the previous function.
Broadcast<BroadcastableValueSets> current = valueSetStack.peek();
spark.udf()
.register("in_valueset",
new InValuesetUdf(current),
DataTypes.BooleanType);
return true;
}
}
}
代码示例来源:origin: com.davidbracewell/mango
@Override
public synchronized void updateConfig() {
if (configBroadcast != null && configBroadcast.isValid()) {
configBroadcast.destroy();
}
configBroadcast = broadcast(Config.getInstance());
}
内容来源于网络,如有侵权,请联系作者删除!