org.apache.spark.broadcast.Broadcast.destroy()方法的使用及代码示例

x33g5p2x  于2022-01-16 转载在 其他  
字(3.1k)|赞(0)|评价(0)|浏览(236)

本文整理了Java中org.apache.spark.broadcast.Broadcast.destroy()方法的一些代码示例,展示了Broadcast.destroy()的具体用法。这些代码示例主要来源于Github/Stackoverflow/Maven等平台,是从一些精选项目中提取出来的代码,具有较强的参考意义,能在一定程度帮忙到你。Broadcast.destroy()方法的具体详情如下:
包路径:org.apache.spark.broadcast.Broadcast
类名称:Broadcast
方法名:destroy

Broadcast.destroy介绍

暂无

代码示例

代码示例来源:origin: apache/tinkerpop

protected void broadcastMemory(final JavaSparkContext sparkContext) {
  this.broadcast.destroy(true); // do we need to block?
  final Map<String, Object> toBroadcast = new HashMap<>();
  this.sparkMemory.forEach((key, object) -> {
    if (!object.value().isEmpty() && this.memoryComputeKeys.get(key).isBroadcast())
      toBroadcast.put(key, object.value());
  });
  this.broadcast = sparkContext.broadcast(toBroadcast);
}

代码示例来源:origin: ai.grakn/grakn-kb

protected void broadcastMemory(final JavaSparkContext sparkContext) {
  this.broadcast.destroy(true); // do we need to block?
  final Map<String, Object> toBroadcast = new HashMap<>();
  this.sparkMemory.forEach((key, object) -> {
    if (!object.value().isEmpty() && this.memoryComputeKeys.get(key).isBroadcast()) {
      toBroadcast.put(key, object.value());
    }
  });
  this.broadcast = sparkContext.broadcast(toBroadcast);
}

代码示例来源:origin: org.apache.tinkerpop/spark-gremlin

protected void broadcastMemory(final JavaSparkContext sparkContext) {
  this.broadcast.destroy(true); // do we need to block?
  final Map<String, Object> toBroadcast = new HashMap<>();
  this.sparkMemory.forEach((key, object) -> {
    if (!object.value().isEmpty() && this.memoryComputeKeys.get(key).isBroadcast())
      toBroadcast.put(key, object.value());
  });
  this.broadcast = sparkContext.broadcast(toBroadcast);
}

代码示例来源:origin: cerner/bunsen

/**
 * Pops a BroadcastableValueSets from the user-defined function stack.
 *
 * @param spark the spark session
 * @return true if there is still a registered in_valuset UDF, false otherwise
 */
public static synchronized boolean popUdf(SparkSession spark) {
 if (valueSetStack.isEmpty()) {
  return false;
 } else {
  // Cleanup the previously broadcast valuesets
  Broadcast<BroadcastableValueSets> old = valueSetStack.pop();
  old.destroy();
  if (valueSetStack.isEmpty()) {
   return false;
  } else {
   // Re-apply the previous function.
   Broadcast<BroadcastableValueSets> current = valueSetStack.peek();
   spark.udf()
     .register("in_valueset",
       new InValuesetUdf(current),
       DataTypes.BooleanType);
   return true;
  }
 }
}

代码示例来源:origin: com.cerner.bunsen/bunsen-core

/**
 * Pops a BroadcastableValueSets from the user-defined function stack.
 *
 * @param spark the spark session
 * @return true if there is still a registered in_valuset UDF, false otherwise
 */
public static synchronized boolean popUdf(SparkSession spark) {
 if (valueSetStack.isEmpty()) {
  return false;
 } else {
  // Cleanup the previously broadcast valuesets
  Broadcast<BroadcastableValueSets> old = valueSetStack.pop();
  old.destroy();
  if (valueSetStack.isEmpty()) {
   return false;
  } else {
   // Re-apply the previous function.
   Broadcast<BroadcastableValueSets> current = valueSetStack.peek();
   spark.udf()
     .register("in_valueset",
       new InValuesetUdf(current),
       DataTypes.BooleanType);
   return true;
  }
 }
}

代码示例来源:origin: com.davidbracewell/mango

@Override
public synchronized void updateConfig() {
 if (configBroadcast != null && configBroadcast.isValid()) {
   configBroadcast.destroy();
 }
 configBroadcast = broadcast(Config.getInstance());
}

相关文章