org.apache.drill.yarn.zk.ZKClusterCoordinatorDriver类的使用及代码示例

x33g5p2x  于2022-02-05 转载在 其他  
字(7.1k)|赞(0)|评价(0)|浏览(119)

本文整理了Java中org.apache.drill.yarn.zk.ZKClusterCoordinatorDriver类的一些代码示例,展示了ZKClusterCoordinatorDriver类的具体用法。这些代码示例主要来源于Github/Stackoverflow/Maven等平台,是从一些精选项目中提取出来的代码,具有较强的参考意义,能在一定程度帮忙到你。ZKClusterCoordinatorDriver类的具体详情如下:
包路径:org.apache.drill.yarn.zk.ZKClusterCoordinatorDriver
类名称:ZKClusterCoordinatorDriver

ZKClusterCoordinatorDriver介绍

[英]Driver class for the ZooKeeper cluster coordinator. Provides defaults for most options, but allows customizing each. Provides a #build() method to create and start the ZK service. Obtains the initial set of Drillbits (which should be empty for a YARN-defined cluster) which can be retrieved after building.

Maintains the ZK connection and monitors for disconnect. This class simply detects a disconnect timeout, it does not send a disconnect event itself to avoid creating a timer thread just for this purpose. Instead, the caller can poll #hasFailed().

Defaults match those in Drill. (Actual Drill defaults are not yet used due to code incompatibility issues.)
[中]ZooKeeper群集协调器的驱动程序类。为大多数选项提供默认值,但允许自定义每个选项。提供一个#build()方法来创建和启动ZK服务。获取可在构建后检索的初始组钻头(对于纱线定义的簇,应为空)。
维护ZK连接并监控断开连接。这个类只是检测一个断开连接超时,它本身并不发送一个断开连接事件,以避免为此目的创建一个计时器线程。相反,调用方可以轮询#hasfiled()。
默认值与演练中的匹配。(由于代码不兼容问题,尚未使用实际的演练默认值。)

代码示例

代码示例来源:origin: apache/drill

/**
 * Convert a Drillbit endpoint to a string key used in the (zk-->task) map.
 * Note that the string format here must match the one used in
 * {@link #toKey(Task)} to map a task to string key.
 *
 * @param dbe
 *          the Drillbit endpoint from ZK
 * @return a string key for this object
 */
private String toKey(DrillbitEndpoint dbe) {
 return ZKClusterCoordinatorDriver.asString(dbe);
}

代码示例来源:origin: apache/drill

public void finish(RegistryHandler handler) {
 zkDriver.removeDrillbitListener(this);
 zkDriver.close();
}

代码示例来源:origin: apache/drill

/**
 * Convert a task to a string key used in the (zk-->task) map. Note that the
 * string format here must match the one used in
 * {@link #toKey(DrillbitEndpoint)} to map a drillbit endpoint to string key.
 *
 * @param task
 *          the task tracked by the cluster controller
 * @return a string key for this object
 */
private String toKey(Task task) {
 return zkDriver.toKey(task.getHostName());
}

代码示例来源:origin: apache/drill

addDrillbit(probeZk, FRED_HOST);
ZKClusterCoordinatorDriver driver = new ZKClusterCoordinatorDriver()
  .setConnect(connStr, ZK_ROOT, CLUSTER_ID).build();
List<DrillbitEndpoint> bits = driver.getInitialEndpoints();
assertEquals(1, bits.size());
assertEquals(makeKey(FRED_HOST),
  ZKClusterCoordinatorDriver.asString(bits.get(0)));
driver.addDrillbitListener(listener);
for (DrillbitEndpoint dbe : listener.added) {
 assertEquals(makeKey(WILMA_HOST),
   ZKClusterCoordinatorDriver.asString(dbe));
for (DrillbitEndpoint dbe : listener.removed) {
 assertEquals(makeKey(FRED_HOST),
   ZKClusterCoordinatorDriver.asString(dbe));
driver.close();
server.stop();
server.close();

代码示例来源:origin: apache/drill

addDrillbit(probeZk, FRED_HOST);
ZKClusterCoordinatorDriver driver = new ZKClusterCoordinatorDriver()
  .setConnect(connStr, ZK_ROOT, CLUSTER_ID)
  .setPorts(TEST_USER_PORT, TEST_CONTROL_PORT, TEST_DATA_PORT).build();
ZKRegistry registry = new ZKRegistry(driver);
TestRegistryHandler handler = new TestRegistryHandler();
assertNull(fredsTracker.task);
assertEquals(fredsKey,
  ZKClusterCoordinatorDriver.asString(fredsTracker.endpoint));
driver.close();
server.stop();
server.close();

代码示例来源:origin: apache/drill

/**
 * Basic setup: start a ZK and verify that the initial endpoint list is empty.
 * Also validates the basics of the test setup (mock server, etc.)
 *
 * @throws Exception
 */
@Test
public void testBasics() throws Exception {
 try (TestingServer server = new TestingServer()) {
  server.start();
  String connStr = server.getConnectString();
  ZKClusterCoordinatorDriver driver = new ZKClusterCoordinatorDriver()
    .setConnect(connStr, "drill", "drillbits").build();
  assertTrue(driver.getInitialEndpoints().isEmpty());
  driver.close();
  server.stop();
 }
}

代码示例来源:origin: apache/drill

int userPort = config.getInt(DrillOnYarnConfig.DRILLBIT_USER_PORT);
int bitPort = config.getInt(DrillOnYarnConfig.DRILLBIT_BIT_PORT);
ZKClusterCoordinatorDriver driver = new ZKClusterCoordinatorDriver()
  .setConnect(zkConnect, zkRoot, clusterId)
  .setFailureTimoutMs(failureTimeoutMs)
  .setRetryCount(retryCount)
  .setRetryDelayMs(retryDelayMs)
  .setPorts(userPort, bitPort, bitPort + 1);
ZKRegistry zkRegistry = new ZKRegistry(driver);
dispatcher.registerAddOn(new ZKRegistryAddOn(zkRegistry));

代码示例来源:origin: apache/drill

server.start();
String connStr = server.getConnectString();
ZKClusterCoordinatorDriver driver = new ZKClusterCoordinatorDriver()
  .setConnect(connStr, TEST_ZK_ROOT, TEST_CLUSTER_ID).build();
driver.register(TEST_AM_HOST, TEST_AM_PORT, TEST_APP_ID);
 ZKClusterCoordinatorDriver driver2 = new ZKClusterCoordinatorDriver()
   .setConnect(connStr, TEST_ZK_ROOT, TEST_CLUSTER_ID).build();
  driver.register(PROBE_AM_HOST, PROBE_AM_PORT, PROBE_APP_ID);
  fail();
 } catch (AMRegistrationException e) {
 driver2.close();
 ZKClusterCoordinatorDriver driver2 = new ZKClusterCoordinatorDriver()
   .setConnect(connStr, TEST_ZK_ROOT, PROBE_CLUSTER_ID).build();
  driver2.register(PROBE_AM_HOST, PROBE_AM_PORT, PROBE_APP_ID);
 } catch (AMRegistrationException e) {
  fail("Registration should be OK");
 driver2.close();
 ZKClusterCoordinatorDriver driver2 = new ZKClusterCoordinatorDriver()
   .setConnect(connStr, PROBE_ZK_ROOT, TEST_CLUSTER_ID).build();
  driver2.register(PROBE_AM_HOST, PROBE_AM_PORT, PROBE_APP_ID);

代码示例来源:origin: apache/drill

/**
 * Validate that the key format used for endpoint is the same as that
 * generated for hosts coming from YARN.
 */
@Test
public void testFormat() {
 DrillbitEndpoint dbe = makeEndpoint(TEST_HOST);
 assertEquals(makeKey(TEST_HOST), ZKClusterCoordinatorDriver.asString(dbe));
 ZKClusterCoordinatorDriver driver = new ZKClusterCoordinatorDriver()
   .setPorts(123, 456, 789);
 assertEquals(makeKey(TEST_HOST), driver.toKey(TEST_HOST));
 // Internal default ports (used mostly for testing.)
 driver = new ZKClusterCoordinatorDriver();
 assertEquals("fred:31010:31011:31012", driver.toKey(FRED_HOST));
}

代码示例来源:origin: apache/drill

/**
 * Called during AM startup to initialize ZK. Checks if any Drillbits are
 * already running. These are "unmanaged" because the AM could not have
 * started them (since they predate the AM.)
 */
public void start(RegistryHandler controller) {
 this.registryHandler = controller;
 try {
  zkDriver.build();
 } catch (ZKRuntimeException e) {
  LOG.error("Failed to start ZK monitoring", e);
  throw new AMWrapperException("Failed to start ZK monitoring", e);
 }
 for (DrillbitEndpoint dbe : zkDriver.getInitialEndpoints()) {
  String key = toKey(dbe);
  registry.put(key, new DrillbitTracker(key, dbe));
  // Blacklist the host for each unmanaged drillbit.
  controller.reserveHost(dbe.getAddress());
  LOG.warn("Host " + dbe.getAddress()
    + " already running a Drillbit outside of YARN.");
 }
 zkDriver.addDrillbitListener(this);
}

代码示例来源:origin: apache/drill

public String toKey(String host) {
 return formatKey(host, userPort, controlPort, dataPort);
}

代码示例来源:origin: apache/drill

/**
 * Convenience method to convert a Drillbit to a string. Note that ZK does not
 * advertise the HTTP port, so it does not appear in the generated string.
 *
 * @param bit
 * @return A string representation of a Drillbit.
 */
public static String asString(DrillbitEndpoint bit) {
 return formatKey(bit.getAddress(), bit.getUserPort(), bit.getControlPort(),
   bit.getDataPort());
}

代码示例来源:origin: apache/drill

bitObj.put("trackingState", task.trackingState);
bitObj.put("endpoint",
  ZKClusterCoordinatorDriver.asString(task.endpoint));
bitObj.put("link", task.getLink());
bitObj.put("startTime", task.getStartTime());

相关文章