如何在我的cloudera集群中找到kudu主机名或kududb所在的端口?

pgpifvop  于 2021-06-26  发布在  Impala
关注(0)|答案(3)|浏览(412)

我试图写一个SparkDataframe到kudu数据库,但我不知道kudu主机。我使用的集群是cloudera集群。
如何在集群中找到苦毒大师?

xe55xuns

xe55xuns1#

我知道这不是一个最好的方法,但它是一个快速的方法。假设我们已经有了一个kudu表(即使您没有通过impala创建一个测试/临时表),只需对该表进行一次描述即可。您将得到一堆详细信息,包括kudu master details(主机名),其中端口将是8051。我相信,一旦您了解了主机和端口的详细信息,就可以为您的sparkDataframe探索许多。
临时表语法:
创建表kudu\ no\ partition\ by\ u子句(id bigint primary key,s string,b boolean)存储为kudu;
descripe语法:descripe格式化表\名称;
财年:
kudu网站管理详细信息:https://kudu.apache.org/releases/0.6.0/docs/administration.html
kudu和spark示例:https://kudu.apache.org/docs/developing.html
干杯!!

kd3sttzy

kd3sttzy2#

下面是一个使用cloudera manager java客户端的非常基本的示例(https://cloudera.github.io/cm_api/docs/java-client-swagger/)

package cloudera.kudu_example;

import java.io.IOException;

import com.cloudera.api.swagger.HostsResourceApi;
import com.cloudera.api.swagger.ServicesResourceApi;
import com.cloudera.api.swagger.client.ApiClient;
import com.cloudera.api.swagger.client.ApiException;
import com.cloudera.api.swagger.client.Configuration;
import com.cloudera.api.swagger.model.ApiHost;
import com.cloudera.api.swagger.model.ApiRole;
import com.cloudera.api.swagger.model.ApiRoleList;

public class App {
    public static void main( String[] args ) throws IOException {
        ApiClient cmClient = Configuration.getDefaultApiClient();

        cmClient.setBasePath(args[0]);
        cmClient.setUsername(args[1]);
        cmClient.setPassword(args[2]);

        cmClient.setVerifyingSsl(false);
        HostsResourceApi hostsApiInstance = new HostsResourceApi();
        ServicesResourceApi servicesApiInstance = new ServicesResourceApi();
        try {
            ApiRoleList apiRoles = servicesApiInstance.readRoles("Cluster 1", "KUDU-1");
            for(ApiRole role : apiRoles.getItems()) {
                if(role.getType().equalsIgnoreCase("KUDU_MASTER")) {
                    ApiHost host = hostsApiInstance.readHost(role.getHostRef().getHostId(), "full");
                    System.out.printf("Kudu master runs at %s. IP: %s, status %s", host.getHostname(), host.getIpAddress(), host.getEntityStatus());
                }
            }

        } catch (ApiException e) {
          System.err.println("Exception when calling ClustersResourceApi#readClusters");
          e.printStackTrace();
        }
    }
}
p5cysglq

p5cysglq3#

下面是使用python客户机v3的python示例(https://cloudera.github.io/cm_api/docs/python-client-swagger/):


# !/usr/local/bin/python

import cm_client

# Configure HTTP basic authorization: basic

# configuration = cm_client.Configuration()

cm_client.configuration.username = 'admin'
cm_client.configuration.password = 'admin'

# Create an instance of the API class

api_client = cm_client.ApiClient("http://your-cdh-cluster-cm-host:7180/api/v30")

# create an instance of the ServicesResourceApi class

service_api_instance = cm_client.ServicesResourceApi(api_client)

# create an instance of the HostsResourceApi class

host_api_instance = cm_client.HostsResourceApi(api_client)

# find KUDU_MASTER roles in the CDH cluster

cluster_roles = service_api_instance.read_roles("Cluster 1", "KUDU-1")
for role in cluster_roles.items:
  if role.type == "KUDU_MASTER":
    role_host = host_api_instance.read_host(role.host_ref.host_id, view="full")
    print("Kudu master is located on %s\n" % role_host.hostname)

相关问题