可以通过直线连接到hive,但不能通过java错误:无法打开带有jdbc uri的客户端传输gss initiate失败

xqnpmsa8  于 2021-05-29  发布在  Hadoop
关注(0)|答案(1)|浏览(1365)

我可以访问启用了kerberos的远程hdfs集群。当我尝试用直线连接时

beeline -u "jdbc:hive2://host:port/arstel;principal=principal"

连接成功。
但是当我编写一个java应用程序并使用相同的JDBCURI时,我得到

ERROR transport.TSaslTransport: SASL negotiation failure
javax.security.sasl.SaslException: GSS initiate failed [Caused by GSSException: No valid credentials provided (Mechanism level: Failed to find any Kerberos tgt)]
Caused by: GSSException: No valid credentials provided (Mechanism level: Failed to find any Kerberos tgt)
jdbc.HiveConnection: Could not open client transport with JDBC Uri: jdbc:hive2://host:port/arstel;principal=principal
Exception in thread "main" java.sql.SQLException: Could not open client transport with JDBC Uri: jdbc:hive2://host:port/arstel;principal=principal: GSS initiate failed
Caused by: org.apache.thrift.transport.TTransportException: GSS initiate failed

我的应用程序

public class RecordController {
private static String driverName = "org.apache.hive.jdbc.HiveDriver";

public static void main(String[] args) throws SQLException, ClassNotFoundException {
    Class.forName(driverName);
    Connection con = DriverManager
                   .getConnection("jdbc:hive2://host:port/arstel;principal=principal");
    Statement stmt = con.createStatement();
    String tableName = "evkuzmin_testHiveDriverTable";
    stmt.execute("drop table if exists " + tableName);
    stmt.execute("create table " + tableName + " (key int, value string)");
  }
}

为什么会发生这种情况,我该如何解决?
我也试过这个。同样的结果。

getConnection("jdbc:hive2://host:port/arstel;principal=principal;" +
                                        "auth=kerberos;" +
                                        "kerberosAuthType=fromSubject");

编辑
我没有keytab。在阅读了stack和这里的相关内容之后,我创建了一个如下的指令。然后我以这种方式向java文件添加属性

Class.forName(driverName);
System.setProperty("java.security.auth.login.config","gss-jaas.conf");
System.setProperty("sun.security.jgss.debug","true");
System.setProperty("javax.security.auth.useSubjectCredsOnly","false");
System.setProperty("java.security.krb5.conf","krb5.conf");

在我加入 resources gss-jaas.conf文件

com.sun.security.jgss.initiate {
  com.sun.security.auth.module.Krb5LoginModule=required
  useKeyTab=true
  useTicketCache=false
  principal="hive/host@DOMAIN"
  doNotPrompt=true
  keyTab="EvKuzmin.keytab"
  debug=true;
}

和krb5.conf

[libdefaults]
renew_lifetime = 7d
forwardable = true
default_realm = DOMAIN 
ticket_lifetime = 24h
dns_lookup_realm = false
dns_lookup_kdc = false

[realms]
DOMAIN = {
  admin_server = ms-dcs.DOMAIN 
  kdc = ms-dcs.DOMAIN 
}

[domain_realm]
.DOMAIN  = DOMAIN 
DOMAIN  = DOMAIN

然后错误就变了

java.sql.SQLException: Could not create secure connection to jdbc:hive2://host:port/arstel;principal=hive/principal;auth=kerberos;kerberosAuthType=fromSubject: Failed to open client transport
Caused by: javax.security.sasl.SaslException: Failed to open client transport [Caused by java.io.IOException: Could not instantiate SASL transport]
Caused by: java.io.IOException: Could not instantiate SASL transport
Caused by: javax.security.sasl.SaslException: Failure to initialize security context [Caused by GSSException: Invalid name provided (Mechanism level: Cannot locate default realm)]
Caused by: GSSException: Invalid name provided (Mechanism level: Cannot locate default realm)


这里有个键盘

Keytab name: FILE:/etc/krb5.keytab

但我不知道它的领域是什么,也不知道如何使用它+我没有访问该文件夹的权限。
版本2 2017-08-09
尝试通过usergroupinformation执行。出现错误

Exception in thread "main" java.lang.IllegalArgumentException: Invalid attribute value for hadoop.security.authentication of Kerberos
        at org.apache.hadoop.security.UserGroupInformation.initialize(UserGroupInformation.java:200)
        at org.apache.hadoop.security.UserGroupInformation.setConfiguration(UserGroupInformation.java:227)
        at com.hive.connect.controller.RecordController.main(RecordController.java:20)

edit3 2017-08-10使用用户组

public class RecordController {
  private static String driverName = "org.apache.hive.jdbc.HiveDriver";

  public static void main(String[] args) throws SQLException, ClassNotFoundException, IOException {
    Configuration conf = new Configuration();
    conf.set("hadoop.security.authentication", "kerberos");
    UserGroupInformation.setConfiguration(conf);
    UserGroupInformation.loginUserFromKeytab("hive/principal", "EvKuzmin.keytab");
    Class.forName("org.apache.hive.jdbc.HiveDriver");
    Connection con = DriverManager
                         .getConnection("jdbc:hive2://host:port/arstel;" +
                                          "principal=hive/principal;" +
                                          "tez.queue.name=adhoc;" +
                                          "hive.execution.engine=tez;" +
                                          "mapreduce.job.reduces=1;");

错误

java.io.IOException: Login failure for hive/principal from keytab EvKuzmin.keytab
    at org.apache.hadoop.security.UserGroupInformation.loginUserFromKeytab(UserGroupInformation.java:695)
    at com.hive.connect.controller.RecordController.main(RecordController.java:22)
Caused by: javax.security.auth.login.LoginException: Unable to obtain password from user
q8l4jmvw

q8l4jmvw1#

最后我把krb5.conf改成这样

[libdefaults]
renew_lifetime = 7d
forwardable = true
default_realm = DOMAIN 
ticket_lifetime = 24h
dns_lookup_realm = false
dns_lookup_kdc = false

[realms]
DOMAIN = {
  admin_server = ms-dcs.DOMAIN 
  kdc = ms-dcs.DOMAIN 
}

[domain_realm]
.DOMAIN  = DOMAIN 
DOMAIN  = DOMAIN

还有我的java代码

public class RecordController {
  private static String driverName = "org.apache.hive.jdbc.HiveDriver";

  public static void main(String[] args) throws SQLException, ClassNotFoundException, IOException {
    Class.forName(driverName);
    System.setProperty("javax.security.auth.useSubjectCredsOnly","false");
    System.setProperty("java.security.krb5.conf","krb5.conf");

    Connection con = DriverManager
                         .getConnection("jdbc:hive2://host:port/arstel;" +
                                          "principal=hive/host@DOMAIN;" +
                                          "auth=kerberos;" +
                                          "kerberosAuthType=fromSubject");

编辑
当我在集群上启动它时,我不需要输入密码和登录名,但在windows的本地机器上我需要。

相关问题