我正在尝试连接h2o和kerberized hive，如下文件所示：http://docs.h2o.ai/h2o/latest-stable/h2o-docs/getting-data-into-h2o.html#connecting-to-hive-in-a-kerberized-hadoop-cluster集群
我确信hivehost、hiveprincipal、principal、keytab都配置得很好。
这是我的环境：
hadoop 2.7版
Hive2.2
h2o-3.26.0.2-hdp2.6.zip和h2o-3.26.0.2-hdp3.0.zip（尝试了hdp2.6和hdp3.0）
以下是错误消息：
08-08 14:08:51.819 192.168.6.33:54321 15189 #dex.html INFO: GET /flow/index.html, parms: {}
08-08 14:08:51.819 192.168.6.33:54321 15189 #dex.html INFO: Trying to load resource /flow/index.html via classloader sun.misc.Launcher$AppClassLoader@7e32c033
08-08 14:08:51.870 192.168.6.33:54321 15189 #flow.css INFO: Trying to load resource /flow/css/flow.css via classloader sun.misc.Launcher$AppClassLoader@7e32c033
08-08 14:08:51.933 192.168.6.33:54321 15189 #/flow.js INFO: Trying to load resource /flow/js/flow.js via classloader sun.misc.Launcher$AppClassLoader@7e32c033
08-08 14:08:52.054 192.168.6.33:54321 15189 #/h2o.png INFO: Trying to load resource /flow/img/h2o.png via classloader sun.misc.Launcher$AppClassLoader@7e32c033
08-08 14:08:52.083 192.168.6.33:54321 15189 #ar.woff2 INFO: GET /flow/fonts/Lato-Regular.woff2, parms: {}
08-08 14:08:52.083 192.168.6.33:54321 15189 #ar.woff2 INFO: Trying to load resource /flow/fonts/Lato-Regular.woff2 via classloader sun.misc.Launcher$AppClassLoader@7e32c033
08-08 14:08:52.094 192.168.6.33:54321 15189 #00.woff2 INFO: GET /flow/fonts/fa-solid-900.woff2, parms: {}
08-08 14:08:52.095 192.168.6.33:54321 15189 #00.woff2 INFO: Trying to load resource /flow/fonts/fa-solid-900.woff2 via classloader sun.misc.Launcher$AppClassLoader@7e32c033
08-08 14:08:52.101 192.168.6.33:54321 15189 #tf.woff2 INFO: GET /flow/fonts/SourceCodePro-Regular.ttf.woff2, parms: {}
08-08 14:08:52.101 192.168.6.33:54321 15189 #tf.woff2 INFO: Trying to load resource /flow/fonts/SourceCodePro-Regular.ttf.woff2 via classloader sun.misc.Launcher$AppClassLoader@7e32c033
08-08 14:08:52.597 192.168.6.33:54321 15189 #dex.html INFO: GET /flow/index.html, parms: {}
08-08 14:08:52.597 192.168.6.33:54321 15189 #dex.html INFO: Trying to load resource /flow/index.html via classloader sun.misc.Launcher$AppClassLoader@7e32c033
08-08 14:08:52.654 192.168.6.33:54321 15189 #ndpoints INFO: GET /3/Metadata/endpoints, parms: {}
08-08 14:08:52.859 192.168.6.33:54321 15189 #notebook INFO: GET /3/NodePersistentStorage/notebook, parms: {}
08-08 14:08:52.862 192.168.6.33:54321 15189 #s/exists INFO: GET /3/NodePersistentStorage/categories/environment/names/clips/exists, parms: {}
08-08 14:08:52.862 192.168.6.33:54321 15189 #log.json INFO: GET /flow/help/catalog.json, parms: {}
08-08 14:08:52.862 192.168.6.33:54321 15189 #log.json INFO: Trying to load resource /flow/help/catalog.json via classloader sun.misc.Launcher$AppClassLoader@7e32c033
08-08 14:08:52.863 192.168.6.33:54321 15189 #/3/About INFO: GET /3/About, parms: {}
08-08 14:08:52.867 192.168.6.33:54321 15189 #Builders INFO: GET /3/ModelBuilders, parms: {}
08-08 14:08:52.889 192.168.6.33:54321 15189 #00.woff2 INFO: GET /flow/fonts/fa-regular-400.woff2, parms: {}
08-08 14:08:52.889 192.168.6.33:54321 15189 #00.woff2 INFO: Trying to load resource /flow/fonts/fa-regular-400.woff2 via classloader sun.misc.Launcher$AppClassLoader@7e32c033
08-08 14:08:52.891 192.168.6.33:54321 15189 #flow.png INFO: Trying to load resource /flow/img/h2o-flow.png via classloader sun.misc.Launcher$AppClassLoader@7e32c033
08-08 14:08:53.035 192.168.6.33:54321 15189 #ic.woff2 INFO: GET /flow/fonts/Lato-Italic.woff2, parms: {}
08-08 14:08:53.035 192.168.6.33:54321 15189 #ic.woff2 INFO: Trying to load resource /flow/fonts/Lato-Italic.woff2 via classloader sun.misc.Launcher$AppClassLoader@7e32c033
08-08 14:08:58.454 192.168.6.33:54321 15189 #SQLTable INFO: POST /99/ImportSQLTable, parms: <hidden>
08-08 14:08:58.472 192.168.6.33:54321 15189 FJ-1-15 ERRR: java.lang.RuntimeException: SQLException: Could not open client transport with JDBC Uri: jdbc:hive2://hive.linktime.cloud:10000/default;auth=delegationToken: java.lang.RuntimeException: class org.apache.hadoop.security.JniBasedUnixGroupsMappingWithFallback not org.apache.hive.org.apache.hadoop.security.GroupMappingServiceProvider
08-08 14:08:58.472 192.168.6.33:54321 15189 FJ-1-15 ERRR: Failed to connect and read from SQL database with connection_url: jdbc:hive2://hive.linktime.cloud:10000/default;auth=delegationToken
08-08 14:08:58.472 192.168.6.33:54321 15189 FJ-1-15 ERRR: at water.jdbc.SQLManager$SQLImportDriver.compute2(SQLManager.java:225)
08-08 14:08:58.472 192.168.6.33:54321 15189 FJ-1-15 ERRR: at water.H2O$H2OCountedCompleter.compute(H2O.java:1417)
08-08 14:08:58.472 192.168.6.33:54321 15189 FJ-1-15 ERRR: at jsr166y.CountedCompleter.exec(CountedCompleter.java:468)
08-08 14:08:58.472 192.168.6.33:54321 15189 FJ-1-15 ERRR: at jsr166y.ForkJoinTask.doExec(ForkJoinTask.java:263)
08-08 14:08:58.472 192.168.6.33:54321 15189 FJ-1-15 ERRR: at jsr166y.ForkJoinPool$WorkQueue.runTask(ForkJoinPool.java:974)
08-08 14:08:58.472 192.168.6.33:54321 15189 FJ-1-15 ERRR: at jsr166y.ForkJoinPool.runWorker(ForkJoinPool.java:1477)
08-08 14:08:58.472 192.168.6.33:54321 15189 FJ-1-15 ERRR: at jsr166y.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:104)
08-08 14:08:58.472 192.168.6.33:54321 15189 FJ-1-15 ERRR: Caused by: java.sql.SQLException: Could not open client transport with JDBC Uri: jdbc:hive2://hive.linktime.cloud:10000/default;auth=delegationToken: java.lang.RuntimeException: class org.apache.hadoop.security.JniBasedUnixGroupsMappingWithFallback not org.apache.hive.org.apache.hadoop.security.GroupMappingServiceProvider
08-08 14:08:58.472 192.168.6.33:54321 15189 FJ-1-15 ERRR: at org.apache.hive.jdbc.HiveConnection.<init>(HiveConnection.java:210)
08-08 14:08:58.472 192.168.6.33:54321 15189 FJ-1-15 ERRR: at org.apache.hive.jdbc.HiveDriver.connect(HiveDriver.java:107)
08-08 14:08:58.472 192.168.6.33:54321 15189 FJ-1-15 ERRR: at java.sql.DriverManager.getConnection(DriverManager.java:664)
08-08 14:08:58.472 192.168.6.33:54321 15189 FJ-1-15 ERRR: at java.sql.DriverManager.getConnection(DriverManager.java:247)
08-08 14:08:58.472 192.168.6.33:54321 15189 FJ-1-15 ERRR: at water.jdbc.SQLManager.getConnectionSafe(SQLManager.java:486)
08-08 14:08:58.472 192.168.6.33:54321 15189 FJ-1-15 ERRR: at water.jdbc.SQLManager.access$000(SQLManager.java:13)
08-08 14:08:58.472 192.168.6.33:54321 15189 FJ-1-15 ERRR: at water.jdbc.SQLManager$SQLImportDriver.compute2(SQLManager.java:134)
08-08 14:08:58.472 192.168.6.33:54321 15189 FJ-1-15 ERRR: ... 6 more
08-08 14:08:58.472 192.168.6.33:54321 15189 FJ-1-15 ERRR: Caused by: java.lang.RuntimeException: java.lang.RuntimeException: class org.apache.hadoop.security.JniBasedUnixGroupsMappingWithFallback not org.apache.hive.org.apache.hadoop.security.GroupMappingServiceProvider
08-08 14:08:58.472 192.168.6.33:54321 15189 FJ-1-15 ERRR: at org.apache.hive.org.apache.hadoop.conf.Configuration.getClass(Configuration.java:2248)
08-08 14:08:58.472 192.168.6.33:54321 15189 FJ-1-15 ERRR: at org.apache.hive.org.apache.hadoop.security.Groups.<init>(Groups.java:106)
08-08 14:08:58.472 192.168.6.33:54321 15189 FJ-1-15 ERRR: at org.apache.hive.org.apache.hadoop.security.Groups.<init>(Groups.java:102)
08-08 14:08:58.472 192.168.6.33:54321 15189 FJ-1-15 ERRR: at org.apache.hive.org.apache.hadoop.security.Groups.getUserToGroupsMappingService(Groups.java:450)
08-08 14:08:58.472 192.168.6.33:54321 15189 FJ-1-15 ERRR: at org.apache.hive.org.apache.hadoop.security.UserGroupInformation.initialize(UserGroupInformation.java:309)
08-08 14:08:58.472 192.168.6.33:54321 15189 FJ-1-15 ERRR: at org.apache.hive.org.apache.hadoop.security.UserGroupInformation.ensureInitialized(UserGroupInformation.java:276)
08-08 14:08:58.472 192.168.6.33:54321 15189 FJ-1-15 ERRR: at org.apache.hive.org.apache.hadoop.security.UserGroupInformation.loginUserFromSubject(UserGroupInformation.java:832)
08-08 14:08:58.472 192.168.6.33:54321 15189 FJ-1-15 ERRR: at org.apache.hive.org.apache.hadoop.security.UserGroupInformation.getLoginUser(UserGroupInformation.java:802)
08-08 14:08:58.472 192.168.6.33:54321 15189 FJ-1-15 ERRR: at org.apache.hive.org.apache.hadoop.security.UserGroupInformation.getCurrentUser(UserGroupInformation.java:675)
08-08 14:08:58.472 192.168.6.33:54321 15189 FJ-1-15 ERRR: at org.apache.hive.org.apache.hadoop.hive.shims.Utils.getTokenStrForm(Utils.java:91)
08-08 14:08:58.472 192.168.6.33:54321 15189 FJ-1-15 ERRR: at org.apache.hive.jdbc.HiveConnection.getClientDelegationToken(HiveConnection.java:539)
08-08 14:08:58.472 192.168.6.33:54321 15189 FJ-1-15 ERRR: at org.apache.hive.jdbc.HiveConnection.createBinaryTransport(HiveConnection.java:464)
08-08 14:08:58.472 192.168.6.33:54321 15189 FJ-1-15 ERRR: at org.apache.hive.jdbc.HiveConnection.openTransport(HiveConnection.java:226)
08-08 14:08:58.472 192.168.6.33:54321 15189 FJ-1-15 ERRR: at org.apache.hive.jdbc.HiveConnection.<init>(HiveConnection.java:183)
08-08 14:08:58.472 192.168.6.33:54321 15189 FJ-1-15 ERRR: ... 12 more
08-08 14:08:58.472 192.168.6.33:54321 15189 FJ-1-15 ERRR: Caused by: java.lang.RuntimeException: class org.apache.hadoop.security.JniBasedUnixGroupsMappingWithFallback not org.apache.hive.org.apache.hadoop.security.GroupMappingServiceProvider
08-08 14:08:58.472 192.168.6.33:54321 15189 FJ-1-15 ERRR: at org.apache.hive.org.apache.hadoop.conf.Configuration.getClass(Configuration.java:2242)
08-08 14:08:58.472 192.168.6.33:54321 15189 FJ-1-15 ERRR: ... 25 more
08-08 14:09:07.241 192.168.6.33:54321 15189 #les/info INFO: GET /3/Logs/nodes/self/files/info, parms: {}
08-08 14:09:07.334 192.168.6.33:54321 15189 #les/info INFO: GET /3/Logs/nodes/192.168.6.33:54321/files/info, parms: {}我错过了一些配置吗？
1条答案
按热度按时间vsaztqbk1#
您是作为hadoop作业运行h2o还是在hadoop之外独立运行h2o？delegationtoken auth仅在h2o作为hadoop作业运行时使用。
如果您是独立运行的，则需要使用kerberos身份验证jdbc:hive2://host:10000/default;主体=Hive/host@realm.com.
如果您是通过h2o\u driver.jar作为hadoop作业运行的，那么您需要确保hive位于mapreduce类路径上，可以通过libjars，也可以通过hadoop conf。文档中提供了更多详细信息。