如何修复java.lang.illegalargumentexception:无法识别的hadoop主要版本号:3.1.1

nxagd54h  于 2021-05-27  发布在  Hadoop
关注(0)|答案(0)|浏览(993)

我收到错误“由于无法识别的hadoop主要版本号:3.1.1,无法处理会话;处理器管理性生成1秒:java.lang.illegalargumentexception:无法识别的hadoop主版本号:3.1.1“,用于nifi的自定义处理器。
我在项目中开发了customselecthiveql,具有以下依赖关系。在这个环境中,hadoop的版本是3.1.1.3.1.5.0-152。我尝试了相同的版本,但无法导入此版本的jar。
这里是依赖关系。

<nifi.version>1.11.4</nifi.version>
      <hadoop.version>3.1.1</hadoop.version>
      <hive.version>3.1.0</hive.version>
      </properties>

 <artifactId>nifi-mddof-processors</artifactId>
  <packaging>jar</packaging>

  <dependencies>
      <dependency>
          <groupId>org.apache.nifi</groupId>
          <artifactId>nifi-api</artifactId>
          <version>${nifi.version}</version>
      </dependency>
      <dependency>
          <groupId>org.apache.nifi</groupId>
          <artifactId>nifi-utils</artifactId>
          <version>${nifi.version}</version>
          <scope>provided</scope>
      </dependency>
      <dependency>
          <groupId>org.apache.nifi</groupId>
          <artifactId>nifi-mock</artifactId>
          <version>${nifi.version}</version>
          <scope>test</scope>
      </dependency>
      <dependency>
          <groupId>org.slf4j</groupId>
          <artifactId>slf4j-simple</artifactId>
          <scope>test</scope>
      </dependency>
      <dependency>
          <groupId>junit</groupId>
          <artifactId>junit</artifactId>
          <scope>test</scope>
      </dependency>
      <dependency>
          <groupId>org.apache.nifi</groupId>
          <artifactId>nifi-record</artifactId>
          <scope>compile</scope>
      </dependency>

      <dependency>
          <groupId>org.apache.nifi</groupId>
          <artifactId>nifi-record-serialization-service-api</artifactId>
          <version>${nifi.version}</version>
      </dependency>
      <dependency>
          <groupId>org.apache.nifi</groupId>
          <artifactId>nifi-hadoop-utils</artifactId>
          <exclusions>
              <exclusion>
                  <groupId>org.apache.hadoop</groupId>
                  <artifactId>hadoop-common</artifactId>
              </exclusion>
          </exclusions>
          <version>${nifi.version}</version>
      </dependency>
      <dependency>
          <groupId>org.apache.hadoop</groupId>
          <artifactId>hadoop-common</artifactId>
          <version>${hadoop.version}</version>
          <scope>provided</scope>
      </dependency>
  </dependencies> ```

**Here the logs of the error:**

```2020-09-03 06:24:36,398 WARN [Timer-Driven Process Thread-21] o.a.n.controller.tasks.ConnectableTask Administratively Yielding CustomSelectHiveQL[id=4ef4abad-0174-1000-ffff-ffff913dd5aa] due to uncaught Exception: java.lang.IllegalArgumentException: Unrecognized Hadoop major version number: 3.1.1
java.lang.IllegalArgumentException: Unrecognized Hadoop major version number: 3.1.1
      at org.apache.hadoop.hive.shims.ShimLoader.getMajorVersion(ShimLoader.java:174)
      at org.apache.hadoop.hive.shims.ShimLoader.loadShims(ShimLoader.java:139)
      at org.apache.hadoop.hive.shims.ShimLoader.getHadoopThriftAuthBridge(ShimLoader.java:125)
      at org.apache.hive.service.auth.KerberosSaslHelper.getKerberosTransport(KerberosSaslHelper.java:54)
      at org.apache.hive.jdbc.HiveConnection.createBinaryTransport(HiveConnection.java:445)
      at org.apache.hive.jdbc.HiveConnection.openTransport(HiveConnection.java:201)
      at org.apache.hive.jdbc.HiveConnection.<init>(HiveConnection.java:176)
      at org.apache.hive.jdbc.HiveDriver.connect(HiveDriver.java:105)
      at java.sql.DriverManager.getConnection(DriverManager.java:664)
      at java.sql.DriverManager.getConnection(DriverManager.java:270)
      at com.o2.edh.mddof.processors.customHive.CustomSelectHiveQL.getConnection(CustomSelectHiveQL.java:799)
      at com.o2.edh.mddof.processors.customHive.CustomSelectHiveQL.onTrigger(CustomSelectHiveQL.java:355)
      at org.apache.nifi.processor.AbstractProcessor.onTrigger(AbstractProcessor.java:27)
      at org.apache.nifi.controller.StandardProcessorNode.onTrigger(StandardProcessorNode.java:1176)
      at org.apache.nifi.controller.tasks.ConnectableTask.invoke(ConnectableTask.java:213)
      at org.apache.nifi.controller.scheduling.TimerDrivenSchedulingAgent$1.run(TimerDrivenSchedulingAgent.java:117)
      at org.apache.nifi.engine.FlowEngine$2.run(FlowEngine.java:110)
      at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
      at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:308)
      at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:180)
      at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:294)
      at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
      at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
      at java.lang.Thread.run(Thread.java:748) ```

暂无答案!

目前还没有任何答案,快来回答吧!

相关问题