hive自定义udf jar为其他插入查询提供错误

r7s23pms  于 2021-05-31  发布在  Hadoop
关注(0)|答案(0)|浏览(284)

我正在检查如何在配置单元中创建自定义自定义自定义项。我已经创建了一个自定义自定义自定义项,它什么也不做,只返回给定的文本。
我可以把这个jar装进Hive里而没有任何问题
我也可以从这个jar创建函数,并且可以执行这个函数。
问题
如果添加了这个jar,我就不能从另一个表执行load table/hdfs。以下简单查询失败。

insert into demo1 select * from demo;

堆栈跟踪:

Vertex failed, vertexName=Map 1, vertexId=vertex_1523501275422_0010_3_00, diagnostics=[Task failed, taskId=task_1523501275422_0010_3_00_000000, diagnostics=[TaskAttempt 0 failed, info=[Container container_1523501275422_0010_01_000007 finished with diagnostics set to [Container completed. ]], TaskAttempt 1 failed, info=[Container container_1523501275422_0010_01_000008 finished with diagnostics set to [Container completed. ]], TaskAttempt 2 failed, info=[Container container_1523501275422_0010_01_000009 finished with diagnostics set to [Container completed. ]], TaskAttempt 3 failed, info=[Container container_1523501275422_0010_01_000010 finished with diagnostics set to [Container completed. ]]], Vertex did not succeed due to OWN_TASK_FAILURE, failedTasks:1 killedTasks:0, Vertex vertex_1523501275422_0010_3_00 [Map 1] killed/failed due to:OWN_TASK_FAILURE]
DAG did not succeed due to VERTEX_FAILURE. failedVertices:1 killedVertices:0

请注意,这两个表的结构相同。
如果我删除了jar,那么我就可以毫无问题地执行上面的查询。
code:ca..demo

public class demo extends UDF {

    public String evaluate(String s) {
        if (s == null) {
            return null;
        }else{
            return s;
        }

    }
}

pom.xml文件

<?xml version="1.0" encoding="UTF-8"?>
<project xmlns="http://maven.apache.org/POM/4.0.0"
         xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
         xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
    <modelVersion>4.0.0</modelVersion>

    <groupId>ca.cantire</groupId>
    <artifactId>hive-normalize</artifactId>
    <version>1.0</version>

    <dependencies>
        <!-- https://mvnrepository.com/artifact/org.apache.hive/hive-exec -->
        <dependency>
            <groupId>org.apache.hive</groupId>
            <artifactId>hive-exec</artifactId>
            <version>2.3.3</version>
        </dependency>

    </dependencies>

    <build>
        <pluginManagement>
            <plugins>
                <plugin>
                    <groupId>org.apache.maven.plugins</groupId>
                    <artifactId>maven-surefire-plugin</artifactId>
                    <version>2.8</version>
                </plugin>
                <plugin>
                    <artifactId>maven-assembly-plugin</artifactId>
                    <configuration>
                        <archive>
                            <manifest>
                                <mainClass>ca.abc.demo</mainClass>
                            </manifest>
                        </archive>
                        <descriptorRefs>
                            <descriptorRef>jar-with-dependencies</descriptorRef>
                        </descriptorRefs>
                    </configuration>
                </plugin>
            </plugins>
        </pluginManagement>
    </build>
</project>

暂无答案!

目前还没有任何答案,快来回答吧!

相关问题