合流Kafkahdfs连接器Hive

xqkwcwgp  于 2021-06-07  发布在  Kafka
关注(0)|答案(0)|浏览(166)

我正在使用confluent将数据从kafka导入到hive,尝试执行与此相同的操作:基于时间的桶记录(kafka hdfs连接器)
我的Flume配置如下:

{
    "name":"yangfeiran_hive_sink_9",
    "config":{
        "connector.class":"io.confluent.connect.hdfs.HdfsSinkConnector",
        "topics":"peoplet_people_1000",
        "name":"yangfeiran_hive_sink_9",
        "tasks.max":"1",
        "hdfs.url":"hdfs://master:8020",
        "flush.size":"3",
        "partitioner.class":"io.confluent.connect.hdfs.partitioner.TimeBasedPartitioner",
        "partition.duration.ms":"300000",
        "path.format":"'year'=YYYY/'month'=MM/'day'=dd/'hour'=HH/'minute'=mm/",
        "locale":"en",
        "logs.dir":"/tmp/yangfeiran",
        "topics.dir":"/tmp/yangfeiran",
        "hive.integration":"true",
        "hive.metastore.uris":"thrift://master:9083",
        "schema.compatibility":"BACKWARD",
        "hive.database":"yangfeiran",
        "timezone": "UTC",
    }
}

一切正常,我可以看到数据在hdfs中,表是在hive中创建的,除非我使用“select*from yang”检查数据是否已经在hive中。
它打印错误:
失败:semanticexception无法确定hdfs://master:8020/tmp/yangfeiran/peoplet\u people\u 1000已加密:java.lang.illegalargumentexception:wrong fs:hdfs://master:8020/tmp/yangfeiran/peoplet1000,预计:hdfs://nsstargate
如何解决这个问题?
飞然

暂无答案!

目前还没有任何答案,快来回答吧!

相关问题