我正在使用confluent将数据从kafka导入到hive,尝试执行与此相同的操作:基于时间的桶记录(kafka hdfs连接器)
我的Flume配置如下:
{
"name":"yangfeiran_hive_sink_9",
"config":{
"connector.class":"io.confluent.connect.hdfs.HdfsSinkConnector",
"topics":"peoplet_people_1000",
"name":"yangfeiran_hive_sink_9",
"tasks.max":"1",
"hdfs.url":"hdfs://master:8020",
"flush.size":"3",
"partitioner.class":"io.confluent.connect.hdfs.partitioner.TimeBasedPartitioner",
"partition.duration.ms":"300000",
"path.format":"'year'=YYYY/'month'=MM/'day'=dd/'hour'=HH/'minute'=mm/",
"locale":"en",
"logs.dir":"/tmp/yangfeiran",
"topics.dir":"/tmp/yangfeiran",
"hive.integration":"true",
"hive.metastore.uris":"thrift://master:9083",
"schema.compatibility":"BACKWARD",
"hive.database":"yangfeiran",
"timezone": "UTC",
}
}
一切正常,我可以看到数据在hdfs中,表是在hive中创建的,除非我使用“select*from yang”检查数据是否已经在hive中。
它打印错误:
失败:semanticexception无法确定hdfs://master:8020/tmp/yangfeiran/peoplet\u people\u 1000已加密:java.lang.illegalargumentexception:wrong fs:hdfs://master:8020/tmp/yangfeiran/peoplet1000,预计:hdfs://nsstargate
如何解决这个问题?
飞然
暂无答案!
目前还没有任何答案,快来回答吧!