nifi puthivestreaming处理器:连接到终结点失败

tyg4sfes  于 2021-06-26  发布在  Hive
关注(0)|答案(0)|浏览(342)

有人会用nifi1.3.0和hive在这个问题上提供帮助。我在hive1.2和hive2.1.1中得到了相同的错误。配置单元表以orc格式进行分区、装箱和存储。
分区是在hdfs上创建的,但数据在写入阶段失败。请检查以下日志:

[5:07 AM] papesdiop: Failed connecting to EndPoint {metaStoreUri='thrift://localhost:9083', database='mydb', table='guys', partitionVals=[dev] }
[5:13 AM] papesdiop: I get in log see next, hope it might help too:
[5:13 AM] papesdiop: Caused by: org.apache.hive.hcatalog.streaming.TransactionError: Unable to acquire lock on {metaStoreUri='thrift://localhost:9083', database='mydb', table='guys', partitionVals=[dev] }
  at org.apache.hive.hcatalog.streaming.HiveEndPoint$TransactionBatchImpl.beginNextTransactionImpl(HiveEndPoint.java:578)

完整跟踪日志:
重新连接。org.apache.thrift.transport.ttTransportException:org.apache.thrift.transport.tiostreamtransport.read(tiostreamtransport)处为空。java:132)在org.apache.thrift.transport.tttransport.readall(tttransport。java:86)在org.apache.thrift.protocol.tbinaryprotocol.readall(tbinaryprotocol。java:429)在org.apache.thrift.protocol.tbinaryprotocol.readi32(tbinaryprotocol。java:318)在org.apache.thrift.protocol.tbinaryprotocol.readmessagebegin(tbinaryprotocol。java:219)位于org.apache.thrift.tserviceclient.receivebase(tserviceclient。java:69)位于org.apache.hadoop.hive.metastore.api.thrifthivemetastore$client.recv\u lock(thrifthivemetastore)。java:3906)在org.apache.hadoop.hive.metastore.api.thrifthivemetastore$client.lock(thrifthivemetastore)。java:3893)位于org.apache.hadoop.hive.metastore.hivemetastoreclient.lock(hivemetastoreclient)。java:1863)位于sun.reflect.generatedmethodaccessor380.invoke(未知源)位于sun.reflect.delegatingmethodaccessorimpl.invoke(delegatingmethodaccessorimpl)。java:43)在java.lang.reflect.method.invoke(方法。java:498)位于org.apache.hadoop.hive.metastore.retryingmetastoreclient.invoke(retryingmetastoreclient。java:152)位于com.sun.proxy.$proxy126.lock(未知源),位于org.apache.hive.hcatalog.streaming.hiveendpoint$transactionbatchimpl.beginnexttransactionimpl(hiveendpoint)。java:573)在org.apache.hive.hcatalog.streaming.hiveendpoint$transactionbatchimpl.beginnexttransaction(hiveendpoint)。java:547)位于org.apache.nifi.util.hive.hivewriter.nexttxnbatch(hivewriter。java:261)在org.apache.nifi.util.hive.hivewriter上。java:73)在org.apache.nifi.util.hive.hiveutils.makehivewriter(hiveutils。java:46)在org.apache.nifi.processors.hive.puthivestreaming.makehivewriter(puthivestreaming)。java:964)在org.apache.nifi.processors.hive.puthivestreaming.getorcreatewriter(puthivestreaming。java:875)在org.apache.nifi.processors.hive.puthivestreaming.lambda$null$40(puthivestreaming)。java:676)在org.apache.nifi.processor.util.pattern.exceptionhandler.execute(exceptionhandler。java:127)在org.apache.nifi.processors.hive.puthivestreaming.lambda$ontrigger$44(puthivestreaming。java:673)在org.apache.nifi.controller.repository.standardprocesssession.read(standardprocesssession。java:2136)在org.apache.nifi.controller.repository.standardprocesssession.read(standardprocesssession。java:2106)位于org.apache.nifi.processors.hive.puthivestreaming.ontrigger(puthivestreaming.com)。java:627)在org.apache.nifi.processors.hive.puthivestreaming.lambda$ontrigger$36(puthivestreaming。java:551)在org.apache.nifi.processor.util.pattern.partialfunctions.ontrigger(partialfunctions。java:114)位于org.apache.nifi.processor.util.pattern.rollbackonfailure.ontrigger(rollbackonfailure)。java:184)位于org.apache.nifi.processors.hive.puthivestreaming.ontrigger(puthivestreaming.com)。java:551)在org.apache.nifi.controller.standardprocessornode.ontrigger(standardprocessornode。java:1120)在org.apache.nifi.controller.tasks.continuallyrunprocessortask.call(continuallyrunprocessortask。java:147)在org.apache.nifi.controller.tasks.continuallyrunprocessortask.call(continuallyrunprocessortask。java:47)在org.apache.nifi.controller.scheduling.timerdrivenschedulingagent$1.run(timerdrivenschedulingagent。java:132)在java.util.concurrent.executors$runnableadapter.call(executors。java:511)在java.util.concurrent.futuretask.runandreset(futuretask。java:308)在java.util.concurrent.scheduledthreadpoolexecutor$scheduledfuturetask.access$301(scheduledthreadpoolexecutor。java:180)在java.util.concurrent.scheduledthreadpoolexecutor$scheduledfuturetask.run(scheduledthreadpoolexecutor。java:294)位于java.util.concurrent.threadpoolexecutor.runworker(threadpoolexecutor。java:1149)在java.util.concurrent.threadpoolexecutor$worker.run(threadpoolexecutor。java:624)在java.lang.thread.run(线程。java:748)2017-09-07 06:41:31015调试[timer-4]o.a.n.processors.hive.puthivestreaming puthivestreaming[id=13ed53d2-015e-1000-c7b1-5af434c38751]开始在所有写入程序上发送心跳2017-09-07 06:41:31,890 info[timer driven process thread-3]hive.metastore正在尝试连接到uri为的metastorethrift://localhost:9083 2017-09-07 06:41:31893 info[计时器驱动的进程线程-3]hive.metastore已连接到metastore。2017-09-07 06:41:31911错误[timer-driven process thread-3]o.a.n.processors.hive.puthivestreaming puthivestreaming[id=13ed53d2-015e-1000-c7b1-5af434c38751]无法为终结点创建hivewriter:{metastoreuri='thrift://localhost:9083',database='default',table='guys',partitionvals=[dev]}:org.apache.nifi.util.hive.hivewriter$connectfailure:连接到端点{metastoreuri='thrift://localhost:9083',database='default',table='guys',partitionvals=[dev]}org.apache.nifi.util.hive.hivewriter$connectfailure:连接到端点{metastoreuri='thrift://localhost:9083',database='default',table='guys',partitionvals=[dev]}位于org.apache.nifi.util.hive.hivewriter.(hivewriter。java:79)在org.apache.nifi.util.hive.hiveutils.makehivewriter(hiveutils。java:46)在org.apache.nifi.processors.hive.puthivestreaming.makehivewriter(puthivestreaming。java:964)在org.apache.nifi.processors.hive.puthivestreaming.getorcreatewriter(puthivestreaming。java:875) 在org.apache.nifi.processors.hive.puthivestreaming.lambda$null$40(puthivestreaming)。java:676)位于org.apache.nifi.processor.util.pattern.exceptionhandler.execute(exceptionhandler)。java:127)在org.apache.nifi.processors.hive.puthivestreaming.lambda$ontrigger$44(puthivestreaming。java:673)在org.apache.nifi.controller.repository.standardprocesssession.read(standardprocesssession。java:2136)在org.apache.nifi.controller.repository.standardprocesssession.read(standardprocesssession。java:2106)位于org.apache.nifi.processors.hive.puthivestreaming.ontrigger(puthivestreaming.com)。java:627)在org.apache.nifi.processors.hive.puthivestreaming.lambda$ontrigger$36(puthivestreaming。java:551)位于org.apache.nifi.processor.util.pattern.partialfunctions.ontrigger(partialfunctions。java:114)位于org.apache.nifi.processor.util.pattern.rollbackonfailure.ontrigger(rollbackonfailure)。java:184)在org.apache.nifi.processors.hive.puthivestreaming.ontrigger(puthivestreaming。java:551)位于org.apache.nifi.controller.standardprocessornode.ontrigger(standardprocessornode。java:1120)在org.apache.nifi.controller.tasks.continuallyrunprocessortask.call(continuallyrunprocessortask。java:147)在org.apache.nifi.controller.tasks.continuallyrunprocessortask.call(continuallyrunprocessortask。java:47)在org.apache.nifi.controller.scheduling.timerdrivenschedulingagent$1.run(timerdrivenschedulingagent。java:132)在java.util.concurrent.executors$runnableadapter.call(executors。java:511)在java.util.concurrent.futuretask.runandreset(futuretask。java:308)在java.util.concurrent.scheduledthreadpoolexecutor$scheduledfuturetask.access$301(scheduledthreadpoolexecutor。java:180)在java.util.concurrent.scheduledthreadpoolexecutor$scheduledfuturetask.run(scheduledthreadpoolexecutor。java:294)在java.util.concurrent.threadpoolexecutor.runworker(threadpoolexecutor。java:1149)在java.util.concurrent.threadpoolexecutor$worker.run(threadpoolexecutor。java:624)在java.lang.thread.run(线程。java:748)原因:org.apache.nifi.util.hive.hivewriter$txnbatchfailure:无法从终结点获取事务批处理:{metastoreuri='thrift://localhost:9083',database='default',table='guys',partitionvals=[dev]}位于org.apache.nifi.util.hive.hivewriter.nexttxnbatch(hivewriter)。java:264)在org.apache.nifi.util.hive.hivewriter上。java:73) ... 24公共帧被省略,原因是:org.apache.hive.hcatalog.streaming.transactionerror:无法获取{metastoreuri='thrift://localhost:9083',database='default',table='guys',partitionvals=[dev]}位于org.apache.hive.hcatalog.streaming.hiveendpoint$transactionbatchimpl.beginnexttransactionimpl(hiveendpoint)。java:578)在org.apache.hive.hcatalog.streaming.hiveendpoint$transactionbatchimpl.beginnexttransaction(hiveendpoint。java:547)位于org.apache.nifi.util.hive.hivewriter.nexttxnbatch(hivewriter。java:261) ... 25由于org.apache.thrift.transport.ttTransportException:null位于org.apache.thrift.transport.tiostreamtransport.read(tiostreamtransport)而忽略的公共帧。java:132)在org.apache.thrift.transport.tttransport.readall(tttransport。java:86)在org.apache.thrift.protocol.tbinaryprotocol.readall(tbinaryprotocol。java:429)在org.apache.thrift.protocol.tbinaryprotocol.readi32(tbinaryprotocol。java:318)在org.apache.thrift.protocol.tbinaryprotocol.readmessagebegin(tbinaryprotocol。java:219)位于org.apache.thrift.tserviceclient.receivebase(tserviceclient。java:69)位于org.apache.hadoop.hive.metastore.api.thrifthivemetastore$client.recv\u lock(thrifthivemetastore)。java:3906)在org.apache.hadoop.hive.metastore.api.thrifthivemetastore$client.lock(thrifthivemetastore)。java:3893)位于org.apache.hadoop.hive.metastore.hivemetastoreclient.lock(hivemetastoreclient)。java:1863)位于sun.reflect.generatedmethodaccessor380.invoke(未知源)位于sun.reflect.delegatingmethodaccessorimpl.invoke(delegatingmethodaccessorimpl)。java:43)在java.lang.reflect.method.invoke(方法。java:498)位于org.apache.hadoop.hive.metastore.retryingmetastoreclient.invoke(retryingmetastoreclient。java:152)位于com.sun.proxy.$proxy126.lock(未知源),位于org.apache.hive.hcatalog.streaming.hiveendpoint$transactionbatchimpl.beginnexttransactionimpl(hiveendpoint)。java:573) ... 27公共帧省略2017-09-07 06:41:31911错误[timer-driven process thread-3]o.a.n.processors.hive.puthivestreaming puthivestreaming[id=13ed53d2-015e-1000-c7b1-5af434c38751]连接到hive终结点时出错:表thrift://localhost:9083 2017-09-07 06:41:31,911 debug[timer driven process thread-3]o.a.n.processors.hive.puthivestreaming puthivestreaming[id=13ed53d2-015e-1000-c7b1-5af434c38751]已选择生成其资源;不会是sch

暂无答案!

目前还没有任何答案,快来回答吧!

相关问题