apache phoenix的json数据接收器出现apache flume错误

n3h0vuf2  于 2021-06-04  发布在  Flume
关注(0)|答案(0)|浏览(446)

我想用ApacheFlume将json数据导入ApachePhoenix,遵循一个在线指南http://kalyanbigdatatraining.blogspot.com/2016/10/how-to-stream-json-data-into-phoenix.html,但遇到以下错误。如何解决?非常感谢!
我的环境列表为:
hadoop-2.7.3版本
hbase-1.3.1版
Phoenix-4.12.0-hbase-1.3-bin
Flume-1.7.0
在flume中,我在$flume\u home/plugins.d/phoenix-sink/lib中添加了与phoenix-sink相关的jar
commons-io-2.4.jar
斜纹-api-0.8.0.jar
twill-discovery-api-0.8.0.jar json-path-2.2.0.jar
斜纹-普通-0.8.0.jar
twill-discovery-core-0.8.0.jarPhoenix-flume-4.12.0-hbase-1.3.jar
斜纹-0.8.0.jar
斜纹-zookeeper-0.8.0.jar
2017-11-11 14:49:54786(lifecyclesupervisor-1-1)[调试-org.apache.phoenix.jdbc.phoenixdriver$2.onremovation(phoenixdriver)。java:159)]到期localhost:2181:/hbase因为明确的2017-11-11 14:49:54,787(lifecyclesupervisor-1-1)[info-org.apache.hadoop.hbase.client.connectionmanager$hconnectionimplementation.closezookeeperwatcher(connectionmanager)。java:1712)]正在关闭zookeeper sessionid=0x15fa8952cea00a6 2017-11-11 14:49:54,787(lifecyclesupervisor-1-1)[debug-org.apache.zookeeper.zookeeper.close(zookeeper。java:673)]结束会话:0x15fa8952cea00a6 2017-11-11 14:49:54787(lifecyclesupervisor-1-1)[调试-org.apache.zookeeper.clientcnxn.close(clientcnxn。java:1306)]正在关闭会话的客户端:0x15fa8952cea00a6 2017-11-11 14:49:54,789(生命周期Supervisor-1-1-sendthread)(localhost:2181))[debug-org.apache.zookeeper.clientcnxn$sendthread.readresponse(clientcnxn。java:818)]阅读回复sessionid:0x15fa8952cea00a6,数据包::clientpath:null serverpath:空finished:false header::3,-11 replyheader::32620,0请求::空响应::空2017-11-11 14:49:54,789(lifecyclesupervisor-1-1)[debug-org.apache.zookeeper.clientcnxn.disconnect(clientcnxn。java:1290)]断开会话的客户端连接:0x15fa8952cea00a6 2017-11-11 14:49:54,789(生命周期Supervisor-1-1-sendthread)(localhost:2181))[debug-org.apache.zookeeper.clientcnxn$sendthread.run(clientcnxn。java:1086)]关闭会话0x15fa8952cea00a6的发送线程时引发异常:无法从服务器会话ID 0x15fa8952cea00a6读取其他数据,可能服务器已关闭套接字2017-11-11 14:49:54,789(lifecyclesupervisor-1-1)[info-org.apache.zookeeper.zookeeper.close(zookeeper。java:684)]会话:0x15fa8952cea00a6关闭2017-11-11 14:49:54789(lifecyclesupervisor-1-1-eventthread)[info-org.apache.zookeeper.clientcnxn$eventthread.run(clientcnxn)。java:512)]eventthread关闭2017-11-11 14:49:54,790(lifecyclesupervisor-1-1)[错误-org.apache.flume.lifecyclesupervisor.lifecyclesupervisor$monitorrunable.run(lifecyclesupervisor。java:251)]无法启动运行程序:{policy:org.apache.flume.sink.defaultsinkprocessor@2d2052a0 计数器组:{name:null counters:{}-异常如下。
java.lang.nosuchmethoderror:org.apache.twill.zookeeper.zkclientservice.startandwait()lcom/google/common/util/concurrent/service$state;

at org.apache.phoenix.transaction.TephraTransactionContext.setTransactionClient(TephraTransactionContext.java:147)
at org.apache.phoenix.query.ConnectionQueryServicesImpl.initTxServiceClient(ConnectionQueryServicesImpl.java:401)
at org.apache.phoenix.query.ConnectionQueryServicesImpl.openConnection(ConnectionQueryServicesImpl.java:415)
at org.apache.phoenix.query.ConnectionQueryServicesImpl.access$500(ConnectionQueryServicesImpl.java:257)
at org.apache.phoenix.query.ConnectionQueryServicesImpl$12.call(ConnectionQueryServicesImpl.java:2384)
at org.apache.phoenix.query.ConnectionQueryServicesImpl$12.call(ConnectionQueryServicesImpl.java:2360)
at org.apache.phoenix.util.PhoenixContextExecutor.call(PhoenixContextExecutor.java:76)
at org.apache.phoenix.query.ConnectionQueryServicesImpl.init(ConnectionQueryServicesImpl.java:2360)
at org.apache.phoenix.jdbc.PhoenixDriver.getConnectionQueryServices(PhoenixDriver.java:255)
at org.apache.phoenix.jdbc.PhoenixEmbeddedDriver.createConnection(PhoenixEmbeddedDriver.java:150)
at org.apache.phoenix.jdbc.PhoenixDriver.connect(PhoenixDriver.java:221)
at java.sql.DriverManager.getConnection(DriverManager.java:664)
at java.sql.DriverManager.getConnection(DriverManager.java:208)
at org.apache.phoenix.flume.serializer.BaseEventSerializer.initialize(BaseEventSerializer.java:140)
at org.apache.phoenix.flume.sink.PhoenixSink.start(PhoenixSink.java:119)
at org.apache.flume.sink.DefaultSinkProcessor.start(DefaultSinkProcessor.java:45)
at org.apache.flume.SinkRunner.start(SinkRunner.java:79)
at org.apache.flume.lifecycle.LifecycleSupervisor$MonitorRunnable.run(LifecycleSupervisor.java:249)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:308)
at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:180)
at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:294)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)

2017-11-11 14:49:54792(lifecyclesupervisor-1-1)[信息-org.apache.flume.instrumentation.monitoredcountergroup.stop(monitoredcountergroup。java:149)]组件类型:Flume,名称:phoenix sink\ u 1 stopped 2017-11-11 14:49:54,792(lifecyclesupervisor-1-1)[info-org.apache.flume.instrumentation.monitoredcountergroup.stop(monitoredcountergroup)。java:155)]类型:sink,名称:phoenix sink\uu 1的关闭度量。sink.start.time==1510382993516
这是我的flume-agent.properties

agent.sources = exec
agent.channels = mem-channel
agent.sinks = phoenix-sink
agent.sources.exec.type = exec
agent.sources.exec.command = tail -F /Users/chenshuai1/tmp/users.json
agent.sources.exec.channels = mem-channel
agent.sinks.phoenix-sink.type = org.apache.phoenix.flume.sink.PhoenixSink
agent.sinks.phoenix-sink.batchSize = 10
agent.sinks.phoenix-sink.zookeeperQuorum = localhost
agent.sinks.phoenix-sink.table = users2
agent.sinks.phoenix-sink.ddl = CREATE TABLE IF NOT EXISTS users2 (userid BIGINT NOT NULL, username VARCHAR, password VARCHAR, email VARCHAR, country VARCHAR, state VARCHAR, city VARCHAR, dt VARCHAR NOT NULL CONSTRAINT PK PRIMARY KEY (userid, dt))
agent.sinks.phoenix-sink.serializer = json
agent.sinks.phoenix-sink.serializer.columnsMapping = {"userid":"userid", "username":"username", "password":"password", "email":"email", "country":"country", "state":"state", "city":"city", "dt":"dt"}
agent.sinks.phoenix-sink.serializer.partialSchema = true
agent.sinks.phoenix-sink.serializer.columns = userid,username,password,email,country,state,city,dt
agent.sinks.phoenix-sink.channel = mem-channel
agent.channels.mem-channel.type = memory
agent.channels.mem-channel.capacity = 1000
agent.channels.mem-channel.transactionCapacity = 100

暂无答案!

目前还没有任何答案,快来回答吧!

相关问题