我在cdh版本5.7.0上使用流集
从hdfs(源)加载文件并在solr(目标)上创建记录的示例工作流。验证失败-solr\u 03-无法连接到solr示例:org.apache.solr.common.cloud.zookeeperexception::zookeeper连接字符串。
但是,从日志中,我看到它可以连接到zookeeper,但是,在调用/live\u nodes zookeeper api之后,它以某种方式关闭了会话。
2016-06-03 16:30:21,336 INFO org.apache.solr.common.cloud.ConnectionManager: Client is connected to ZooKeeper
2016-06-03 16:30:21,336 INFO org.apache.solr.common.cloud.SolrZkClient: Using default ZkACLProvider
2016-06-03 16:30:21,338 DEBUG org.apache.zookeeper.ClientCnxn: Reading reply sessionid:0x355187691370039, packet:: clientPath:null serverPath:null finished:false header:: 1,3 replyHeader:: 1,42949673777,0 request:: '/clusterstate.json,F response:: s{38654846788,38654846788,1464989414801,1464989414801,0,0,0,0,0,0,38654846788}
2016-06-03 16:30:21,339 DEBUG org.apache.zookeeper.ClientCnxn: Reading reply sessionid:0x355187691370039, packet:: clientPath:null serverPath:null finished:false header:: 2,3 replyHeader:: 2,42949673777,0 request:: '/aliases.json,F response:: s{38654846789,38654846789,1464989414814,1464989414814,0,0,0,0,0,0,38654846789}
2016-06-03 16:30:21,339 INFO org.apache.solr.common.cloud.ZkStateReader: Updating cluster state from ZooKeeper...
2016-06-03 16:30:21,339 DEBUG org.apache.zookeeper.ClientCnxn: Reading reply sessionid:0x355187691370039, packet:: clientPath:null serverPath:null finished:false header:: 3,3 replyHeader:: 3,42949673777,0 request:: '/clusterstate.json,T response:: s{38654846788,38654846788,1464989414801,1464989414801,0,0,0,0,0,0,38654846788}
2016-06-03 16:30:21,340 DEBUG org.apache.zookeeper.ClientCnxn: Reading reply sessionid:0x355187691370039, packet:: clientPath:null serverPath:null finished:false header:: 4,8 replyHeader:: 4,42949673777,-101 request:: '/live_nodes,T response:: v{}
2016-06-03 16:30:21,340 DEBUG org.apache.zookeeper.ZooKeeper: Closing session: 0x355187691370039
2016-06-03 16:30:21,340 DEBUG org.apache.zookeeper.ClientCnxn: Closing client for session: 0x355187691370039
2016-06-03 16:30:21,352 DEBUG org.apache.zookeeper.ClientCnxn: Reading reply sessionid:0x355187691370039, packet:: clientPath:null serverPath:null finished:false header:: 5,-11 replyHeader:: 5,42949673778,0 request:: null response:: null
2016-06-03 16:30:21,352 DEBUG org.apache.zookeeper.ClientCnxn: Disconnecting client for session: 0x355187691370039
2016-06-03 16:30:21,352 INFO org.apache.zookeeper.ZooKeeper: Session: 0x355187691370039 closed
2016-06-03 16:30:21,352 INFO org.apache.zookeeper.ClientCnxn: EventThread shut down
2016-06-03 16:30:21,352 INFO com.streamsets.pipeline.cluster.ControlChannel: Producer complete
2016-06-03 16:30:21,352 DEBUG com.streamsets.datacollector.stagelibrary.ClassLoaderStageLibraryTask: Returning private ClassLoader for 'streamsets-datacollector-cdh_5_7-lib'
2016-06-03 16:30:21,352 DEBUG com.streamsets.datacollector.stagelibrary.ClassLoaderStageLibraryTask: Returned a private ClassLoader for 'streamsets-datacollector-cdh_5_7-lib', active private ClassLoaders='0'
2016-06-03 16:30:21,353 DEBUG org.apache.http.impl.conn.PoolingClientConnectionManager: Connection manager is shutting down
2016-06-03 16:30:21,353 DEBUG org.apache.http.impl.conn.PoolingClientConnectionManager: Connection manager shut down
2016-06-03 16:30:21,353 DEBUG com.streamsets.datacollector.execution.manager.standalone.StandaloneAndClusterPipelineManager: Status of previewer with id: '0bec83af-c152-4519-98f4-201394bebc36' changed to status: 'INVALID'
2016-06-03 16:30:21,353 INFO com.streamsets.pipeline.cluster.ControlChannel: Producer complete
2016-06-03 16:30:21,353 DEBUG com.streamsets.datacollector.stagelibrary.ClassLoaderStageLibraryTask: Returning private ClassLoader for 'streamsets-datacollector-cdh_5_7-lib'
2016-06-03 16:30:21,353 WARN com.streamsets.datacollector.stagelibrary.ClassLoaderStageLibraryTask: Could not return a private ClassLoader for 'streamsets-datacollector-cdh_5_7-lib', active private ClassLoaders='0'
2016-06-03 16:30:21,353 WARN com.streamsets.datacollector.runner.Pipeline: Exception thrown during pipe 'StagePipe[instance='com_streamsets_pipeline_stage_origin_hdfs_cluster_ClusterHdfsDSource_01' input='[]' output='[com_streamsets_pipeline_stage_origin_hdfs_cluster_ClusterHdfsDSource_01OutputLane14649766295360::s]']' destroy: java.lang.RuntimeException: java.lang.IllegalStateException: Object has already been returned to this pool or is invalid
java.lang.RuntimeException: java.lang.IllegalStateException: Object has already been returned to this pool or is invalid
at com.streamsets.datacollector.stagelibrary.ClassLoaderStageLibraryTask.releaseStageClassLoader(ClassLoaderStageLibraryTask.java:448)
at com.streamsets.datacollector.runner.preview.PreviewStageLibraryTask.releaseStageClassLoader(PreviewStageLibraryTask.java:92)
at com.streamsets.datacollector.creation.StageBean.releaseClassLoader(StageBean.java:60)
at com.streamsets.datacollector.runner.StageRuntime.destroy(StageRuntime.java:203)
at com.streamsets.datacollector.runner.StagePipe.destroy(StagePipe.java:275)
at com.streamsets.datacollector.runner.Pipeline.destroy(Pipeline.java:187)
at com.streamsets.datacollector.runner.preview.PreviewPipeline.destroy(PreviewPipeline.java:71)
at com.streamsets.datacollector.execution.preview.sync.SyncPreviewer.start(SyncPreviewer.java:205)
at com.streamsets.datacollector.execution.preview.async.AsyncPreviewer$2.call(AsyncPreviewer.java:96)
at com.streamsets.pipeline.lib.executor.SafeScheduledExecutorService$SafeCallable.call(SafeScheduledExecutorService.java:233)
at java.util.concurrent.FutureTask.run(FutureTask.java:262)
at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(ScheduledThreadPoolExecutor.java:178)
at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:292)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
Caused by: java.lang.IllegalStateException: Object has already been returned to this pool or is invalid
at org.apache.commons.pool2.impl.GenericKeyedObjectPool.returnObject(GenericKeyedObjectPool.java:484)
at com.streamsets.datacollector.stagelibrary.ClassLoaderStageLibraryTask.releaseStageClassLoader(ClassLoaderStageLibraryTask.java:442)
... 15 more
2016-06-03 16:30:21,353 DEBUG org.apache.http.impl.conn.PoolingClientConnectionManager: Connection manager is shutting down
2016-06-03 16:30:21,353 DEBUG org.apache.http.impl.conn.PoolingClientConnectionManager: Connection manager shut down
因此,为了验证,我连接到zookeeper并查询了活动的\u节点:
[root@hadoop3 ~]# zookeeper-client
Connecting to localhost:2181
Welcome to ZooKeeper!
[zk: localhost:2181(CONNECTED) 5] ls /solr
[configs, overseer, aliases.json, live_nodes, collections, overseer_elect, clusterstate.json, solr.xml, zkdtsm, clusterprops.json, token]
[zk: localhost:2181(CONNECTED) 6] ls /solr/live_nodes
[hadoop3:8983_solr, hadoop4:8983_solr, hadoop2:8983_solr, hadoop5:8983_solr]
不确定,为什么streamset调用api时响应为空,为什么它在能够连接时抱怨“zookeeper connection string”。。。有什么帮助吗?
1条答案
按热度按时间uxh89sit1#
添加
/solr
到你的zookeeper url,看起来像:host:port/solr
.