datastax pig-无法从cassandra加载任何数据

7cwmlq89  于 2021-06-21  发布在  Pig
关注(0)|答案(0)|浏览(249)

我试图在pig中运行一个test pig脚本,从datasax企业的cassandra加载数据,但是我得到了一个错误。让我展示整个场景:
cassandra架构:创建keyspace libdata,replication={'class':'simplestrategy','replication\u factor':1};
创建表libout(“stabr”文本、“fscskey”文本、“fscs\ seq”文本、“libid”文本、“libname”文本、“address”文本、“city”文本、“zip”文本、“zip4”文本、“cnty”文本、“phone”文本、“c\ u out\ ty”文本、“c\ msa”文本、“sq\ ft”文本、“l\ u num\ u bm”int、“f\ u bkmob”文本、“hours”int、“f\ u hours”文本、“wks\ u open”int、“f\ u wksopn”文本、“yr\ u sub”int,“statstru”int、“statname”int、“statadr”int、“longitud”float、“latitude”float、“fipsst”int、“fipsco”int、“fipsplac”int、“cntypop”int、“locale”text、“centract”float、“cenblock”int、“cdcode”text、“mat\cent”text、“mat\u type”int、“cbsa”int、“microf”text、主键(“fscskey”、“fscs\ seq”);
cqlsh:libdata>创建表libsqft(year int,state text,sqft bigint,主键(year,state));第二个表将用于存储从pig到cassandra的数据。
在pig grunt:grunt>libdata=load'cql://libdata/libout'使用cqlstorage();咕噜>转储libdata;
这是我的输出:
2014-08-18 23:02:11603[main]info org.apache.pig.tools.pigstats.scriptstate-脚本中使用的pig功能:未知2014-08-18 23:02:11607[main]info org.apache.pig.backend.hadoop.executionengine.mapreducelayer.mrcompiler-文件连接阈值:100?false 2014-08-18 23:02:11608[main]info org.apache.pig.backend.hadoop.executionengine.mapreducelayer.multiqueryoptimizer-优化前mr计划大小:1 2014-08-18 23:02:11608[main]info org.apache.pig.backend.hadoop.executionengine.mapreducelayer.multiqueryoptimizer-优化后mr计划大小:1 2014-08-18 23:02:11,613[main]info org.apache.pig.tools.pigstats.scriptstate-pig脚本设置已添加到作业2014-08-18 23:02:11613[main]info org.apache.pig.backend.hadoop.executionengine.mapreducelayer.jobcontrolcompiler-mapred.job.reduce.markreset.buffer.percent未设置,设置为默认值0.3 2014-08-18 23:02:11,613[main]info org.apache.pig.backend.hadoop.executionengine.mapreducelayer.jobcontrolcompiler-创建jar文件job51353284935577655.jar 2014-08-18 23:02:14378[main]info org.apache.pig.backend.hadoop.executionengine.mapreducelayer.jobcontrolcompiler-创建jar文件job51353249315577655.jar 2014-08-18 23:02:14,386[main]info org.apache.pig.backend.hadoop.executionengine.mapreducelayer.jobcontrolcompiler-设置单个存储作业2014-08-18 23:02:14400[main]info org.apache.pig.backend.hadoop.executionengine.mapreducelayer.mapreducelauncher-1个map reduce作业正在等待提交。2014-08-18 23:02:14783[thread-12]info org.apache.pig.backend.hadoop.executionengine.util.mapredutil-要处理的总输入路径(组合):1 2014-08-18 23:02:14901[main]info org.apache.pig.backend.hadoop.executionengine.mapreducelayer.mapreducelauncher-完成0%2014-08-18 23:02:15,439[main]info org.apache.pig.backend.hadoop.executionengine.mapreducelayer.mapreducelauncher-hadoopjobid:job\u 201408182033\u 0011 2014-08-18 23:02:15439[main]info org.apache.pig.backend.hadoop.executionengine.mapreducelayer.mapreducelauncher-更多信息请访问:http:// IP :50030/jobdetails.jsp?jobid=job\u 201408182033\u 0011 2014-08-18 23:03:00167[main]info org.apache.pig.backend.hadoop.executionengine.mapreducelayer.mapreducelauncher-作业\u 201408182033\u 0011失败!停止运行所有相关作业2014-08-18 23:03:00167[main]info org.apache.pig.backend.hadoop.executionengine.mapreducelayer.mapreducelauncher-100%完成2014-08-18 23:03:00169[main]warn org.apache.pig.backend.hadoop.executionengine.mapreducelayer.launcher-没有要写入的日志文件。2014-08-18 23:03:00,169[main]error org.apache.pig.backend.hadoop.executionengine.mapreducelayer.launcher-后端错误消息java.lang.runtimeexception位于org.apache.cassandra.hadoop.cql3.cqlpagingrecordreader$roweiterator.executequery(cqlpagingrecordreader)。java:657)在org.apache.cassandra.hadoop.cql3.cqlpagingrecordreader$行迭代器(cqlpagingrecordreader)。java:301)位于org.apache.cassandra.hadoop.cql3.cqlpagingrecordreader.initialize(cqlpagingrecordreader)。java:167)位于org.apache.pig.backend.hadoop.executionengine.mapreducelayer.pigrecordreader.initialize(pigrecordreader。java:181)在org.apache.hadoop.mapred.maptask$newtrackingrecordreader.initialize(maptask。java:522)在org.apache.hadoop.mapred.maptask.runnewmapper(maptask。java:763)在org.apache.hadoop.mapred.maptask.run(maptask。java:370)在org.apache.hadoop.mapred.child$4.run(child。java:266)位于java.security.accesscontroller.doprivileged(本机方法)javax.security.auth.subject.doas(主题。java:415)在org.apache.hadoop.security.usergroupinformation.doas(usergroupinformation。java:1121)在org.apache.hadoop.mapred.child.main(child。java:260)原因:处的unavailableexception()cassandra$execute\u prepared\u cql3\u query\u result$execute\u prepared\u cql3\u query\u resultstandardscheme.read(cassandra。java:53662)在org.apache.cassandra.thrift.cassandra$execute\u prepared\u cql3\u query\u result$execute\u prepared\u cql3\u query\u resultstandardscheme.read(cassandra。java:53630)在cassandra$execute\u prepared\u cql3\u query\u result.read(cassandra。java:53545)位于org.apache.thrift.tserviceclient.receivebase(tserviceclient。java:78)在org.apache.cassandra.thrift.cassandra$client.recv\u execute\u prepared\u cql3\u query(cassandra。java:1820)在org.apache.cassandra.thrift.cassandra$client.execute\u prepared\u cql3\u query(cassandra。java:1805)在org.apache.cassandra.hadoop.cql3.cqlpagingrecordreader$roweiterator.executequery(cqlpagingrecordreader)上。java:635) ... 还有11个
2014-08-18 23:03:00,173[main]error org.apache.pig.tools.pigstats.simplepigstats-错误2997:无法从备份的错误中重新创建异常:java.lang.runtimeexception位于org.apache.cassandra.hadoop.cql3.cqlpagingrecordreader$rowiiterator.executequery(cqlpagingrecordreader)。java:657)在org.apache.cassandra.hadoop.cql3.cqlpagingrecordreader$行迭代器(cqlpagingrecordreader)。java:301)位于org.apache.cassandra.hadoop.cql3.cqlpagingrecordreader.initialize(cqlpagingrecordreader)。java:167)位于org.apache.pig.backend.hadoop.executionengine.mapreducelayer.pigrecordreader.initialize(pigrecordreader。java:181)在org.apache.hadoop.mapred.maptask$newtrackingrecordreader.initialize(maptask。java:522)在org.apache.hadoop.mapred.maptask.runnewmapper(maptask。java:763)在org.apache.hadoop.mapred.maptask.run(maptask。java:370)在org.apache.hadoop.mapred.child$4.run(child。java:266)位于java.security.accesscontroller.doprivileged(本机方法)javax.security.auth.subject.doas(主题。java:415)在org.apache.hadoop.security.usergroupinformation.doas(usergroupinformation。java:1121)在org.apache.hadoop.mapred.child.main(child。java:260)原因:处的unavailableexception()cassandra$execute\u prepared\u cql3\u query\u result$execute\u prepared\u cql3\u query\u resultstandardscheme.read(cassandra。java:53662)在org.apache.cassandra.thrift.cassandra$execute\u prepared\u cql3\u query\u result$execute\u prepared\u cql3\u query\u resultstandardscheme.read(cassandra。java:53630)在cassandra$execute\u prepared\u cql3\u query\u result.read(cassandra。java:53545)位于org.apache.thrift.tserviceclient.receivebase(tserviceclient。java:78)在org.apache.cassandra.thrift.cassandra$client.recv\u execute\u prepared\u cql3\u query(cassandra。java:1820)在org.apache.cassandra.thrift.cassandra$client.execute\u prepared\u cql3\u query(cassandra。java:1805)在org.apache.cassandra.hadoop.cql3.cqlpagingrecordreader$roweiterator.executequery(cqlpagingrecordreader)上。java:635) ... 还有11个
2014-08-18 23:03:00173[main]error org.apache.pig.tools.pigstats.pigstattil-1 map reduce作业失败!2014-08-18 23:03:00174[main]信息org.apache.pig.tools.pigstats.simplepigstats-脚本统计:
hadoopversion pigversion userid startedat finishedat功能1.0.4.13 0.10.1 ubuntu 2014-08-18 23:02:11 2014-08-18 23:03:00未知
失败!
失败的作业:jobid alias feature message outputs job\u 201408182033\u 0011 libdata map\u only message:作业失败!错误-#个失败的Map任务超出了允许的限制。失败计数:1。上次失败任务:任务\u 201408182033 \u 0011 \u m\u000000cfs://10.82.31.13/tmp/temp-1734707970/tmp1694465949,
输入:未能从“”读取数据cql://libdata/libout"
输出:未能在“”中生成结果cfs://10.82.31.13/tmp/temp-1734707970/tmp1694465949"
计数器:写入的记录总数:写入的字节总数:0可溢出内存管理器溢出计数:0主动溢出的包总数:0主动溢出的记录总数:0
作业编号:job\ U 201408182033\ U 0011
2014-08-18 23:03:00174[main]info org.apache.pig.backend.hadoop.executionengine.mapreducelayer.mapreducelauncher-失败!2014-08-18 23:03:00,215[main]error org.apache.pig.tools.grunt.grunt-错误2997:无法从备份的错误中重新创建异常:java.lang.runtimeexception位于org.apache.cassandra.hadoop.cql3.cqlpagingrecordreader$roweiterator.executequery(cqlpagingrecordreader)。java:657)在org.apache.cassandra.hadoop.cql3.cqlpagingrecordreader$行迭代器(cqlpagingrecordreader)。java:301)位于org.apache.cassandra.hadoop.cql3.cqlpagingrecordreader.initialize(cqlpagingrecordreader)。java:167)位于org.apache.pig.backend.hadoop.executionengine.mapreducelayer.pigrecordreader.initialize(pigrecordreader。java:181)在org.apache.hadoop.mapred.maptask$newtrackingrecordreader.initialize(maptask。java:522)在org.apache.hadoop.mapred.maptask.runnewmapper(maptask。java:763)在org.apache.hadoop.mapred.maptask.run(maptask。java:370)在org.apache.hadoop.mapred.child$4.run(child。java:266)位于java.security.accesscontroller.doprivileged(本机方法)javax.security.auth.subject.doas(主题。java:415)在org.apache.hadoop.security.usergroupinformation.doas(usergroupinformation。java:1121)在org.apache.hadoop.mapred.child.main(child。java:260)原因:org.apache.cassandra.thrift.cassandra$execute\u prepared\u cql3\u query\u result$exec上的unavailableexception()

暂无答案!

目前还没有任何答案,快来回答吧!

相关问题