hbase批量加载失败| kosmosfilesystem文件系统异常

xpcnnkqh  于 2021-05-30  发布在  Hadoop
关注(0)|答案(1)|浏览(401)

我刚才提到下面的博客是为了运行简单的批量加载,但它给出了一些kosmosfilesystem异常。
http://www.thecloudavenue.com/2013/04/bulk-loading-data-in-hbase.html
下面是运行以下命令后生成的日志。
hadoop jar hbasebulkimport.jar/user/hduser/hbase/input/rowfeeder.csv/user/hduser/hbase/output/nbafinal2010

15/04/16 12:40:35 INFO zookeeper.ZooKeeper: Client environment:java.library.path=:/usr/hdp/2.2.0.0-2041/hadoop/lib/native/Linux-amd64-64:/usr/hdp/2.2.0.0-2041/hadoop/lib/native
15/04/16 12:40:35 INFO zookeeper.ZooKeeper: Client environment:java.io.tmpdir=/tmp
15/04/16 12:40:35 INFO zookeeper.ZooKeeper: Client environment:java.compiler=<NA>
15/04/16 12:40:35 INFO zookeeper.ZooKeeper: Client environment:os.name=Linux
15/04/16 12:40:35 INFO zookeeper.ZooKeeper: Client environment:os.arch=amd64
15/04/16 12:40:35 INFO zookeeper.ZooKeeper: Client environment:os.version=2.6.32-504.12.2.el6.x86_64
15/04/16 12:40:35 INFO zookeeper.ZooKeeper: Client environment:user.name=hduser
15/04/16 12:40:35 INFO zookeeper.ZooKeeper: Client environment:user.home=/home/hduser
15/04/16 12:40:35 INFO zookeeper.ZooKeeper: Client environment:user.dir=/home/hduser/user/shashi/hbase/bulkLoad
15/04/16 12:40:35 INFO zookeeper.ZooKeeper: Initiating client connection, connectString=localhost:2181 sessionTimeout=90000 watcher=hconnection-0x2dd06f21, quorum=localhost:2181, baseZNode=/hbase-unsecure
15/04/16 12:40:35 INFO zookeeper.RecoverableZooKeeper: Process identifier=hconnection-0x2dd06f21 connecting to ZooKeeper ensemble=localhost:2181
15/04/16 12:40:35 INFO zookeeper.ClientCnxn: Opening socket connection to server localhost/127.0.0.1:2181. Will not attempt to authenticate using SASL (unknown error)
15/04/16 12:40:35 INFO zookeeper.ClientCnxn: Socket connection established to localhost/127.0.0.1:2181, initiating session
15/04/16 12:40:35 INFO zookeeper.ClientCnxn: Session establishment complete on server localhost/127.0.0.1:2181, sessionid = 0x34cb73f98050030, negotiated timeout = 40000
Exception in thread "main" java.io.IOException: java.lang.reflect.InvocationTargetException
        at org.apache.hadoop.hbase.client.ConnectionManager.createConnection(ConnectionManager.java:426)
        at org.apache.hadoop.hbase.client.ConnectionManager.createConnection(ConnectionManager.java:403)
        at org.apache.hadoop.hbase.client.ConnectionManager.getConnectionInternal(ConnectionManager.java:281)
        at org.apache.hadoop.hbase.client.HTable.<init>(HTable.java:207)
        at org.apache.hadoop.hbase.client.HTable.<init>(HTable.java:169)
        at Driver.main(Driver.java:48)
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.lang.reflect.Method.invoke(Method.java:606)
        at org.apache.hadoop.util.RunJar.run(RunJar.java:221)
        at org.apache.hadoop.util.RunJar.main(RunJar.java:136)
Caused by: java.lang.reflect.InvocationTargetException
        at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
        at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:57)
        at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
        at java.lang.reflect.Constructor.newInstance(Constructor.java:526)
        at org.apache.hadoop.hbase.client.ConnectionManager.createConnection(ConnectionManager.java:424)
        ... 11 more
Caused by: java.lang.ExceptionInInitializerError
        at org.apache.hadoop.hbase.ClusterId.parseFrom(ClusterId.java:64)
        at org.apache.hadoop.hbase.zookeeper.ZKClusterId.readClusterIdZNode(ZKClusterId.java:75)
        at org.apache.hadoop.hbase.client.ZooKeeperRegistry.getClusterId(ZooKeeperRegistry.java:106)
        at org.apache.hadoop.hbase.client.ConnectionManager$HConnectionImplementation.retrieveClusterId(ConnectionManager.java:858)
        at org.apache.hadoop.hbase.client.ConnectionManager$HConnectionImplementation.<init>(ConnectionManager.java:662)
        ... 16 more
Caused by: java.lang.UnsupportedOperationException: Not implemented by the KosmosFileSystem FileSystem implementation
        at org.apache.hadoop.fs.FileSystem.getScheme(FileSystem.java:216)
        at org.apache.hadoop.fs.FileSystem.loadFileSystems(FileSystem.java:2564)
        at org.apache.hadoop.fs.FileSystem.getFileSystemClass(FileSystem.java:2574)
        at org.apache.hadoop.fs.FileSystem.createFileSystem(FileSystem.java:2591)
        at org.apache.hadoop.fs.FileSystem.access$200(FileSystem.java:91)
        at org.apache.hadoop.fs.FileSystem$Cache.getInternal(FileSystem.java:2630)
        at org.apache.hadoop.fs.FileSystem$Cache.get(FileSystem.java:2612)
        at org.apache.hadoop.fs.FileSystem.get(FileSystem.java:370)
        at org.apache.hadoop.fs.FileSystem.get(FileSystem.java:169)
        at org.apache.hadoop.fs.FileSystem.get(FileSystem.java:354)
        at org.apache.hadoop.fs.Path.getFileSystem(Path.java:296)
        at org.apache.hadoop.hbase.util.DynamicClassLoader.<init>(DynamicClassLoader.java:104)
        at org.apache.hadoop.hbase.protobuf.ProtobufUtil.<clinit>(ProtobufUtil.java:226)
        ... 21 more
xesrikrc

xesrikrc1#

看起来是和jar有关的问题。在使用hbase lib的最新jar之后,问题得到了解决。

相关问题