我正在使用ApacheNutch2.3和Hadoop1.2.1以及hbase 0.94。我已经为分布式模式配置了apachenutch。当我运行爬网时,它会给出以下错误
15/12/21 12:06:56 INFO zookeeper.ClientCnxn: Session establishment complete on server node1/1.11.1.2:2181, sessionid = 0x151ba038ac60250, negotiated timeout = 180000
Exception in thread "main" java.lang.NoClassDefFoundError: com/google/protobuf/Message
at org.apache.hadoop.hbase.io.HbaseObjectWritable.<clinit>(HbaseObjectWritable.java:265)
at org.apache.hadoop.hbase.ipc.Invocation.write(Invocation.java:139)
at org.apache.hadoop.hbase.ipc.HBaseClient$Connection.sendParam(HBaseClient.java:620)
at org.apache.hadoop.hbase.ipc.HBaseClient.call(HBaseClient.java:989)
at org.apache.hadoop.hbase.ipc.WritableRpcEngine$Invoker.invoke(WritableRpcEngine.java:87)
at com.sun.proxy.$Proxy5.getProtocolVersion(Unknown Source)
at org.apache.hadoop.hbase.ipc.WritableRpcEngine.getProxy(WritableRpcEngine.java:141)
at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.getMaster(HConnectionManager.java:813)
at org.apache.hadoop.hbase.client.HBaseAdmin.<init>(HBaseAdmin.java:127)
at org.apache.gora.hbase.store.HBaseStore.initialize(HBaseStore.java:115)
at org.apache.gora.store.DataStoreFactory.initializeDataStore(DataStoreFactory.java:104)
at org.apache.gora.store.DataStoreFactory.createDataStore(DataStoreFactory.java:163)
at org.apache.gora.store.DataStoreFactory.createDataStore(DataStoreFactory.java:137)
at org.apache.nutch.storage.StorageUtils.createWebStore(StorageUtils.java:78)
at org.apache.nutch.crawl.InjectorJob.run(InjectorJob.java:218)
at org.apache.nutch.crawl.InjectorJob.inject(InjectorJob.java:252)
at org.apache.nutch.crawl.InjectorJob.run(InjectorJob.java:275)
at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:65)
at org.apache.nutch.crawl.InjectorJob.main(InjectorJob.java:284)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:483)
at org.apache.hadoop.util.RunJar.main(RunJar.java:160)
Caused by: java.lang.ClassNotFoundException: com.google.protobuf.Message
at java.net.URLClassLoader$1.run(URLClassLoader.java:372)
at java.net.URLClassLoader$1.run(URLClassLoader.java:361)
at java.security.AccessController.doPrivileged(Native Method)
at java.net.URLClassLoader.findClass(URLClassLoader.java:360)
at java.lang.ClassLoader.loadClass(ClassLoader.java:424)
at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:308)
at java.lang.ClassLoader.loadClass(ClassLoader.java:357)
... 24 more
在nutch的lib中,有一个jar protobuf-java-2.4.1.jar如何解决?
1条答案
按热度按时间70gysomp1#
我认为应该将protobufjava{version}.jar添加到hadoop lib目录中。它应该能解决你的问题。用于类路径验证。运行以下命令
如果它包含repective jar,则表示它已被添加到类路径。