在hadoop上,将数据从hdfs导入hbase表时出错

vqlkdk9b  于 2021-06-02  发布在  Hadoop
关注(0)|答案(0)|浏览(285)

我从命令行运行这个命令

hbase org.apache.hadoop.hbase.mapreduce.Driver import inputtable /backup/inputtable/part-m-00000

它失败了,因为Yarn日志问题(如果我得到它的权利)。
hdfs上的文件:/backup/inputable/part-m-00000存在
而mapreduce作业作为导出成功

hbase org.apache.hadoop.hbase.mapreduce.Driver export output /backup/

当我尝试导入时,由于原木Yarn输出,我面临一个问题:

log4j:ERROR setFile(null,true) call failed.
java.io.FileNotFoundException: /usr/local/hadoop/logs/userlogs/application_1466671180097_0001/container_1466671180097_0001_01_000001 (Is a directory)
    at java.io.FileOutputStream.open0(Native Method)
    at java.io.FileOutputStream.open(FileOutputStream.java:270)
    at java.io.FileOutputStream.<init>(FileOutputStream.java:213)
    at java.io.FileOutputStream.<init>(FileOutputStream.java:133)
    at org.apache.log4j.FileAppender.setFile(FileAppender.java:294)
    at org.apache.log4j.FileAppender.activateOptions(FileAppender.java:165)
    at org.apache.hadoop.yarn.ContainerLogAppender.activateOptions(ContainerLogAppender.java:55)
    at org.apache.log4j.config.PropertySetter.activate(PropertySetter.java:307)
    at org.apache.log4j.config.PropertySetter.setProperties(PropertySetter.java:172)
    at org.apache.log4j.config.PropertySetter.setProperties(PropertySetter.java:104)
    at org.apache.log4j.PropertyConfigurator.parseAppender(PropertyConfigurator.java:842)
    at org.apache.log4j.PropertyConfigurator.parseCategory(PropertyConfigurator.java:768)
    at org.apache.log4j.PropertyConfigurator.configureRootCategory(PropertyConfigurator.java:648)
    at org.apache.log4j.PropertyConfigurator.doConfigure(PropertyConfigurator.java:514)
    at org.apache.log4j.PropertyConfigurator.doConfigure(PropertyConfigurator.java:580)
    at org.apache.log4j.helpers.OptionConverter.selectAndConfigure(OptionConverter.java:526)
    at org.apache.log4j.LogManager.<clinit>(LogManager.java:127)
    at org.apache.log4j.Logger.getLogger(Logger.java:104)
    at org.apache.commons.logging.impl.Log4JLogger.getLogger(Log4JLogger.java:262)
    at org.apache.commons.logging.impl.Log4JLogger.<init>(Log4JLogger.java:108)
    at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
    at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
    at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
    at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
    at org.apache.commons.logging.impl.LogFactoryImpl.createLogFromClass(LogFactoryImpl.java:1025)
    at org.apache.commons.logging.impl.LogFactoryImpl.discoverLogImplementation(LogFactoryImpl.java:844)
    at org.apache.commons.logging.impl.LogFactoryImpl.newInstance(LogFactoryImpl.java:541)
    at org.apache.commons.logging.impl.LogFactoryImpl.getInstance(LogFactoryImpl.java:292)
    at org.apache.commons.logging.impl.LogFactoryImpl.getInstance(LogFactoryImpl.java:269)
    at org.apache.commons.logging.LogFactory.getLog(LogFactory.java:657)
    at org.apache.hadoop.service.AbstractService.<clinit>(AbstractService.java:43)
SLF4J: Class path contains multiple SLF4J bindings.
SLF4J: Found binding in [jar:file:/usr/local/hadoop/share/hadoop/common/lib/slf4j-log4j12-1.7.10.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: Found binding in [jar:file:/tmp/hadoop-hduser/nm-local-dir/usercache/hduser/filecache/16/phoenix-4.6.0-HBase-1.1-client.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an explanation.
SLF4J: Actual binding is of type [org.slf4j.impl.Log4jLoggerFactory]
giu 23, 2016 10:43:22 AM com.sun.jersey.guice.spi.container.GuiceComponentProviderFactory register
INFORMAZIONI: Registering org.apache.hadoop.mapreduce.v2.app.webapp.JAXBContextResolver as a provider class
giu 23, 2016 10:43:22 AM com.sun.jersey.guice.spi.container.GuiceComponentProviderFactory register
INFORMAZIONI: Registering org.apache.hadoop.yarn.webapp.GenericExceptionHandler as a provider class
giu 23, 2016 10:43:22 AM com.sun.jersey.guice.spi.container.GuiceComponentProviderFactory register
INFORMAZIONI: Registering org.apache.hadoop.mapreduce.v2.app.webapp.AMWebServices as a root resource class
giu 23, 2016 10:43:22 AM com.sun.jersey.server.impl.application.WebApplicationImpl _initiate
INFORMAZIONI: Initiating Jersey application, version 'Jersey: 1.9 09/02/2011 11:17 AM'
giu 23, 2016 10:43:22 AM com.sun.jersey.guice.spi.container.GuiceComponentProviderFactory getComponentProvider
INFORMAZIONI: Binding org.apache.hadoop.mapreduce.v2.app.webapp.JAXBContextResolver to GuiceManagedComponentProvider with the scope "Singleton"
giu 23, 2016 10:43:23 AM com.sun.jersey.guice.spi.container.GuiceComponentProviderFactory getComponentProvider
INFORMAZIONI: Binding org.apache.hadoop.yarn.webapp.GenericExceptionHandler to GuiceManagedComponentProvider with the scope "Singleton"
giu 23, 2016 10:43:23 AM com.sun.jersey.guice.spi.container.GuiceComponentProviderFactory getComponentProvider
INFORMAZIONI: Binding org.apache.hadoop.mapreduce.v2.app.webapp.AMWebServices to GuiceManagedComponentProvider with the scope "PerRequest"
log4j:WARN No appenders could be found for logger (org.apache.hadoop.ipc.Server).
log4j:WARN Please initialize the log4j system properly.
log4j:WARN See http://logging.apache.org/log4j/1.2/faq.html#noconfig for more info.

log4j.属性

log4j.rootLogger=INFO, FILE
log4j.rootCategory=INFO, FILE
log4j.appender.console=org.apache.log4j.ConsoleAppender
log4j.appender.console.target=System.err
log4j.appender.console.layout=org.apache.log4j.PatternLayout
log4j.appender.console.layout.ConversionPattern=%d{yy/MM/dd HH:mm:ss} %p %c{1}: %m%n

# Ignore messages below warning level from Jetty, because it's a bit verbose

log4j.logger.org.eclipse.jetty=WARN

# Set the appender named FILE to be a File appender

log4j.appender.FILE=org.apache.log4j.FileAppender

# Change the path to where you want the log file to reside

log4j.appender.FILE.File=/usr/local/spark/logs/SparkOut.log

# Prettify output a bit

log4j.appender.FILE.layout=org.apache.log4j.PatternLayout
log4j.appender.FILE.layout.ConversionPattern=%d{yy/MM/dd HH:mm:ss} %p %c{1}: %m%n

# Settings to quiet third party logs that are too verbose

log4j.logger.org.spark-project.jetty=WARN
log4j.logger.org.spark-project.jetty.util.component.AbstractLifeCycle=ERROR
log4j.logger.org.apache.spark.repl.SparkIMain$exprTyper=INFO
log4j.logger.org.apache.spark.repl.SparkILoop$SparkILoopInterpreter=INFO
log4j.logger.org.apache.parquet=ERROR
log4j.logger.parquet=ERROR

# SPARK-9183: Settings to avoid annoying messages when looking up nonexistent UDFs in SparkSQL with Hive support

log4j.logger.org.apache.hadoop.hive.metastore.RetryingHMSHandler=FATAL
log4j.logger.org.apache.hadoop.hive.ql.exec.FunctionRegistry=ERROR

当导入mapreduce作业失败时,导出mapreduce作业如何可能成功?

暂无答案!

目前还没有任何答案,快来回答吧!

相关问题