我正在尝试为日志文件定义一个系统属性，例如'logger.file'='/tmp/some_name.log'，以便为不同的spark作业动态分配不同的日志文件。我试着把测试用例减少到最低限度。在使用属性时失败，如果我在log4j.properties中硬编码该值，一切都可以。
有人能开导我吗？。。
谢谢！
python代码测试\u log4j.py：

from pyspark.context import SparkContext
from pyspark import SparkConf
from pyspark.sql import SparkSession

SparkContext.setSystemProperty('logger.File', '/tmp/some_name.log')
spark = SparkSession.builder.config(conf=SparkConf()).getOrCreate()

logger = spark._jvm.org.apache.log4j.LogManager.getLogger('default')
logger.warn('hey')

log4j.属性：

log4j.rootLogger=INFO, spark_jobs
log4j.logger.spark_jobs=INFO

log4j.appender.spark_jobs=org.apache.log4j.RollingFileAppender
log4j.appender.spark_jobs.layout=org.apache.log4j.PatternLayout
log4j.appender.spark_jobs.layout.ConversionPattern=%d{yy/MM/dd HH:mm:ss} %p %c{1}: %m%n
log4j.appender.spark_jobs.Threshold=INFO

log4j.appender.spark_jobs.File=${logger.File}

测试：

/usr/hdp/current/spark2-client/bin/spark-submit --master local --conf spark.driver.extraJavaOptions="-Dlog4j.debug -Dlog4j.configuration=file:/tmp/log4j.properties" --conf spark.executor.extraJavaOptions="-Dlog4j.debug -Dlog4j.configuration=file:/tmp/log4j.properties" test_log4j.py

结果：

log4j: Using URL [file:/tmp/log4j.properties] for automatic log4j configuration.
log4j: Reading configuration from URL file:/tmp/log4j.properties
log4j: Parsing for [root] with value=[INFO, spark_jobs].
log4j: Level token is [INFO].
log4j: Category root set to INFO
log4j: Parsing appender named "spark_jobs".
log4j: Parsing layout options for "spark_jobs".
log4j: Setting property [conversionPattern] to [%d{yy/MM/dd HH:mm:ss} %p %c{1}: %m%n].
log4j: End of parsing for "spark_jobs".
log4j: Setting property [file] to [].
log4j: Setting property [threshold] to [INFO].
log4j: setFile called: , true
log4j:ERROR setFile(null,true) call failed.
java.io.FileNotFoundException:  (No such file or directory)
        at java.io.FileOutputStream.open0(Native Method)
        at java.io.FileOutputStream.open(FileOutputStream.java:270)
        at java.io.FileOutputStream.<init>(FileOutputStream.java:213)
        at java.io.FileOutputStream.<init>(FileOutputStream.java:133)
        at org.apache.log4j.FileAppender.setFile(FileAppender.java:294)
        at org.apache.log4j.RollingFileAppender.setFile(RollingFileAppender.java:207)
        at org.apache.log4j.FileAppender.activateOptions(FileAppender.java:165)
        at org.apache.log4j.config.PropertySetter.activate(PropertySetter.java:307)
        at org.apache.log4j.config.PropertySetter.setProperties(PropertySetter.java:172)
        at org.apache.log4j.config.PropertySetter.setProperties(PropertySetter.java:104)
        at org.apache.log4j.PropertyConfigurator.parseAppender(PropertyConfigurator.java:842)
        at org.apache.log4j.PropertyConfigurator.parseCategory(PropertyConfigurator.java:768)
        at org.apache.log4j.PropertyConfigurator.configureRootCategory(PropertyConfigurator.java:648)
        at org.apache.log4j.PropertyConfigurator.doConfigure(PropertyConfigurator.java:514)
        at org.apache.log4j.PropertyConfigurator.doConfigure(PropertyConfigurator.java:580)
        at org.apache.log4j.helpers.OptionConverter.selectAndConfigure(OptionConverter.java:526)
        at org.apache.log4j.LogManager.<clinit>(LogManager.java:127)
        at org.apache.spark.internal.Logging$class.initializeLogging(Logging.scala:120)
        at org.apache.spark.internal.Logging$class.initializeLogIfNecessary(Logging.scala:108)
        at org.apache.spark.deploy.SparkSubmit$.initializeLogIfNecessary(SparkSubmit.scala:71)
        at org.apache.spark.deploy.SparkSubmit$.main(SparkSubmit.scala:128)
        at org.apache.spark.deploy.SparkSubmit.main(SparkSubmit.scala)
log4j: Parsed "spark_jobs" options.
log4j: Parsing for [spark_jobs] with value=[INFO].
log4j: Level token is [INFO].
log4j: Category spark_jobs set to INFO
log4j: Handling log4j.additivity.spark_jobs=[null]
log4j: Finished configuring.