java 如何使用log4j appender for Kafka编写应用程序日志？

gajydyqb 于 2023-04-28 发布在 Java

关注(0)|答案(4)|浏览(176)

我想使用log4j kafka appender将我的应用日志消息流到Kafka。
我的应用程序使用了Sping Boot 和log4j2。
我在主题中看到了启动日志消息，但没有看到应用程序中使用的日志语句。
我在log4j2配置中遗漏了什么吗？

<?xml version="1.0" encoding="UTF-8"?>
<Configuration status="debug" name="kafka-logs" 
packages="myapp.kafka.test">
<Appenders>
    <Kafka name="Kafka" topic="test-kafka-topic">
        <JSONLayout />
        <Property name="bootstrap.servers">server1,server2</Property>
    </Kafka>
    <Async name="Async">
        <AppenderRef ref="Kafka"/>
    </Async>

    <Console name="stdout" target="SYSTEM_OUT">
        <PatternLayout pattern="%d{HH:mm:ss.SSS} %-5p [%-7t] %F:%L - %m%n"/>
    </Console>

</Appenders>
<Loggers>
    <Root level="INFO">
        <AppenderRef ref="Kafka"/>
        <AppenderRef ref="stdout"/>
    </Root>
    <Logger name="org.apache.kafka" level="WARN" />
</Loggers>

pom.xml

<dependency>
        <groupId>org.apache.logging.log4j</groupId>
        <artifactId>log4j-api</artifactId>
        <version>2.11.2</version>
    </dependency>

    <dependency>
        <groupId>org.apache.logging.log4j</groupId>
        <artifactId>log4j-core</artifactId>
        <version>2.11.2</version>
    </dependency>

    <dependency>
        <groupId>org.slf4j</groupId>
        <artifactId>slf4j-simple</artifactId>
        <version>1.7.10</version>
    </dependency>

    <dependency>
        <groupId>org.apache.kafka</groupId>
        <artifactId>kafka-log4j-appender</artifactId>
        <version>1.0.0</version>
        <exclusions>
            <exclusion>
                <groupId>log4j</groupId>
                <artifactId>log4j</artifactId>
            </exclusion>
            <exclusion>
                <groupId>org.slf4j</groupId>
                <artifactId>slf4j-log4j12</artifactId>
            </exclusion>
        </exclusions>
    </dependency>

Java代码

@SpringBootApplication
public class TestApplication {
private static final Logger logger = 
LoggerFactory.getLogger(TestApplication.class);
public static void main(String[] args) {
  SpringApplication application = new SpringApplication(TestApplication.class);
  logger.info("Before Application Startup");
  application.run(args);
  logger.info("After Application Startup");
}
}

Kafka中的日志：

{
  "thread" : "main",
  "level" : "INFO",
  "loggerName" : "TestApplication",
  "message" : "Started TestApplication in 17.19 seconds (JVM running for 22.67)",
  "endOfBatch" : false,
  "loggerFqcn" : "org.apache.commons.logging.LogFactory$Log4jLog",
  "instant" : {
  "epochSecond" : 1555422668,
  "nanoOfSecond" : 856000000
},
"threadId" : 1,
"threadPriority" : 5
}

控制台输出：

08:51:08.856 [main] INFO  TestApplication - Started TestApplication in 17.19 seconds (JVM running for 22.67)
[main] INFO TestApplication - Testing my application from TestApplication
[main] ERROR TestApplication - Testing my application from TestApplication

Java

来源：https://stackoverflow.com/questions/55697278/how-to-write-application-logs-using-log4j-appender-for-kafka

4条答案

按热度按时间

lyr7nygr1#

首先，你需要在你的pom中删除spring Boot 应用程序中的默认日志提供程序。xml文件，其中Logback和Log-classic，那么你需要添加log4j 2作为新的日志提供程序，并添加Kafka appender。添加依赖项后，您需要xml配置文件，您可以在其中添加Kafka appender配置。默认情况下，您需要在项目的资源路径中找到配置文件，并将其命名为“log4j 2”。xml”。
您可以找到许多其他Log4j2 appenders，如Cassandra或Failover appender，并将它们添加到配置文件中的Kafka appender旁边。您可以在下面找到一个适用且正确的示例。

<!--excluding logback -->
        <dependency>
            <groupId>org.springframework.boot</groupId>
            <artifactId>spring-boot-starter</artifactId>
            <exclusions>
                <exclusion>
                    <groupId>org.springframework.boot</groupId>
                    <artifactId>spring-boot-starter-logging</artifactId>
                </exclusion>
                <exclusion>
                    <groupId>org.springframework.boot</groupId>
                    <artifactId>logback-classic</artifactId>
                </exclusion>
            </exclusions>
        </dependency>

<!--adding log4j2 and kafka appender-->
        <dependency>
            <groupId>org.springframework.boot</groupId>
            <artifactId>spring-boot-starter-log4j2</artifactId>
        </dependency>
    <dependency>
        <groupId>org.apache.kafka</groupId>
        <artifactId>kafka-log4j-appender</artifactId>
        <exclusions>
            <exclusion>
                <groupId>org.slf4j</groupId>
                <artifactId>slf4j-log4j12</artifactId>
            </exclusion>
        </exclusions>
    </dependency>

Kafka appender配置

<?xml version="1.0" encoding="UTF-8"?>
<Configuration status="info" name="kafka-appender" packages="Citydi.ElasticDemo">

    <Appenders>

        <Kafka name="kafkaLogAppender" topic="Second-Topic">
            <JSONLayout />
            <Property name="bootstrap.servers">localhost:9092</Property>
            <MarkerFilter marker="Recorder" onMatch="DENY" onMismatch="ACCEPT"/>
        </Kafka>

        <Console name="stdout" target="SYSTEM_OUT">
            <PatternLayout pattern="%d{HH:mm:ss.SSS} stdout %highlight{%-5p} [%-7t] %F:%L - %m%n"/>
            <MarkerFilter marker="Recorder" onMatch="DENY" onMismatch="ACCEPT"/>
        </Console>

    </Appenders>
    <Loggers>
        <Root level="INFO">
            <AppenderRef ref="kafkaLogAppender"/>
            <AppenderRef ref="stdout"/>
        </Root>
        <Logger name="org.apache.kafka" level="warn" />
    </Loggers>

</Configuration>

赞(0）回复(0）举报 2023-04-28

blpfk2vs2#

我想log4j2已经有了Kafka appender。至于我，我在Gradle中添加了Kafka客户端。所以不需要添加kafka-log4j-appender

compile group: 'org.apache.kafka', name: 'kafka-clients', version: '2.0.1'

Maven应该差不多

赞(0）回复(0）举报 2023-04-28

lpwwtiir3#

你需要排除spring-boot-starter-logging如下

<dependency>
            <groupId>org.springframework.boot</groupId>
            <artifactId>spring-boot-starter</artifactId>
             <exclusions>
                <exclusion>
                    <groupId>org.springframework.boot</groupId>
                    <artifactId>spring-boot-starter-logging</artifactId>
                </exclusion>
                <exclusion>
                    <groupId>org.springframework.boot</groupId>
                    <artifactId>logback-classic</artifactId>
                </exclusion>
            </exclusions>
        </dependency>

赞(0）回复(0）举报 2023-04-28

eeq64g8w4#

试试这个：

<appender name="KAFKA" class="org.apache.kafka.log4jappender.KafkaLog4jAppender">
    <param name="Topic" value="logs"/>
    <param name="BrokerList" value="localhost:9092"/>
    <layout class="org.apache.log4j.PatternLayout">
        <param name="ConversionPattern" value="%d{yyyy-MM-dd HH:mm:ss} %-5p %c{1}:%L - %m%n"/>
    </layout>
</appender>

<logger name="org.apache.kafka" additivity="false">
    <level value="INFO"/>
    <appender-ref ref="KAFKA"/>
</logger>

赞(0）回复(0）举报 2023-04-28

我来回答

java 如何使用log4j appender for Kafka编写应用程序日志？

4条答案

相关问题

热门标签

最新问答