我有一个使用springboot设置的项目,我使用springkafka来使用消息。应用程序部署在一个独立的tomcat示例上。SpringKafka正在生成大量日志消息,它们会自动转到catalina.out。有没有办法将这些kafka日志消息重定向到我为应用程序创建的单独日志(datariver log)中?
我正在使用logback进行日志记录。
我的logback-spring.xml就是这样的:
<property name="LOGS" value="${catalina.base}/logs" />
<include resource="org/springframework/boot/logging/logback/base.xml" />
<appender name="DATA-RIVER" class="ch.qos.logback.core.rolling.RollingFileAppender">
<file>${LOGS}/DataRiver.log</file>
<append>true</append>
<encoder class="ch.qos.logback.classic.encoder.PatternLayoutEncoder">
<Pattern>%p %d %C{1.} [%t-[%X{threadid}]] %m%n</Pattern>
</encoder>
<rollingPolicy class="ch.qos.logback.core.rolling.TimeBasedRollingPolicy">
<fileNamePattern>${LOGS}/archived/DataRiver-%d{yyyy-MM-dd}.log</fileNamePattern>
</rollingPolicy>
</appender>
<appender name="ERROR" class="ch.qos.logback.core.rolling.RollingFileAppender">
<file>${LOGS}/DataRiver-Err.log</file>
<append>true</append>
<encoder class="ch.qos.logback.classic.encoder.PatternLayoutEncoder">
<Pattern>%p %d{ISO8601} [%t-[%X{threadid}]] - %m%n</Pattern>
</encoder>
<rollingPolicy class="ch.qos.logback.core.rolling.TimeBasedRollingPolicy">
<fileNamePattern>${LOGS}/archived/DataRiver-Err-%d{yyyy-MM-dd}.log</fileNamePattern>
</rollingPolicy>
</appender>
<logger name="dataRiver" level="INFO" additivity="false">
<appender-ref ref="DATA-RIVER"/>
</logger>
<logger name="error" level="WARN" additivity="false">
<appender-ref ref="ERROR"/>
</logger>
这是我的日志服务:
public class LoggingService {
public static final Logger LOGGER_DATA_RIVER = LoggerFactory.getLogger("dataRiver");
public static final Logger LOGGER_ERROR = LoggerFactory.getLogger("error");
}
以下是我的消费者配置:
@Bean
public ConsumerFactory<String, GenericData.Record> testConsumerFactoryFirst() {
Map<String, Object> dataRiverProps = setTestDataRiverProps();
dataRiverProps.put(ConsumerConfig.BOOTSTRAP_SERVERS_CONFIG, env.getProperty("test1.bootstrap.servers"));
dataRiverProps.put(KafkaAvroDeserializerConfig.SCHEMA_REGISTRY_URL_CONFIG, env.getProperty("test1.schema.registry.url"));
return new DefaultKafkaConsumerFactory<>(dataRiverProps);
}
private ConcurrentKafkaListenerContainerFactory<String, GenericData.Record> testKafkaListenerContainerFactory(ConsumerFactory<String, GenericData.Record> consumerFactory) {
ConcurrentKafkaListenerContainerFactory<String, GenericData.Record> factory = new ConcurrentKafkaListenerContainerFactory<>();
factory.setConsumerFactory(consumerFactory);
factory.setBatchListener(true);
return factory;
}
@Bean
public ConcurrentKafkaListenerContainerFactory<String, GenericData.Record> testKafkaListenerContainerFactoryFirst() {
return testKafkaListenerContainerFactory(testConsumerFactoryFirst());
}
以下是消费者:
@KafkaListener(topics = "#{'${test.kafka.topics}'.split(',')}", containerFactory = "testKafkaListenerContainerFactoryFirst")
public void consumeAvroFirst(List<Message<GenericData.Record>> list) {
consumeJsonMessageBatch(convertAvroToJsonBatch(list), ""Kafka Consumer test-1");
}
private List<String> convertAvroToJsonBatch(List<Message<GenericData.Record>> list) {
return list.stream().map(record -> record.getPayload().toString()).collect(Collectors.toList());
}
以下是catalina.out的摘录:
2019-04-20 02:16:54.190 WARN 18286 --- [ntainer#1-0-C-1] org.apache.kafka.clients.NetworkClient : [Consumer clientId=consumer-12, groupId=DataRiver1] 10 partitions have leader brokers without a matching listener, including [test.autoEvents-8, test.autoEvents-2, test.autoEvents-4, test.loginEvaluationEvents-17, test.loginEvaluationEvents-11, test.loginEvaluationEvents-5, test.loginEvaluationEvents-13, test.loginEvaluationEvents-7, test.loginEvaluationEvents-1, test.loginEvaluationEvents-19]
2019-04-20 02:16:54.320 WARN 18286 --- [ntainer#1-0-C-1] org.apache.kafka.clients.NetworkClient : [Consumer clientId=consumer-12, groupId=DataRiver1] 10 partitions have leader brokers without a matching listener, including [test.autoEvents-8, test.autoEvents-2, test.autoEvents-4, test.loginEvaluationEvents-17, test.loginEvaluationEvents-11, test.loginEvaluationEvents-5, test.loginEvaluationEvents-13, test.loginEvaluationEvents-7, test.loginEvaluationEvents-1, test.loginEvaluationEvents-19]
2019-04-20 02:16:54.320 WARN 18286 --- [ntainer#0-0-C-1] org.apache.kafka.clients.NetworkClient : [Consumer clientId=consumer-10, groupId=DataRiver1] 10 partitions have leader brokers without a matching listener, including [test.autoEvents-8, test.autoEvents-2, test.autoEvents-4, test.loginEvaluationEvents-17, test.loginEvaluationEvents-11, test.loginEvaluationEvents-5, test.loginEvaluationEvents-13, test.loginEvaluationEvents-7, test.loginEvaluationEvents-1, test.loginEvaluationEvents-19]
2019-04-20 02:16:54.346 WARN 18286 --- [ntainer#2-0-C-1] org.apache.kafka.clients.NetworkClient : [Consumer clientId=consumer-2, groupId=DataRiver1] 10 partitions have leader brokers without a matching listener, including [test.autoEvents-8, test.autoEvents-2, test.autoEvents-4, test.loginEvaluationEvents-17, test.loginEvaluationEvents-11, test.loginEvaluationEvents-5, test.loginEvaluationEvents-13, test.loginEvaluationEvents-7, test.loginEvaluationEvents-1, test.loginEvaluationEvents-19]
谢谢你的帮助!
1条答案
按热度按时间uttx8gqw1#
在logback-spring.xml中创建新的appender。在appender中添加一个过滤器。创建一个过滤器类来匹配kafka日志。将appender输出到所需的位置。
附录:
筛选器类:
我建议大家看看这个答案:https://stackoverflow.com/a/5653532/10368579
logback文档在此处提供了有关匹配器的更多信息:https://logback.qos.ch/manual/filters.html#matcher