如何向prometheus报告kafka制作人的度量(使用spring boot)

lymgl2op  于 2021-06-06  发布在  Kafka
关注(0)|答案(1)|浏览(401)

我正在使用spring集成从udp端点到kafka的数据流。我已经在@configuration中将replyingkafkatemplate初始化为@bean,同时使用consumer和producer配置。当我的服务器启动并且发送了一些udp请求之后,我可以看到消费者的指标。但是,即使在producer配置中设置了jmx reporter,我也看不到producer的度量。
我试着不设置producer metrics reporter,假设它将自动显示为consumer metrics(没有额外的配置)
生产者配置

Map<String, Object> configProps = new HashMap<>();
        configProps.put(
                ProducerConfig.BOOTSTRAP_SERVERS_CONFIG,
                bootstrapAddress);
        configProps.put(
                ProducerConfig.KEY_SERIALIZER_CLASS_CONFIG,
                StringSerializer.class);
        configProps.put(
                ProducerConfig.VALUE_SERIALIZER_CLASS_CONFIG,
                KafkaAvroSerializer.class);
        configProps.put("schema.registry.url", "http://schema-regisry-server:8081");
        configProps.put(
                ProducerConfig.RETRIES_CONFIG,
                3);
        configProps.put(ProducerConfig.RECONNECT_BACKOFF_MS_CONFIG, 500);
        configProps.put(ProducerConfig.RECONNECT_BACKOFF_MAX_MS_CONFIG, 5000);
        configProps.put(ProducerConfig.METRIC_REPORTER_CLASSES_CONFIG, "org.apache.kafka.common.metrics.JmxReporter");
        configProps.put(ProducerConfig.METRICS_RECORDING_LEVEL_CONFIG, "INFO");

        printConfigProps(configProps);
        return new DefaultKafkaProducerFactory<>(configProps);

消费者配置

Map<String, Object> properties = new HashMap<>();
        properties.put(ConsumerConfig.BOOTSTRAP_SERVERS_CONFIG, bootstrapAddress);
        properties.put(ConsumerConfig.KEY_DESERIALIZER_CLASS_CONFIG, StringDeserializer.class);
        properties.put(ConsumerConfig.VALUE_DESERIALIZER_CLASS_CONFIG, KafkaAvroDeserializer.class);
        properties.put("schema.registry.url", "http://schema-regisry-server:8081");
        properties.put(ConsumerConfig.GROUP_ID_CONFIG, "spring-integration");
        // automatically reset the offset to the earliest offset
        properties.put(ConsumerConfig.AUTO_OFFSET_RESET_CONFIG, "earliest");

        return properties;

Kafka模板创建

@Bean
    public ReplyingKafkaTemplate<String, DataModel, DataModel> replyKafkaTemplate(ProducerFactory<String, DataModel> pf, KafkaMessageListenerContainer<String, DataModel> container) {
        ReplyingKafkaTemplate<String, DataModel, DataModel> template = new ReplyingKafkaTemplate<>(pf, container);
        template.start();
        return template;
    }

侦听器容器创建:

@Bean
    public KafkaMessageListenerContainer<String, DataModel> replyContainer(ConsumerFactory<String, DataModel> cf) {
        ContainerProperties containerProperties = new ContainerProperties(destinationTopic);
        containerProperties.setGroupId("test");
        return new KafkaMessageListenerContainer<>(cf, containerProperties);
    }

消费工厂创建

@Bean
    public ConsumerFactory<?, ?> consumerFactory() {
        return new DefaultKafkaConsumerFactory<>(consumerConfigs());
    }
cwxwcias

cwxwcias1#

SpringBoot2 2.3.0之前的版本在默认情况下只公开消费者指标。springboot2.3.0(几周前发布)依赖于micrometer 1.4,默认情况下,micrometer 1.4公开了消费者和生产者的指标。如果你不能使用最新版本的springboot,你就必须自己实现它。

相关问题