使用自定义编码器示例化kafka生产者时,无法示例化

lnxxn5zx  于 2021-06-07  发布在  Kafka
关注(0)|答案(3)|浏览(385)

我是Kafka的新人。我想在我的spring项目中使用kafka生产者/消费者来替换activemq(jms)。我需要的是一个kafka生产者将我的消息对象发布到一个主题,然后一个消费者从该主题订阅它。
首先是我的自定义编码器,解码器也是如此(对于我的消息类configurationactionmsg):

@Component
public class ConfigActionMessageEncoder implements Encoder<ConfigurationActionMsg> {

    public ConfigActionMessageEncoder() {
    /* This constructor must be present for successful compile. */
    }
    public ConfigActionMessageEncoder(VerifiableProperties verifiableProperties) {
    /* This constructor must be present for successful compile. */
    }

    @Override
    public byte[] toBytes(ConfigurationActionMsg actionMsg){
        return SerializationUtils.serialize(actionMsg);
    }}

下面是我对处理器和使用者的配置

@Configuration
@ComponentScan(basePackages = {"XXX"})
public class KafkaConfig {
@Bean
public KafkaProducer<String,ConfigurationActionMsg> kafkaProducer(){
    Properties props = new Properties();
    props.put("zk.connect", "127.0.0.1:2181");
    props.put("bootstrap.servers", "localhost:9092");
    props.put("acks", "all");
    props.put("retries", 0);
    props.put("batch.size", 16384);
    props.put("linger.ms", 1);
    props.put("buffer.memory", 33554432);
    props.put("key.serializer", "org.apache.kafka.common.serialization.StringSerializer");
    props.put("value.serializer", "com.atlas.configengine2.XXX.ConfigActionMessageEncoder");

    return new KafkaProducer<>(props);
}

@Bean
public KafkaConsumer<String, ConfigurationActionMsg> kafkaConsumer(){

    Properties props = new Properties();
    props.put("zk.connect", "127.0.0.1:2181");
    props.put("bootstrap.servers", "localhost:9092");
    //We should only have one process running for consumer
    props.put("group.id", "resolverActionTrigger");
    props.put("enable.auto.commit", "true");
    props.put("auto.commit.interval.ms", "1000");
    props.put("session.timeout.ms", "30000");
    props.put("key.deserializer", "org.apache.kafka.common.serialization.StringDeserializer");
    props.put("value.deserializer", "com.atlas.configengine2.XXX.ConfigActionMessageDecoder");
    KafkaConsumer<String, ConfigurationActionMsg> consumer = new KafkaConsumer<>(props);
    consumer.subscribe(Arrays.asList("configAction"));
    return consumer;
}}

我不确定这是否是示例化生产者/消费者的正确方法。但这种方法行不通。因为我的kafkaproducer无法示例化。
一些调试信息:
原因:org.apache.kafka.common.kafkae异常:com.atlas.configengine2.jms.configactionmessageencoder不是org.apache.kafka.common.serialization.serializer的示例
但我不确定这是否是唯一的问题?那么如何编写自定义编码器呢?

mbzjlibv

mbzjlibv1#

//create properties
        //https://kafka.apache.org/documentation/#producerconfigs
        Properties prop = new Properties();
        prop.setProperty(ProducerConfig.BOOTSTRAP_SERVERS_CONFIG, "127.0.0.1:9092,127.0.0.1:9094,127.0.0.1:9098");
        prop.setProperty(ProducerConfig.KEY_SERIALIZER_CLASS_CONFIG, StringSerializer.class.getName());
        prop.setProperty(ProducerConfig.VALUE_SERIALIZER_CLASS_CONFIG, Employee.class.getName());

        //Create Producer
        //For String
        //KafkaProducer<String, String> producer= new KafkaProducer(prop);
        //For Object
        KafkaProducer<String, Employee> producer = new KafkaProducer(prop);
        Address add = new Address("India");
        Employee emp = new Employee(1, "Arun", add);
        //For String
    // ProducerRecord prodRecord = new ProducerRecord("aryan_topic", "Any String Value");
        ProducerRecord prodRecord = new ProducerRecord("aryan_topic", emp);
    //Send Record

        producer.send(prodRecord);
        System.out.println("Data Send");

}
}
import org.apache.kafka.common.header.Headers;

导入org.apache.kafka.common.serialization.serializer;
导入java.io.serializable;导入java.util.map;
//由arun singh开发

public class Employee implements Serializable, Serializer {
//add all unImplemented methods here
}
 and it will work as
//
[main] INFO org.apache.kafka.common.utils.AppInfoParser - Kafka version: 2.4.0
[main] INFO org.apache.kafka.common.utils.AppInfoParser - Kafka commitId: 77a89fcf8d7fa018
[main] INFO org.apache.kafka.common.utils.AppInfoParser - Kafka startTimeMs: 1583515298948
[kafka-producer-network-thread | producer-1] INFO org.apache.kafka.clients.Metadata - [Producer clientId=producer-1] Cluster ID: h_EDcwYmThiz-iBTc0AsGw
Data Send
oipij1gg

oipij1gg2#

你应该实施 org.apache.kafka.common.serialization.Serializer 不是 Encoder .
请参见customserializer示例。

xienkqul

xienkqul3#

首先,应该使用 org.apache.kafka.common.serialization.Serializer .
第二,我认为您可以发送对象的字符串表示,例如json,并跟上 StringSerializer 在producer中,以避免实现自定义 org.apache.kafka.common.serialization.Serializer .

相关问题