kafkaproducer第一次尝试发送记录时无法创建主题无效的\u复制\u因子

kiayqfof  于 2021-06-07  发布在  Kafka
关注(0)|答案(2)|浏览(535)

尝试将第一条消息发布到新主题时,日志中出现以下错误。
warn][o.a.kafka.clients.networkclient][[producer clientid=producer-1]获取相关id为766890的元数据时出错:{mytopic-1=invalid\u replication\u factor,mytopic-2=invalid\u replication\u factor}][]
Kafka失败的原因在于:

"Hashed wheel timer #1" #521 prio=5 os_prio=0 tid=0x00007f932cd7d000 nid=0x199fa in Object.wait() [0x00007f9322b79000]
   java.lang.Thread.State: TIMED_WAITING (on object monitor)
    at java.lang.Object.wait(Native Method)
    at org.apache.kafka.clients.Metadata.awaitUpdate(Metadata.java:177)
    - locked <0x000000047838b990> (a org.apache.kafka.clients.Metadata)
    at org.apache.kafka.clients.producer.KafkaProducer.waitOnMetadata(KafkaProducer.java:903)
    at org.apache.kafka.clients.producer.KafkaProducer.doSend(KafkaProducer.java:794)
    at org.apache.kafka.clients.producer.KafkaProducer.send(KafkaProducer.java:784)

我们使用的kafka jar版本是:0.10.0.1
代理上的server.properties是:

broker.id=3
host.name=<>
port=9092

message.max.bytes=20971520
num.partitions=30
auto.create.topics.enable=true

# Replication configurations

default.replication.factor=2
num.replica.fetchers=2
replica.fetch.max.bytes=20971520

log.dirs=/mnt1/data/kafka/kafka-logs-3
log.retention.hours=48
log.flush.interval.ms=10000
log.flush.interval.messages=20000
log.flush.scheduler.interval.ms=2000
log.cleanup.interval.mins=30

zookeeper.connect=<>
zookeeper.connection.timeout.ms=1000000

# Socket server configuration

num.io.threads=8
num.network.threads=8
socket.request.max.bytes=20971520
socket.receive.buffer.bytes=20971520
socket.send.buffer.bytes=20971520
queued.max.requests=32
fetch.purgatory.purge.interval.requests=100
producer.purgatory.purge.interval.requests=100

inter.broker.protocol.version=0.10.2.0
log.message.format.version=0.10.0
delete.topic.enable=true

为什么我不能发送消息?没有创建主题本身!正在使用的生产商属性是:

Properties props = new Properties();
        props.put("acks", "0");
        props.put("retries", "0");
        props.put("batch.size", "16384");
        props.put("linger.ms", "100");
        props.put("buffer.memory", "33554432");
        props.put("key.serializer", "org.apache.kafka.common.serialization.IntegerSerializer");
        props.put("value.serializer", "org.apache.kafka.common.serialization.StringSerializer");
        props.put("max.request.size", String.valueOf(ByteUnit.MB.toBytes(10)));
        return props;
lf3rwulv

lf3rwulv1#

我也遇到了同样的问题,在搜索Kafka代码库时,我意识到在某些情况下, Errors.COORDINATOR_NOT_AVAILABLE 报告为 Errors.INVALID_REPLICATION_FACTOR 在我的例子中,实际的潜在问题是使用者配置中指定的主题不存在
裁判:https://github.com/apache/kafka/blob/322b10964ce71eac81da9e574be7dec59c0fc93d/core/src/main/scala/kafka/server/kafkaapis.scala#l1106

bq9c1y66

bq9c1y662#

您在集群中有多少经纪人?这个 default.replication.factor2 ,这意味着您必须至少有2个代理正在运行。

相关问题