如何解决kafka producer中的recordtoolargeexception?

rdlzhqv9  于 2021-06-07  发布在  Kafka
关注(0)|答案(1)|浏览(687)

我用flinkkafkaproducer08把唱片寄给Kafka。但有时我会遇到以下异常,即使我在错误消息中打印的记录太小,大小为0.02MB。

java.lang.RuntimeException: Could not forward element to next operator
Caused by: java.lang.RuntimeException: Could not forward element to next operator
Caused by: java.lang.Exception: Failed to send data to Kafka: The message is 1513657 bytes when serialized which is larger than the maximum request size you have configured with the max.request.size configuration.
Caused by: org.apache.kafka.common.errors.RecordTooLargeException: The message is 1513657 bytes when serialized which is larger than the maximum request size you have configured with the max.request.size configuration

试图更改producer上的max.request.size,但这需要代理更改并重新启动代理。

bis0qfac

bis0qfac1#

还有一个最大消息大小的代理设置:

message.max.bytes : 1,000,000 (default)

因此,您需要重新启动代理--但这不应该是一个问题。Kafka是建立在一个强大的方式就经纪人反弹。
查阅http://kafka.apache.org/082/documentation.html#producerconfigs

相关问题