bin/kafka-producer-perf-test.sh --messages 1000000 --message-size 1000 --topics kafka_producertest_1replication_1partitions --broker-list mesos006:9092,mesos007:9092,mesos008:9092 --threads 1 --compression-codec 3 --batch-size 100000例外情况：
Exception in thread "main" kafka.common.InvalidConfigException: Batch size = 100000 can't be larger than queue size = 10000
at kafka.producer.ProducerConfig$.validateBatchSize(ProducerConfig.scala:39)
at kafka.producer.ProducerConfig$.validate(ProducerConfig.scala:29)
at kafka.producer.ProducerConfig.<init>(ProducerConfig.scala:116)
at kafka.producer.ProducerConfig.<init>(ProducerConfig.scala:56)
at kafka.producer.OldProducer.<init>(BaseProducer.scala:59)
at kafka.tools.ProducerPerformance$ProducerThread.<init>(ProducerPerformance.scala:210)
at kafka.tools.ProducerPerformance$$anonfun$main$1.apply$mcVI$sp(ProducerPerformance.scala:58)
at scala.collection.immutable.Range.foreach$mVc$sp(Range.scala:141)
at kafka.tools.ProducerPerformance$.main(ProducerPerformance.scala:57)
at kafka.tools.ProducerPerformance.main(ProducerPerformance.scala)配置/producer.properties：
queue.buffering.max.messages=100000使用kafka-producer-perf-test.sh时，如何设置producer的所有参数。或者给它一个配置文件（例如：config/producer.properties）如何使config/producer.properties生效？
kafka-producer-perf-test.sh参数
[root@mesos006 kafka_2.10-0.8.2.0]# bin/kafka-producer-perf-test.sh
Missing required argument "[topics]"
Option Description
------ -----------
--batch-size <Integer: size> Number of messages to write in a
single batch. (default: 200)
--broker-list <hostname:port,.., REQUIRED: broker info (the list of
hostname:port> broker host and port for bootstrap.
--compression-codec <Integer: If set, messages are sent compressed
supported codec: NoCompressionCodec (default: 0)
as 0, GZIPCompressionCodec as 1,
SnappyCompressionCodec as 2,
LZ4CompressionCodec as 3>
--csv-reporter-enabled If set, the CSV metrics reporter will
be enabled
--date-format <date format> The date format to use for formatting
the time field. See java.text.
SimpleDateFormat for options.
(default: yyyy-MM-dd HH:mm:ss:SSS)
--help Print usage.
--hide-header If set, skips printing the header for
the stats
--initial-message-id <Integer: initial The is used for generating test data,
message id> If set, messages will be tagged with
an ID and sent by producer starting
from this ID sequentially. Message
content will be String type and in
the form of 'Message:000...1:xxx...'
--message-send-gap-ms <Integer: If set, the send thread will wait for
message send time gap> specified time between two sends
(default: 0)
--message-size <Integer: size> The size of each message. (default:
100)
--messages <Long: count> The number of messages to send or
consume (default:
9223372036854775807)
--metrics-dir <metrics dictory> If csv-reporter-enable is set, and
this parameter isset, the csv
metrics will be outputed here
--new-producer Use the new producer implementation.
--producer-num-retries <Integer> The producer retries number (default:
3)
--producer-retry-backoff-ms <Integer> The producer retry backoff time in
milliseconds (default: 100)
--reporting-interval <Integer: size> Interval at which to print progress
info. (default: 5000)
--request-num-acks <Integer> Number of acks required for producer
request to complete (default: -1)
--request-timeout-ms <Integer> The produce request timeout in ms
(default: 3000)
--show-detailed-stats If set, stats are reported for each
reporting interval as configured by
reporting-interval
--sync If set, messages are sent
synchronously.
--threads <Integer: number of threads> Number of sending threads. (default: 1)
--topics <topic1,topic2..> REQUIRED: The comma separated list of
topics to produce to
--vary-message-size If set, message size will vary up to
the given maximum.
2条答案
按热度按时间jdgnovmf1#
raogr8fs2#
检查附加到您正在使用的脚本的文档：
我认为你要找的论点是：
我没有试过，但值得一试。