如果我从c++应用程序或从本地计算机的命令行生成和使用kafka服务器,那么我的kafka服务器工作得很好。
但是它在外部ip地址上工作不好:创建主题时,我看到使用tcpdump和acks的网络流量(意味着kafka正在应答),但是我在队列中没有发现消息,java也没有给出错误。日志上什么也找不到。或者谷歌。这是我的应用程序:
public class KafkaPBJProducer {
private static String KAFKA_TOPIC = "test";
public static void main(String[] args) {
Properties properties = new Properties();
properties.put("bootstrap.servers", "192.168.1.131:9092");
properties.put("acks", "all");
properties.put("retries", 0);
properties.put("batch.size", 16384);
properties.put("linger.ms", 1);
properties.put("buffer.memory", 33554432);
properties.put("key.serializer", "org.apache.kafka.common.serialization.StringSerializer");
properties.put("value.serializer", "org.apache.kafka.common.serialization.StringSerializer");
properties.put("kafka.topic", KAFKA_TOPIC);
runMainLoop(properties);
}
static void runMainLoop(Properties properties) {
@SuppressWarnings("resource")
KafkaProducer<String, String> producer = new KafkaProducer<String, String>(properties);
for (int count = 0; count < 10; count++) {
String msg = getMsg(String.valueOf(count));
System.out.println("Producing message: " + msg);
producer.send(new ProducerRecord<String, String>(KAFKA_TOPIC, 0, "dev-" + count, msg));
producer.flush();
}
}
public static String getMsg(String id) {
JsonObject obj = new JsonObject();
try {
obj.addProperty("id", id);
obj.addProperty("timestamp", new Timestamp(System.currentTimeMillis()).toString());
obj.addProperty("data", Base64.getEncoder().encodeToString("Hello, World!".getBytes("utf-8")));
} catch (UnsupportedEncodingException e) {
e.printStackTrace();
}
return new Gson().toJson(obj);
}
}
当它运行时,每一条消息都会阻塞一分钟(不会显示错误,它们只是需要很长时间才能被发送):
Producing message: {"id":"0","timestamp":"2018-08-21 13:49:10.697","data":"SGVsbG8sIFdvcmxkIQ\u003d\u003d"}
Producing message: {"id":"1","timestamp":"2018-08-21 13:50:10.794","data":"SGVsbG8sIFdvcmxkIQ\u003d\u003d"}
Producing message: {"id":"2","timestamp":"2018-08-21 13:51:10.797","data":"SGVsbG8sIFdvcmxkIQ\u003d\u003d"}
Producing message: {"id":"3","timestamp":"2018-08-21 13:52:10.813","data":"SGVsbG8sIFdvcmxkIQ\u003d\u003d"}
我可以看到主题是:
> bin/kafka-topics.sh --list --zookeeper localhost:2181
__consumer_offsets
test
我可以使用命令行手动向主题添加内容:
$ bin/kafka-console-producer.sh --broker-list localhost:9092 --topic test
>one
>two
>four
但如果我选中主题,则只列出手动发送的消息:
$ bin/kafka-console-consumer.sh --bootstrap-server 192.168.1.131:9092 --topic test --from-beginning
one
two
four
(after several minutes, I press CTRL-C)
^CProcessed a total of 3 messages
这里发生了什么?为什么创建了主题,但服务器上不接受任何消息?为什么Kafka没有报告任何错误?
蒂亚!
1条答案
按热度按时间lf5gs5x21#
找到了。基于kafka-无法使用java向远程服务器发送消息,问题出在配置文件conf/servers.properties上;需要取消对此行的注解:
无论如何谢谢你。