如果在没有kerberos安全性的情况下使用kafka到kafka运行一个简单的作业,那么一切都可以工作。我需要做同样的事情,但背后的kerberos安全。请看下面我的工作代码:
job.name=Kafka2KafkaSimple
job.group=Kafka
job.description=This is a job that runs forever, copies an input Kafka topic to an output Kafka topic
job.lock.enabled=true
kafka.brokers=[host1]:9092
source.class=org.apache.gobblin.source.extractor.extract.kafka.UniversalKafkaSource
gobblin.source.kafka.extractorType=DESERIALIZER
kafka.deserializer.type=BYTE_ARRAY
extract.namespace=org.apache.gobblin.extract.kafka
bootstrap.with.offset=latest
topic.whitelist=logging_application_access
writer.builder.class=org.apache.gobblin.kafka.writer.KafkaDataWriterBuilder
writer.kafka.topic=logging_application_access
writer.kafka.producerConfig.bootstrap.servers=[host2]:9092
writer.kafka.producerConfig.value.serializer=org.apache.kafka.common.serialization.ByteArraySerializer
writer.output.format=TEXT
data.publisher.type=org.apache.gobblin.publisher.NoopPublisher
gobblin.yarn.keytab.file.path="/etc/security/keytabs/[secret].keytab"
gobblin.yarn.keytab.principal.name=[secret]
输出问题如下:
2019-01-21 14:40:34 BRST WARN [kafka-producer-network-thread | gobblin] org.apache.kafka.common.network.Selector 276 - Error in I/O with [hostname]/[ip]
java.io.EOFException
at org.apache.kafka.common.network.NetworkReceive.readFrom(NetworkReceive.java:62)
at org.apache.kafka.common.network.Selector.poll(Selector.java:248)
at org.apache.kafka.clients.NetworkClient.poll(NetworkClient.java:192)
at org.apache.kafka.clients.producer.internals.Sender.run(Sender.java:191)
at org.apache.kafka.clients.producer.internals.Sender.run(Sender.java:122)
at java.lang.Thread.run(Thread.java:748)
暂无答案!
目前还没有任何答案,快来回答吧!