我是一个新手与汇合地面军事系统连接器。我遵循了中概述的文档https://docs.confluent.io/4.1.2/connect/kafka-connect-gcs/gcs_connector.html 一切都按计划进行,除了Kafka的主题数据没有出现在gcs桶里。目录结构类似于Kafka主题名,但没有实际的主题记录。我使用上述文档中包含的默认属性文件作为avro格式模式,刷新大小为3。我下载了凭据文件,并且可以从vm示例看到它。我还可以将本地文件从我的googlevm示例复制到bucket,这样我就知道我有正确的权限。
{"name": "gcs",
"config": {
"connector.class": "io.confluent.connect.gcs.GcsSinkConnector",
"tasks.max": "1",
"topics": "gcs_topic",
"gcs.bucket.name": "#bucket-name",
"gcs.part.size": "5242880",
"flush.size": "3",
"gcs.credentials.path": "#/path/to/credentials/keys.json",
"storage.class": "io.confluent.connect.gcs.storage.GcsStorage",
"format.class": "io.confluent.connect.gcs.format.avro.AvroFormat",
"partitioner.class": "io.confluent.connect.storage.partitioner.DefaultPartitioner",
"schema.compatibility": "NONE",
"confluent.topic.bootstrap.servers": "localhost:9092",
"confluent.topic.replication.factor": "1",
"name": "gcs"
},
"tasks": [],
"type": null
}
连接属性文件如下:
bootstrap.servers=localhost:9092
group.id=connect-cluster
key.converter=io.confluent.connect.avro.AvroConverter
key.converter.schema.registry.url=http://localhost:8081
value.converter=io.confluent.connect.avro.AvroConverter
value.converter.schema.registry.url=http://localhost:8081
config.storage.topic=connect-configs
offset.storage.topic=connect-offsets
status.storage.topic=connect-statuses
config.storage.replication.factor=1
offset.storage.replication.factor=1
status.storage.replication.factor=1
internal.key.converter=org.apache.kafka.connect.json.JsonConverter
internal.value.converter=org.apache.kafka.connect.json.JsonConverter
internal.key.converter.schemas.enable=false
internal.value.converter.schemas.enable=false
Plugin.path=~/confluent/share/java,~/confluent/share/confluent-hub-components,~/confluent/etc,~/confluent/etc/
rest.extension.classes=io.confluent.connect.replicator.monitoring.ReplicatorMonitoringExtension
producer.interceptor.classes=io.confluent.monitoring.clients.interceptor.MonitoringProducerInterceptor
consumer.interceptor.classes=io.confluent.monitoring.clients.interceptor.MonitoringConsumerInterceptor
暂无答案!
目前还没有任何答案,快来回答吧!