嘿,我已经试着解决这个问题两个星期了。基本上,我希望kafka在ssl上,模式注册表在https上。没有要使用的kerberos。我有两个spring服务,一个是生产者,另一个是消费者(avro)
这是我当前的docker compose,当我向生产者发送请求时,它不会在应用程序中抛出任何错误,请求超时,但kafka日志显示 kafka_1 | [2019-12-03 09:53:27,454] INFO [SocketServer brokerId=1] Failed authentication with /172.18.0.1 (SSL handshake failed) (org.apache.kafka.common.network.Selector)
当我取消docker compose中的行的注解时,我得到 PKIX path building failed
以及其他一些错误,指定avro不能序列化或类似的东西
zookeeper:
image: confluentinc/cp-zookeeper:5.3.0
ports:
- 2181:2181
environment:
ZOOKEEPER_CLIENT_PORT: "2181"
ZOOKEEPER_TICK_TIME: "2000"
kafka:
image: confluentinc/cp-kafka:5.3.0
ports:
- 29094:29094
environment:
KAFKA_BROKER_ID: 1
KAFKA_LISTENER_SECURITY_PROTOCOL_MAP: PLAINTEXT:PLAINTEXT,SSL:SSL
KAFKA_SECURITY_PROTOCOL: SSL
KAFKA_INTER_BROKER_PROTOCOL: SSL
KAFKA_INTER_BROKER_LISTENER_NAME: SSL
KAFKA_LISTENERS: SSL://kafka:29094,PLAINTEXT://kafka:9092
KAFKA_ADVERTISED_LISTENERS: SSL://kafka:29094,PLAINTEXT://kafka:9092
KAFKA_ZOOKEEPER_CONNECT: zookeeper:2181
KAFKA_AUTO_CREATE_TOPICS_ENABLE: "true"
KAFKA_OFFSETS_TOPIC_REPLICATION_FACTOR: 1
# KAFKA_SSL_CLIENT_AUTH: required
KAFKA_SSL_KEYSTORE_FILENAME: kafka.server.keystore.jks
KAFKA_SSL_TRUSTSTORE_FILENAME: kafka.server.truststore.jks
KAFKA_SSL_KEY_CREDENTIALS: key_credential
KAFKA_SSL_KEYSTORE_CREDENTIALS: key_credential
KAFKA_SSL_KEYSTORE_LOCATION: /etc/kafka/secrets/kafka.server.keystore.jks
KAFKA_SSL_KEYSTORE_PASSWORD: PASSWORD
KAFKA_SSL_KEY_PASSWORD: PASSWORD
KAFKA_SSL_TRUSTSTORE_LOCATION: /etc/kafka/secrets/kafka.server.truststore.jks
KAFKA_SSL_TRUSTSTORE_PASSWORD: PASSWORD
KAFKA_SSL_TRUSTSTORE_CREDENTIALS: key_credential
KAFKA_HEAP_OPTS: -Xmx456M
KAFKA_SSL_ENDPOINT_IDENTIFICATION_ALGORITHM: ""
# KAFKA_AUTHORIZER_CLASS_NAME: kafka.security.auth.SimpleAclAuthorizer
KAFKA_ALLOW_EVERYONE_IF_NO_ACL_FOUND: "true"
KAFKA_SUPER_USERS: User:CN=Kafka-domain
volumes:
- /var/run/docker.sock:/var/run/docker.sock
- /home/xubu/Documents:/etc/kafka/secrets
schema-registry:
image: confluentinc/cp-schema-registry:5.3.0
depends_on:
- zookeeper
- kafka
ports:
- 8181:8181
- 8085:8085
- 8086:8086
environment:
SCHEMA_REGISTRY_HOST_NAME: schema-registry
SCHEMA_REGISTRY_LISTENERS: http://schema-registry:8085, https://schema-registry:8086
SCHEMA_REGISTRY_KAFKASTORE_CONNECTION_URL: zookeeper:2181
SCHEMA_REGISTRY_KAFKASTORE_SECURITY_PROTOCOL: SSL
SCHEMA_REGISTRY_KAFKASTORE_BOOTSTRAP_SERVERS: SSL://kafka:29094
SCHEMA_REGISTRY_KAFKASTORE_SSL_TRUSTSTORE_LOCATION: /etc/kafka/client/kafka.client.truststore.jks
SCHEMA_REGISTRY_KAFKASTORE_SSL_TRUSTSTORE_PASSWORD: PASSWORD
SCHEMA_REGISTRY_KAFKASTORE_SSL_KEYSTORE_LOCATION: /etc/kafka/client/kafka.client.keystore.jks
SCHEMA_REGISTRY_KAFKASTORE_SSL_KEYSTORE_PASSWORD: PASSWORD
SCHEMA_REGISTRY_KAFKASTORE_SSL_KEY_PASSWORD: PASSWORD
SCHEMA_REGISTRY_KAFKASTORE_SSL_ENDPOINT_IDENTIFICATION_ALGORITHM: ""
SCHEMA_REGISTRY_SSL_TRUSTSTORE_LOCATION: /etc/kafka/client/kafka.client.truststore.jks
SCHEMA_REGISTRY_SSL_TRUSTSTORE_PASSWORD: PASSWORD
SCHEMA_REGISTRY_SSL_KEYSTORE_LOCATION: /etc/kafka/client/kafka.client.keystore.jks
SCHEMA_REGISTRY_SSL_KEYSTORE_PASSWORD: PASSWORD
SCHEMA_REGISTRY_SSL_KEY_PASSWORD: PASSWORD
SCHEMA_REGISTRY_SSL_ENDPOINT_IDENTIFICATION_ALGORITHM: ""
SCHEMA_REGISTRY_SECURITY_PROTOCOL: SSL
SCHEMA_REGISTRY_INTER_INSTANCE_PROTOCOL: "https"
#SCHEMA_REGISTRY_SSL_CLIENT_AUTH: 'true'
volumes:
- /home/xubu/Documents:/etc/kafka/client
- /home/xubu/Documents:/etc/kafka/consumer
下面是我的spring boot application.yaml的一部分
spring:
kafka:
bootstrap-servers: kafka:29094
producer:
key-serializer: org.apache.kafka.common.serialization.StringSerializer
value-serializer: io.confluent.kafka.serializers.KafkaAvroSerializer
ssl:
key-store-location: /home/xubu/Documents/kafka.client.keystore.jks
key-password: PASSWORD
key-store-password: PASSWORD
trust-store-location: /home/xubu/Documents/kafka.client.truststore.jks
trust-store-password: PASSWORD
protocol: SSL
properties:
value:
subject:
name:
strategy: io.confluent.kafka.serializers.subject.RecordNameStrategy
value-serializer: io.confluent.kafka.serializers.KafkaAvroSerializer
ssl.endpoint.identification.algorithm: https
schema.registry.url: https://schema-registry:8086
ssl:
trust-store-location: /home/xubu/Documents/kafka.client.truststore.jks
trust-store-password: PASSWORD
key-store-location: /home/xubu/Documents/kafka.client.keystore.jks
key-store-password: PASSWORD
key-password: PASSWORD
protocol: SSL
key-store-type: jks
trust-store-type: jks
这是我过去两周来的苦衷,这是一篇介绍如何在schema registry上尝试访问控制列表的文章
暂无答案!
目前还没有任何答案,快来回答吧!