如何将高级kafka配置添加到landoop“fast data dev”映像

3pvhb19x  于 2021-06-07  发布在  Kafka
关注(0)|答案(2)|浏览(330)

有人知道如何将自定义Kafka配置选项传递给 landoop/fast-data-dev Docker 形象?
因为没有办法puss一个定制的配置文件和/或配置参数,所以到目前为止我尝试的是挂载我自己的配置文件和/或配置参数 server.properties 将文件配置到 /opt/confluent/etc/kafka 通过将以下内容添加到我的docker compose文件中

landoop:
hostname: 'landoop'
image: 'landoop/fast-data-dev:latest'
expose:
  - '3030'
ports:
  - '3030:3030'
environment:
  - RUNTESTS=0
  - RUN_AS_ROOT=1
volumes:
  - ./docker/landoop/tmp:/tmp
  - ./docker/landoop/opt/confluent/etc/kafka:/opt/confluent/etc/kafka

但是,这会导致Kafka抛出以下日志:

landoop_1       | 2017-09-28 11:53:03,886 INFO exited: broker (exit status 1; not expected)
landoop_1       | 2017-09-28 11:53:04,749 INFO spawned: 'broker' with pid 281
landoop_1       | 2017-09-28 11:53:05,851 INFO success: broker entered RUNNING state, process has stayed up for > than 1 seconds (startsecs)
landoop_1       | 2017-09-28 11:53:11,867 INFO exited: rest-proxy (exit status 1; not expected)
landoop_1       | 2017-09-28 11:53:12,604 INFO spawned: 'rest-proxy' with pid 314
landoop_1       | 2017-09-28 11:53:13,024 INFO exited: schema-registry (exit status 1; not expected)
landoop_1       | 2017-09-28 11:53:13,735 INFO spawned: 'schema-registry' with pid 341
landoop_1       | 2017-09-28 11:53:13,739 INFO success: rest-proxy entered RUNNING state, process has stayed up for > than 1 seconds (startsecs)

另外,当我去http://localhost:3030/kafka topics ui/,我看到以下内容:

KAFKA REST
/api/kafka-rest-proxy 
CONNECTIVITY ERROR

有什么建议吗?谢谢您。

oxiaedzo

oxiaedzo1#

目前,使用最新版本的landoop/fast data dev,可以通过将配置选项转换为大写、用下划线替换点以及用kafka前缀来指定自定义kafka配置选项。例如,如果要为“log.retention.bytes”和“log.retention.hours”设置特定值,则应将以下内容添加到docker compose环境部分:

environment:
  KAFKA_LOG_RETENTION_BYTES: 1073741824
  KAFKA_LOG_RETENTION_HOURS: 48
  ADV_HOST: 127.0.0.1                  
  RUNTESTS: 0   
  BROWSECONFIGS: 1

您还可以通过这种方式为其他服务(schema registry、connect、rest proxy)指定配置选项。查看文档了解详细信息https://hub.docker.com/r/landoop/fast-data-dev/.
一旦容器启动,您可以通过查看容器内以下路径的配置文件来确认这一点: /run/broker/server.properties 如果在环境参数上将“browseconfigs”设置为1,也可以通过以下url的landoop ui:
http://127.0.0.1:3030/config/broker/server.properties

lqfhib0f

lqfhib0f2#

我做了几件事,简化了整个过程。这只对dev环境有效
以交互式shell作为入口点启动docker
启动主机网络上的docker
对server.properties文件进行必要的更改,包括docker运行的主机ip(如(2)docker在主机网络上运行中所述)

4. 如果您想进行任何高级配置,可以现在进行

运行实际入口点“/usr/local/bin/setup并运行.sh”
实际使用的命令:
启动容器
sudo docker run-it--entrypoint/bin/bash--net=host--rm-e adv\u host=hostip landoop/fast数据-dev:latest
将以下内容添加到/run/broker/server.properties
advised.host.name=主机ip
advised.port=9092
运行/usr/local/bin/setup-and-run.sh

相关问题