无法使用azure函数cosmosdb触发器创建kafka主题?

lokaqttq  于 2021-06-04  发布在  Kafka
关注(0)|答案(1)|浏览(304)

我有一个azure函数cosmosdb触发器,我正在创建一个主题并将更新发送到该主题,而创建kafka主题时,我得到下面的错误,但无法得到下面错误的实际原因。
我共享了一个我正在使用的示例代码。。
我的python代码:

import logging
from confluent_kafka.admin import AdminClient, NewTopic
def getKafkaConfiguration():
    kafkaConfig = {
        "bootstrap.servers": "10.42.122.183:9094,10.42.122.184:9094,10.42.122.188
5:9094",
        "security.protocol": "SASL_SSL",
        "ssl.ca.location": "./ca-cert",
        "sasl.mechanism": "PLAIN",
        "sasl.username": "****",
        "sasl.password": "*********",
        "debug": "broker,admin"
    }
    return kafkaConfig
admin_client = AdminClient(getKafkaConfiguration())
topic_list = [NewTopic("TestAzureFunction", 1,1)]
fs = admin_client.create_topics(topic_list)
for topic, f in fs.items():
    try:
        f.result()
        logging.info("Topic {} created".format(topic))
    except Exception as exp:
        logging.exception("Failed to create topic {}: {}".format(topic, exp))

运行时出错

destroyed: (errno: Success)
%7|1595442943.207|FAIL|rdkafka#producer-1| [thrd:sasl_ssl://10.42.122.183:9094/bootstrap]: sasl_ssl://10.42.122.183:9094/bootstrap: Client is terminating (after 1436ms in
state TRY_CONNECT)
%7|1595442943.208|STATE|rdkafka#producer-1| [thrd:sasl_ssl://10.42.122.183:9094/bootstrap]: sasl_ssl://10.42.122.183:9094/bootstrap: Broker changed state TRY_CONNECT -> DO
WN
%7|1595442943.208|BROKERFAIL|rdkafka#producer-1| [thrd:sasl_ssl://10.42.122.183:9094/bootstrap]: sasl_ssl://10.42.122.183:9094/bootstrap: failed: err: Local: Broker handle
 destroyed: (errno: Success)
%7|1595442943.207|TERM|rdkafka#producer-1| [thrd::0/internal]: :0/internal: Received TERMINATE op in state INIT: 1 refcnts, 0 toppar(s), 0 active toppar(s), 0 outbufs, 0 w
aitresps, 0 retrybufs
%7|1595442943.208|BROKERFAIL|rdkafka#producer-1| [thrd::0/internal]: :0/internal: failed: err: Local: Broker handle destroyed: (errno: Success)
%7|1595442943.208|FAIL|rdkafka#producer-1| [thrd::0/internal]: :0/internal: Client is terminating (after 60049ms in state INIT)
%7|1595442943.208|STATE|rdkafka#producer-1| [thrd::0/internal]: :0/internal: Broker changed state INIT -> DOWN
%7|1595442943.208|TERMINATE|rdkafka#producer-1| [thrd::0/internal]: :0/internal: Handle is terminating in state DOWN: 1 refcnts (0x55e833e02ef0), 0 toppar(s), 0 active top
par(s), 0 outbufs, 0 waitresps, 0 retrybufs: failed 0 request(s) in retry+outbuf
%7|1595442943.208|BROKERFAIL|rdkafka#producer-1| [thrd::0/internal]: :0/internal: failed: err: Local: Broker handle destroyed: (errno: Success)
%7|1595442943.207|TERM|rdkafka#producer-1| [thrd:sasl_ssl://10.42.122.185:9094/bootstrap]: sasl_ssl://10.42.122.185:9094/bootstrap: Received TERMINATE op in state TRY_CONN
ECT: 1 refcnts, 0 toppar(s), 0 active toppar(s), 0 outbufs, 0 waitresps, 0 retrybufs
%7|1595442943.208|BROKERFAIL|rdkafka#producer-1| [thrd:sasl_ssl://10.42.122.185:9094/bootstrap]: sasl_ssl://10.42.122.185:9094/bootstrap: failed: err: Local: Broker handle
 destroyed: (errno: Success)
%7|1595442943.208|FAIL|rdkafka#producer-1| [thrd:sasl_ssl://10.42.122.185:9094/bootstrap]: sasl_ssl://10.42.122.185:9094/bootstrap: Client is terminating (after 2912ms in
state TRY_CONNECT)
%7|1595442943.208|STATE|rdkafka#producer-1| [thrd:sasl_ssl://10.42.122.185:9094/bootstrap]: sasl_ssl://10.42.122.185:9094/bootstrap: Broker changed state TRY_CONNECT -> DO
WN
%7|1595442943.208|BROKERFAIL|rdkafka#producer-1| [thrd:sasl_ssl://10.42.122.185:9094/bootstrap]: sasl_ssl://10.42.122.185:9094/bootstrap: failed: err: Local: Broker handle
 destroyed: (errno: Success)
%7|1595442943.207|TERM|rdkafka#producer-1| [thrd:sasl_ssl://10.42.122.184:9094/bootstrap]: sasl_ssl://10.42.122.184:9094/bootstrap: Received TERMINATE op in state TRY_CONN
ECT: 1 refcnts, 0 toppar(s), 0 active toppar(s), 0 outbufs, 0 waitresps, 0 retrybufs
%7|1595442943.208|BROKERFAIL|rdkafka#producer-1| [thrd:sasl_ssl://10.42.122.184:9094/bootstrap]: sasl_ssl://10.42.122.184:9094/bootstrap: failed: err: Local: Broker handle
 destroyed: (errno: Success)
%7|1595442943.208|FAIL|rdkafka#producer-1| [thrd:sasl_ssl://10.42.122.184:9094/bootstrap]: sasl_ssl://10.42.122.184:9094/bootstrap: Client is terminating (after 2437ms in
state TRY_CONNECT)
%7|1595442943.208|STATE|rdkafka#producer-1| [thrd:sasl_ssl://10.42.122.184:9094/bootstrap]: sasl_ssl://10.42.122.184:9094/bootstrap: Broker changed state TRY_CONNECT -> DO
WN
%7|1595442943.208|BROKERFAIL|rdkafka#producer-1| [thrd:sasl_ssl://10.42.122.184:9094/bootstrap]: sasl_ssl://10.42.122.184:9094/bootstrap: failed: err: Local: Broker handle
 destroyed: (errno: Success)
f1tvaqid

f1tvaqid1#

您应该尝试将此参数设置为true:

auto.create.topics.enable = true

这是一个代理端配置,所以找到 server.propreties 文件并添加参数,以便能够以编程方式生成主题。
afaik azure使用 Ambari ,所以您应该遵循以下步骤:
选择Kafka
选择configs并搜索 auto.create.topics.enable 参数。
将值设置为 true 重新启动所有受影响的程序。

相关问题