msk-kafka主题吗?

r9f1avp5  于 2021-06-04  发布在  Kafka
关注(0)|答案(1)|浏览(365)

我认为在aws-msk-kafka集群中使用lambda来消费指向某个主题的消息是微不足道的,但是我找不到从aws-docs中执行此操作的方法。有可能吗?

4urapxun

4urapxun1#

对。您可以在lambda代码中使用kafka客户机库。下面是python的一个示例:


# !/bin/env python

import json
import logging
import time
import os
from kafka import KafkaConsumer               

"""
Read data from MSK and console it out.

Required environment variables:
MSK_BOOTSTRAP_SRV: MSK Bootstrap servers. 
MSK_TOPIC_NAME: MSK topic. 

"""
kafka_client = None
logger = logging.getLogger()
logger.setLevel(logging.DEBUG)

def lambda_handler(event, context):
    logger.debug('MSK consumer starts.')
    try:
        consumer = KafkaConsumer(os.environ['MSK_TOPIC_NAME'], auto_offset_reset='earliest',
                             bootstrap_servers=os.environ['MSK_BOOTSTRAP_SRV'], api_version=(0, 10), consumer_timeout_ms=1000)

        for msg in consumer:
            print(msg.value)

        if consumer is not None:
            consumer.close()

    except Exception as ex:
        logger.error('Exception: {}'.format(ex))
    return

相关问题