如何将数据从API发送到ElasticSearch?

2eafrhcq  于 2023-03-01  发布在  ElasticSearch
关注(0)|答案(1)|浏览(234)

我想知道是否有一种方法可以将API数据发送到Elastic Search,并使用Lambda每隔x分钟从API中提取数据?您知道任何python代码示例吗?

5ktev3wc

5ktev3wc1#

你需要设置一个Lambda,用CloudWatch作为触发器(scheduled/cron job),这将每x分钟运行一次你的python lambda。
从API提取数据:

import requests
url = 'API url to pull data from.(string)'
params = 'Query Parameters.(dict)'
api_response = requests.get(url=url, params=params)
if api_response.status_code != 200:
  raise Exception('API call failed')

发送数据到ElasticSearch(python ElasticSearch客户端的示例)

from datetime import datetime
from elasticsearch import Elasticsearch
es = Elasticsearch()

doc = {
    'author': 'kimchy',
    'text': 'Elasticsearch: cool. bonsai cool.',
    'timestamp': datetime.now(),
}
resp = es.index(index="test-index", id=1, document=doc)
print(resp['result'])

resp = es.get(index="test-index", id=1)
print(resp['_source'])

es.indices.refresh(index="test-index")

resp = es.search(index="test-index", query={"match_all": {}})
print("Got %d Hits:" % resp['hits']['total']['value'])
for hit in resp['hits']['hits']:
    print("%(timestamp)s %(author)s: %(text)s" % hit["_source"])

你需要安装这些python模块

pip install elasticsearch
pip install requests

相关问题