我有一个Elasticsearch示例,我使用API Gateway + Lambda来设置一个小后端。ES包含一些Word插件的嵌入,这些插件是我为“与文档聊天”LLM项目生成的。
这是embedding_search_handler.py:
import os
import json
import requests
from elasticsearch import Elasticsearch, RequestsHttpConnection
from requests_aws4auth import AWS4Auth
from decouple import config
import sys
sys.path.append('./')
# Elasticsearch setup
host = 'https://myhost.us-east-1.es.amazonaws.com'
region = 'us-east-1'
# Retrieve keys from the .env file using python-decouple
ACCESS_KEY = config('ACCESS_KEY')
SECRET_KEY = config('SECRET_KEY')
OPENAI_API_KEY = config('OPENAI_API_KEY')
awsauth = AWS4Auth(ACCESS_KEY, SECRET_KEY, region, 'es')
es = Elasticsearch(
hosts=[{'host': host, 'port': 443}],
http_auth=awsauth,
use_ssl=True,
verify_certs=True,
connection_class=RequestsHttpConnection
)
def query_elasticsearch(embedding):
try:
body = {
"size": 5,
"query": {
"script_score": {
"query": {"match_all": {}},
"script": {
"source": "cosineSimilarity(params.query_vector, 'embedding_vector') + 1.0",
"params": {"query_vector": embedding}
}
}
}
}
response = es.search(index="document_embeddings", body=body)
return response['hits']['hits']
except Exception as e:
print(f"Error querying Elasticsearch: {e}")
return []
def lambda_handler(event, context):
query_text = event['query']
query_embedding = convert_query_to_embedding(query_text)
if not query_embedding:
return {
"statusCode": 500,
"body": json.dumps({"error": "Failed to get embedding for query"})
}
results = query_elasticsearch(query_embedding)
return {
"statusCode": 200,
"body": json.dumps(results)
}
def convert_query_to_embedding(query):
headers = {
"Authorization": f"Bearer {OPENAI_API_KEY}",
"Content-Type": "application/json"
}
try:
response = requests.post(
"https://api.openai.com/v1/embed",
headers=headers,
json={
"model": "text-embedding-ada-002",
"data": [query]
}
)
response.raise_for_status() # Raises a HTTPError if the HTTP request returned an unsuccessful status code
data = response.json()
embedding = data['data'][0]['embedding']
return embedding
except requests.RequestException as e:
print(f"Error calling OpenAI API: {e}")
return None
当我尝试在Lambda中运行这个测试时:
{
"query": "what is this?"
}
我得到这个错误:
START RequestId: 5cdd60ba-562b-4cb5-a5f9-b02bbbb26a97 Version: $LATEST
[ERROR] Runtime.ImportModuleError: Unable to import module 'embedding_search_handler': attempted relative import beyond top-level package
Traceback (most recent call last):END RequestId: 5cdd60ba-562b-4cb5-a5f9-b02bbbb26a97
REPORT RequestId: 5cdd60ba-562b-4cb5-a5f9-b02bbbb26a97 Duration: 1.10 ms Billed Duration: 2 ms Memory Size: 128 MB Max Memory Used: 50 MB
这是我的目录的样子:
当我在目录“lambda_package”中运行zip -r ../lambda_package.zip
时,它也显示为正确压缩。这是压缩和解压内容的屏幕截图:
当我转到“编辑运行时设置”时,我的配置文件被设置为“embedding_search_handler.lambda_handler”
我不明白我做错了什么。
1条答案
按热度按时间p1iqtdky1#
我建议创建一个包含库的lambda层,或者从ECR镜像构建lambda函数,其中包含requirements.txt和dockerfile。