elasticsearch Lambda函数问题:“无法导入模块'embedding_search_handler':尝试相对导入超出顶级包”,

f8rj6qna  于 2023-10-17  发布在  ElasticSearch
关注(0)|答案(1)|浏览(99)

我有一个Elasticsearch示例,我使用API Gateway + Lambda来设置一个小后端。ES包含一些Word插件的嵌入,这些插件是我为“与文档聊天”LLM项目生成的。
这是embedding_search_handler.py:

import os
import json
import requests
from elasticsearch import Elasticsearch, RequestsHttpConnection
from requests_aws4auth import AWS4Auth
from decouple import config
import sys
sys.path.append('./')

# Elasticsearch setup
host = 'https://myhost.us-east-1.es.amazonaws.com'
region = 'us-east-1'

# Retrieve keys from the .env file using python-decouple
ACCESS_KEY = config('ACCESS_KEY')
SECRET_KEY = config('SECRET_KEY')
OPENAI_API_KEY = config('OPENAI_API_KEY')

awsauth = AWS4Auth(ACCESS_KEY, SECRET_KEY, region, 'es')

es = Elasticsearch(
    hosts=[{'host': host, 'port': 443}],
    http_auth=awsauth,
    use_ssl=True,
    verify_certs=True,
    connection_class=RequestsHttpConnection
)

def query_elasticsearch(embedding):
    try:
        body = {
            "size": 5,
            "query": {
                "script_score": {
                    "query": {"match_all": {}},
                    "script": {
                        "source": "cosineSimilarity(params.query_vector, 'embedding_vector') + 1.0",
                        "params": {"query_vector": embedding}
                    }
                }
            }
        }
        response = es.search(index="document_embeddings", body=body)
        return response['hits']['hits']
    except Exception as e:
        print(f"Error querying Elasticsearch: {e}")
        return []

def lambda_handler(event, context):
    query_text = event['query']
    query_embedding = convert_query_to_embedding(query_text)
    if not query_embedding:
        return {
            "statusCode": 500,
            "body": json.dumps({"error": "Failed to get embedding for query"})
        }
    
    results = query_elasticsearch(query_embedding)
    return {
        "statusCode": 200,
        "body": json.dumps(results)
    }

def convert_query_to_embedding(query):
    headers = {
        "Authorization": f"Bearer {OPENAI_API_KEY}",
        "Content-Type": "application/json"
    }
    
    try:
        response = requests.post(
            "https://api.openai.com/v1/embed",
            headers=headers,
            json={
                "model": "text-embedding-ada-002",
                "data": [query]
            }
        )
        response.raise_for_status()  # Raises a HTTPError if the HTTP request returned an unsuccessful status code
        
        data = response.json()
        embedding = data['data'][0]['embedding']
        return embedding
    except requests.RequestException as e:
        print(f"Error calling OpenAI API: {e}")
        return None

当我尝试在Lambda中运行这个测试时:

{
  "query": "what is this?"
}

我得到这个错误:

START RequestId: 5cdd60ba-562b-4cb5-a5f9-b02bbbb26a97 Version: $LATEST
[ERROR] Runtime.ImportModuleError: Unable to import module 'embedding_search_handler': attempted relative import beyond top-level package
Traceback (most recent call last):END RequestId: 5cdd60ba-562b-4cb5-a5f9-b02bbbb26a97
REPORT RequestId: 5cdd60ba-562b-4cb5-a5f9-b02bbbb26a97  Duration: 1.10 ms   Billed Duration: 2 ms   Memory Size: 128 MB Max Memory Used: 50 MB

这是我的目录的样子:

当我在目录“lambda_package”中运行zip -r ../lambda_package.zip时,它也显示为正确压缩。这是压缩和解压内容的屏幕截图:

当我转到“编辑运行时设置”时,我的配置文件被设置为“embedding_search_handler.lambda_handler”
我不明白我做错了什么。

p1iqtdky

p1iqtdky1#

我建议创建一个包含库的lambda层,或者从ECR镜像构建lambda函数,其中包含requirements.txt和dockerfile。

相关问题