目前，我想为生产构建RAG聊天机器人。我已经有了我的LLM API，我想创建一个自定义LLM，然后在RetrievalQA.from_chain_type函数中使用它。我不知道Langchain是否支持我的情况。
我在reddit上读到了这个主题：https://www.reddit.com/r/LangChain/comments/17v1rhv/integrating_llm_rest_api_into_a_langchain/，在langchain文档中：https://python.langchain.com/docs/modules/model_io/llms/custom_llm
但是当我将自定义LLM应用到qa_chain时，这仍然不起作用。下面是我的代码，希望得到你的支持，对不起我的语言，英语不是我的母语。

from pydantic import Extra
import requests
from typing import Any, List, Mapping, Optional

from langchain.callbacks.manager import CallbackManagerForLLMRun
from langchain.llms.base import LLM

class LlamaLLM(LLM):
    llm_url = 'https:/myhost/llama/api'

    class Config:
        extra = Extra.forbid

    @property
    def _llm_type(self) -> str:
        return "Llama2 7B"

    def _call(
        self,
        prompt: str,
        stop: Optional[List[str]] = None,
        run_manager: Optional[CallbackManagerForLLMRun] = None,
        **kwargs: Any,
    ) -> str:
        if stop is not None:
            raise ValueError("stop kwargs are not permitted.")

        payload = {
            "inputs": prompt,
            "parameters": {"max_new_tokens": 100},
            "token": "abcdfejkwehr"
        }

        headers = {"Content-Type": "application/json"}

        response = requests.post(self.llm_url, json=payload, headers=headers, verify=False)
        response.raise_for_status()

        # print("API Response:", response.json())

        return response.json()['generated_text']  # get the response from the API

    @property
    def _identifying_params(self) -> Mapping[str, Any]:
        """Get the identifying parameters."""
        return {"llmUrl": self.llm_url}

llm = LlamaLLM()

#Testing
prompt = "[INST] Question: Who is Albert Einstein? \n Answer: [/INST]"
result = llm._call(prompt)
print(result)

Albert Einstein (1879-1955) was a German-born theoretical physicist who is widely regarded as one of the most influential scientists of the 20th century. He is best known for his theory of relativity, which revolutionized our understanding of space and time, and his famous equation E=mc².

# Build prompt
from langchain.prompts import PromptTemplate
template = """[INST] <<SYS>>

Answer the question base on the context below.

<</SYS>>

Context: {context}
Question: {question}
Answer:
[/INST]"""
QA_CHAIN_PROMPT = PromptTemplate(input_variables=["context", "question"],template=template,)

# Run chain
from langchain.chains import RetrievalQA

qa_chain = RetrievalQA.from_chain_type(llm,
                                       verbose=True,
                                       # retriever=vectordb.as_retriever(),
                                       retriever=custom_retriever,
                                       return_source_documents=True,
                                       chain_type_kwargs={"prompt": QA_CHAIN_PROMPT})

question = "Is probability a class topic?"
result = qa_chain({"query": question})
result["result"]

Encountered some errors. Please recheck your request!

在我的例子中，自定义检索结合了检索和重新排序。我已经测试过了，没问题。
我也用正常的检索进行了测试，但仍然不起作用。所以我认为检索不是错误的原因。

retriever=vectordb.as_retriever()

型
此外，它也有与不安全的请求有关的问题，但它是否会影响请求。（我也不知道如何修复它）

/usr/local/lib/python3.10/dist-packages/urllib3/connectionpool.py:1061: InsecureRequestWarning: Unverified HTTPS request is being made to host 'myhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings
  warnings.warn(
Encountered some errors. Please recheck your request!

型
另外，下面是我的API格式，有什么问题吗？

curl --location 'https:/myhost:10001/llama/api' -k \
--header 'Content-Type: application/json' \
--data-raw '{
    "inputs": "[INST] Question: Who is Albert Einstein? \n Answer: [/INST]",
    "parameters": {"max_new_tokens":100},
    "token": "abcdfejkwehr"
}

型
这是因为API的上下文长度设置。所以我已经修复了它，它的工作正常。