next.js 流式传输时使用OpenAI API从函数调用返回参数？

rvpgvaaj 于 11个月前发布在其他

关注(0)|答案(3)|浏览(115)

我做了一个简单的函数调用OpenAI API示例，我只使用函数调用来格式化响应，我没有调用多个函数或任何外部API。
当我没有流响应时，我可以返回函数参数，这是我需要的数据。
在我的NextJS路由处理程序中：

export async function POST(request: Request) {
  try {
    const openai = new OpenAI({
      apiKey: process.env["OPENAI_API_KEY"],
    });
    const response = await openai.chat.completions.create({
      model: "gpt-4",
      // stream: true,
      messages: [
        {
          role: "user",
          content: "Give me 5 questions and answers for a pub quiz",
        },
      ],
      tools: [
        {
          type: "function",
          function: {
            name: "get_questions_and_answers",
            description: "Get questions and answers",
            parameters: simpleJsonSchema,
          },
        },
      ],
      tool_choice: {
        type: "function",
        function: { name: "get_questions_and_answers" },
      },
    });
    return Response.json(
       JSON.parse(
         response.choices[0].message.tool_calls?.[0].function.arguments || "",
       ),
    );
  } catch (serverError) {
    console.error({ serverError });
    throw new Error();
  }
}

字符串
simpleJsonSchema.json:

{
  "type": "object",
  "properties": {
    "getQuestions": {
      "type": "array",
      "items": {
        "type": "object",
        "properties": {
          "Question": {"type": "string"},
          "Answer": {"type": "string"}
        },
        "required": ["Question", "Answer"]
      }
    }
  },
  "required": ["getQuestions"]
}

型
来自API的响应：

{"getQuestions":[{"Question":"What is the capital of Australia?","Answer":"Canberra"},{"Question":"Who wrote 'To Kill a Mockingbird'?","Answer":"Harper Lee"},{"Question":"What is the highest peak in the world?","Answer":"Mount Everest"},{"Question":"Who is known as the 'Father of Computers'?","Answer":"Charles Babbage"},{"Question":"What is the largest ocean in the world?","Answer":"Pacific Ocean"}]}

型
这在本地开发时很好，但是当部署到Vercel时，请求有时会超时。我已经尝试添加流，因为这是推荐的解决方案：

const response = await openai.chat.completions.create({
  model: "gpt-4",
  stream: true,
  messages: [
    {
      role: "user",
      content: "Give me 5 questions and answers for a pub quiz",
    },
  ],
  tools: [
    {
      type: "function",
      function: {
        name: "get_questions_and_answers",
        description: "Get questions and answers",
        parameters: simpleJsonSchema,
      },
    },
  ],
  tool_choice: {
    type: "function",
    function: { name: "get_questions_and_answers" },
  },
});

const stream = OpenAIStream(response);
return new StreamingTextResponse(stream);

型
但是现在响应中有很多不必要的数据。当我尝试在客户端解析JSON时，我会得到错误。
来自API的响应：

{"tool_calls":[ {"id": "call_IhxvzkZ5EsmZpHc6tOznTmzb", "type": "function", "function": {"name": "get_questions_and_answers", "arguments": "{\n  \"getQuestions\": [\n    {\n      \"Question\": \"Question 1\",\n      \"Answer\": \"Answer 1\"\n    },\n    {\n      \"Question\": \"Question 2\",\n      \"Answer\": \"Answer 2\"\n    },\n    {\n      \"Question\": \"Question 3\",\n      \"Answer\": \"Answer 3\"\n    },\n    {\n      \"Question\": \"Question 4\",\n      \"Answer\": \"Answer 4\"\n    },\n    {\n      \"Question\": \"Question 5\",\n      \"Answer\": \"Answer 5\"\n    }\n  ]\n}"}}

型
据我所知，文档只涉及使用useChat，但我有一些特殊的要求，所以我需要自己处理抓取和表单状态：https://sdk.vercel.ai/docs/api-reference/use-chat
为什么我会得到无效的JSON？
以下是重现该问题的存储库：https://github.com/jameschetwood/openai-function-calling

next.js

来源：https://stackoverflow.com/questions/77728888/return-arguments-from-function-calling-with-openai-api-when-streaming

3条答案

按热度按时间

mrwjdhj31#

这是你得到的回应：

{"tool_calls":[ {"id": "call_HRxqlP3yzeHsoN43tMyZjMlr", "type": "function", "function": {"name": "get_questions_and_answers", "arguments": "{\n  \"getQuestions\": [\n    {\n      \"Question\": \"What is the capital city of France?\",\n      \"Answer\": \"Paris\"\n    },\n    {\n      \"Question\": \"Who painted the Mona Lisa?\",\n      \"Answer\": \"Leonardo da Vinci\"\n    },\n    {\n      \"Question\": \"What is the largest planet in our solar system?\",\n      \"Answer\": \"Jupiter\"\n    },\n    {\n      \"Question\": \"What is the national flower of England?\",\n      \"Answer\": \"Rose\"\n    },\n    {\n      \"Question\": \"Which country is famous for its tulips?\",\n      \"Answer\": \"Netherlands\"\n    }\n  ]\n}"}}

字符串
我用https://jsoneditoronline.org/来自动纠正json，它只是添加了“]}"。由于某种原因，openai没有发送正确的json响应。你必须添加它

accumulatedText += "]}";

型
然后响应工作：
x1c 0d1x的数据
如果openai更新了它的响应API，它可能会正确地发送json数据。所以更好的方法是在try/catch中解析

try {
      const parsed = JSON.parse(accumulatedText);
      console.log({ parsed });
    } catch (error) {
      // you should error for each specific case
      accumulatedText += "]}";
      console.log("correct accumulatedText in catch block", accumulatedText);
    }

型

赞(0）回复(0）举报 11个月前

mrzz3bfm2#

问题

正如您在其他StackOverflow成员的帮助下已经发现的那样，问题是以下JSON在结尾缺少]}。

{"tool_calls":[ {"id": "call_IhxvzkZ5EsmZpHc6tOznTmzb", "type": "function", "function": {"name": "get_questions_and_answers", "arguments": "{\n  \"getQuestions\": [\n    {\n      \"Question\": \"Question 1\",\n      \"Answer\": \"Answer 1\"\n    },\n    {\n      \"Question\": \"Question 2\",\n      \"Answer\": \"Answer 2\"\n    },\n    {\n      \"Question\": \"Question 3\",\n      \"Answer\": \"Answer 3\"\n    },\n    {\n      \"Question\": \"Question 4\",\n      \"Answer\": \"Answer 4\"\n    },\n    {\n      \"Question\": \"Question 5\",\n      \"Answer\": \"Answer 5\"\n    }\n  ]\n}"}}

字符串

溶液

当然，硬编码accumulatedText += "]}";并不是一个好的解决方案。正如你所说，有一个包来修复JSON将是一个目标。
实际上，https://jsoneditoronline.org可以作为一个NPM包使用，它被称为jsonrepair。
我分叉了你的GitHub存储库，并对你的流page.tsx文件做了以下更改（请参阅下面代码中的注解）：

"use client";
// import JSON5 from "json5";
import { jsonrepair } from "jsonrepair"; // Added

export default function Home() {
  async function callApi() {
    const response = await fetch(`streamed/api/chat2`, {
      method: "POST",
    });
    console.log({ response });

    if (!response.body) throw new Error("no body from response");

    const reader = response.body.getReader();
    const decoder = new TextDecoder();
    let accumulatedText = "";
    while (true) {
      const readValues = await reader.read();
      const { value, done } = readValues;
      if (done) break;
      const text = decoder.decode(value, { stream: true });
      accumulatedText += text;
    }
    console.log({ accumulatedText });

    const repairedJSON = jsonrepair(accumulatedText); // Added

    const parsed = JSON.parse(repairedJSON); // Changed
    console.log({ parsed });
  }
  return (
    <main className={"p-10"}>
      <h1 className={"pb-4"}>OpenAI Test - streamed</h1>
      <button
        className={"bg-orange-400 rounded px-4 py-2"}
        onClick={callApi}
        type={"button"}
      >
        Test
      </button>
    </main>
  );
}

型
这样做，我能够得到一个没有错误的响应：

的数据