langchain4j [功能]在AI服务接口中除了TokenStream之外，还添加对Flux< String>的支持,

snz8szmq  于 3个月前  发布在  其他
关注(0)|答案(8)|浏览(63)

您的功能请求是否与问题相关？请描述。

我通常使用流式响应。因为我处于Spring Boot环境中，与TokenStream相比，使用Flux数据类型更方便。

描述您希望的解决方案

理想情况下，我可以在接口中定义一个方法来返回Flux,或者我可以调用tokenStream.asFlux()。

描述您考虑过的替代方案

我现在手动创建一个Sink并在其上发出next/complete/error以获得Flux。

langchain4j

8条答案

按热度按时间
puruo6ea

puruo6ea1#

同意：)

赞(0）分享  回复(0）举报  3个月前
mjqavswn

mjqavswn2#

如何在Spring Boot环境中使用tokenstream?现在将数据打印到控制台。
如何将流式数据返回给API服务或前端服务？
谢谢！

赞(0）分享  回复(0）举报  3个月前
n6lpvg4x

n6lpvg4x3#

@GetMapping("/assistant")
public TokenStream assistant(@RequestParam(value = "message", defaultValue = "hello") String message) {
        TokenStream tokenStream = assistant.chat(message);
        tokenStream.onNext(System.out::println)
                .onComplete(System.out::println)
                .onError(Throwable::printStackTrace)
                .start();
        return tokenStream;
    }
赞(0）分享  回复(0）举报  3个月前
smdncfj3

smdncfj34#

这里是一个将 TokenStream 转换为 Flux<String> 的示例。

赞(0）分享  回复(0）举报  3个月前
tquggr8v

tquggr8v5#

我习惯使用Flux,发现它返回的是FE而不是流式数据，我想使用类似SSE的请求，有没有办法实现呢？谢谢！

控制台输出：

public Flux assistantFlux(String message) {
assistant = AiServices.builder(Assistant.class)
.streamingChatLanguageModel(zhiPuAi.getZhiPuAiStreamChatModel())
.tools(new AssistantToolExecution())
.chatMemory(MessageWindowChatMemory.withMaxMessages(10))
.build();
Sinks.Many sink = Sinks.many().unicast().onBackpressureBuffer();
assistant.chatWithStream(message)
.onNext(sink::tryEmitNext)
.onComplete(aiMessageResponse -> sink.tryEmitComplete())
.onError(sink::tryEmitError)
.start();
return sink.asFlux();
}

Postman输出：

2024-04-15T11:02:43.249+08:00 DEBUG 19733 --- [bigmodel.cn/...] d.langchain4j.model.zhipu.ZhipuAiClient : onOpen()
2024-04-15T11:02:43.249+08:00 DEBUG 19733 --- [bigmodel.cn/...] d.langchain4j.model.zhipu.ZhipuAiClient : onEvent() {"id":"8561163774422257761","created":1713150161,"model":"glm-4","choices":[{"index":0,"delta":{"role":"assistant","content":"I"}}]}
2024-04-15T11:02:43.250+08:00 DEBUG 19733 --- [bigmodel.cn/...] d.langchain4j.model.zhipu.ZhipuAiClient : onEvent() {"id":"8561163774422257761","created":1713150161,"model":"glm-4","choices":[{"index":0,"delta":{"role":"assistant","content":"'m"}}]}
2024-04-15T11:02:43.250+08:00 DEBUG 19733 --- [bigmodel.cn/...] d.langchain4j.model.zhipu.ZhipuAiClient : onEvent() {"id":"8561163774422257761","created":1713150161,"model":"glm-4","choices":[{"index":0,"delta":{"role":"assistant","content":" good"}}]}
2024-04-15T11:02:43.251+08:00 DEBUG 19733 --- [bigmodel.cn/...] d.langchain4j.model.zhipu.ZhipuAiClient : onEvent() {"id":"8561163774422257761","created":1713150161,"model":"glm-4","choices":[{"index":0,"delta":{"role":"assistant","content":","}}]}
2024-04-15T11:02:43.251+08:00 DEBUG 19733 --- [bigmodel.cn/...] d.langchain4j.model.zhipu.ZhipuAiClient : onEvent() {"id":"8561163774422257761","created":1713150161,"model":"glm-4","choices":[{"index":0,"delta":{"role":"assistant","content":" thank"}}]}
2024-04-15T11:02:43.251+08:00 DEBUG 19733 --- [bigmodel.cn/...] d.langchain4j.model.zhipu.ZhipuAiClient : onEvent() {"id":"8561163774422257761","created":1713150161,"model":"glm-4","choices":[{"index":0,"delta":{"role":"assistant","content":" you"}}]}
2024-04-15T11:02:43.251+08:00 DEBUG 19733 --- [bigmodel.cn/...] d.langchain4j.model.zhipu.ZhipuAiClient : onEvent() {"id":"8561163774422257761","created":1713150161,"model":"glm-4","choices":[{"index":0,"delta":{"role":"assistant","content":"!"}}]}
2024-04-15T11:02:43.251+08:00 DEBUG 19733 --- [bigmodel.cn/...] d.langchain4j.model.zhipu.ZhipuAiClient : onEvent() {"id":"8561163774422257761","created":1713150161,"model":"glm-4","choices":[{"index":0,"delta":{"role":"assistant","content":" How"}}]}
2024-04-15T11:02:43.251+08:00 DEBUG 19733 --- [bigmodel.cn/...] d.langchain4j.model.zhipu.ZhipuAiClient : onEvent() {"id":"8561163774422257761","created":1713150161,"model":"glm-4","choices":[{"index":0,"delta":{"role":"assistant","content":" can"}}]}
2024-04-15T11:02:43.251+08:00 DEBUG 19733 --- [bigmodel.cn/...] d.langchain4j.model.zhipu.ZhipuAiClient : onEvent() {"id":"8561163774422257761","created":1713150161,"model":"glm-4","choices":[{"index":0,"delta":{"role":"assistant","content":" I"}}]}
2024-04-15T11:02:43.251+08:00 DEBUG 19733 --- [bigmodel.cn/...] d.langchain4j.model.zhipu.ZhipuAiClient : onEvent() {"id":"8561163774422257761","created":1713150161,"model":"glm-4","choices":[{"index":0,"delta":{"role":"assistant","content":" assist"}}]}
2024-04-15T11:02:43.251+08:00 DEBUG 19733 --- [bigmodel.cn/...] d.langchain4j.model.zhipu.ZhipuAiClient : onEvent() {"id":"8561163774422257761","created":1713150161,"model":"glm-4","choices":[{"index":0,"delta":{"role":"assistant","content":" you"}}]}
2024-04-15T11:02:43.252+08:00 DEBUG 19733 --- [bigmodel.cn/...] d.langchain4j.model.zhipu.ZhipuAiClient : onEvent() {"id":"8561163774422257761","created":1713150161,"model":"glm-4","choices":[{"index":0,"delta":{"role":"assistant","content":" today"}}]}
2024-04-15T11:02:43.259+08:00 DEBUG 19733 --- [bigmodel.cn/...] d.langchain4j.model.zhipu.ZhipuAiClient : onEvent() {"id":"8561163774422257761","created":1713150161,"model":"glm-4","choices":[{"index":0,"delta":{"role":"assistant","content":"?"}}]}
2024-04-15T11:02:43.260+08:00 DEBUG 19733 --- [bigmodel.cn/...] d.langchain4j.model.zhipu.ZhipuAiClient : onEvent() {"id":"8561163774422257761","created":1713150161,"model":"glm-4","choices":[{"index":0,"delta":{"role":"assistant","content":" If"}}]}
2024-04-15T11:02:43.260+08:00 DEBUG 19733 --- [bigmodel.cn/...] d.langchain4j.model.zhipu.ZhipuAiClient : onEvent() {"id":"8561163774422257761","created":1713150161,"model":"glm-4","choices":[{"index":0,"delta":{"role":"assistant","content":" you"}}]}
2024-04-15T11:02:43.260+08:00 DEBUG 19733 --- [bigmodel.cn/...] d.langchain4j.model.zhipu.ZhipuAiClient : onEvent() {"id":"8561163774422257761","created":1713150161,"model":"glm-4","choices":[{"index":0,"delta":{"role":"assistant","content":" have"}}]}
2024-04-15T11:02:43.260+08:00 DEBUG 19733 --- [bigmodel.cn/...] d.langchain4j.model.zhipu.ZhipuAiClient : onEvent() {"id":"8561163774422257761","created":1713150161,"model":"glm-4","choices":[{"index":0,"delta":{"role":"assistant","content":" any"}}]}
2024-04-15T11:02:43.260+08:00 DEBUG 19733 --- [bigmodel.cn/...] d.langchain4j.model.zhipu.ZhipuAiClient : onEvent() {"id":"8561163774422257761","created":1713150161,"model":"glm-4","choices":[{"index":0,"delta":{"role":"assistant","content":" questions"}}]}
2024-04-15T11:02:43.260+08:00 DEBUG 19733 --- [bigmodel.cn/...] d.langchain4j.model.zhipu.ZhipuAiClient : onEvent() {"id":"8561163774422257761","created":1713150161,"model":"glm-4","choices":[{"index":0,"delta":{"role":"assistant","content":" or"}}]}
2024-04-15T11:02:43.260+08:00 DEBUG 19733 --- [bigmodel.cn/...] d.langchain4j.model.zhipu.ZhipuAiClient : onEvent() {"id":"8561163774422257761","created":1713150161,"model":"glm-4","choices":[{"index":0,"delta":{"role":"assistant","content":" need"}}]}
2024-04-15T11:02:43.260+08:00 DEBUG 19733 --- [bigmodel.cn/...] d.langchain4j.model.zhipu.ZhipuAiClient : onEvent() {"id":"8561163774422257761","created":1713150161,"model":"glm-4","choices":[{"index":0,"delta":{"role":"assistant","content":" help"}}]}
2024-04-15T11:02:43.260+08:00 DEBUG 19733 --- [bigmodel.cn/...] d.langchain4j.model.zhipu.ZhipuAiClient : onEvent() {"id":"8561163774422257761","created":1713150161,"model":"glm-4","choices":[{"index":0,"delta":{"role":"assistant","content":" with"}}]}
2024-04-15T11:02:43.260+08:00 DEBUG 19733 --- [bigmodel.cn/...] d.langchain4j.model.zhipu.ZhipuAiClient : onEvent() {"id":"8561163774422257761","created":1713150161,"model":"glm-4","choices":[{"index":0,"delta":{"role":"assistant","content":" something"}}]}
2024-04-15T11:02:43.260+08:00 DEBUG 19733 --- [bigmodel.cn/...] d.langchain4j.model.zhipu.ZhipuAiClient : onEvent() {"id":"8561163774422257761","created":1713150161,"model":"glm-4","choices":[{"index":0,"delta":{"role":"assistant","content":","}}]}
2024-04-15T11:02:43.260+08:00 DEBUG 19733 --- [bigmodel.cn/...] d.langchain4j.model.zhipu.ZhipuAiClient : onEvent() {"id":"8561163774422257761","created":1713150161,"model":"glm-4","choices":[{"index":0,"delta":{"role":"assistant","content":" feel"}}]}
2024-04-15T11:02:43.261+08:00 DEBUG 19733 --- [bigmodel.cn/...] d.langchain4j.model.zhipu.ZhipuAiClient : onEvent() {"id":"8561163774422257761","created":1713150161,"model":"glm-4","choices":[{"index":0,"delta":{"role":"assistant","content":" free"}}]}
2024-04-15T11:02:43.261+08:00 DEBUG 19733 --- [bigmodel.cn/...] d.langchain4j.model.zhipu.ZhipuAiClient : onEvent() {"id":"8561163774422257761","created":1713150161,"model":"glm-4","choices":[{"index":0,"delta":{"role":"assistant","content":" to"}}]}
2024-04-15T11:02:43.261+08:00 DEBUG 19733 --- [bigmodel.cn/...] d.langchain4j.model.zhipu.ZhipuAiClient : onEvent() {"id":"8561163774422257761","created":1713150161,"model":"glm-4","choices":[{"index":0,"delta":{"role":"assistant","content":" ask"}}]}
2024-04-15T11:02:43.261+08:00 DEBUG 19733 --- [bigmodel.cn/...] d.langchain4j.model.zhipu.ZhipuAiClient : onEvent() {"id":"8561163774422257761","created":1713150161,"model":"glm-4","choices":[{"index":0,"delta":{"role":"assistant","content":"."}}]}
2024-04-15T11:02:43.261+08:00 DEBUG 19733 --- [bigmodel.cn/...] d.langchain4j.model.zhipu.ZhipuAiClient : onEvent() {"id":"8561163774422257761","created":1713150161,"model":"glm-4","choices":[{"index":0,"finish_reason":"stop","delta":{"role":"assistant","content":""}}],"usage":{"prompt_tokens":35,"completion_tokens":32,"total_tokens":67}}
2024-04-15T11:02:43.261+08:00 DEBUG 19733 --- [bigmodel.cn/...] d.langchain4j.model.zhipu.ZhipuAiClient : onEvent() [DONE]
2024-04-15T11:02:43.261+08:00 DEBUG 19733 --- [bigmodel.cn/...] d.langchain4j.model.zhipu.ZhipuAiClient : onClosed()

赞(0）分享  回复(0）举报  3个月前
41zrol4v

41zrol4v6#

使用以下代码解决了我的问题，修改类型
MediaType.TEXT_EVENT_STREAM_VALUE

@PostMapping(path = "/assistantFlux", produces = MediaType.TEXT_EVENT_STREAM_VALUE)
    public Flux<String> assistantFlux(@RequestBody AssistantDto assistantDto) {
          assistant = AiServices.builder(Assistant.class)
                .streamingChatLanguageModel(zhiPuAi.getZhiPuAiStreamChatModel())
                .tools(new AssistantToolExecution())
                .chatMemory(MessageWindowChatMemory.withMaxMessages(10))
                .build();

        Sinks.Many<String> sink = Sinks.many().unicast().onBackpressureBuffer();
        assistant.chatWithStream(message)
                .onNext(sink::tryEmitNext)
                .onComplete(aiMessageResponse -> sink.tryEmitComplete())
                .onError(sink::tryEmitError)
                .start();
        return sink.asFlux();
    }

赞(0）分享  回复(0）举报  3个月前
af7jpaap

af7jpaap7#

相关： #322

赞(0）分享  回复(0）举报  3个月前
uhry853o

uhry853o8#

相关： #689

赞(0）分享  回复(0）举报  3个月前
我来回答

相关问题

    查看更多

    热门标签

    更多
    JavaquerypythonNode开发语言requestUtil数据库Table后端算法LoggerMessageElementParser

    最新问答

    更多
    • 蜀ICP备2022004421号-2 NULL123 https://www.null123.com © All rights reserved
    • 本站内容来源互联网,如果侵犯您的权益请联系我们删除, 联系方式：448109455@qq.com