Ⅰ. Issue Description

Tengine buffers the entire response body into RAM before sending it.

Ⅱ. Describe what happened

If serving a file, Tengine will read the entire file into RAM (visible from htop ), and after that is done, it will send the file to the client.

Ⅲ. Describe what you expected to happen

Tengine should not buffer the entire file in RAM. It should stream the file as it is read from the disk. Memory usage should not increase significantly.

Ⅳ. How to reproduce it (as minimally and precisely as possible)

1. Compile Tengine with XQUIC support
2. Acquire QUIC capable HTTP client (I used docker image ymuski/curl-http3 for local testing, but Firefox also triggers this)
3. Configure Tengine to serve files from a directory
4. Create a large file, or truncate -s 2G 2.bin
5. curl --http3 -v -k https://localhost/2.bin . There is a substantial delay before curl begins to receive the response. It is visible from htop that nginx will rapidly consume memory until it buffers the entire file in RAM.

Ⅴ. Anything else we need to know?

It appears the issue is caused in part by ngx_http_xquic_send_chain reading the entire response into another chain. This prevents normal backpressure from happening, which results in the entire response being read into the buffer. The patch below makes it so that Tengine will no longer wait until the entire file is in RAM to begin sending the response. However, it does not fully fix the issue.

diff --git a/modules/ngx_http_xquic_module/ngx_http_xquic_filter_module.c b/modules/ngx_http_xquic_module/ngx_http_xquic_filter_module.c
index 6eff9254..8b0fb707 100644
--- a/modules/ngx_http_xquic_module/ngx_http_xquic_filter_module.c
+++ b/modules/ngx_http_xquic_module/ngx_http_xquic_filter_module.c
@@ -648,6 +648,9 @@ ngx_http_xquic_send_chain(ngx_connection_t *c, ngx_chain_t *in, off_t limit)
     /* update h3_stream->output_queue here */
     h3_stream = r->xqstream;
 
+    // HACK: stop accepting new buffers if we have been limited by flow control
+    if (!h3_stream->wait_to_write) {
+
     last_chain = h3_stream->output_queue;
 
     while (last_chain != NULL) {
@@ -705,10 +708,12 @@ ngx_http_xquic_send_chain(ngx_connection_t *c, ngx_chain_t *in, off_t limit)
         r->xqstream->queued++; /* used to count buffers not sent*/
     }
 
+    // end hack
+    }
 
     last_out = h3_stream->output_queue;
     send = 0;

With the patch, Tengine will still consume the same amount of RAM, but it will also busy-wait on epoll_wait . I am not sure how to fix this.

Ⅵ. Environment:

• Tengine version (use sbin/nginx -V ): Commit 04baff4 on branch master (latest commit as of issue creation)
• OS (e.g. from /etc/os-release): Fedora 39
• Kernel (e.g. uname -a ): 6.5.12-300.fc39.x86_64
• Others: XQUIC commit alibaba/xquic@4d024a6