nginx+php-fpm大文件上传缓冲

总结：
我试图在没有nginx fastcgi缓冲的情况下直接将大文件上传到PHP，这样我就可以立即处理从php://input上传的数据（即作为流）。
详细内容：
我想要的行为是让nginx在每个字节（或字节块）到达nginx时将其传递给上游，在本例中即php-fpm。原因是能够获得非常大的文件上传并尽快处理它们（即比如说一旦每个块可用就将数据移动到S3）。现在nginx在1.7.11 http://nginx.org/en/CHANGES中以fastcgi_request_buffering配置选项的形式添加了对禁用fastcgi缓冲的支持。
当实现这个时，似乎nginx没有执行这个指令，或者php-fpm没有将数据流传输到worker。下面是curl请求的情况：

[root@localhost app]# ll -h large.txt 
-rw-r--r--. 1 nginx nginx 307M Nov 10  2015 large.txt
[root@localhost app]# time curl -vs -XPOST --data-binary @large.txt http://127.0.0.1:80
* About to connect() to 127.0.0.1 port 80 (#0)
*   Trying 127.0.0.1...
* Connected to 127.0.0.1 (127.0.0.1) port 80 (#0)
> POST / HTTP/1.1
> User-Agent: curl/7.29.0
> Host: 127.0.0.1
> Accept: */*
> Content-Length: 321912832
> Content-Type: application/x-www-form-urlencoded
> Expect: 100-continue
> 
< HTTP/1.1 100 Continue
< HTTP/1.1 404 Not Found
< Server: nginx
< Date: Fri, 06 Nov 2015 19:18:27 GMT
< Content-Type: text/html
< Content-Length: 162
< Connection: keep-alive
* HTTP error before end of send, stop sending
< 
<html>
<head><title>404 Not Found</title></head>
<body bgcolor="white">
<center><h1>404 Not Found</h1></center>
<hr><center>nginx</center>
</body>
</html>
* Closing connection 0

real    7m43.438s
user    0m0.653s
sys 0m1.219s
[root@localhost app]#

字符串
在这样做的同时，我跟踪了nginx和php-fpm日志：

==> /var/log/php-fpm/www-error.log <==
[06-Nov-2015 12:18:27 America/Denver] PHP Fatal error:  Allowed memory size of 268435456 bytes exhausted (tried to allocate 268173441 bytes) in Unknown on line 0

==> /var/log/nginx/error.log <==
2015/11/06 14:18:27 [error] 31657#0: *1 upstream sent unexpected FastCGI record: 3 while reading response header from upstream, client: 127.0.0.1, server: myserver.whatever.com, request: "POST / HTTP/1.1", upstream: "fastcgi://127.0.0.1:9000", host: "127.0.0.1"
2015/11/06 14:18:27 [error] 31657#0: *1 open() "/etc/nginx/html/50x.html" failed (2: No such file or directory), client: 127.0.0.1, server: myserver.whatever.com, request: "POST / HTTP/1.1", upstream: "fastcgi://127.0.0.1:9000", host: "127.0.0.1"

==> /var/log/nginx/access.log <==
127.0.0.1 - - [06/Nov/2015:14:18:27 -0500] "POST / HTTP/1.1" 404 187 "-" "curl/7.29.0"

==> /var/log/php-fpm/error.log <==
[06-Nov-2015 14:18:27] WARNING: [pool www] child 31619 exited with code 70 after 479.683142 seconds from start
[06-Nov-2015 14:18:27] NOTICE: [pool www] child 31809 started

型
所以这一行的问题很明显：PHP Fatal error: Allowed memory size of 268435456 bytes exhausted (tried to allocate 268173441 bytes) in Unknown on line 0个
问题是，是nginx没有遵守fastcgi_request_buffering off;指令，还是php-fpm进程管理器搞砸了，没有直接将数据流传输到php worker，而是作为整个请求对象传递。
或者，也许我错过了一些php-fpm的配置：http://php.net/manual/en/install.fpm.configuration.php的
以下是一些相关信息：
PHP fpm configs：

[root@localhost nginx]# cat /etc/php-fpm.conf | grep -vE '^;' | grep -v '^$'
include=/etc/php-fpm.d/*.conf
[global]
pid = /run/php-fpm/php-fpm.pid
error_log = /var/log/php-fpm/error.log
daemonize = yes

型
池配置：

[root@localhost nginx]# cat /etc/php-fpm.d/www.conf | grep -vE '^;' | grep -v '^$'
[www]
user = apache
group = apache
listen = 127.0.0.1:9000
listen.allowed_clients = 127.0.0.1
pm = dynamic
pm.max_children = 50
pm.start_servers = 5
pm.min_spare_servers = 5
pm.max_spare_servers = 35
slowlog = /var/log/php-fpm/www-slow.log
php_admin_value[error_log] = /var/log/php-fpm/www-error.log
php_admin_flag[log_errors] = on
php_value[session.save_handler] = files
php_value[session.save_path]    = /var/lib/php/session
php_value[soap.wsdl_cache_dir]  = /var/lib/php/wsdlcache
php_value[upload_max_filesize] = 100G
php_value[post_max_size] = 100G
php_value[max_execution_time] = 3600
request_terminate_timeout = 3600
request_slowlog_timeout = 60
php_value[max_input_time] = 3600

型
版本：

[root@localhost nginx]# php -v
PHP 5.6.15 (cli) (built: Oct 29 2015 14:18:11) 
Copyright (c) 1997-2015 The PHP Group
Zend Engine v2.6.0, Copyright (c) 1998-2015 Zend Technologies
    with Xdebug v2.3.3, Copyright (c) 2002-2015, by Derick Rethans
[root@localhost nginx]# php-fpm -v
PHP 5.6.15 (fpm-fcgi) (built: Oct 29 2015 14:18:34)
Copyright (c) 1997-2015 The PHP Group
Zend Engine v2.6.0, Copyright (c) 1998-2015 Zend Technologies
    with Xdebug v2.3.3, Copyright (c) 2002-2015, by Derick Rethans
[root@localhost nginx]# nginx -v
nginx version: nginx/1.8.0
[root@localhost nginx]# uname -a
Linux localhost.localdomain 3.10.0-229.14.1.el7.x86_64 #1 SMP Tue Sep 15 15:05:51 UTC 2015 x86_64 x86_64 x86_64 GNU/Linux
[root@localhost nginx]# cat /etc/redhat-release 
CentOS Linux release 7.1.1503 (Core)

型
nginx配置：

server {
    listen       0.0.0.0:80 default_server;
    server_name               myserver.com;
    location / {
        root   /opt/my/app/public;
        index  index.html index.htm index.php;
        try_files $uri $uri/ /index.php?$args;
    }

    error_page   500 502 503 504  /50x.html;
    location = /50x.html {
        root   html;
    }

    location ~ \.php$ {
        root /opt/my/app/public;
        try_files      $uri = 404;
        fastcgi_pass   127.0.0.1:9000;
        fastcgi_index  index.php;
        fastcgi_param  SCRIPT_FILENAME  $document_root$fastcgi_script_name;
    fastcgi_request_buffering off;
    fastcgi_read_timeout 3600;
        include        fastcgi_params;
    }
}

型

nginx+php-fpm大文件上传缓冲

1条答案

相关问题

热门标签

最新问答