NodeJS如何从aws s3 bucket下载文件到磁盘?

oxiaedzo  于 2022-11-03  发布在  Node.js
关注(0)|答案(7)|浏览(474)

我的目标:
显示一个对话框,提示用户保存正在从aws下载的文件。
我的问题:
我目前正在使用awssum-amazon-s3创建下载流。但是我只设法将文件保存到我的服务器或流到命令行...正如您从我的代码中所看到的,我最后的尝试是尝试手动设置内容处理头,但失败了。我不能使用res.download(),因为头已经设置好了。
我怎样才能实现我的目标?
我的节点代码:

app.post('/dls/:dlKey', function(req, res, next){
        // download the file via aws s3 here
        var dlKey = req.param('dlKey');

        Dl.findOne({key:dlKey}, function(err, dl){
            if (err) return next(err);
            var files = dl.dlFile;

            var options = {
                BucketName    : 'xxxx',
                ObjectName    : files,
            };

            s3.GetObject(options, { stream : true }, function(err, data) {
                // stream this file to stdout
                fmt.sep();
                data.Headers['Content-Disposition'] = 'attachment';
                console.log(data.Headers);
                data.Stream.pipe(fs.createWriteStream('test.pdf'));
                data.Stream.on('end', function() {
                    console.log('File Downloaded!');
                });
            });
        });

        res.end('Successful Download Post!');
    });

我的代码为棱角分明:

$scope.dlComplete = function (dl) {
        $scope.procDownload = true;
        $http({
            method: 'POST',
            url: '/dls/' + dl.dlKey
        }).success(function(data/*, status, headers, config*/) {
            console.log(data);
            $location.path('/#!/success');
        }).error(function(/*data, status, headers, config*/) {
            console.log('File download failed!');
        });
    };

此代码的目的是让用户使用生成的密钥下载一次文件。

ghhaqwfi

ghhaqwfi1#

这是在aws-sdk的最新版本上使用流的完整代码

var express = require('express');
var app = express();
var fs = require('fs');

app.get('/', function(req, res, next){
    res.send('You did not say the magic word');
});

app.get('/s3Proxy', function(req, res, next){
    // download the file via aws s3 here
    var fileKey = req.query['fileKey'];

    console.log('Trying to download file', fileKey);
    var AWS = require('aws-sdk');
    AWS.config.update(
      {
        accessKeyId: "....",
        secretAccessKey: "...",
        region: 'ap-southeast-1'
      }
    );
    var s3 = new AWS.S3();
    var options = {
        Bucket    : '/bucket-url',
        Key    : fileKey,
    };

    res.attachment(fileKey);
    var fileStream = s3.getObject(options).createReadStream();
    fileStream.pipe(res);
});

var server = app.listen(3000, function () {
    var host = server.address().address;
    var port = server.address().port;
    console.log('S3 Proxy app listening at http://%s:%s', host, port);
});
t9aqgxwy

t9aqgxwy2#

下面的代码对我来说适用于最新的库:

var s3 = new AWS.S3();
var s3Params = {
    Bucket: 'your bucket',
    Key: 'path/to/the/file.ext'
};
s3.getObject(s3Params, function(err, res) {
    if (err === null) {
       res.attachment('file.ext'); // or whatever your logic needs
       res.send(data.Body);
    } else {
       res.status(500).send(err);
    }
});
8mmmxcuj

8mmmxcuj3#

只需从S3创建一个ReadStream,然后将WriteStream写到你想下载的位置。找到下面的代码。对我来说非常有效:

var AWS = require('aws-sdk');
var path = require('path');
var fs = require('fs');

AWS.config.loadFromPath(path.resolve(__dirname, 'config.json'));
AWS.config.update({
  accessKeyId: AWS.config.credentials.accessKeyId,
  secretAccessKey: AWS.config.credentials.secretAccessKey,
  region: AWS.config.region
});

var s3 = new AWS.S3();
var params = {
  Bucket: '<your-bucket>', 
  Key: '<path-to-your-file>'
};
let readStream = s3.getObject(params).createReadStream();
let writeStream = fs.createWriteStream(path.join(__dirname, 's3data.txt'));
readStream.pipe(writeStream);
yh2wf1be

yh2wf1be4#

您已经知道解决问题最重要的是什么:您可以将来自S3的文件流通过管道传输到任何可写流,无论是文件流...还是将发送到客户端的响应流!

s3.GetObject(options, { stream : true }, function(err, data) {
    res.attachment('test.pdf');
    data.Stream.pipe(res);
});

注意res.attachment的使用,它将设置正确的头。你也可以检查关于流和S3的this answer

hgc7kmma

hgc7kmma5#

使用aws SDK v3

npm install @aws-sdk/client-s3

下载代码

import { GetObjectCommand } from "@aws-sdk/client-s3";
/**
 * download a file from AWS and send to your rest client
 */
app.get('/download', function(req, res, next){
    var fileKey = req.query['fileKey'];

    var bucketParams = {
        Bucket: 'my-bucket-name',
        Key: fileKey,
    };

    res.attachment(fileKey);
    var fileStream = await s3Client.send(new GetObjectCommand(bucketParams));
    // for TS you can add: if (fileStream.Body instanceof Readable)
    fileStream.Body.pipe(res)
});
k4emjkb1

k4emjkb16#

为此,我使用React frontendnode js backend。前端我使用Axios。我用这个点击按钮下载文件。
====节点js后端代码(AWS S3)======
//在GET方法内部调用了此函数

public download = (req: Request, res: Response) => {
    const keyName = req.query.keyName as string;
    if (!keyName) {
        throw new Error('key is undefined');
    }
    const downloadParams: AWS.S3.GetObjectRequest = {
        Bucket: this.BUCKET_NAME,
        Key: keyName
    };

    this.s3.getObject(downloadParams, (error, data) => {
        if (error) {
            return error;
        }
        res.send(data.Body);
        res.end();
    });
};

======Reactjs前端代码========
//此函数处理下载按钮onClick

const downloadHandler = async (keyName: string) => {
  const response = await axiosInstance.get( //here use axios interceptors
    `papers/paper/download?keyName=${keyName}`,{
      responseType:'blob', //very very important dont miss (if not downloaded file unsupported to view)
    }
  );
  const url = window.URL.createObjectURL(new Blob([response.data]));
  const link = document.createElement("a");
  link.href = url;
  link.setAttribute("download", "file.pdf"); //change "file.pdf" according to saved name you want, give extension according to filetype
  document.body.appendChild(link);
  link.click();
  link.remove();
};

------ OR(如果您使用的是标准axios而不是axios拦截器)-----

axios({
   url: 'http://localhost:5000/static/example.pdf',
   method: 'GET',
   responseType: 'blob', // very very important
}).then((response) => {
   const url = window.URL.createObjectURL(new Blob([response.data]));
   const link = document.createElement('a');
   link.href = url;
   link.setAttribute('download', 'file.pdf');
   document.body.appendChild(link);
   link.click();
});

有关更多信息,请参阅下面的文章1. article 1 2. article 2

2q5ifsrm

2q5ifsrm7#

使用express,基于Jushua的答案和https://docs.aws.amazon.com/AmazonS3/latest/userguide/example_s3_GetObject_section.html

public downloadFeedFile = (req: IFeedUrlRequest, res: Response) => {
    const downloadParams: GetObjectCommandInput = parseS3Url(req.s3FileUrl.replace(/\s/g, ''));
    logger.info("requesting S3 file  " + JSON.stringify(downloadParams));
    const run = async () => {
      try {
        const fileStream = await this.s3Client.send(new GetObjectCommand(downloadParams));
        if (fileStream.Body instanceof Readable){
          fileStream.Body.once('error', err => {
            console.error("Error downloading s3 file")
            console.error(err);
          });

          fileStream.Body.pipe(res);

        }
      } catch (err) {
        logger.error("Error", err);
      }
    };

  run();

  };

相关问题