NodeJS 不正确的标题检查当尝试到解压缩的s3对象主体?

anhgbhbe  于 2023-02-18  发布在  Node.js
关注(0)|答案(1)|浏览(130)

我使用以下代码压缩并上传一个对象到s3:

let data: string | Buffer = JSON.stringify(rules);
  let contentType = "application/json";
  let encoding = null;
  let filename = `redirector-rules.json`;
  if (format === "gz") {
    contentType = "application/gzip";
    encoding = "gzip";
    filename = `redirector-rules.gz`;
    const buf = Buffer.from(data, "utf-8");
    data = zlib.gzipSync(buf);
  }

  // res.end(data);
  // return res.status(200).send(data);
  await s3.upload(filename, data, contentType, encoding);

我假设这是正确的工作,因为当我使用aws s3 cp命令下载结果文件时,它工作得很好,我能够在我的机器上解压缩它。此外,可能不相关的事实是,如果我通过s3的conole下载,我的系统无法解压缩它,它可能会损坏或截断。
在另一端我有一个lambda代码读取get对象并试图解压缩它:

const getRules = async (rulesCommand: GetObjectCommand): Promise<Config> => {

    const resp = await fetchRulesFile(rulesCommand);
    const data = await parseResponse(resp, rulesCommand);

    return data;

};

const fetchRulesFile = async (rulesCommand: GetObjectCommand): Promise<GetObjectCommandOutput> => {
  try {
    console.log(`Retrieving rules file with name ${rulesCommand.input.Key}`);
    const resp = await client.send(rulesCommand);
    return resp;
  } catch (err) {
    throw new Error(`Error retrieving rules file: ${err}`);
  }
};

const parseResponse = async (resp: GetObjectCommandOutput, rulesCommand: GetObjectCommand): Promise<Config> => {
  const { Body } = resp;
  if (!Body) {
    throw new Error("No body in response");
  }

  let data: string = await Body.transformToString();

  if (rulesCommand.input.Key?.endsWith(".gz")) {
    console.log(`Uncompressing rules file with name ${rulesCommand.input.Key}`);
    try {
      data = zlib.gunzipSync(data).toString("utf-8");
    } catch (err) {
      throw new Error(`Error decompressing rules file: ${err}`);
    }
  }

  return JSON.parse(data) as Config;
};

但我一直收到这个错误:Error: incorrect header check

k97glaaz

k97glaaz1#

我通过在parseResponse函数中使用Readable和streams解决了这个问题:

const parseResponse = async (
  resp: GetObjectCommandOutput,
  rulesCommand: GetObjectCommand
): Promise<Config> => {
  const { Body } = resp;
  if (!Body) {
    throw new Error("No body in response");
  }

  let data = "";
  const readableStream = new Readable();
  readableStream._read = () => {}; // noop
  // @ts-ignore
  Body.on("data", (chunk : any) => {
    readableStream.push(chunk);
    data += chunk;
  });
  // @ts-ignore
  Body.on("end", () => {
    readableStream.push(null);
  });
  const gunzip = zlib.createGunzip();
  const result = await new Promise((resolve, reject) => {
    let buffer = "";
    readableStream.pipe(gunzip);
    gunzip.on("data", (chunk) => {
      buffer += chunk;
    });
    gunzip.on("end", () => {
      resolve(buffer);
    });
    gunzip.on("error", reject);
  });
  return result as Config;
};

由于类型不匹配,我不得不在Body.on处添加@ts-ignore。但是它仍然可以在编译后的JS处理程序中工作,并且用转换来修复它似乎有点复杂。

相关问题