一个简单的文件分割器。在for循环中写入的第一行被写入writestream。在“for await readline”的后续迭代中写入的行不是。接下来的几行怎么写?
const tempWriteStream = (temporary: string): WriteStream => {
console.log(`Temporary file is ${temporary}`);
return fs.createWriteStream(temporary);
}
const fileSplitter = async (sourceBucket: GetObjectCommandInput) => {
const data = await s3Client.send(new GetObjectCommand(sourceBucket));
const rl = readline.createInterface({
input: data.Body,
crlfDelay: Infinity
});
let count = 0
let header = ""
let temporary = createTemporaryFile();
let writeStream = await tempWriteStream(temporary)
for await (const line of rl) {
if (count === 0) {
header = line;
}
count++;
// Handle drain
const writeCanContinue = writeStream.write(line)
if (count > limit || !writeCanContinue) {
console.log('Starting a new file')
writeStream.end()
handleCreateNewFile()
}
}
await writeStream.end(async () => {
})
await writeStream.on('finish', async () =>{
await s3Put(temporary, sourceBucket)
})
// data.Body is a readable stream
}
1条答案
按热度按时间gwbalxhn1#
作为others have said,不要使用readline。我也放弃了writeStream。这是整个lambda。AWS javascript SDK 3.