我最近在heroku上建立了一个平台来托管 puppet 消除脚本(express,node js)。在剧本开始花越来越长的时间完成之前,一切都很顺利。根据heroku的文档,任何超过30秒的时间都会超时请求,并且失败。
解决这个问题的方法是使用redis服务器(特别是npm模块bull)。
我以前从没听说过排队,所以这将是我学习的机会。
下面是我当前的代码(可以使用)。
我将(彻底)向您介绍整个过程:
服务器从客户端获取post请求,请求它运行存储在数据库中的以下脚本:
router.post(
"/get-data/:id", // route
ensureAuth, // ensure the person making the request is allowed to make the request in the first place
fsController.createFile, // creates a .js file where the script will be copied to
scrapperController.getData, // runs the function in the newly created .js file
csvController.buildCSVAndDeliver // adds that data to a csv file and downloads it
);
将创建该文件,并将数据库中的代码复制到该文件中
const Script = require("../models/Script");
const fs = require("fs");
exports.createFile = async (req, res, next) => {
try {
const script = await Script.findById({ _id: req.params.id });
let filename = `./scrapper/${script._id}.js`;
// While loop to check if file exists and to append a digit if it does.
let counter = 0;
while (fs.existsSync(filename)) {
filename = `./scrapper/${script._id}${counter}.js`;
counter++;
}
fs.writeFileSync(filename, script.script);
req.filename = filename;
next();
} catch (error) {
fs.unlink(filename);
res.send(error, "Please Try Again");
res.redirect('/dashboard')
}
};
scrapercontroller获取该文件并执行它
const fs = require("fs");
exports.getData = async (req, res, next) => {
try {
let scrapper = require(`.${req.filename}`);
const response = await scrapper(req.body);
req.response = response;
fs.unlinkSync(req.filename);
next();
} catch (error) {
res.send(error, "Please Try Again");
fs.unlinkSync(req.filename);
}
};
最后,下载csv文件
const fs = require("fs");
const { Parser } = require("json2csv");
exports.buildCSVAndDeliver = async (req, res) => {
let csvFile = `./csv/${req.params.id}.csv`;
try {
const parser = new Parser();
const csv = parser.parse(req.response);
fs.writeFileSync(csvFile, csv);
res.download(csvFile, function () {
fs.unlinkSync(csvFile);
});
} catch (error) {
console.log(error);
}
}
既然您已经了解了流程,我应该在哪里添加查询代码?这个代码结构还能用吗?
谢谢,如果您需要我提供更多信息,请随时询问。
暂无答案!
目前还没有任何答案,快来回答吧!