NodeJS Artillery:从CSV文件加载JSON文档

fhity93d  于 2023-03-01  发布在  Node.js
关注(0)|答案(2)|浏览(90)

我想测试的lambda函数需要一个复杂的json,根据我的理解,json需要进入csv文件。我的问题是我尝试了各种方法从csv加载json,但总是得到错误。我不确定这在炮兵中是否可行。我的示例csv看起来像这样。

    • 后期数据. csv**

column1 {"个人资料":{"姓名":"irfan","电子邮件":"www.example.com "},"地址":["地址1 ","地址2 "]}{"个人资料":{"姓名":" Tomas ","电子邮件":" www.example.com "},"地址":["地址1","地址2"]}{"个人资料":{"姓名":"Joel","电子邮件":"www.example.com "},"地址":["地址1 ","地址2 "]} irfan@email.com "},"address":["address1","address2"]} { "profile":{"name":"Tomas","email":" tomas@email.com "},"address":["address1","address2"]} { "profile":{"name":"Joel","email":" joel@email.com "},"address":["address1","address2"]}
我只有一列,因为我想要的只是这个json文档作为请求体传递给我的hello world lambda,这样我就可以对它进行负载测试。
下面是我的炮兵脚本文件的内容。

config:
  target: "https://api-gateway-load-testing-tst-ap-southeast-2.xxxxxxxxxxx.com"
  phases:
    - 
      duration: 5
      arrivalRate: 1
  defaults:
    headers:
      x-api-key: "xxxxxxxxxxxxxxxxxxxxxxxxxxx"
      Content-Type: "application/json"
  payload:
    # path is relative to the location of the test script
    path: "post-data.csv"
    fields:
      - "column1"
    order: sequence
    delimiter: "~"
    skipHeader: true
    cast: false
  plugins:
    cloudwatch:
      namespace: "serverless-artillery-loadtest"
scenarios:
  - flow:
      - post:
          url: "/v1/hello-world"
          json:
            data: {{ column1 }}

当我在json中的键和值周围加上双引号时,我收到错误消息"执行任务时出错:在1579692773908中执行来自1579692773908的加载时遇到错误异常:炮兵已退出,代码为非零:"
有没有办法从csv加载json,让我的hello world lambda函数以json的形式接收请求主体,格式如下:
{"数据":{"个人资料":{"姓名":" irfan ","电子邮件":" www.example.com "},"地址":["地址1","地址2"]}}irfan@email.com"},"address":["address1","address2"]}}
任何帮助都将不胜感激。

v2g6jxz6

v2g6jxz61#

我自己设法通过编写自定义JavaScript来加载json有效负载而不是从csv文件来解决这个问题。我使用config.processor沿着beforeScenario钩子来定义我的自定义逻辑。
对于任何可能面临类似问题的人,以下是我的解决方案:

脚本.yml

config:
  target: "https://api-ap-southeast-2.aws.my-domain.com"
  processor: "./post-body.js"
  # Following phases test a scenario where 0 tps ramps up to 50 tps in 1 minutes, and then ramps up to 1000 tps every 30 seconds in 50 tps increment

  phases:
    - 
      duration: 60
      arrivalRate: 10
      rampTo: 50
    -
      duration: 30
      arrivalRate: 50
    -
      duration: 30
      arrivalRate: 100
    -
      duration: 30
      arrivalRate: 150
    -
      duration: 30
      arrivalRate: 200
    -
      duration: 30
      arrivalRate: 250
    -
      duration: 30
      arrivalRate: 300
    -
      duration: 30
      arrivalRate: 350
    -
      duration: 30
      arrivalRate: 400
    -
      duration: 30
      arrivalRate: 450
    -
      duration: 30
      arrivalRate: 500
    -
      duration: 30
      arrivalRate: 550
    -
      duration: 30
      arrivalRate: 600
    -
      duration: 30
      arrivalRate: 650
    -
      duration: 30
      arrivalRate: 700
    -
      duration: 30
      arrivalRate: 750
    -
      duration: 30
      arrivalRate: 800
    -
      duration: 30
      arrivalRate: 850
    -
      duration: 30
      arrivalRate: 900
    -
      duration: 30
      arrivalRate: 950
    -
      duration: 270
      arrivalRate: 1000
  defaults:
    headers:
      x-api-key: "fake-x-api-key"
      Content-Type: "application/json"
  plugins:
    cloudwatch:
      namespace: "my-service-name"
    influxdb:
      testName: "my-service Load Test Results"
      influx:
        host: "fake-ip-address"
        username: "fake-username"
        password: "fake-password"
        database: "influx"

scenarios:
  - name: my-service-name load test with varying load
    beforeScenario: generatePostBody
    flow:
      - post:
          url: "/my-fake-endpoint"
          json:
            "{{ data }}"

下面的post-body.js包含了我自定义的JS逻辑。我引入了一个新的txt文件post-data.txt,它基本上取代了我在问题中提到的csv文件,以托管数千行,其中每行都是作为json的请求负载。每次执行一个场景时,都会拾取一个随机的json负载字符串。转换为json对象并作为POST请求的一部分发送。我还使用CloudWatch和InfluxDB来输出结果。

后正文.js

const fs = require("fs");
const filePath = "./post-data.txt";
let postData;

/**
 * Generates post body
 */
const generatePostBody = async (userContext, events, done) => {
  try{
    // add variables to virtual user's context:
    if(postData === undefined || postData === null || postData.length === 0) {
      postData = await loadDataFromTxt(filePath);
    }
    const postBodyStr = postData[Math.floor(Math.random()*postData.length)];
    userContext.vars.data = JSON.parse(postBodyStr);

    // continue with executing the scenario:
    return done();

  } catch(err) {
    console.log(`Error occurred in function generatePostBody. Detail: ${err}`);
    throw(err);
  }
}

/**
 * Loads post body from csv file
 * @param {object} filePath - The path of csv file
 */
const loadDataFromCsv = async filePath => {
  const data = [];

  return new Promise((resolve, reject) => {
    fs.createReadStream(filePath)
      .pipe(csv({delimiter: '||'}))
      .on("data", data => data.push(data))
      .on("end", () => {
        return resolve(data);
      })
      .on("error", error => {
        return reject(error);
      });
  });
};

/**
 * Loads post body from text file
 * @param {object} filePath - The path of text file
 */
const loadDataFromTxt = async (path) => {
  return new Promise((resolve, reject) => {
    fs.readFile(path, 'utf8', function (err, data) {
      if (err) {
        reject(err);
      }
      resolve(data.toString().split("\n"));
    });
  });
}

// Load data from txt file once at the start of the execution
// and save the results in a global variable
(async () => {
  try {
  postData = await loadDataFromTxt(filePath);
  //console.log(JSON.parse(postData[0]));
  } catch (error) {
    console.log(`Error occurred in main. Detail: ${err}`);
  }
})();

module.exports.generatePostBody = generatePostBody;

后期数据.txt

{ "profile":{"name":"irfan","email":"irfan@email.com"},"address":["address1","address2"]}
{ "profile":{"name":"Tomas","email":"tomas@email.com"},"address":["address1","address2"]}
{ "profile":{"name":"Joel","email":"joel@email.com"},"address":["address1","address2"]}

高温加热

xriantvc

xriantvc2#

只是想发布一个额外的解决方案,我的工作,将工作,如果你的JSON是相当简单的。问题是围绕引号和逗号在CSV文件时,分配JSON到一个字段。我改变了CSV分隔符从逗号到'|'。我还将JSON双引号改为单引号,以避免被误解为CSV字符串的开始/结束。我的CSV数据行如下所示:

||A|B|0.5|1.5|C|[{'timeIncrement':2, 'percentSlip': 0}]

然后在我的场景的处理器中,在解析JSON字段之前,我只需要将所有单引号替换为双引号。

var paymentScheduleReqs = context.vars.paymentScheduleReqs.replaceAll("'", '"');
  var paymentScheduleReqsInstance = JSON.parse(paymentScheduleReqs);

下面是CSV文件的配置:

processor: "./api-payment-test-helper.js"
  payload:
    # path is relative to the location of the test script
    path: "comprehensive.csv"
    delimiter: "|"
    fields:
      - "field1"
      - "field2"
      - "field3"
      - "field4"
      - "field5"
      - "field6"
      - "field7"
      - "paymentScheduleReqs"

我会第一个承认它很笨拙,但它足够简单,并且完成了任务。如果您的JSON中有合法的单引号或管道,那么您需要处理它。

相关问题