NodeJS js-如何继续描述特定查询的Completed状态的Cloudwatch日志查询

a0zr77ik  于 2022-12-18  发布在  Node.js
关注(0)|答案(3)|浏览(125)

因此,对于任何熟悉CW日志的人来说......这不是典型的返回结果的DB查询调用。您发送一个API调用来启动一个查询,它返回一个查询ID。然后您发送一个不同的API调用来获取查询结果,“希望”查询完成,如果它没有完成,您就不走运了。这就是我现在所做的。
我有一个查询需要花费一些时间,我“猜测”处理这个问题的方法是不断循环DescribeQueries调用,直到我在返回的Completed查询数组中找到匹配项,然后继续执行其余代码。我无法完成这个任务!
我试过完全不起作用的While和Do.. While循环。我试过在找到匹配项时设置转义条件值..但它从未被设置,Lambda函数超时。

function checkQueryStatus (logGroup, qID, callback) {

      var params = {
      logGroupName: logGroup, 
      maxResults: '3',
      status: 'Complete'
    };

let found = 0;

      do {
      cwlogs.describeQueries(params, function(err, data) {
        if (err) console.log(err, err.stack); // an error occurred
        else {
          // console.log(data.queries);           // successful response
            const qList = data.queries;
            if (qList.some(query => query.queryId === qID)) {  
              console.log('Query Done');
              callback(1);
              found = 1;
          } else {
            console.log('Query not done');
          }
        }
      });
      } while (found == 0); 
}

checkQueryStatus(logGroupName, queryID, function(qStatus) {
  console.log('Query Status: ', qStatus);

  if (qStatus == 1) { 
  console.log('Query Done');

  <do other code...>

我该怎么做呢?我现在正在查看Promises,看看这是怎么回事..如果DescribeQueries找到匹配项,我想触发GetQueryResults API调用。

vdgimpew

vdgimpew1#

我在AWS Athena中遇到过类似的问题。当我启动一个查询时,我会得到已经启动的响应,但在查询完成时却没有得到任何通知。我想到的最好的解决方案是使用setTimeout每隔100ms左右检查一次状态,并在查询完成时继续。希望这能有所帮助。

nkoocmlb

nkoocmlb2#

下面是一个通用函数,您可以使用它等待查询完成并返回查询结果。

async function queryCloudWatch(queryRequest: StartQueryRequest): Promise<GetQueryResultsResponse> {
    const queryResponse: StartQueryResponse = await cloudwatchLogs.startQuery(queryRequest).promise()

    if (!queryResponse.queryId) return {}

    let response: GetQueryResultsResponse | undefined = undefined

    while (!response || response.status === 'Running') {
        response = await cloudwatchLogs.getQueryResults({
            "queryId": queryResponse.queryId
        }).promise()
    }

    return response;
}
dxxyhpgq

dxxyhpgq3#

您有两种选择:查询日志洞察或查询日志组
如果要查询日志洞察:

const AWS = require('aws-sdk');
AWS.config.setPromisesDependency(require('bluebird'));
AWS.config.update({region: 'us-west-2'});
const cloudWatchLogs = new AWS.CloudWatchLogs({apiVersion: '2014-03-28'});
exports.handler = async (event) => {

  // Cloudwatch Log Group name
  const logGroupName = '/aws/lambda/<Name of your Log Group>';
  const timestamp = new Date();

  const params = {
    endTime: timestamp.getTime(),
    queryString: `fields @message, @timestamp
    | sort @timestamp desc
    | limit 10
    | filter @message like /(?i)("Error")/
    | stats count() by bin(1d)`, // Group by Day
    startTime: timestamp.setDate( timestamp.getDate() - 3 ), // Last 3 days
    logGroupName: logGroupName
  };
  
  // 1. Start the query. When we start a query, this returns a queryId for us to use on our next step.
  const data = await cloudwatchlogs.startQuery(params).promise();
  const { queryId } = data;
  console.debug('query id', queryId);

  while (true) {
    
    // 2. Send Insight query to CloudwatchLogs
    const insightData = await cloudwatchlogs.getQueryResults({ queryId })
        .promise();
    
    // 3. Check if it is available    
    if (Array.isArray(insightData.results) && insightData.status === 'Complete') {
      const insightResult = insightData.results;
      
      // Change this line to publish to SNS or send to Slack
      console.log(JSON.stringify(insightResult, null, 4))
      break;
    }
    
    // 4. Otherwise, Wait for 100 ms for insight api result
    await new Promise((resolve, reject) => setTimeout(resolve, 100));
  } 

  return 'ok';
}

要使用filterLogEvents API查询日志组:

const AWS = require('aws-sdk');
AWS.config.setPromisesDependency(require('bluebird'));
AWS.config.update({region: 'us-west-2'});
const cloudWatchLogs = new AWS.CloudWatchLogs({apiVersion: '2014-03-28'});

const timestamp = new Date();
const endtTime = timestamp.getTime();
const params = {
    endTime: endtTime,
    filterPattern: `"${stringToSearch}"`,
    startTime: new Date (endtTime - 5 * 60 * 60* 24 * 1000).getTime(), // Last 5 days
    logGroupName: 'myLogGroup',
    limit : 10
};

const events = await cloudWatchLogs.filterLogEvents(params).promise();
console.log(`successfully queryCloudWatchLogs ${stringToSearch} results: ${JSON.stringify(events)}`);
const results = events.events.map(e => e.message)
console.log(`successfully queryCloudWatchLogs ${stringToSearch} results (${results.length}): ${JSON.stringify(results)}`);

相关问题