如何确定已完成作业使用的hadoop集群资源?
我们的群集资源管理器 yarn
. 可以通过http访问特定的yarn-api端点;例如:
curl -L http://my.hadoop.instance:8088/ws/v1/cluster/apps/application_1547448533998_502644
将返回:
{
"app": {
"allocatedMB": -1,
"allocatedVCores": -1,
"amContainerLogs": "http://someNode.hadoop.instance:8042/node/containerlogs/container_e149_1547448533998_502644_01_000001/someUser",
"amHostHttpAddress": "someNode.hadoop.instance:8042",
"amNodeLabelExpression": "",
"applicationTags": "",
"applicationType": "SPARK",
"clusterId": 1547448533998,
"clusterUsagePercentage": 0.0,
"diagnostics": "",
"elapsedTime": 583889,
"finalStatus": "SUCCEEDED",
"finishedTime": 1550621490747,
"id": "application_1547448533998_502644",
"logAggregationStatus": "TIME_OUT",
"memorySeconds": 15821179,
"name": "ProjectCantor",
"numAMContainerPreempted": 0,
"numNonAMContainerPreempted": 0,
"preemptedResourceMB": 0,
"preemptedResourceVCores": 0,
"priority": 0,
"progress": 100.0,
"queue": "dsg",
"queueUsagePercentage": 0.0,
"runningContainers": -1,
"startedTime": 1550620906858,
"state": "FINISHED",
"trackingUI": "History",
"trackingUrl": "http://my.hadoop.instance:8088/proxy/application_1547448533998_502644/",
"unmanagedApplication": false,
"user": "someUser",
"vcoreSeconds": 14713
}
}
但是,有几个属性,例如 allocatedMB
以及 allocatedVCores
设置为 -1
而不是任何有意义的价值。
提前谢谢。
暂无答案!
目前还没有任何答案,快来回答吧!