确定已完成作业使用的hadoop集群资源

nhhxz33t  于 2021-05-27  发布在  Hadoop
关注(0)|答案(0)|浏览(265)

如何确定已完成作业使用的hadoop集群资源?
我们的群集资源管理器 yarn . 可以通过http访问特定的yarn-api端点;例如:

curl -L http://my.hadoop.instance:8088/ws/v1/cluster/apps/application_1547448533998_502644

将返回:

{
    "app": {
        "allocatedMB": -1,
        "allocatedVCores": -1,
        "amContainerLogs": "http://someNode.hadoop.instance:8042/node/containerlogs/container_e149_1547448533998_502644_01_000001/someUser",
        "amHostHttpAddress": "someNode.hadoop.instance:8042",
        "amNodeLabelExpression": "",
        "applicationTags": "",
        "applicationType": "SPARK",
        "clusterId": 1547448533998,
        "clusterUsagePercentage": 0.0,
        "diagnostics": "",
        "elapsedTime": 583889,
        "finalStatus": "SUCCEEDED",
        "finishedTime": 1550621490747,
        "id": "application_1547448533998_502644",
        "logAggregationStatus": "TIME_OUT",
        "memorySeconds": 15821179,
        "name": "ProjectCantor",
        "numAMContainerPreempted": 0,
        "numNonAMContainerPreempted": 0,
        "preemptedResourceMB": 0,
        "preemptedResourceVCores": 0,
        "priority": 0,
        "progress": 100.0,
        "queue": "dsg",
        "queueUsagePercentage": 0.0,
        "runningContainers": -1,
        "startedTime": 1550620906858,
        "state": "FINISHED",
        "trackingUI": "History",
        "trackingUrl": "http://my.hadoop.instance:8088/proxy/application_1547448533998_502644/",
        "unmanagedApplication": false,
        "user": "someUser",
        "vcoreSeconds": 14713
    }
}

但是,有几个属性,例如 allocatedMB 以及 allocatedVCores 设置为 -1 而不是任何有意义的价值。
提前谢谢。

暂无答案!

目前还没有任何答案,快来回答吧!

相关问题