instanceprofile

j1dl9f46  于 2021-06-02  发布在  Hadoop
关注(0)|答案(3)|浏览(432)

我试图从eclipse运行elasticmapreduce,但是做不到。
我的代码如下:

public class RunEMR {

    /**
     * @param args
     */
    public static void main(String[] args) {
        // TODO Auto-generated method stub
         AWSCredentials credentials = new BasicAWSCredentials("xxxx","xxxx");
            AmazonElasticMapReduceClient emr = new AmazonElasticMapReduceClient(credentials);

            StepFactory stepFactory = new StepFactory();

            StepConfig enableDebugging = new StepConfig()
                .withName("Enable Debugging")
                .withActionOnFailure("TERMINATE_JOB_FLOW")
                .withHadoopJarStep(stepFactory.newEnableDebuggingStep());

            StepConfig installHive = new StepConfig()
                .withName("Install Hive")
                .withActionOnFailure("TERMINATE_JOB_FLOW")
                .withHadoopJarStep(stepFactory.newInstallHiveStep());

            StepConfig hiveScript = new StepConfig().withName("Hive Script")
                .withActionOnFailure("TERMINATE_JOB_FLOW")
                .withHadoopJarStep(stepFactory.newRunHiveScriptStep("s3://mywordcountbuckett/binary/WordCount.jar"));

            RunJobFlowRequest request = new RunJobFlowRequest()
                .withName("Hive Interactive")
                .withSteps(enableDebugging, installHive)
                .withLogUri("s3://mywordcountbuckett/")
                .withInstances(new JobFlowInstancesConfig()
                    .withEc2KeyName("xxxx")
                    .withHadoopVersion("0.20")
                    .withInstanceCount(3)
                    .withKeepJobFlowAliveWhenNoSteps(true)
                    .withMasterInstanceType("m1.small")
                    .withSlaveInstanceType("m1.small"));

            RunJobFlowResult result = emr.runJobFlow(request);

    }

}

我得到的错误是:

Exception in thread "main" com.amazonaws.AmazonServiceException: InstanceProfile is required for creating cluster. (Service: AmazonElasticMapReduce; Status Code: 400; Error Code: ValidationException; Request ID: 7a96ee32-9744-11e5-947d-65ca8f7db0a5

我试了几个小时,但没能修好。有人知道怎么做吗?

li9yvcax

li9yvcax1#

对于ec2角色(这里是jobflowrole),将在内部创建具有相同名称的示例概要文件。因此,它可以互换地使用这些名称。如果您使用boto3从头开始创建emr集群,那么还应该创建emr服务角色、一个ec2jobflow角色、一个链接到ec2jobflow角色的示例概要文件。aws文件

dw1jzc5e

dw1jzc5e2#

您尝试使用的版本已弃用,需要iam角色。遵循文档中给出的示例http://docs.aws.amazon.com/elasticmapreduce/latest/managementguide/calling-emr-with-java-sdk.html.

mmvthczy

mmvthczy3#

我也有同样的例外 InstanceProfile is required for creating cluster .
必须设置服务角色和作业流角色,如下所示

aRunJobFlowRequest.setServiceRole("EMR_DefaultRole")
aRunJobFlowRequest.setJobFlowRole("EMR_EC2_DefaultRole")

之后我就没事了。
用于emr iam角色的aws文件说
aws身份和访问管理(iam)角色为iam用户或aws服务提供了一种对资源具有特定权限和访问权限的方法。例如,这可能允许用户访问资源或其他服务以代表您行事。必须为集群指定两个iam角色:amazon emr服务的角色(service role)和amazon emr管理的ec2示例的角色(instance profile)。
所以说这个词 InstanceProfile 在异常消息中可能意味着 a role for the EC2 instances (instance profile) 但我在指定 JobFlowRole . 有点奇怪。

相关问题