如何使用eclipse在amazonemr中运行hadoop作业?

vx6bjr1n  于 2021-06-02  发布在  Hadoop
关注(0)|答案(0)|浏览(222)

我遵循了amazon提供的教程,但是我的代码似乎没有运行。
我得到的错误是:

Exception in thread "main" java.lang.Error: Unresolved compilation problem: 
    The method withJobFlowRole(String) is undefined for the type AddJobFlowStepsRequest

    at main.main(main.java:38)

我的完整代码:

import java.io.IOException;
import com.amazonaws.auth.AWSCredentials;
import com.amazonaws.auth.PropertiesCredentials;
import com.amazonaws.services.elasticmapreduce.*;
import com.amazonaws.services.elasticmapreduce.model.AddJobFlowStepsRequest;
import com.amazonaws.services.elasticmapreduce.model.AddJobFlowStepsResult;
import com.amazonaws.services.elasticmapreduce.model.HadoopJarStepConfig;
import com.amazonaws.services.elasticmapreduce.model.StepConfig;
import com.amazonaws.services.elasticmapreduce.util.StepFactory;

public class main {

    public static void main(String[] args) {

        AWSCredentials credentials = null;
        try {
            credentials = new PropertiesCredentials(
                main.class.getResourceAsStream("AwsCredentials.properties"));
        } catch (IOException e1) {
            System.out.println("Credentials were not properly entered into AwsCredentials.properties.");
            System.out.println(e1.getMessage());
            System.exit(-1);
        }

        AmazonElasticMapReduce client = new AmazonElasticMapReduceClient(credentials);

        // predefined steps. See StepFactory for list of predefined steps
        StepConfig hive = new StepConfig("Hive", new StepFactory().newInstallHiveStep());

        // A custom step
        HadoopJarStepConfig hadoopConfig1 = new HadoopJarStepConfig()
            .withJar("s3://mywordcountbuckett/binary/WordCount.jar")
            .withMainClass("com.my.Main1") // optional main class, this can be omitted if jar above has a manifest
            .withArgs("--verbose"); // optional list of arguments
        StepConfig customStep = new StepConfig("Step1", hadoopConfig1);

        AddJobFlowStepsResult result = client.addJobFlowSteps(new AddJobFlowStepsRequest()  
            .withJobFlowRole("jobflow_role")
            .withServiceRole("service_role")
            .withSteps(hive, customStep));
        System.out.println(result.getStepIds());

    }
}

代码没有运行的原因是什么?有基于最新版本的教程吗?

暂无答案!

目前还没有任何答案,快来回答吧!

相关问题