启动flink会话并在docker compose中将作业提交给flink

2nc8po8w  于 2021-06-21  发布在  Flink
关注(0)|答案(0)|浏览(295)

我有一个docker从大数据欧洲yaml文件组成,我想启动resourcemanager节点上的flink。我用下载的flink v添加音量。11,增加了卷和开放端口,所以docker compose看起来像

resourcemanager:
    image: bde2020/hadoop-resourcemanager:2.0.0-hadoop3.2.1-java8
    container_name: resourcemanager
    restart: always
    environment:
      SERVICE_PRECONDITION: "namenode:9000 namenode:9870 datanode:9864"
    env_file:
      - ./hadoop.env
    volumes:
    - ./flink/:/flink
    ports:
      - 8081:8081

根据flink文档设置,我尝试在容器内使用以下命令启动flink群集:

./bin/yarn-session.sh -n 10 -tm 8192 -s 32

并使用命令提交作业:

/bin/flink run ./examples/batch/WordCount.jar hdfs:///..../LICENSE-2.0.txt hdfs:///.../wordcount-result.txt

对吗?我还需要做些什么来正确启动它?
我接到第一个命令后:

Error: A JNI error has occurred, please check your installation and try again
Exception in thread "main" java.lang.NoClassDefFoundError: org/apache/hadoop/yarn/exceptions/YarnException
        at java.lang.Class.getDeclaredMethods0(Native Method)
        at java.lang.Class.privateGetDeclaredMethods(Class.java:2701)
        at java.lang.Class.privateGetMethodRecursive(Class.java:3048)
        at java.lang.Class.getMethod0(Class.java:3018)
        at java.lang.Class.getMethod(Class.java:1784)
        at sun.launcher.LauncherHelper.validateMainClass(LauncherHelper.java:544)
        at sun.launcher.LauncherHelper.checkAndLoadMain(LauncherHelper.java:526)
Caused by: java.lang.ClassNotFoundException: org.apache.hadoop.yarn.exceptions.YarnException
        at java.net.URLClassLoader.findClass(URLClassLoader.java:382)
        at java.lang.ClassLoader.loadClass(ClassLoader.java:418)
        at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:352)
        at java.lang.ClassLoader.loadClass(ClassLoader.java:351)
        ... 7 more

暂无答案!

目前还没有任何答案,快来回答吧!

相关问题