无法在EKS docker镜像上的EMR上启动Flink SQL客户端

enxuqcxy  于 12个月前  发布在  Apache
关注(0)|答案(1)|浏览(155)

我在EKS - * public.ecr.aws/emr-on-eks/flink/emr-6.15.0-flink:latest*上使用EMR的Flink docker镜像。当我尝试启动SQL客户端时,我看到以下错误。(当我使用官方flink镜像时,我没有看到此错误。)

06:33:00.495 [main] ERROR org.apache.flink.table.client.SqlClient - SQL Client must stop. Unexpected exception. This is a bug. Please consider filing an issue.
java.lang.RuntimeException: Could not find a free permitted port on the machine.
        at org.apache.flink.util.NetUtils.getAvailablePort(NetUtils.java:177) ~[flink-dist-1.17.1-amzn-1.jar:1.17.1-amzn-1]
        at org.apache.flink.table.client.SqlClient$EmbeddedGateway.create(SqlClient.java:257) ~[flink-sql-client-1.17.1-amzn-1.jar:1.17.1-amzn-1]
        at org.apache.flink.table.client.SqlClient.start(SqlClient.java:108) ~[flink-sql-client-1.17.1-amzn-1.jar:1.17.1-amzn-1]
        at org.apache.flink.table.client.SqlClient.startClient(SqlClient.java:228) [flink-sql-client-1.17.1-amzn-1.jar:1.17.1-amzn-1]
        at org.apache.flink.table.client.SqlClient.main(SqlClient.java:179) [flink-sql-client-1.17.1-amzn-1.jar:1.17.1-amzn-1]
Exception in thread "main" org.apache.flink.table.client.SqlClientException: Unexpected exception. This is a bug. Please consider filing an issue.
        at org.apache.flink.table.client.SqlClient.startClient(SqlClient.java:242)
        at org.apache.flink.table.client.SqlClient.main(SqlClient.java:179)
Caused by: java.lang.RuntimeException: Could not find a free permitted port on the machine.
        at org.apache.flink.util.NetUtils.getAvailablePort(NetUtils.java:177)
        at org.apache.flink.table.client.SqlClient$EmbeddedGateway.create(SqlClient.java:257)
        at org.apache.flink.table.client.SqlClient.start(SqlClient.java:108)
        at org.apache.flink.table.client.SqlClient.startClient(SqlClient.java:228)
        ... 1 more

字符串
它看起来是由 NetUtils 类(* NetUtils.java *)的getAvailablePort()引起的。

public static Port getAvailablePort() {
    for (int i = 0; i < 50; i++) {
        try (ServerSocket serverSocket = new ServerSocket(0)) {
            int port = serverSocket.getLocalPort();
            if (port != 0) {
                FileLock fileLock = new FileLock(NetUtils.class.getName() + port);
                if (fileLock.tryLock()) {
                    return new Port(port, fileLock);
                } else {
                    fileLock.unlockAndDestroy();
                }
            }
        } catch (IOException ignored) {
        }
    }

    throw new RuntimeException("Could not find a free permitted port on the machine.");
}


我看到一个相关的Jira ticket(https://issues.apache.org/jira/browse/FLINK-28613),它似乎通过更改文件权限解决了。但是它对我不起作用。基本上我不知道为什么它无法在/tmp文件夹上执行FileLock。
你能告诉我怎么才能修好它吗?
谢谢你,谢谢

s3fp2yjn

s3fp2yjn1#

我进一步检查了sql-client.sh,它写入/mnt/tmp而不是/tmp。在此之前创建临时目录修复了这个问题,我通过添加以下内容创建了一个自定义Docker镜像。

RUN mkdir -p /mnt/tmp

字符串
这不仅会影响SQL客户端,还会影响其他应用getAvailablePort()的地方,因此需要更新Docker镜像。

相关问题