Vertex webclient使用了太多的文件描述符

neekobn8  于 2021-08-20  发布在  Java
关注(0)|答案(0)|浏览(240)

我正在尝试在aws lambda中运行Vertex webclient。aws lambda最多有1024个文件描述符,不可调整。我很难找到到底是什么耗尽了我所有的文件描述符。我只使用vertx的webclient,没有运行任何verticles。这是我的共享网络客户端:

var webClientOptions = new WebClientOptions()
                .setSsl(true)
                .setUseAlpn(true)
                .setReusePort(true)
                .setDefaultPort(443)
                .setHttp2MaxPoolSize(30)
                .setProtocolVersion(HttpVersion.HTTP_2)
                .setSslHandshakeTimeout(30)
                .setIdleTimeout(5)
                .setIdleTimeoutUnit(TimeUnit.SECONDS);

        return WebClient.create(vertx, webClientOptions);

此客户端在lambda调用之间重复使用,并且从不关闭。
我如何使用它:

var requestFutures = new ArrayList<CompletableFuture<Foo>>();
for (var request : requests) {
  CompletableFuture<Foo> future = new CompletableFuture();
  requestFutures.add(future);

  Buffer buffer = Buffer.buffer(request.getPayload());
  httpRequest.sendBuffer(buffer, (res) -> {
    future.complete(res);
  });
}

for (var requestFuture : requestFutures) {
  requestFuture.get();
}

我有45k-65k个请求要发送。我得到的是:

io.netty.channel.ChannelException: Failed to open a socket.
at io.netty.channel.socket.nio.NioSocketChannel.newSocket(NioSocketChannel.java:70)
at io.netty.channel.socket.nio.NioSocketChannel.<init>(NioSocketChannel.java:87)
at io.netty.channel.socket.nio.NioSocketChannel.<init>(NioSocketChannel.java:80)
at io.netty.bootstrap.AbstractBootstrap.initAndRegister(AbstractBootstrap.java:320)
at io.netty.bootstrap.Bootstrap.doResolveAndConnect(Bootstrap.java:163)
at io.netty.bootstrap.Bootstrap.connect(Bootstrap.java:145)
at io.vertx.core.net.impl.ChannelProvider.handleConnect(ChannelProvider.java:140)
at io.vertx.core.net.impl.ChannelProvider.connect(ChannelProvider.java:93)
at io.vertx.core.http.impl.HttpChannelConnector.doConnect(HttpChannelConnector.java:171)
at io.vertx.core.http.impl.HttpChannelConnector.connect(HttpChannelConnector.java:105)
at io.vertx.core.http.impl.pool.Pool$Holder.connect(Pool.java:125)
at io.vertx.core.http.impl.pool.Pool.checkPendingTasks(Pool.java:250)
at io.netty.util.concurrent.AbstractEventExecutor.safeExecute(AbstractEventExecutor.java:163)
at io.netty.util.concurrent.SingleThreadEventExecutor.runAllTasks(SingleThreadEventExecutor.java:404)
at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:474)
at io.netty.util.concurrent.SingleThreadEventExecutor$5.run(SingleThreadEventExecutor.java:909)
at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30)
at java.base/java.lang.Thread.run(Unknown Source)
Caused by: java.net.SocketException: Too many open files
at java.base/sun.nio.ch.Net.socket0(Native Method)
at java.base/sun.nio.ch.Net.socket(Unknown Source)
at java.base/sun.nio.ch.Net.socket(Unknown Source)
at java.base/sun.nio.ch.SocketChannelImpl.<init>(Unknown Source)
at java.base/sun.nio.ch.SelectorProviderImpl.openSocketChannel(Unknown Source)
at io.netty.channel.socket.nio.NioSocketChannel.newSocket(NioSocketChannel.java:68)
... 17 more

当我在本地运行这个程序,并在osx活动监视器中查看打开的文件/套接字时,它会出现数千个。我不明白它为什么要这样做,因为我已经配置了 WebClientOptions .
有人能从这些信息中看出我为什么要使用这么多的文件描述符吗?

暂无答案!

目前还没有任何答案,快来回答吧!

相关问题