我在flink中使用ec2角色假定方法进行身份验证时遇到问题。正在通过提交作业 aws emr add-steps
.
下面的代码在main方法中工作,允许我查看流的详细信息。
val k = new AmazonKinesisClient(new InstanceProfileCredentialsProvider())
k.setRegion(Region.getRegion(Regions.fromName("my-region")))
println("Stream: " + k.describeStream("stream-name"))
但是,当尝试使用相同的方法在同一个emr集群上进行身份验证时,我得到以下错误。
val config = new Properties()
config.put(AWSConfigConstants.AWS_REGION, "my-region")
config.put(AWSConfigConstants.AWS_CREDENTIALS_PROVIDER, "INSTANCE")
config.put(ConsumerConfigConstants.STREAM_INITIAL_POSITION, "LATEST")
val consumer = new FlinkKinesisConsumer[String](
"stream-name", new SimpleStringSchema, config)
错误
Caused by: java.io.IOException: Server returned HTTP response code: 403 for URL: http://169.254.169.254/latest/meta-data/iam/security-credentials/
at sun.net.www.protocol.http.HttpURLConnection.getInputStream0(HttpURLConnection.java:1894)
at sun.net.www.protocol.http.HttpURLConnection.getInputStream(HttpURLConnection.java:1492)
at java.net.HttpURLConnection.getResponseCode(HttpURLConnection.java:480)
at org.apache.flink.kinesis.shaded.com.amazonaws.internal.EC2MetadataClient.readResponse(EC2MetadataClient.java:111)
我还尝试更改amazonkinesisclient.createkinesisclient方法以强制使用 InstanceProfileCredentialsProvider
. 但不幸的是,这给出了相同的403错误。
public static AmazonKinesisClient createKinesisClient(Properties configProps) {
AmazonKinesisClient client = new AmazonKinesisClient(new InstanceProfileCredentialsProvider());
client.setRegion(Region.getRegion(Regions.fromName("my-region")));
return client;
}
再次出错。
Caused by: java.io.IOException: Server returned HTTP response code: 403 for URL: http://169.254.169.254/latest/meta-data/iam/security-credentials/
at sun.net.www.protocol.http.HttpURLConnection.getInputStream0(HttpURLConnection.java:1894)
at sun.net.www.protocol.http.HttpURLConnection.getInputStream(HttpURLConnection.java:1492)
at java.net.HttpURLConnection.getResponseCode(HttpURLConnection.java:480)
at org.apache.flink.kinesis.shaded.com.amazonaws.internal.EC2MetadataClient.readResponse(EC2MetadataClient.java:111)
1条答案
按热度按时间whitzsjs1#
如果需要代理来访问外部资源,请确保添加
-Dhttp.nonProxyHosts 169.254.169.254
允许服务器访问元数据存储。