s3上amazon emr上的配置单元任务顶点失败

iezvtpos  于 2021-06-26  发布在  Hive
关注(0)|答案(2)|浏览(470)

我已经在emr上创建了Hive表

create external table tests3(
transaction_id String,
order_id String,
user_id String,
amount String,
subscriber_number String,
product_type String,
provider String,
region String,
status String,
created_time String,
last_updated_time BIGINT,

)
row format 
    serde 'org.openx.data.jsonserde.JsonSerDe'
location 's3://locationtobucket';

当我从tests3中选择*时,我会得到记录。但是运行任何tez任务都会给我以下错误。

Query : select count(*) from tests3;

Vertex failed, vertexName=Map 1, vertexId=vertex_1513582536692_0022_1_00
, diagnostics=[Vertex vertex_1513582536692_0022_1_00 [Map 1] killed/failed due to:ROOT_INPUT_INIT_FAILURE, Vertex Input: tests3 initializer fail
ed, vertex=vertex_1513582536692_0022_1_00 [Map 1], com.amazon.ws.emr.hadoop.fs.shaded.com.amazonaws.services.s3.model.AmazonS3Exception: Bad Req
uest (Service: Amazon S3; Status Code: 400; Error Code: 400 Bad Request; Request ID: DEBA2E241B9DE8C3),
monwx1rj

monwx1rj1#

只要将配置单元执行引擎更改为mr,它就会正常启动。

zbwhf8kr

zbwhf8kr2#

请检查您的帐户是否具有访问s3和您尝试访问的bucket的适当权限。另外,检查s3 bucket url是否正确,例如s3://locationtobucket/missingfile
标准的第一步是:通过以下命令尝试使用具有相同凭据的aws命令行工具:
hdfs fs-ls s3a://locationtobucket/

相关问题