amazon雅典娜在geo-json上返回“hive\u cursor\u error:hive\u cursor\u error”

li9yvcax  于 2021-06-27  发布在  Hive
关注(0)|答案(1)|浏览(362)

我正在尝试使用amazon athena读取geojson文件。
我的输入数据的头部如下所示:

{
"type": "FeatureCollection",
"name": "sql_statement",
"features": [
{ "type": "Feature", "properties": { "gridsize": 500.0, "big_gid": 353, "little_gid": 22482 }, "geometry": { "type": "Polygon", "coordinates": [ [ [ -981739.267883020918816, 30855.609566356935829 ], [ -981739.267883020918816, 31355.354737498135364 ], [ -981241.022986860014498, 31355.354737498135364 ], [ -981241.022986860014498, 30855.609566356935829 ], [ -981739.267883020918816, 30855.609566356935829 ] ] ] } },
{ "type": "Feature", "properties": { "gridsize": 500.0, "big_gid": 353, "little_gid": 22483 }, "geometry": { "type": "Polygon", "coordinates": [ [ [ -981241.022986860014498, 30855.609566356935829 ], [ -981241.022986860014498, 31355.354737498135364 ], [ -980742.778090699226595, 31355.354737498135364 ], [ -980742.778090699226595, 30855.609566356935829 ], [ -981241.022986860014498, 30855.609566356935829 ] ] ] } },
{ "type": "Feature", "properties": { "gridsize": 500.0, "big_gid": 353, "little_gid": 22484 }, "geometry": { "type": "Polygon", "coordinates": [ [ [ -980742.778090699226595, 30855.609566356935829 ], [ -980742.778090699226595, 31355.354737498135364 ], [ -980244.533194538322277, 31355.354737498135364 ], [ -980244.533194538322277, 30855.609566356935829 ], [ -980742.778090699226595, 30855.609566356935829 ] ] ] } },
{ "type": "Feature", "properties": { "gridsize": 500.0, "big_gid": 353, "little_gid": 22485 }, "geometry": { "type": "Polygon", "coordinates": [ [ [ -980244.533194538322277, 30855.609566356935829 ], [ -980244.533194538322277, 31355.354737498135364 ], [ -979746.288298377417959, 31355.354737498135364 ], [ -979746.288298377417959, 30855.609566356935829 ], [ -980244.533194538322277, 30855.609566356935829 ] ] ] } },
{ "type": "Feature", "properties": { "gridsize": 500.0, "big_gid": 353, "little_gid": 22486 }, "geometry": { "type": "Polygon", "coordinates": [ [ [ -979746.288298377417959, 30855.609566356935829 ], [ -979746.288298377417959, 31355.354737498135364 ], [ -979248.043402216513641, 31355.354737498135364 ], [ -979248.043402216513641, 30855.609566356935829 ], [ -979746.288298377417959, 30855.609566356935829 ] ] ] } },
{ "type": "Feature", "properties": { "gridsize": 500.0, "big_gid": 353, "little_gid": 22487 }, "geometry": { "type": "Polygon", "coordinates": [ [ [ -979248.043402216513641, 30855.609566356935829 ], [ -979248.043402216513641, 31355.354737498135364 ], [ -978749.798506055609323, 31355.354737498135364 ], [ -978749.798506055609323, 30855.609566356935829 ], [ -979248.043402216513641, 30855.609566356935829 ] ] ] } },

我使用地理空间文档作为模板定义了表:

CREATE external TABLE IF NOT EXISTS testdb.grid_500
 (
 gridsize double,
 big_gid int, 
 little_gid int, 
 geometry binary
 )
ROW FORMAT SERDE 'com.esri.hadoop.hive.serde.JsonSerde'
STORED AS INPUTFORMAT 'com.esri.json.hadoop.EnclosedJsonInputFormat'
OUTPUTFORMAT 'org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat'
LOCATION 's3://jdl-athena/grid/'
;

但是,我对该表运行的任何查询都会返回相同的、但毫无帮助的错误:

Your query has the following error(s):

HIVE_CURSOR_ERROR: HIVE_CURSOR_ERROR

This query ran against the "testdb" database, unless qualified by the query.
Please post the error message on our forum or contact customer support with 
Query Id: 25d3da93-5cfd-46bb-9f77-4eb014679ba6.

你知道我为什么会收到这个错误吗?或者我该如何调试它?我甚至不知道如何获得足够的信息来诊断可能发生的事情。

6ioyuze2

6ioyuze21#

所以在googlefu上花了很多时间之后,我发现(当然)esri已经为地理数据创建了自己的json格式。我使用的是开放标准 GeoJson esri格式称为 ESRI Enclosed JSON . 对于geometric json,athena只支持esri格式,不支持geojson。
我在这里发现了这种区别:https://github.com/esri/gis-tools-for-hadoop/issues/29
所以看起来我需要不同格式的输入数据。

相关问题