我正在尝试使用pyspark的hivecontext以avro格式加载一个外部表。外部表创建查询在配置单元中运行。但是，相同的查询在配置单元上下文中失败，错误为， org.apache.hadoop.hive.serde2.SerDeException: Encountered exception determining schema. Returning signal schema to indicate problem: null 我的avro模式如下。

{
  "type" : "record",
  "name" : "test_table",
  "namespace" : "com.ent.dl.enh.test_table",
  "fields" : [ {
    "name" : "column1",
    "type" : [ "null", "string" ] , "default": null
  }, {
    "name" : "column2",
    "type" : [ "null", "string" ] , "default": null
  }, {
    "name" : "column3",
    "type" : [ "null", "string" ] , "default": null
  }, {
    "name" : "column4",
    "type" : [ "null", "string" ] , "default": null
  } ]
}

我的create table脚本是，

CREATE EXTERNAL TABLE test_table_enh ROW FORMAT SERDE 'org.apache.hadoop.hive.serde2.avro.AvroSerDe' STORED AS INPUTFORMAT 'org.apache.hadoop.hive.ql.io.avro.AvroContainerInputFormat' OUTPUTFORMAT 'org.apache.hadoop.hive.ql.io.avro.AvroContainerOutputFormat' LOCATION 's3://Staging/test_table/enh' TBLPROPERTIES ('avro.schema.url'='s3://Staging/test_table/test_table.avsc')

我用spark submit运行下面的代码，

from pyspark import SparkConf, SparkContext
from pyspark.sql import HiveContext

print "Start of program"
sc = SparkContext()
hive_context = HiveContext(sc)

hive_context.sql("CREATE EXTERNAL TABLE test_table_enh ROW FORMAT SERDE 'org.apache.hadoop.hive.serde2.avro.AvroSerDe' STORED AS INPUTFORMAT 'org.apache.hadoop.hive.ql.io.avro.AvroContainerInputFormat' OUTPUTFORMAT 'org.apache.hadoop.hive.ql.io.avro.AvroContainerOutputFormat' LOCATION 's3://Staging/test_table/enh' TBLPROPERTIES ('avro.schema.url'='s3://Staging/test_table/test_table.avsc')")

print "end"

spark版本：2.2.0 openjdk版本：1.8.0 hive版本：2.3.0