我有一个具有不同列和id的已排序数据集。该数据集已排序（也使用Parquet工具验证）：示例：

file 1: ID 1-10
file 2: ID 10-12
file 3: ID 12-33
....

我还生成并编写了\u元数据和\u公共\u元数据文件。我尝试使用过滤器查询（非常大的）数据集

val mydata=spark.read.parquet("s3a://.../mylocation")
val result = mydata.filter(mydata("id") === 11)
result.explain(true)

解释告诉我：

== Parsed Logical Plan ==
Filter (id#14L = 11)
+- Relation[fieldA#12, fieldB#13,id#14L] parquet

== Analyzed Logical Plan ==
fieldA: int, fieldB: string, id: bigint
Filter (id#14L = 11)
+- Relation[fieldA#12, fieldB#13,id#14L] parquet

== Optimized Logical Plan ==
Filter (isnotnull(id#14L) && (id#14L = 11))
+- Relation[fieldA#12, fieldB#13,id#14L] parquet

== Physical Plan ==

* (1) Project [fieldA#12, fieldB#13,id#14L]

+- *(1) Filter (isnotnull(id#14L) && (id#14L = 11))
   +- *(1) FileScan parquet [fieldA#12,fieldB#13,id#14L] Batched: true, Format: Parquet, Location: InMemoryFileIndex[s3a://mybucket/path/to/data], PartitionFilters: [], PushedFilters: [IsNotNull(id), EqualTo(id,11)], ReadSchema: struct<fieldA:int,fieldB:string,id:bigint>

我还启用了日志记录，可以看到读取多个文件以获取每个文件的元数据。我在s3的这个“目录”中有10000个文件，所以从这些文件中检索所有元数据需要很多时间
为什么spark不从\元数据文件获取元数据？是否有启用此功能的选项？我已经尝试了以下选项：

spark.conf.set("parquet.summary.metadata.level","ALL")
spark.conf.set("parquet.filter.statistics.enabled","true")
spark.conf.set("parquet.filter.dictionary.enabled","true")
spark.conf.set("spark.sql.parquet.filterPushdown","true")
spark.conf.set("spark.sql.hive.convertMetastoreParquet","true")
spark.conf.set("spark.sql.parquet.respectSummaryFiles","true")
spark.conf.set("spark.sql.parquet.mergeSchema","false")
spark.conf.set("spark.sql.hive.convertMetastoreParquet.mergeSchema","false")
spark.conf.set("spark.sql.optimizer.metadataOnly", "true")