本文整理了Java中org.apache.parquet.schema.Types.buildMessage()
方法的一些代码示例,展示了Types.buildMessage()
的具体用法。这些代码示例主要来源于Github
/Stackoverflow
/Maven
等平台,是从一些精选项目中提取出来的代码,具有较强的参考意义,能在一定程度帮忙到你。Types.buildMessage()
方法的具体详情如下:
包路径:org.apache.parquet.schema.Types
类名称:Types
方法名:buildMessage
[英]Returns a builder to construct a MessageType.
[中]返回构造MessageType的生成器。
代码示例来源:origin: apache/hive
/**
* Searches column names by name on a given Parquet message schema, and returns its projected
* Parquet schema types.
*
* @param schema Message type schema where to search for column names.
* @param colNames List of column names.
* @param colTypes List of column types.
* @return A MessageType object of projected columns.
*/
public static MessageType getSchemaByName(MessageType schema, List<String> colNames, List<TypeInfo> colTypes) {
List<Type> projectedFields = getProjectedGroupFields(schema, colNames, colTypes);
Type[] typesArray = projectedFields.toArray(new Type[0]);
return Types.buildMessage()
.addFields(typesArray)
.named(schema.getName());
}
代码示例来源:origin: apache/hive
@Test
public void testUnannotatedListOfPrimitives() throws Exception {
MessageType fileSchema = Types.buildMessage()
.repeated(INT32).named("list_of_ints")
.named("UnannotatedListOfPrimitives");
Path test = writeDirect("UnannotatedListOfPrimitives",
fileSchema,
new DirectWriter() {
@Override
public void write(RecordConsumer rc) {
rc.startMessage();
rc.startField("list_of_ints", 0);
rc.addInteger(34);
rc.addInteger(35);
rc.addInteger(36);
rc.endField("list_of_ints", 0);
rc.endMessage();
}
});
ArrayWritable expected = list(
new IntWritable(34), new IntWritable(35), new IntWritable(36));
List<ArrayWritable> records = read(test);
Assert.assertEquals("Should have only one record", 1, records.size());
assertEquals("Should match expected record",
expected, records.get(0));
}
代码示例来源:origin: apache/hive
@Test
public void testAvroPrimitiveInList() throws Exception {
Path test = writeDirect("AvroPrimitiveInList",
Types.buildMessage()
.requiredGroup().as(LIST)
.repeated(INT32).named("array")
.named("list_of_ints")
.named("AvroPrimitiveInList"),
new DirectWriter() {
@Override
public void write(RecordConsumer rc) {
rc.startMessage();
rc.startField("list_of_ints", 0);
rc.startGroup();
rc.startField("array", 0);
rc.addInteger(34);
rc.addInteger(35);
rc.addInteger(36);
rc.endField("array", 0);
rc.endGroup();
rc.endField("list_of_ints", 0);
rc.endMessage();
}
});
ArrayWritable expected = list(
new IntWritable(34), new IntWritable(35), new IntWritable(36));
List<ArrayWritable> records = read(test);
Assert.assertEquals("Should have only one record", 1, records.size());
assertEquals("Should match expected record", expected, records.get(0));
}
代码示例来源:origin: apache/hive
@Test
public void testThriftPrimitiveInList() throws Exception {
Path test = writeDirect("ThriftPrimitiveInList",
Types.buildMessage()
.requiredGroup().as(LIST)
.repeated(INT32).named("list_of_ints_tuple")
.named("list_of_ints")
.named("ThriftPrimitiveInList"),
new DirectWriter() {
@Override
public void write(RecordConsumer rc) {
rc.startMessage();
rc.startField("list_of_ints", 0);
rc.startGroup();
rc.startField("list_of_ints_tuple", 0);
rc.addInteger(34);
rc.addInteger(35);
rc.addInteger(36);
rc.endField("list_of_ints_tuple", 0);
rc.endGroup();
rc.endField("list_of_ints", 0);
rc.endMessage();
}
});
ArrayWritable expected = list(
new IntWritable(34), new IntWritable(35), new IntWritable(36));
List<ArrayWritable> records = read(test);
Assert.assertEquals("Should have only one record", 1, records.size());
assertEquals("Should match expected record", expected, records.get(0));
}
代码示例来源:origin: apache/hive
@Test
public void testStringMapRequiredPrimitive() throws Exception {
Path test = writeDirect("StringMapRequiredPrimitive",
Types.buildMessage()
.optionalGroup().as(MAP)
.repeatedGroup()
代码示例来源:origin: apache/hive
@Test
public void testStringMapOptionalPrimitive() throws Exception {
Path test = writeDirect("StringMapOptionalPrimitive",
Types.buildMessage()
.optionalGroup().as(MAP)
.repeatedGroup()
代码示例来源:origin: apache/hive
Types.buildMessage()
.optionalGroup().as(LIST)
.repeatedGroup()
代码示例来源:origin: apache/hive
Types.buildMessage()
.optionalGroup().as(MAP)
.repeatedGroup()
代码示例来源:origin: apache/hive
@Test
public void testDoubleMapWithStructValue() throws Exception {
Path test = writeDirect("DoubleMapWithStructValue",
Types.buildMessage()
.optionalGroup().as(MAP)
.repeatedGroup()
代码示例来源:origin: apache/hive
Types.buildMessage()
.optionalGroup().as(LIST)
.repeatedGroup()
代码示例来源:origin: apache/hive
Types.buildMessage()
.optionalGroup().as(LIST)
.repeatedGroup()
代码示例来源:origin: apache/hive
Types.buildMessage()
.optionalGroup().as(MAP)
.repeatedGroup()
代码示例来源:origin: apache/hive
@Test
public void testMapWithComplexKey() throws Exception {
Path test = writeDirect("MapWithComplexKey",
Types.buildMessage()
.optionalGroup().as(MAP)
.repeatedGroup()
代码示例来源:origin: apache/hive
@Test
public void testNestedMap() throws Exception {
Path test = writeDirect("DoubleMapWithStructValue",
Types.buildMessage()
.optionalGroup().as(MAP)
.repeatedGroup()
代码示例来源:origin: apache/hive
@Test
public void testUnannotatedListOfGroups() throws Exception {
Path test = writeDirect("UnannotatedListOfGroups",
Types.buildMessage()
.repeatedGroup()
.required(FLOAT).named("x")
代码示例来源:origin: apache/hive
Types.buildMessage()
.optionalGroup().as(LIST)
.repeatedGroup()
代码示例来源:origin: apache/hive
Types.buildMessage()
.optionalGroup().as(LIST)
.repeatedGroup()
代码示例来源:origin: apache/hive
@Test
public void testNewRequiredGroupInList() throws Exception {
Path test = writeDirect("NewRequiredGroupInList",
Types.buildMessage()
.optionalGroup().as(LIST)
.repeatedGroup()
代码示例来源:origin: apache/hive
@Test
public void testNewOptionalGroupInList() throws Exception {
Path test = writeDirect("NewOptionalGroupInList",
Types.buildMessage()
.optionalGroup().as(LIST)
.repeatedGroup()
代码示例来源:origin: org.apache.spark/spark-sql_2.11
} else {
if (columns.size() > 0) {
Types.MessageTypeBuilder builder = Types.buildMessage();
for (String s: columns) {
if (!fileSchema.containsField(s)) {
内容来源于网络,如有侵权,请联系作者删除!