从java读写parquet和hdfs文件

wmomyfyw  于 2021-05-26  发布在  Spark
关注(0)|答案(0)|浏览(585)

目前我可以用java读写hdfs文件,但是除了hdfs之外我不知道如何读apache parquet文件,我的想法是能够用java读写这两个文件

package com.leerhdfs;

//import org.apache.commons.io.IOUtils;
import org.apache.hadoop.conf.Configuration;
import org.apache.hadoop.fs.FSDataInputStream;
import org.apache.hadoop.fs.FSDataOutputStream;
import org.apache.hadoop.fs.FileSystem;
import org.apache.hadoop.fs.Path;
import org.apache.hadoop.io.IOUtils;
import org.apache.hadoop.util.Progressable;

import java.io.*;
import java.net.URI;
import java.nio.charset.StandardCharsets;

public class ReadWriteHDFSExample {

public static void main(String[] args) throws IOException {

    String localsrc = args[0];
    String destinosrc = args[1];

    InputStream in = new BufferedInputStream(new FileInputStream(localsrc));
    Configuration conf = new Configuration();

    FileSystem fs = FileSystem.get(URI.create(destinosrc), conf);

    OutputStream out = fs.create(new Path(destinosrc), new Progressable() {

        public void progress() {
            System.out.println(".");

        }
    });

    IOUtils.copyBytes(in, out, 4096, true);
 }

}
请帮帮我!!!谢谢!!

暂无答案!

目前还没有任何答案,快来回答吧!

相关问题