读取snappy压缩文件时出错

brccelvz 于 2021-06-03 发布在 Hadoop

关注(0)|答案(1)|浏览(641)

我正在读一本书 Snappy Compressed file 从本地到 java .

File snappyFile = new File(fileName);  
Configuration conf = new Configuration();               
CompressionCodec codec = (CompressionCodec)
ReflectionUtils.newInstance(SnappyCodec.class, conf);
FileInputStream is2 = new FileInputStream(snappyFile);
CompressionInputStream cis = codec.createInputStream(is2);
BufferedReader cr = new BufferedReader(new InputStreamReader(cis));

我的代码在reflectionutils.newinstance处出现异常

Exception in thread "main" java.lang.RuntimeException: java.lang.NoSuchMethodException: org.xerial.snappy.SnappyCodec.<init>()
    at org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:128)
    at org.finra.oats.AWS.AWSnappyRead.run(AWSnappyRead.java:64)
    at org.finra.oats.AWS.AWSnappyRead.main(AWSnappyRead.java:48)
Caused by: java.lang.NoSuchMethodException: org.xerial.snappy.SnappyCodec.<init>()
    at java.lang.Class.getConstructor0(Class.java:2849)
    at java.lang.Class.getDeclaredConstructor(Class.java:2053)
    at org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:122)
    ... 2 more

是因为版本不匹配的问题。我正在使用 snappy 1.1.1.2 版本

Java hadoop snappy compression

来源：https://stackoverflow.com/questions/29821662/error-while-reading-snappy-compressed-file

1条答案

按热度按时间

yqkkidmi1#

要读写snappy压缩文件，可以使用提供的snappyinputstream/snappyoutputstream类。

String fileName = "foo.snap";

// write a snappy compressed file
try (OutputStream os = new SnappyOutputStream(new FileOutputStream(fileName))) {
    os.write("Hello Snappy-World".getBytes(Charset.defaultCharset()));
}

// read a snappy compressed file
try (InputStream is = new SnappyInputStream(new FileInputStream(fileName))) {
    byte[] bytes = new byte[100];
    is.read(bytes);
    System.out.println(new String(bytes, Charset.defaultCharset()));
}

// check if the file is compressed with the snappy algorithm
try (InputStream is = new FileInputStream(fileName)) {
    SnappyCodec readHeader = SnappyCodec.readHeader(is);
    if (readHeader.isValidMagicHeader()) {
        System.out.println("is a Snappy compressed file");
        System.out.printf("%s: %d%n%s: %d%n", 
                "compatible version", readHeader.compatibleVersion,
                "version", readHeader.version
        );
    } else {
        System.out.println("is not a Snappy compressed file");                
    }
}

赞(0）回复(0）举报 2021-06-03

我来回答

读取snappy压缩文件时出错

1条答案

相关问题

热门标签

最新问答