本文整理了Java中org.apache.hadoop.io.Text.getBytes()
方法的一些代码示例,展示了Text.getBytes()
的具体用法。这些代码示例主要来源于Github
/Stackoverflow
/Maven
等平台,是从一些精选项目中提取出来的代码,具有较强的参考意义,能在一定程度帮忙到你。Text.getBytes()
方法的具体详情如下:
包路径:org.apache.hadoop.io.Text
类名称:Text
方法名:getBytes
[英]Returns the raw bytes; however, only data up to #getLength() is valid. Please use #copyBytes() if you need the returned array to be precisely the length of the data.
[中]返回原始字节;但是,只有#getLength()以下的数据有效。如果需要返回的数组正好是数据的长度,请使用#copyBytes()。
代码示例来源:origin: apache/hive
public static Text transformTextToUTF8(Text text, Charset previousCharset) {
return new Text(new String(text.getBytes(), 0, text.getLength(), previousCharset));
}
代码示例来源:origin: apache/hive
public static Text transformTextFromUTF8(Text text, Charset targetCharset) {
return new Text(new String(text.getBytes(), 0, text.getLength()).getBytes(targetCharset));
}
代码示例来源:origin: apache/hive
/**
* Convert String to SHA-1
*/
public Text evaluate(Text n) {
if (n == null) {
return null;
}
digest.reset();
digest.update(n.getBytes(), 0, n.getLength());
byte[] shaBytes = digest.digest();
String shaHex = Hex.encodeHexString(shaBytes);
result.set(shaHex);
return result;
}
代码示例来源:origin: apache/hive
/**
* Convert String to md5
*/
public Text evaluate(Text n) {
if (n == null) {
return null;
}
digest.reset();
digest.update(n.getBytes(), 0, n.getLength());
byte[] md5Bytes = digest.digest();
String md5Hex = Hex.encodeHexString(md5Bytes);
result.set(md5Hex);
return result;
}
代码示例来源:origin: apache/hive
public static BytesWritable getBinaryFromText(Text text) {
BytesWritable bw = new BytesWritable();
bw.set(text.getBytes(), 0, text.getLength());
return bw;
}
代码示例来源:origin: apache/hive
@Override public byte[] getBytes(Text writable) {
//@TODO There is no reason to decode then encode the string to bytes really
//@FIXME this issue with CTRL-CHAR ^0 added by Text at the end of string and Json serd does not like that.
try {
return Text.decode(writable.getBytes(), 0, writable.getLength()).getBytes(Charset.forName("UTF-8"));
} catch (CharacterCodingException e) {
throw new RuntimeException(e);
}
}
代码示例来源:origin: apache/hive
public static int getTextUtfLength(Text t) {
byte[] data = t.getBytes();
int len = 0;
for (int i = 0; i < t.getLength(); i++) {
if (isUtfStartByte(data[i])) {
len++;
}
}
return len;
}
代码示例来源:origin: apache/hive
/**
* CRC32 for string
*/
public LongWritable evaluate(Text n) {
if (n == null) {
return null;
}
crc32.reset();
crc32.update(n.getBytes(), 0, n.getLength());
result.set(crc32.getValue());
return result;
}
代码示例来源:origin: apache/hive
public void internalWriteString(String str) throws TException {
if (str != null) {
tmpText.set(str);
trans_.write(tmpText.getBytes(), 0, tmpText.getLength());
} else {
trans_.write(nullText.getBytes(), 0, nullText.getLength());
}
}
代码示例来源:origin: apache/hive
public BytesWritable evaluate(Text value){
if (value == null) {
return null;
}
byte[] bytes = new byte[value.getLength()];
System.arraycopy(value.getBytes(), 0, bytes, 0, value.getLength());
byte[] decoded = Base64.decodeBase64(bytes);
result.set(decoded, 0, decoded.length);
return result;
}
}
代码示例来源:origin: apache/hive
/**
* Convert every character in s to two hex digits.
*
*/
public Text evaluate(Text s) {
if (s == null) {
return null;
}
byte[] str = s.getBytes();
return evaluate(str, s.getLength());
}
代码示例来源:origin: apache/hive
private static void setString(BytesColumnVector outputColVector, int i, Text t) {
if (t == null) {
outputColVector.noNulls = false;
outputColVector.isNull[i] = true;
return;
}
outputColVector.setVal(i, t.getBytes(), 0, t.getLength());
}
代码示例来源:origin: apache/kylin
@Override
public String[] call(Text text) throws Exception {
String s = Bytes.toString(text.getBytes(), 0, text.getLength());
return s.split(BatchConstants.SEQUENCE_FILE_DEFAULT_DELIMITER, -1);
}
});
代码示例来源:origin: apache/kylin
@Override
public Collection<String[]> parseMapperInput(Object mapperInput) {
Text text = (Text) mapperInput;
String[] columns = Bytes.toString(text.getBytes(), 0, text.getLength()).split(delimiter, -1);
return Collections.singletonList(columns);
}
代码示例来源:origin: apache/hive
public void writeText(Text text) throws TException {
writeTextBytes(text.getBytes(), 0, text.getLength());
}
代码示例来源:origin: apache/kylin
public KeyValue create(Text key, byte[] value, int voffset, int vlen) {
return create(key.getBytes(), 0, key.getLength(), value, voffset, vlen);
}
代码示例来源:origin: apache/hive
@Override
public void write(Writable r) throws IOException {
if (r instanceof Text) {
Text tr = (Text) r;
outStream.write(tr.getBytes(), 0, tr.getLength());
outStream.write(finalRowSeparator);
} else {
// DynamicSerDe always writes out BytesWritable
BytesWritable bw = (BytesWritable) r;
outStream.write(bw.get(), 0, bw.getSize());
outStream.write(finalRowSeparator);
}
}
代码示例来源:origin: apache/hive
@Override
public void write(Writable r) throws IOException {
if (r instanceof Text) {
Text tr = (Text) r;
outStream.write(tr.getBytes(), 0, tr.getLength());
} else {
// DynamicSerDe always writes out BytesWritable
BytesWritable bw = (BytesWritable) r;
outStream.write(bw.get(), 0, bw.getSize());
}
}
代码示例来源:origin: apache/hive
public void write(Writable row) throws IOException {
Text text = (Text) row;
Text escapeText = text;
if (HiveConf.getBoolVar(conf, HiveConf.ConfVars.HIVESCRIPTESCAPE)) {
escapeText = HiveUtils.escapeText(text);
}
out.write(escapeText.getBytes(), 0, escapeText.getLength());
out.write(Utilities.newLineCode);
}
代码示例来源:origin: apache/kylin
@Override
public void doMap(NullWritable key, Text value, Context context) throws IOException, InterruptedException {
tmpBuf.clear();
int size = value.getLength()+ 1;
if (size >= tmpBuf.capacity()) {
tmpBuf = ByteBuffer.allocate(countNewSize(tmpBuf.capacity(), size));
}
tmpBuf.put(Bytes.toBytes(index)[3]);
tmpBuf.put(value.getBytes(), 0, value.getLength());
outputKey.set(tmpBuf.array(), 0, tmpBuf.position());
sortableKey.init(outputKey, type);
context.write(sortableKey, NullWritable.get());
}
内容来源于网络,如有侵权,请联系作者删除!