本文整理了Java中com.clearspring.analytics.stream.cardinality.HyperLogLog.getBytes()
方法的一些代码示例,展示了HyperLogLog.getBytes()
的具体用法。这些代码示例主要来源于Github
/Stackoverflow
/Maven
等平台,是从一些精选项目中提取出来的代码,具有较强的参考意义,能在一定程度帮忙到你。HyperLogLog.getBytes()
方法的具体详情如下:
包路径:com.clearspring.analytics.stream.cardinality.HyperLogLog
类名称:HyperLogLog
方法名:getBytes
暂无
代码示例来源:origin: apache/incubator-pinot
@Override
public byte[] serialize(HyperLogLog hyperLogLog) {
try {
return hyperLogLog.getBytes();
} catch (IOException e) {
throw new RuntimeException("Caught exception while serializing HyperLogLog", e);
}
}
代码示例来源:origin: apache/incubator-pinot
public static byte[] toBytes(HyperLogLog hll) {
try {
return hll.getBytes();
} catch (IOException e) {
throw new RuntimeException(e);
}
}
代码示例来源:origin: apache/incubator-pinot
@Test
public void testHllFieldSerializedSize()
throws Exception {
for (int i = 5; i < 10; i++) {
HyperLogLog hll = new HyperLogLog(i);
Assert.assertEquals(HllSizeUtils.getHllFieldSizeFromLog2m(i), hll.getBytes().length);
LOGGER.info("Estimated: " + hll.cardinality());
for (int j = 0; j < 100; j++) {
hll.offer(rand.nextLong());
}
Assert.assertEquals(HllSizeUtils.getHllFieldSizeFromLog2m(i), hll.getBytes().length);
LOGGER.info("Estimated: " + hll.cardinality());
for (int j = 0; j < 9900; j++) {
hll.offer(rand.nextLong());
}
Assert.assertEquals(HllSizeUtils.getHllFieldSizeFromLog2m(i), hll.getBytes().length);
LOGGER.info("Estimated: " + hll.cardinality());
}
}
}
代码示例来源:origin: addthis/stream-lib
@Test
public void testSerializationUsingBuilder() throws IOException {
HyperLogLog hll = new HyperLogLog(8);
hll.offer("a");
hll.offer("b");
hll.offer("c");
hll.offer("d");
hll.offer("e");
HyperLogLog hll2 = HyperLogLog.Builder.build(hll.getBytes());
assertEquals(hll.cardinality(), hll2.cardinality());
}
代码示例来源:origin: com.github.ddth/ddth-simplehll-core
/**
* {@inheritDoc}
*/
@Override
public byte[] toBytes() {
if (hll == null) {
throw new IllegalStateException();
}
try {
return hll.getBytes();
} catch (IOException e) {
throw new RuntimeException(e);
}
}
代码示例来源:origin: Big-Data-Manning/big-data-code
public void operate(FlowProcess process, BufferCall call) {
HyperLogLog hll = new HyperLogLog(14);
Iterator<TupleEntry> it = call.getArgumentsIterator();
while(it.hasNext()) {
TupleEntry tuple = it.next();
hll.offer(tuple.getObject(0));
}
try {
call.getOutputCollector().add(
new Tuple(hll.getBytes()));
} catch (IOException e) {
throw new RuntimeException(e);
}
}
}
代码示例来源:origin: Big-Data-Manning/big-data-code
public void operate(FlowProcess process, BufferCall call) {
Iterator<TupleEntry> it = call.getArgumentsIterator();
HyperLogLog curr = null;
try {
while(it.hasNext()) {
TupleEntry tuple = it.next();
byte[] serialized = (byte[]) tuple.getObject(0);
HyperLogLog hll = HyperLogLog.Builder.build(
serialized);
if(curr==null) {
curr = hll;
} else {
curr = (HyperLogLog) curr.merge(hll);
}
}
call.getOutputCollector().add(
new Tuple(curr.getBytes()));
} catch (IOException e) {
throw new RuntimeException(e);
} catch(CardinalityMergeException e) {
throw new RuntimeException(e);
}
}
}
代码示例来源:origin: LiveRamp/cascading_ext
@Override
public void cleanup(FlowProcess flowProcess, OperationCall operationCall) {
JobConf conf = (JobConf) flowProcess.getConfigCopy();
try {
LOG.info("HLL counter found " + approxCounter.cardinality() + " distinct keys");
Hfs tap = new Hfs(new SequenceFile(new Fields("bytes")), BloomProps.getApproxCountsDir(conf));
TupleEntryCollector out = tap.openForWrite(new HadoopFlowProcess(conf));
out.add(new Tuple(new BytesWritable(approxCounter.getBytes())));
out.close();
} catch (IOException e) {
throw new RuntimeException("couldn't write approximate counts to side bucket", e);
}
}
代码示例来源:origin: Big-Data-Manning/big-data-code
@Override
public void execute(Tuple tuple, BasicOutputCollector collector) {
String url = tuple.getString(1);
int bucket = tuple.getInteger(2);
PersonID user = (PersonID) tuple.getValue(3);
HColumn<Integer, byte[]> hcol =
_template.querySingleColumn(
url,
bucket,
BytesArraySerializer.get());
HyperLogLog hll;
try {
if(hcol==null) hll = new HyperLogLog(14);
else hll = HyperLogLog.Builder.build(hcol.getValue());
hll.offer(user);
ColumnFamilyUpdater<String, Integer> updater =
_template.createUpdater(url);
updater.setByteArray(bucket, hll.getBytes());
_template.update(updater);
} catch(IOException e) {
throw new RuntimeException(e);
}
}
内容来源于网络,如有侵权,请联系作者删除!