我的hadoop程序读取数千行(.log)文件并对其进行解析。之后,我使用output.collect()函数将它们写入一个文件。但是在每个reducer输出之后,它都将输出写入newline。如何将所有内容写入同一行,如何确保我的输出文件仅为一行?
减速器等级
public class Reduce extends MapReduceBase
implements Reducer<Text, Text, Text, Text> {
public void reduce(Text key, Iterator<Text> values,OutputCollector<Text, Text> output,
Reporter reporter) throws IOException {
Text t2 = new Text("");
output.collect(key, t2);
}
}
Map类
public void map(LongWritable key, Text value,
OutputCollector<Text, Text> output,
Reporter reporter) throws IOException {
Path("xxxx");//Location of file in HDFS
InputStreamReader(fs.open(pt)));
String line = value.toString();
String bePublished="";
String patternString = "xxxx;
Pattern pattern = Pattern.compile(patternString);
Matcher matcher = pattern.matcher(line);
for(int u=0;u<48;u++){
//Here i update my bepublished string
}
Text t2 = new Text("");
output.collect(new Text(bePublished), t2);
}
暂无答案!
目前还没有任何答案,快来回答吧!