2011年5月8日 星期日

Coverting Text file to Binary Sequence File

1.In the Main Class
job.setOutputKeyClass(BytesWritable.class);
job.setOutputValueClass(BytesWritable.class);

job.setInputFormatClass(TextInputFormat.class);
job.setOutputFormatClass(SequenceFileAsBinaryOutputFormat.class);

job.setNumReduceTasks(0); //don't use reduce class

2.In the Map class
public class Map extends
Mapper< LongWritable, Text, BytesWritable, BytesWritable>
{
private BytesWritable one = new BytesWritable("1".getBytes());
private BytesWritable val = new BytesWritable();
public void map(LongWritable key, Text value, Context context)
throws IOException, InterruptedException
{
String s = value.toString();
val.set(s.getBytes(),0,s.length());
context.write(one,val);

}
}

沒有留言:

張貼留言