View gzipped file content in hadoop

Question

How can I decompress and view few lines of a compressed file in hdfs. The below command displays the last few lines of the compressed data

hadoop fs -tail /myfolder/part-r-00024.gz

Is there a way I can use the -text command and pipe the output to tail command? I tried this but this doesn't work.

hadoop fs -text /myfolder/part-r-00024.gz > hadoop fs -tail /myfolder/

Answer 1

The following will show you the specified number of lines without decompressing the whole file:

hadoop fs -cat /hdfs_location/part-00000.gz | zcat | head -n 20

The following will page the file, also without first decompressing the whole of it:

hadoop fs -cat /hdfs_location/part-00000.gz | zmore

Answer 2

Try the following, should work as long as your file isn't too big (since the whole thing will be decompressed):

hadoop fs -text /myfolder/part-r-00024.gz | tail

Answer 3

I ended up writing a pig script.

A = LOAD '/myfolder/part-r-00024.gz' USING PigStorage('\t');
B = LIMIT A 10;
DUMP B;