Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

how to use header(first row) as field names in Pig

Given a csv file with first row which can be taken as the header, how can one load the field names dynamically in Pig using these headers? i.e.

id,year,total
1,1999,190
2,1998,20

a = LOAD '/path/to/file.csv' USING PigStorage() AS --use first row as field names
> describe a;
> id:bytearray,year:bytearray,total:bytearray 
like image 502
DevEx Avatar asked Nov 01 '22 05:11

DevEx


1 Answers

As this is a CSV file and you want to use first row as a header, you should use CSVLoader() for it.It will treat first row as header. Your script will be like this.

--Register the piggybank jar
REGISTER piggybank.jar
define CSVLoader org.apache.pig.piggybank.storage.CSVLoader();  

A = LOAD '/path/to/file.csv' using CSVLoader AS(id:int,year:chararray,total:int);
like image 131
Sandeep Singh Avatar answered Nov 15 '22 08:11

Sandeep Singh