Menu
I just got my hands on pandas and am figuring out how I can read a file. The file is from the WRDS database and is the SP500 constituents list all the way back to the 1960s. I checked the file and no matter what I do to import it using read_csv, I still can't display the data correctly.
df = read_csv('sp500-sb.txt')
df
<class 'pandas.core.frame.DataFrame'>
Int64Index: 1231 entries, 0 to 1230
Data columns: gvkeyx from thru conm
gvkey co_conm
...(the column names)
dtypes: object(1)
What does the above chunk of output mean? Anything would be helpful.
763
To efficiently parse fixed width files with Python, we can use the Pandas' read_fwf method. to define the col_specification list with the column specifications for filename. txt. Then we call read.
Method 1: Using read.fwf function from utils package. We have to use column widths for reading. Syntax: read. fwf(file, widths, header = FALSE, sep = “\t”, skip = 0, row.
We will read the text file with pandas using the read_csv() function. Along with the text file, we also pass separator as a single space (' ') for the space character because, for text files, the space character will separate each field. There are three parameters we can pass to the read_csv() function.
A fixed width file is similar to a csv file, but rather than using a delimiter, each field has a set number of characters. This creates files with all the data tidily lined up with an appearance similar to a spreadsheet when opened in a text editor.
pandas.read_fwf() was added in pandas 0.7.3 (April 2012) to handle fixed-width files.
API reference
An example from other question
92
Wes answered me in an email. Cheers.
This is a fixed-width-format file (not delimited by commas or tabs as usual). I realize that pandas does not have a fixed-width reader like R does, though one can be fashioned very easily. I'll see what I can do. In the meantime if you can export the data in another format (like csv--truly comma separated) you'll be able to read it with read_csv. I suspect with some unix magic you can transform a FWF file into a CSV file.
I recommend following the issue on github as your e-mail is about to disappear from my inbox :)
https://github.com/pydata/pandas/issues/920
best, Wes
41
If you love us? You can donate to us via Paypal or buy me a coffee so we can maintain and grow! Thank you!
Donate Us With
