Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

Skipping more than one row in Python csv

Tags:

python

csv

I am connecting to an API to snag some data. The output is a report that includes a multi-line header combined with a traditional single-line header.

Example:

1. Document Name: Test
2. Document Date: 8/7/2015
3. Document ID: 3804804
4. Document Author: Joe Blow
5.
6. Date, ID, Name, Age, Sex, Result
7. 8/7/2015, 2808380, Sara Jenkings, 33, F, 208.20

In the example, I want to skip over lines 1 - 5 and write the row on line 6 as the header row and all other rows after that as normal rows.

Now, I know how to skip over one line with next(reader, None), but how do I skip more than one row if I know the number of rows to skip will be consistently 5 rows like in the example?

I normally would use a database to skip the rows, but I want to see if I can have Python save the data correctly without the database doing more work.

like image 711
Fastidious Avatar asked Aug 08 '15 14:08

Fastidious


2 Answers

Skipping 5 header lines using list comprehension:

import csv

nheaderlines = 5

with open(path + file) as csvfile:
    reader = csv.DictReader(csvfile)

    [next(reader, None) for item in range(nheaderlines)]

    for row in reader:
        print(row)
like image 41
Siegfried Heide Avatar answered Oct 19 '22 04:10

Siegfried Heide


You can use itertools.islice, passing the line you want to begin writing from as the second parameter so for line 6 being 0 based you use 5, If stop is None, then iteration continues until the iterator is exhausted

import  csv

from itertools import islice

with open("in.csv") as f, open("out.csv","w") as out:
    r = csv.reader(islice(f, start=5,stop=None))
    wr = csv.writer(out)
    wr.writerows(r)

You don't necessarily need the csv module if you are keeping the lines as is:

with open("in.csv") as f, open("out.csv","w") as out:
    r = islice(f, 5 ,None)
    out.writelines(r)
like image 75
Padraic Cunningham Avatar answered Oct 19 '22 05:10

Padraic Cunningham