Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

Can only iterate once through csv reader

So I basically have an extremely long list of strings, and a CSV file that contains a column of strings and a column of numbers. I need to loop through the extremely long list of strings, and for each one, loop through the rows of the CSV file checking each string in the first column of the CSV to see if it occurs in my string, and if it does, add the number in the other column to something. A minimal sort of example would be

import csv
sList = ['a cat', 'great wall', 'mediocre wall']
vals = []
with open('file.csv', 'r') as f:
    r = csv.reader(f)
    for w in sList:
        val = 0
        for row in r:
            if row[0] in w:
                val += 1
        vals.append(val)

An example of a CSV file with which I might use this could be

a, 1
great, 2

Of course csv.reader(f) creates an iterable that I can loop through only once. I've seen recommendations elsewhere to use itertools but all of the recommendations I've found have been for problems that involve looping through the CSV file a small number of times, usually just twice. If I tried to use this to loop through the CSV many times I'm unsure of what that would mean for memory consumption, and in general I'm just wondering about the smartest way to approach this problem.

like image 345
Addem Avatar asked Dec 08 '14 03:12

Addem


People also ask

How many ways can you read a csv file?

There are two common ways to read a . csv file when using Python. The first by using the csv library, and the second by using the pandas library.

What is the difference between CSV reader and CSV DictReader?

csv. Reader() allows you to access CSV data using indexes and is ideal for simple CSV files. csv. DictReader() on the other hand is friendlier and easy to use, especially when working with large CSV files.

Is CSV reader an iterator?

The csv. reader object is an iterator. An iterator is an object with a next() method that will return the next value available or raise StopIteration if no value is available.

How do I iterate through a csv file in Python?

Step 1: Load the CSV file using the open method in a file object. Step 2: Create a reader object with the help of DictReader method using fileobject. This reader object is also known as an iterator can be used to fetch row-wise data. Step 3: Use for loop on reader object to get each row.


1 Answers

You need to "reset" the file iterator:

import csv
sList = ['a cat', 'great wall', 'mediocre wall']
vals = []
with open('data.csv', 'r') as f:
    r = csv.reader(f)
    for w in sList:
        val = 0
        f.seek(0)  #<-- set the iterator to beginning of the input file
        for row in r:
            print(row)
            if row[0] in w:
                val += 1
        vals.append(val)
like image 152
Marcin Avatar answered Oct 26 '22 13:10

Marcin