Reading file string into an array (In a pythonic way)

Question

I'm reading lines from a file to then work with them. Each line is composed solely by float numbers.

I have pretty much everything sorted up to convert the lines into arrays.

I basically do (pseudopython code)

 line=file.readlines()
 line=line.split(' ') # Or whatever separator
 array=np.array(line)
 #And then iterate over every value casting them as floats
      newarray[i]=array.float(array[i])

This works, buts seems a bit counterintuitive and antipythonic, I wanted to know if there is a better way to handle the inputs from a file to have at the end an array full of floats.

Answer 1

Quick answer:

arrays = []
for line in open(your_file): # no need to use readlines if you don't want to store them
    # use a list comprehension to build your array on the fly
    new_array = np.array((array.float(i) for i in line.split(' '))) 
    arrays.append(new_array)

If you process often this kind of data, the csv module will help.

import csv

arrays = []
# declare the format of you csv file and Python will turn line into
# lists for you 
parser = csv.reader(open(your_file), delimiter=' '))
for l in parser: 
    arrays.append(np.array((array.float(i) for i in l)))

If you feel wild, you can even make this completly declarative:

import csv

parser = csv.reader(open(your_file), delimiter=' '))
make_array = lambda row : np.array((array.float(i) for i in row)) 
arrays = [make_array(row) for row in parser]

And if you realy want you colleagues to hate you, you can make a one liner (NOT PYTHONIC AT ALL :-):

arrays = [np.array((array.float(i) for i in r)) for r in csv.reader(open(your_file), delimiter=' '))]

Stripping all the boiler plate and flexibility, you can end up with a clean and quite readable one liner. I wouldn't use it because I like the refatoring potential of using csv, but it can be good enought. It's a grey zone here, so I wouldn't say it's Pythonic, but it's definitly handy.

arrays = [np.array((array.float(i) for i in l.split())) for l in open(your_file))]