How should I use Numpy's vstack method?

Question

Firstly, here is the relevant part of the code:

stokes_list = np.zeros(shape=(numrows,1024)) # 'numrows' defined earlier
for i in range(numrows):
    epoch_name = y['filename'][i] # 'y' is an array from earlier
    os.system('pdv -t {0} > temp.txt '.format(epoch_name)) # 'pdv' is a command from another piece of software - here I copy the output into a temporary file
    stokes_line = np.genfromtxt('temp.txt', usecols=3, dtype=[('stokesI','float')], skip_header=1)
    stokes_list = np.vstack((stokes_line,stokes_line))

So, basically, every time the code loops around, stokes_line pulls one of the columns (4th one) from the file temp.txt, and I want it to add a line to stokes_list each time.

For example, if the first stokes_line is

1.1 2.2 3.3  

and the second is

4.4 5.5 6.6  

then stokes_list will be

1.1 2.2 3.3  
4.4 5.5 6.6  

and will keep growing...

It's not working at the moment, because I think that the line:

stokes_list = np.vstack((stokes_line,stokes_line))

is not correct. It's only stacking 2 lists - which makes sense as I only have 2 arguments. I basically would like to know how I keep stacking again and again.

Any help would be very gratefully received!
If it is needed, here is an example of the format of the temp.txt file:

File: t091110_065921.SFTC Src: J1903+0925 Nsub: 1 Nch: 1 Npol: 4 Nbin: 1024 RMS: 0.00118753  
0 0 0 0.00148099 -0.00143755 0.000931365 -0.00296775  
0 0 1 0.000647476 -0.000896698 0.000171287 0.00218597  
0 0 2 0.000704697 -0.00052846 -0.000603842 -0.000868739  
0 0 3 0.000773361 -0.00234724 -0.0004112 0.00358033  
0 0 4 0.00101559 -0.000691062 0.000196023 -0.000163109  
0 0 5 -0.000220367 -0.000944024 0.000181002 -0.00268215  
0 0 6 0.000311783 0.00191545 -0.00143816 -0.00213856  

Answer 1

vstacking again and again is not good, because it copies the whole arrays.

Create a normal Python list, .append to it and then pass it whole to np.vstack to create a new array once.

stokes_list = []
for i in xrange(numrows):
    ...
    stokes_line = ...
    stokes_list.append(stokes_line)

big_stokes = np.vstack(stokes_list)

Answer 2

You already know the final size of the stokes_list array since you know numrows. So it seems you don't need to grow an array (which is very inefficient). You can simply assign the correct row at each iteration. Simply replace your last line by :

stokes_list[i] = stokes_line

By the way, about your non-working line I think you meant :

stokes_list = np.vstack((stokes_list, stokes_line))

where you're replacing stokes_list by its new value.