Cannot convert array to floats python

Question

I'm having a problem that seems like the answer would be easily explained. I'm struggling to convert my array elements to floats (so that I can multiply, add on them etc)

import csv
import os
import glob
import numpy as np

def get_data(filename):
    with open(filename, 'r') as f:
        reader = csv.reader(f)                      
        return list(reader)

all_data = []

path=raw_input('What is the directory?')       
for infile in glob.glob(os.path.join(path, '*.csv')):
     all_data.extend(get_data(infile))
a = np.array(all_data)
current_track_data=a[0:,[8,9,10,11,12]]
abs_track_data=a[0:,7]

and I get the error:

> --------------------------------------------------------------------------- ValueError                                Traceback (most recent call last) C:\Users\AClayton\AppData\Local\Enthought\Canopy\App\appdata\canopy-1.0.3.1262.win-x86_64\lib\site-packages\IPython\utils\py3compat.pyc in execfile(fname, glob, loc)
    174             else:
    175                 filename = fname
--> 176             exec compile(scripttext, filename, 'exec') in glob, loc
    177     else:
    178         def execfile(fname, *where):
> 
> C:\Users\AClayton\Current\python begin\code_tester2.py in <module>()
>      18 for infile in glob.glob(os.path.join(path, '*.csv')): # performs loop for each file in the specified path with extension .csv
>      19      all_data.extend(get_data(infile))
> ---> 20      a = np.ndarray(all_data, dtype=float)
>      21 
>      22      current_track_data=a[0:,[8,9,10,11,12]]
> 
> ValueError: sequence too large; must be smaller than 32

Answer 1

Your script is not the same as the code you've posted... As the traceback of your error shows, in line 20 you are calling np.ndarray. That's the numpy array object, not the np.array factory function. Unless you know very well what you are doing, follow the docs advice and:

Arrays should be constructed using array, zeros or empty (refer to the See Also section below). The parameters given here refer to a low-level method (ndarray(...)) for instantiating an array.

So change your line #20 to:

 a = np.array(all_data, dtype=float)

and you should be fine.

The error you are getting comes because ndarray takes your first input as the shape of the array to be created. There is a harcoded limit to the number of dimensions set to 32 on my Windows system (may be platform dependent, not sure). Your all_data list has more than 32 entries (or whatever that value is in your system), misinterpreted as sizes of dimensions, and that's what triggers the error.