Efficient way to convert delimiter separated string to numpy array

Question

I have a String as follows :

1|234|4456|789

I have to convert it into numpy array.I would like to know the most efficient way.Since I will be calling this function for more than 50 million times!

Answer 1

@jterrace wins one (1) internet.

In the measurements below the example code has been shortened to allow the tests to fit on one line without scrolling where possible.

For those not familiar with timeit the -s flag allows you to specify a bit of code which will only be executed once.

---

The fastest and least-cluttered way is to use numpy.fromstring as jterrace suggested:

python -mtimeit -s"import numpy;s='1|2'" "numpy.fromstring(s,dtype=int,sep='|')"
100000 loops, best of 3: 1.85 usec per loop

---

The following three examples use string.split in combination with another tool.

string.split with numpy.fromiter

python -mtimeit -s"import numpy;s='1|2'" "numpy.fromiter(s.split('|'),dtype=int)"
100000 loops, best of 3: 2.24 usec per loop

string.split with int() cast via generator-expression

python -mtimeit -s"import numpy;s='1|2'" "numpy.array(int(x) for x in s.split('|'))"
100000 loops, best of 3: 3.12 usec per loop

string.split with NumPy array of type int

python -mtimeit -s"import numpy;s='1|2'" "numpy.array(s.split('|'),dtype=int)"
100000 loops, best of 3: 9.22 usec per loop

Answer 2

The fastest way is to use the numpy.fromstring method:

>>> import numpy
>>> data = "1|234|4456|789"
>>> numpy.fromstring(data, dtype=int, sep="|")
array([   1,  234, 4456,  789])