How to read stdin to a 2d python array of integers?

Question

I would like to read a 2d array of integers from stdin (or from a file) in Python.

Non-working code:

from StringIO import StringIO
from array import array

# fake stdin
stdin = StringIO("""1 2
3 4
5 6""")

a = array('i')
a.fromstring(stdin.read())

This gives me an error: a.fromstring(stdin.read()) ValueError: string length not a multiple of item size

Answer 1

Several approaches to accomplish this are available. Below are a few of the possibilities.

Using an array

From a list

Replace the last line of code in the question with the following.

a.fromlist([int(val) for val in stdin.read().split()])

Now:

>>> a
array('i', [1, 2, 3, 4, 5, 6])

Con: does not preserve 2d structure (see comments).

From a generator

Note: this option is incorporated from comments by eryksun.

A more efficient way to do this is to use a generator instead of the list. Replace the last two lines of the code in the question with:

a = array('i', (int(val) for row in stdin for val in row.split()))

This produces the same result as the option above, but avoids creating the intermediate list.

Using a NumPy array

If you want the preserve the 2d structure, you could use a NumPy array. Here's the whole example:

from StringIO import StringIO
import numpy as np

# fake stdin
stdin = StringIO("""1 2
3 4
5 6""")

a = np.loadtxt(stdin, dtype=np.int)

Now:

>>> a
array([[1, 2],
       [3, 4],
       [5, 6]])

Using standard lists

It is not clear from the question if a Python list is acceptable. If it is, one way to accomplish the goal is replace the last two lines of the code in the question with the following.

a = [map(int, row.split()) for row in stdin]

After running this, we have:

>>> a
[[1, 2], [3, 4], [5, 6]]

Answer 2

I've never used array.array, so I had to do some digging around.

The answer is in the error message -

ValueError: string length not a multiple of item size

How do you determine the item size? Well it depends on the type you initialized it with. In your case you initialized it with i which is a signed int. Now, how big is an int? Ask your python interpreter..

>>> a.itemsize
4

The value above provides insight into the problem. Your string is only 11 bytes wide. 11 isn't a multiple of 4. But increasing the length of the string will not give you an array of {1,2,3,4,5,6}... I'm not sure what it would give you. Why the uncertainty? Well, read the docstring below... (It's late, so I highlighted the important part, in case you're getting sleepy, like me!)

array.fromfile(f, n) Read n items (as machine values) from the file object f and append them to the end of the array. If less than n items are available, EOFError is raised, but the items that were available are still inserted into the array. f must be a real built-in file object; something else with a read() method won’t do.

array.fromstring reads data in the same manner as array.fromfile. Notice the bold above. "as machine values" means "reads as binary". So, to do what you want to do, you need to use the struct module. Check out the code below.

import struct
a = array.array('i')
binary_string = struct.pack('iiii', 1, 2, 3, 4)
a.fromstring(binary_string)

The code snippet above loads the array with tlhe values 1, 2, 3, 4; like we expect.

Hope it helps.