What is the most efficient way to subdivide a large list?

Question

I have a list that contains the RGBA color data for a 1024*1024 image, meaning 4,194,304 integers in that list. I need to split it down into 1024 sub-lists with each of those having 1024 sub-lists containing the 4 channels in order to be able to use it for what I need.

I have tried using for loops to append data to new lists, but that is a very slow process. I just need the list to be divided every 4 integers. What is the most efficient way of doing this? I have numpy if that can be used for this somehow.

I suppose I should mention that the list comes from unpacking a struct from a .raw image, so if there is a way to have the list split upon creation while unpacking that would also work.

Answer 1

It sounds like you could use numpy.reshape to get what you're after. Say you have a list of 12 elements:

>>> import numpy as np
>>> x = np.arange(12)
>>> x
array([ 0,  1,  2,  3,  4,  5,  6,  7,  8,  9, 10, 11])

We'll reshape it to give rows of four elements each:

>>> x.reshape(-1,4)
array([[ 0,  1,  2,  3],
       [ 4,  5,  6,  7],
       [ 8,  9, 10, 11]])

You can give reshape more than two dimensions, too, so say x was 5x5 RGBA image in a 100-element 1-d array, you could do y = x.reshape(5,5,4), so that y[0][0] gives the four channels of the (0,0) pixel, y[0][1] contains the four channels of the (0,1) pixel, and so on.