Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

Memory error while slicing an array

Tags:

python

numpy

I have object d connected to h5 dataset:

>>> data = d[:, :, 0].astype(np.float32)
>>> data.shape
(17201, 10801)
>>> data[data==-32768] = data[data>0].min()
Traceback (most recent call last):
  File "<interactive input>", line 1, in <module>
MemoryError

Can I do some other slicing trick to avoid this error?

like image 818
theta Avatar asked Jan 22 '26 11:01

theta


1 Answers

OK, I'm writing answer myself, as there is acceptable solution, gained after @mgilson questioned data type.

If data allows, memory error can be avoided by using simpler data type while operating on array. Considering initial question this worked for me:

>>> data = d[:, :, 0].astype(np.short)
>>> data[data==-32768] = data[data>0].min()
>>> data = data.astype(np.float32)
like image 106
theta Avatar answered Jan 24 '26 00:01

theta



Donate For Us

If you love us? You can donate to us via Paypal or buy me a coffee so we can maintain and grow! Thank you!