I have object d connected to h5 dataset:
>>> data = d[:, :, 0].astype(np.float32)
>>> data.shape
(17201, 10801)
>>> data[data==-32768] = data[data>0].min()
Traceback (most recent call last):
File "<interactive input>", line 1, in <module>
MemoryError
Can I do some other slicing trick to avoid this error?
OK, I'm writing answer myself, as there is acceptable solution, gained after @mgilson questioned data type.
If data allows, memory error can be avoided by using simpler data type while operating on array. Considering initial question this worked for me:
>>> data = d[:, :, 0].astype(np.short)
>>> data[data==-32768] = data[data>0].min()
>>> data = data.astype(np.float32)
If you love us? You can donate to us via Paypal or buy me a coffee so we can maintain and grow! Thank you!
Donate Us With