Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

Item assignment to Python dask array objects

I've created a Python dask array and I'm trying to modify a slice of the array as follows:

import numpy as np
import dask.array as da

x = np.random.random((20000, 100, 100)) # Create numpy array
dx = da.from_array(x, chunks=(x.shape[0], 10, 10)) # Create dask array from numpy array

dx[:50, :, :] = 0 # Modify a slice of the dask array

Such an attempt to modify the dask array raises the exception:

TypeError: 'Array' object does not support item assignment

Is there a way to modify a dask array slice without raising an exception?

like image 850
Lcg3 Avatar asked Mar 21 '16 22:03

Lcg3


1 Answers

Currently dask.array does not support item assignment or any other mutation operation.

In the case above I recommend concatenating with zeros

In [1]: import dask.array as da

In [2]: dx = da.random.random((20000 - 50, 100, 100), chunks=(None, 10, 10))

In [3]: z = da.zeros((50, 100, 100), chunks=(50, 10, 10))

In [4]: dx2 = da.concatenate([z, dx], axis=0)

In [5]: dx2
Out[5]: dask.array<concate..., shape=(20000, 100, 100), dtype=float64, chunksize=(50, 10, 10)>

In [6]: (dx2 == 0)[0:100, 0, 0].compute()
Out[6]: 
array([ True,  True,  True,  True,  True,  True,  True,  True,  True,
        True,  True,  True,  True,  True,  True,  True,  True,  True,
        True,  True,  True,  True,  True,  True,  True,  True,  True,
        True,  True,  True,  True,  True,  True,  True,  True,  True,
        True,  True,  True,  True,  True,  True,  True,  True,  True,
        True,  True,  True,  True,  True, False, False, False, False,
       False, False, False, False, False, False, False, False, False,
       False, False, False, False, False, False, False, False, False,
       False, False, False, False, False, False, False, False, False,
       False, False, False, False, False, False, False, False, False,
       False, False, False, False, False, False, False, False, False, False], dtype=bool)

The da.where(condition, iftrue, iffalse) function can also be quite useful in working around cases where mutation is often desired.

like image 122
MRocklin Avatar answered Oct 30 '22 20:10

MRocklin