initialize pandas SparseArray

Question

Is it possible to initialize a pandas SparseArray by providing only the dense entries? I could not figure this out from the documentation: http://pandas.pydata.org/pandas-docs/stable/sparse.html .

For example, say I want a length 1000 SparseArray with a one at index 9 and zeros everywhere else, how would I go about creating it? This is one way:

a = [0] * 1000
a[9] = 1
sparse_a = pd.SparseArray(data=a, fill_value=0) 

But, in the above, we have to create the dense array before the sparse one. Is there a way to specify only the indices and the dense entries to create the SparseArray directly?

Answer 1

A length 10 SparseArray with a one at index 9 and zeros everywhere else:

pd.SparseArray(1, index= range(1), kind='block', 
               sparse_index= BlockIndex(10, [8], [1]), 
               fill_value=0)

Notes:

1. index could be any list as long as its length is equal to all non-sparsed part of the array (the smaller part of the data), in this case, number of 1 in the sparse array
2. BlockIndex(10, [8], [1]) is the object pointing to the positions of the non-parsed part of the data where the first argument is the TOTAL length of the array (sparse + non-sparse), the second argument is a list of starting positions of the non-sparse data and the third argument is a list of how long each block of non-sparse lasts. Notice: that the length of the array mentioned in point 1 is the sum of all elements of the list in the third argument of this BlockIndex

So a more general example is: to make a length 20 SparseArray where the 2nd, 3rd, 6th,7th,8th elements are 1 and the rest is 0 is:

pd.SparseArray(1, index= range(5), kind='block', 
               sparse_index= BlockIndex(20, [1,5], [2,3]), 
               fill_value=0)

or

pd.SparseArray(1, index= [None, 3, 2, 7, np.inf], kind='block',
               sparse_index= BlockIndex(20, [1,5], [2,3]),
               fill_value=0)

Sadly, I don't know any good way to specify an array of non-sparsed data as the first argument for SparseArray-- it does not mean that it can't be done, this is only a disclaimer. I think as long as you specify index=... pandas will require a scalar for the first argument (the data).

Tested on Windows 7, pandas version 0.20.2 installed by Aconda.