Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

Concatenating views in NumPy

Indexing NumPy array through slices/indexes creates a view that is lightweight (doesn't copy data) and allows assigning to elements of original array. I.e.

import numpy as np
a = np.array([1, 2, 3, 4, 5])
a[2:4] = [6, 7]
print(a)
# [1 2 6 7 5]

But how about multiple views, how do I concatenate them to create a bigger view that still assigns to original first array. E.g. for imaginary function concatenate_views(...):

a = np.array([1, 2, 3, 4, 5, 6, 7, 8, 9, 10])
concatenate_views((a[1:3], a[4:6], a[7:9])) = [11, 12, 13, 14, 15, 16]
print(a)
# should print [1 11 12 4 13 14 7 15 16 10]

Of course, I can create a list of indexes for each view that it views to, just by converting slices to indexes and then concatenating these indexes. This way I'll get all indexes of concatenated views and can use these indexes to create a combined view. But this is not what I want. I want NumPy to keep the notion of slices representation, because all slices can be very long and it will be inefficient to convert and store these slices as indexes. I want NumPy to be aware of underlying slices of all concatenated views to make internally just simple looping of slices ranges.

Also would be nice to generalize the problem. Not only concatenate views, but also allow to form any arbitrary tree of slicing/indexing operations, e.g. concatenate views, then apply some slicing, then indexing, then slicing, then concatenate again. Also N-dimensional slicing/indexing. I.e. all fancy stuff that can be done with single un-concatenated view.

The main point of concatenated views is only efficiency. Of cause we can represent any view or slicing operation by N-D array of integer indexes (coordinates, like meshgrid) and then can use this array to make a view of source array. But if numpy can keep a notion of source set of slices instead of array of integers, then first it will be lightweight (much less memory consumption), second instead of reading indexes from memory numpy internally can loop (iterate) through each slice more efficiently in C++ loops.

By having concatenated view I wanted to be able to apply any numpy operation like np.mean(...) to the combined view in efficient way.

Full procedure of concatenating views of N-D slicing based on 2D example is described down below:

    1 Step described below:
    
    2D array slicing using 3 slices for each axis
    
    a,b,c - sizes of "slices" along axis 0
    d,e,f - sizes of "slices" along axis 1
    
    Each "slice" - is either slice(start, stop, step) or 1D array of integer indexes
            
      d e f
     .......
    a.0.1.2.
     .......
    b.3.4.5.
     .......
    c.6.7.8.
     .......

    Above 0 1 2 3 4 5 6 7 8 mean not a single integer but some 2D sub-array.
    Dots (`.`) also mean some 2D sub-arrays.
    
    Sub-views shapes:
    0:(a, d), 1:(a, e), 2:(a, f)
    3:(b, d), 4:(b, e), 5:(b, f)
    6:(c, d), 7:(c, e), 8:(c, f)

    Final aggregated (concatenated) view shape:
    ((a + b + c), (d + e + f))
    containing 2D array
    012
    345
    678
    
    There can be more than one Steps, each next Step applies new sequence of slicing
    to the final view obtained on previous Step. Each Step has different set of sizes
    of slices and different amount of slices per each dimension.
    In general each next Step reduces number of total elements, except the case
    when slices or indexes overlap then you may get more elements but with duplicates.
like image 462
Arty Avatar asked Feb 23 '26 21:02

Arty


1 Answers

You can use np.r_ to concatenate slice objects and assign back to the indexed array:

a[np.r_[1:3, 4:6, 7:9]] = [11, 12, 13, 14, 15, 16]

print(a)
array([ 1, 11, 12,  4, 13, 14,  7, 15, 16, 10])

Update

Based on your update, I think you might want something like:

from itertools import islice

it = iter([11, 12, 13, 14, 15, 16])
for s in slice(1,3), slice(4,6), slice(7,9):
    a[s] = list(islice(it, s.stop-s.start))

print(a)
array([ 1, 11, 12,  4, 13, 14,  7, 15, 16, 10])
like image 162
yatu Avatar answered Feb 26 '26 10:02

yatu