numpy - explanation of the 3rd string integer of numpy.r_['string integer', array]

Question

Question

Please help understand in layman's words what the 3rd string integer in numpy.r_['1,2,0', array] is and how it works.

numpy documentation says about the 3rd integer as below, but cannot figure out what it is exactly.

which axis should contain the start of the arrays which are less than the specified number of dimensions. In other words the third integer allows you to specify where the 1’s should be placed in the shape of the arrays that have their shapes upgraded.

numpy.r_

A string integer specifies which axis to stack multiple comma separated arrays along. A string of two comma-separated integers allows indication of the minimum number of dimensions to force each entry into as the second integer (the axis to concatenate along is still the first integer).

A string with three comma-separated integers allows specification of the axis to concatenate along, the minimum number of dimensions to force the entries to, and which axis should contain the start of the arrays which are less than the specified number of dimensions. In other words the third integer allows you to specify where the 1’s should be placed in the shape of the arrays that have their shapes upgraded. By default, they are placed in the front of the shape tuple. The third argument allows you to specify where the start of the array should be instead. Thus, a third argument of ‘0’ would place the 1’s at the end of the array shape. Negative integers specify where in the new shape tuple the last dimension of upgraded arrays should be placed, so the default is ‘-1’.

Without the 3rd integer, the behavior makes sense. Frame a 2D shape and place the array elements along the 1st axis or 2nd axis.

print(np.r_['0,2', [1,2,3], [4,5,6]])
print(np.r_['1,2', [1,2,3], [4,5,6]])
---
[[1 2 3]
 [4 5 6]]
[[1 2 3 4 5 6]]

However, not sure how the 3rd integer is converting the shape. It appears '1,2,0' is a transpose of '0,2,1', and '1,2,1' is that of '0,2,0' but no idea how this occurs.

print(np.r_['0,2', [1,2,3], [4,5,6]])
print()
print(np.r_['0,2,0', [1,2,3], [4,5,6]])
print(np.r_['0,2,1', [1,2,3], [4,5,6]])
---
[[1 2 3]
 [4 5 6]]

[[1]
 [2]
 [3]
 [4]
 [5]
 [6]]
[[1 2 3]
 [4 5 6]]

print(np.r_['1,2', [1,2,3], [4,5,6]])
print()
print(np.r_['1,2,0', [1,2,3], [4,5,6]])
print(np.r_['1,2,1', [1,2,3], [4,5,6]])
---
[[1 2 3 4 5 6]]

[[1 4]
 [2 5]
 [3 6]]
[[1 2 3 4 5 6]]

The answer in Understanding the syntax of numpy.r_() concatenation is touching on the part, but not sure what it is telling completely. It seems like a switch to make the shape (1, 3) or its transpose (3, 1) when doing np.newaxis.

np.r_['0,2,-1', [1,2,3], [[4,5,6]]] has shape (3,) and shape (1, 3). np.r_ needs to add a 1 to its shape tuple to make it either (1,3) or (3,1). That is where the -1 in 0,2,-1 comes into play. It basically decides where the extra 1 needs to be placed in the shape tuple of the array.

---

Update

Based on the answer from hpaulj, it is much easier to think of the 3rd integer as shift operator.

Plus(+) is shifting the original shape from the highest dimension side to the lower in the expanded shape.

print(np.r_['0,5,0',np.ones((2,3))].shape)  # 0 places the original shape to the highest dimension side.
print(np.r_['0,5,1',np.ones((2,3))].shape)  # shift 1 to the right from highest to lower dimension.
print(np.r_['0,5,2',np.ones((2,3))].shape)  # shift 2 to the right from highest to lower dimension.
print(np.r_['0,5,3',np.ones((2,3))].shape)  # shift 3 to the right from highest to lower dimension.
print(np.r_['0,5,4',np.ones((2,3))].shape)  # Cannot shift shape (2, 3) further than 3 in 5 dimension shape.
---
(2, 3, 1, 1, 1)
(1, 2, 3, 1, 1)
(1, 1, 2, 3, 1)
(1, 1, 1, 2, 3)
ValueError: axes don't match array

Minus(-) is shifting the original shape from the lowest dimension side to the higher in the expanded shape.

print(np.r_['0,5,-1',np.ones((2,3))].shape)  # -1 places the original shape at the lowest dimension side.
print(np.r_['0,5,-2',np.ones((2,3))].shape)  # shift 1 to the left from lowest to higher dimension.
print(np.r_['0,5,-3',np.ones((2,3))].shape)  # shift 2 to the left from lowest to higher dimension.
print(np.r_['0,5,-4',np.ones((2,3))].shape)  # shift 3 to the left from lowest to higher dimension.
print(np.r_['0,5,-5',np.ones((2,3))].shape)  # Cannot shift shape (2, 3) further than 3 in 5 dimension shape.
---
(1, 1, 1, 2, 3)
(1, 1, 2, 3, 1)
(1, 2, 3, 1, 1)
(2, 3, 1, 1, 1)
ValueError: axes don't match array

Answer 1

With np.stack I can add a new axis in 3 places relative to a 2d array. Working with a single element tuple to highlight the new axis:

In [37]: np.stack((np.ones((2,3)),),0).shape     # before
Out[37]: (1, 2, 3)
In [38]: np.stack((np.ones((2,3)),),1).shape     # mid
Out[38]: (2, 1, 3)
In [39]: np.stack((np.ones((2,3)),),2).shape     # after
Out[39]: (2, 3, 1)

But with r_ I can't get the (2,1,3) version:

In [40]: np.r_['0,3',np.ones((2,3))].shape
Out[40]: (1, 2, 3)
In [41]: np.r_['0,3,0',np.ones((2,3))].shape
Out[41]: (2, 3, 1)
In [42]: np.r_['0,3,1',np.ones((2,3))].shape
Out[42]: (1, 2, 3)

But with r_ I can add more than one new axis:

In [43]: np.r_['0,4',np.ones((2,3))].shape
Out[43]: (1, 1, 2, 3)
In [44]: np.r_['0,4,0',np.ones((2,3))].shape
Out[44]: (2, 3, 1, 1)
In [45]: np.r_['0,4,1',np.ones((2,3))].shape
Out[45]: (1, 2, 3, 1)
In [46]: np.r_['0,4,2',np.ones((2,3))].shape
Out[46]: (1, 1, 2, 3)

With newaxis I can achieve all these plus:

In [48]: np.ones((2,3))[:,None,:,None].shape
Out[48]: (2, 1, 3, 1)

The latest expand_dims allows me to do the same:

In [51]: np.expand_dims(np.ones((2,3)),(1,3)).shape
Out[51]: (2, 1, 3, 1)

While r_ is clever, I'm not sure it has aged well. stack and expand_dims have been added later. What's more valuable in r_ is its ability to translate slice notation into arange or linspace calls.

Don't forget that you can study the code at:

np.lib.index_tricks.AxisConcatenator

Looks like the 2nd value is handled with np.array(..., ndmin), and the 3rd value with a transpose. The code's a bit involved.

edit

ndmin adds leading dimensions as needed

In [82]: np.array([1,2,3], ndmin=3).shape
Out[82]: (1, 1, 3)

So this corresponds to the default (-1) value in r_.

By default, they are placed in the front of the shape tuple

In [105]: np.r_['0,3',[1,2,3]].shape
Out[105]: (1, 1, 3)
In [106]: np.r_['0,3,-1',[1,2,3]].shape
Out[106]: (1, 1, 3)

Thus, a third argument of ‘0’ would place the 1’s at the end of the array shape.

In [111]: np.r_['0,3,0',[1,2,3]].shape
Out[111]: (3, 1, 1)
In [112]: np.r_['0,3,0',np.ones((2,3))].shape
Out[112]: (2, 3, 1)

In both of these the array dimensions start at 0

The third argument allows you to specify where the start of the array should be instead

Thus start at 1 (i.e. with one leading 1):

In [113]: np.r_['0,3,1',[1,2,3]].shape
Out[113]: (1, 3, 1)
In [114]: np.r_['0,3,1',np.ones((2,3))].shape
Out[114]: (1, 2, 3)

and starting at 2:

In [115]: np.r_['0,3,2',[1,2,3]].shape
Out[115]: (1, 1, 3)

In [116]: np.r_['0,3,2',np.ones((2,3))].shape
Traceback (most recent call last):
  File "<ipython-input-116-88fa40ceb503>", line 1, in <module>
    np.r_['0,3,2',np.ones((2,3))].shape
  File "/usr/local/lib/python3.8/dist-packages/numpy/lib/index_tricks.py", line 395, in __getitem__
    newobj = newobj.transpose(axes)
ValueError: axes don't match array

It can't start the (2,3) dimension at 2 in a 3d array, hence the error. The error occurs while attempting to transpose the 3d array produced by ndmin. It probably should have caught this error when constructing the axes argument, rather allowing transpose to fail.

Negative integers specify where in the new shape tuple the last dimension of upgraded arrays should be placed

In [117]: np.r_['0,3,-2',[1,2,3]].shape            # 3 at -2
Out[117]: (1, 3, 1)
In [118]: np.r_['0,3,-2',np.ones((2,3))].shape     # 3 at -2
Out[118]: (2, 3, 1)

'-3' produces (3,1,1), over another, and error for (2,3).

Instead of focusing on where it adds the 1's, this third number makes most sense when focusing on where the original dimension(s) occur in the expanded array. The start of the array shape for positives, and 'end of the array shape` for negatives'.

In other words, when an n ndim array is placed in a m ndim result, the default has leading 1's. The n is at the right. The third number shifts that n to the left.

Let's test that by placing a (2,) in a (5,):

In [121]: np.r_['0,5',np.ones((2,3))].shape
Out[121]: (1, 1, 1, 2, 3)                      # all the way right
In [122]: np.r_['0,5,0',np.ones((2,3))].shape
Out[122]: (2, 3, 1, 1, 1)                      # all the way left
In [123]: np.r_['0,5,1',np.ones((2,3))].shape
Out[123]: (1, 2, 3, 1, 1)
In [124]: np.r_['0,5,2',np.ones((2,3))].shape
Out[124]: (1, 1, 2, 3, 1)
In [125]: np.r_['0,5,-3',np.ones((2,3))].shape    
Out[125]: (1, 2, 3, 1, 1)                        # same as [123]
In [126]: np.r_['0,5,-2',np.ones((2,3))].shape
Out[126]: (1, 1, 2, 3, 1)