Starred expression inside square brackets

Question

It seems that starred expressions can not be used inside square brackets the way they are inside parentheses:

>>> import numpy as np
>>> x = np.ones((3, 4))
>>> x[:, *(None, None), :]
  File "<stdin>", line 1
    x[:, *(None, None), :]
         ^
SyntaxError: invalid syntax

though one would expect the latter to output the same as

>>> x[(slice(None), *(None, None), slice(None))]
array([[[[ 1.,  1.,  1.,  1.]]],


       [[[ 1.,  1.,  1.,  1.]]],


       [[[ 1.,  1.,  1.,  1.]]]])

Do you know any good reason why this is not allowed, and if there are plans for supporting it in the next Python releases?

Answer 1

As for how to do that, the answer probably is x[:, None, None, :]. But maybe you have a tuple that contains nones = (None, None), in which case you can do: x[:, nones[0], nones[1], :]. But I agree this would be nicer to have x[:, *nones, :] being valid.

On the question, why isn't it possible, we can have a look at the Python grammar to see why this doesn't work:

trailer: '(' [arglist] ')' | '[' subscriptlist ']' | '.' NAME
subscriptlist: subscript (',' subscript)* [',']
subscript: test | [test] ':' [test] [sliceop]
sliceop: ':' [test]

If you follow the longest and most promising grammar branch from test, searching for a '*' literal (I don't present the whole subtree because most other branches stop very early) : test -> or_test -> and_test -> comparison -> xor_expr -> and_expr -> shift_expr -> arith_expr.

Note that the rule we are searching for is this one

star_expr: '*' expr

Let's see if we can find it from here (arith_expr):

arith_expr: term (('+'|'-') term)*
term: factor (('*'|'@'|'/'|'%'|'//') factor)*
factor: ('+'|'-'|'~') factor | power
power: atom_expr ['**' factor]
atom_expr: ['await'] atom trailer*

Remember, trailer is where we started from, so we already know there is nothing that leads to star_expr down here, and atom closes all these different paths:

atom: ('(' [yield_expr|testlist_comp] ')' |
       '[' [testlist_comp] ']' |
       '{' [dictorsetmaker] '}' |
       NAME | NUMBER | STRING+ | '...' | 'None' | 'True' | 'False')

Basically you should only be allowed to use as a subscriptlist any of the expressions we saw down the branch (test -> ... -> arith_expr -> ... -> atom) or any one from NAME | NUMBER | STRING+ | '...' | 'None' | 'True' | 'False').

There are lot of branches that I didn't went through here. For most of them you can understand why just by looking at the rules' name. On the other hand it's probably not useless to double check.

For example when reading expression test:

test: or_test ['if' or_test 'else' test] | lambdef

We can see that ['if' or_test 'else' test] closes the path (as we are looking for * something). On the other hand, I could have included lambdef as this is perfectly valid for our quest, I simply ignored such paths because they close right after we explore them (applying a single rule invalidades *...), in this case (as you might have guessed):

lambdef: 'lambda' [varargslist] ':' test

We see a 'lambda' here, it's not an expression that starts with '*' so the path closes. Interestingly, that implies that x[lambda e: e] is perfectly valid syntax (I wouldn't have guessed and I never saw it but it makes sense).

In the end, we didn't saw a single leading * expression in the process, thus there should be no ambiguity in what you propose (unless I missed one combination of rules). Maybe that would make sense to ask to someone who've actually worked on this to check if there is a good reason for not having this outside potential grammar ambiguities.