Why does Python insert None into slice steps?

Question

This can best be illustrated by example (all examples assume ast is imported; note that I am using Python 2.7.1):

# Outputs: Slice(lower=Num(n=1), upper=Num(n=10), step=None)
ast.dump(ast.parse("l[1:10]").body[0].value.slice)

# Outputs: Slice(lower=Num(n=1), upper=Num(n=10), step=Name(id='None', ctx=Load()))
ast.dump(ast.parse("l[1:10:]").body[0].value.slice)

# Outputs: Slice(lower=Num(n=1), upper=None, step=None)
ast.dump(ast.parse("l[1:]").body[0].value.slice)

# Outputs: Slice(lower=None, upper=None, step=None)
ast.dump(ast.parse("l[:]").body[0].value.slice)

So, as we can see, l[1:10] results in an AST node whose slice has two children—lower and upper both set to numeric literals—and an empty third step child. But [1:10:], which we would think of as identical, sets its slice's step child to be the None literal expression (Name(id='None', ctx=Load())).

Okay, I thought. Maybe Python treats l[1:10:] and l[1:10] as completely different kinds of expressions. The Python expression reference (link) certainly seemed to indicate so; l[1:10] is a simple slicing, but l[1:10:] is an extended slicing (with only one slice item).

But, even in the context of extended slicing, the step argument is treated specially. If we try ignoring the upper or lower bound in an extended slicing with one slice item, we just end up with empty children:

# Outputs: Slice(lower=Num(n=1), upper=None, step=Name(id='None', ctx=Load()))
ast.dump(ast.parse("l[1::]").body[0].value.slice)

# Outputs: Slice(lower=None, upper=Num(n=10), step=Name(id='None', ctx=Load()))
ast.dump(ast.parse("l[:10:]").body[0].value.slice)

Moreover, upon further inspection the AST doesn't even treat these slicings as extended slicings. Here's what extended slicings actually look like:

# Outputs: ExtSlice(dims=[Slice(lower=None, upper=None, step=Name(id='None', ctx=Load())), Slice(lower=None, upper=None, step=Name(id='None', ctx=Load()))])
ast.dump(ast.parse("l[::, ::]").body[0].value.slice)

So here is my conclusion: the AST always treats the step parameter special for some reason and, unrelatedly, the Slice AST node represents a long slice (I guess so there don't have to be two different base Slice classes—ShortSlice and LongSlice—though I would think that would be preferred) and so a single-item extended slice can be represented as a normal Slice node and is done so for some reason. It just seems wrong to me to allow None parameters to be interpreted as defaults, but I understand that that was a purposeful design decision; the None literal insertion and treating long slices as Slice nodes just seem kind of like accidents (or artifacts of old designs).

Does anyone else have a more informed explanation?

Answer 1

Without such treatment in extended slice notation you wouldn't be able to distinguish between l[1:] and l[1::], and you couldn't invoke different special methods - __getslice__ can be invoked for a normal slice, but __getitem__ must be invoked for an extended slice.

So it's mostly a backward-compatibility thing for Python 2.x, which has disappeared in Python 3.x:

Python 3.2 (r32:88445, Mar 25 2011, 19:28:28) 
[GCC 4.5.2] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> import ast
>>> ast.dump(ast.parse("l[1:]").body[0].value.slice)
'Slice(lower=Num(n=1), upper=None, step=None)'
>>> ast.dump(ast.parse("l[1::]").body[0].value.slice)
'Slice(lower=Num(n=1), upper=None, step=None)'
>>> 

See python2.7 source for ast.c and data model description for more info.