ValueError: source code string cannot contain null bytes

Question

import hashlib


def my_function(bytes_):
    """
    >>> my_function(b'\0')
    '6e340b9cffb37a989ca544e6bb780a2c78901d3fb33738768511a30617afa01d'
    """
    return hashlib.sha256(bytes_).hexdigest()


if __name__ == "__main__":
    import doctest
    doctest.testmod()

Expected behaviour: 1 test passes

Actual behaviour: ValueError exception is raised from within doctest.py.

I've tried to use b'\x00' literal as well with the same result. How to use null bytes in doctest? What exactly is the problem here, and is there any fix or not-too-ugly workaround?

Answer 1

Your doctest is getting double-parsed. First, Python applies backslash processing to convert \0 to a null byte in the docstring itself, and then doctest tries to run the thing after the >>> as Python code and encounters an error because backslash processing has already been applied.

This would also be a problem for anyone just trying to print your docstring, or trying to call help on your function.

As with regexes, to avoid the first layer of backslash processing, use raw string notation:

def my_function(bytes_):
    r"""
    >>> my_function(b'\0')
    '6e340b9cffb37a989ca544e6bb780a2c78901d3fb33738768511a30617afa01d'
    """
    return hashlib.sha256(bytes_).hexdigest()