Mysterious "embedded null byte" error

Question

Working on a fairly large/complex Django project with a team, we occasionally see runserver crash with ValueError: embedded null byte. We restart runserver and it's fine - either for a few minutes or a few days. We can detect no pattern to what causes the crashes (seems totally random). Fortunately it's only happening in local development, not on our servers, but I'm worried that it will bite us down the road.

The stack trace below does not point to any location in our code -- seems to come either from Django or the virtualenv itself.

Using Django 1.9.8, Python 3.5.0, on El Capitan.

I can't see any way to debug this. Theories?

Traceback (most recent call last):
  File "manage.py", line 10, in <module>
    execute_from_command_line(sys.argv)
  File "/path/to/virtualenvs/ourproj/lib/python3.5/site-packages/django/core/management/__init__.py", line 353, in execute_from_command_line
    utility.execute()
  File "/path/to/virtualenvs/ourproj/lib/python3.5/site-packages/django/core/management/__init__.py", line 345, in execute
    self.fetch_command(subcommand).run_from_argv(self.argv)
  File "/path/to/virtualenvs/ourproj/lib/python3.5/site-packages/django/core/management/base.py", line 348, in run_from_argv
    self.execute(*args, **cmd_options)
  File "/path/to/virtualenvs/ourproj/lib/python3.5/site-packages/django/core/management/commands/runserver.py", line 54, in execute
    super(Command, self).execute(*args, **options)
  File "/path/to/virtualenvs/ourproj/lib/python3.5/site-packages/django/core/management/base.py", line 399, in execute
    output = self.handle(*args, **options)
  File "/path/to/virtualenvs/ourproj/lib/python3.5/site-packages/django/core/management/commands/runserver.py", line 93, in handle
    self.run(**options)
  File "/path/to/virtualenvs/ourproj/lib/python3.5/site-packages/django/core/management/commands/runserver.py", line 102, in run
    autoreload.main(self.inner_run, None, options)
  File "/path/to/virtualenvs/ourproj/lib/python3.5/site-packages/django/utils/autoreload.py", line 333, in main
    reloader(wrapped_main_func, args, kwargs)
  File "/path/to/virtualenvs/ourproj/lib/python3.5/site-packages/django/utils/autoreload.py", line 299, in python_reloader
    reloader_thread()
  File "/path/to/virtualenvs/ourproj/lib/python3.5/site-packages/django/utils/autoreload.py", line 275, in reloader_thread
    change = fn()
  File "/path/to/virtualenvs/ourproj/lib/python3.5/site-packages/django/utils/autoreload.py", line 204, in code_changed
    for filename in gen_filenames():
  File "/path/to/virtualenvs/ourproj/lib/python3.5/site-packages/django/utils/autoreload.py", line 114, in gen_filenames
    basedirs = [os.path.abspath(basedir) for basedir in basedirs
  File "/path/to/virtualenvs/ourproj/lib/python3.5/site-packages/django/utils/autoreload.py", line 115, in <listcomp>
    if os.path.isdir(basedir)]
  File "/path/to/virtualenvs/ourproj/bin/../lib/python3.5/genericpath.py", line 42, in isdir
    st = os.stat(s)
ValueError: embedded null byte

Answer 1

• One of the AppConfig objects has null byte in its path attribute.
• One of the LOCALE_PATHS has null byte.
• One of the files is in the "wrong" encoding, so Django treats something has null byte (e. g. AppConfig.path).
• One of the files or directories in project directory has null byte (\x00) in its name.
• Other reason.

autoreload.py lines related to this issue. os.path.isdir() raises ValueError: embedded null byte if its argument has null byte, e. g. os.path.isdir('foo\x00bar').

You can try to edit autoreload.py, comment out these lines temporarily:

basedirs = [os.path.abspath(basedir) for basedir in basedirs
            if os.path.isdir(basedir)]

and add this:

temp_basedirs = []
for basedir in basedirs:
    try:
        if os.path.isdir(basedir):
            temp_basedirs.append(os.path.abspath(basedir))
    except ValueError:
        print(basedir)
        raise
basedirs = temp_basedirs