Python 3.9.8 fails using Black and importing `typed_ast.ast3`

Question

Since updating to Python 3.9.8, we get an error while using Black in our CI pipeline.

black....................................................................Failed
- hook id: black
- exit code: 1
Traceback (most recent call last):
  File "../.cache/pre-commit/repol9drvp84/py_env-python3/bin/black", line 5, in <module>
    from black import patched_main
  File "../.cache/pre-commit/repol9drvp84/py_env-python3/lib/python3.9/site-packages/black/__init__.py", line 52, in <module>
    from typed_ast import ast3, ast27
  File "../.cache/pre-commit/repol9drvp84/py_env-python3/lib/python3.9/site-packages/typed_ast/ast3.py", line 40, in <module>
    from typed_ast import _ast3
ImportError: ../.cache/pre-commit/repol9drvp84/py_env-python3/lib/python3.9/site-packages/typed_ast/_ast3.cpython-39-x86_64-linux-gnu.so: undefined symbol: _PyUnicode_DecodeUnicodeEscape

The error can be easily reproduced with:

% pip install typed_ast
% python3 -c 'from typed_ast import ast3'
Traceback (most recent call last):
  File "<string>", line 1, in <module>
ImportError:
/usr/lib/python3/dist-packages/typed_ast/_ast3.cpython-39-x86_64-linux-gnu.so:
undefined symbol: _PyUnicode_DecodeUnicodeEscape

Currently the only workaround is downgrading to Python 3.9.7.

Is another fix available?

See also Bug#998854: undefined symbol: _PyUnicode_DecodeUnicodeEscape

Answer 1

The initial error was a failing Python Black pipeline. Black failed because it was pinned to an older version which now fails with Python 3.9.8.

Updating Black to the latest version, 21.10b0, fixed the error for me.

See also typed_ast issue #169:

For others who may find this in a search, I ran into this problem via black because I had black pinned to an older version. The current version of black appears to no longer use typed-ast and thus won't encounter this issue.

Using the latest typed-ast version >=1.5.0 seems to work as well.

E.g., pip install typed-ast --upgrade