"SyntaxError: invalid character" when using pipreqs to generate requirements.txt

Question

Stack Trace:

(realestate_env) PS C:\Users\Jun\Desktop\Jupyter Projects\Real Estate Price Prediction> pipreqs                   
ERROR: Failed on file: C:\Users\Jun\Desktop\Jupyter Projects\Real Estate Price Prediction\realestate_env\Lib\site-packages\black\linegen.py
Traceback (most recent call last):
  File "C:\Users\Jun\AppData\Local\Programs\Python\Python39\lib\runpy.py", line 197, in _run_module_as_main
    return _run_code(code, main_globals, None,
  File "C:\Users\Jun\AppData\Local\Programs\Python\Python39\lib\runpy.py", line 87, in _run_code
    exec(code, run_globals)
  File "C:\Users\Jun\Desktop\Jupyter Projects\Real Estate Price Prediction\realestate_env\Scripts\pipreqs.exe\__main__.py", line 7, in <module>
  File "C:\Users\Jun\Desktop\Jupyter Projects\Real Estate Price Prediction\realestate_env\lib\site-packages\pipreqs\pipreqs.py", line 488, in main
    init(args)
  File "C:\Users\Jun\Desktop\Jupyter Projects\Real Estate Price Prediction\realestate_env\lib\site-packages\pipreqs\pipreqs.py", line 415, in init
    candidates = get_all_imports(input_path,
  File "C:\Users\Jun\Desktop\Jupyter Projects\Real Estate Price Prediction\realestate_env\lib\site-packages\pipreqs\pipreqs.py", line 131, in get_all_imports
    raise exc
  File "C:\Users\Jun\Desktop\Jupyter Projects\Real Estate Price Prediction\realestate_env\lib\site-packages\pipreqs\pipreqs.py", line 117, in get_all_imports
    tree = ast.parse(contents)
  File "C:\Users\Jun\AppData\Local\Programs\Python\Python39\lib\ast.py", line 50, in parse
    return compile(source, filename, mode, flags,
  File "<unknown>", line 283
    Ø: Set[str] = set()
     ^
SyntaxError: invalid character '˜' (U+02DC)
(realestate_env) PS C:\Users\Jun\Desktop\Jupyter Projects\Real Estate Price Prediction> 

I am having this issue when I am trying to use pipreqs to create a requirement.txt file. I have done an extensive search on the Internet to solve this problem but then hardly found an effective solution until I found the below link: https://blog.csdn.net/Pragmatism_3568/article/details/105414950

It seems to be related to the byte-order mark (BOM) issue, but I can't fully understand it because it's in Chinese.

Answer 1

I was able to solve this issue by specifying the encoding utf-8-sig when executing pipreqs:

pipreqs.exe <path> --encoding=utf-8-sig

see pipreqs usage on pypi for more information. I hope this helps.