vscode not autocompleting python from module made with pybind11

Question

I have a python module named relatpy.cpython-38-x86_64-linux-gnu.so that I created using pybind11 with a custom made C++ library. When I import this module in the terminal using something like ipython I get functional autocompletion like the figure shows.

However in vscode I don't get any autocompletion, I just get the message Import "relatpy" could not be resolved. The python file that imports this module can still be run just fine in vscode, just no autocompletion.

Do I have to add this module to my PYTHONPATH or how do I go about doing this? This module is also rebuilt pretty often so is there a way to get autocompletion to work even if the module changes?

I'm using pylance as the python language server in vscode.

I created the pybind11 module using the following CMakeLists.txt:

project(relatpy)

find_package(pybind11 REQUIRED)

file(GLOB SOURCES "../relativity/src/*.cpp")
file(GLOB HEADERS "../relativity/include/*.h")
file(GLOB BINDINGS "*.cpp")

pybind11_add_module(${PROJECT_NAME} ${SOURCES} ${HEADERS} ${BINDINGS})

set_target_properties(${PROJECT_NAME} PROPERTIES LIBRARY_OUTPUT_DIRECTORY ${PYTHON_DIR})

Answer 1

Per this comment on the pybind issue creating stub pyi files automatically from pybind projects, you can use stubgen, from mypy, to generate stub files, that Pylance can read.

pip install mypy

then

PYTHONPATH=<path to your python module> stubgen -m <your python module name>

(in your case, the path should be to the directory containing relatpy.cpython-38-x86_64-linux-gnu.so, and your python module name is relatpy)

Stubgen tells you where it's put the .pyi file, in my case out/<python module name>.pyi. You need to move the .pyi file to sit beside your Python module, i.e. next to relatpy.cpython-38-x86_64-linux-gnu.so.

When I looked at the .pyi files produced, it had an import flags line, which VS Code had underlined. So I also did:

pip install flags

(Sometime ago, I had tried the other solution mentioned in the pybind issue, pybind11-stubgen, but could not get it to work, whereas stubgen worked without errors.)

Edit: the CMake code required is pretty trivial: after building your module, simply figure out the location of stubgen (in my case, in my virtual environment as I'd pip installed it via a requirements.txt file), and run it using a custom command.

pybind11_add_module(${PYMODULE_NAME} ...)
target_link_libraries(${PYMODULE_NAME} PRIVATE ... pybind11::pybind11)

# Use stubgen to create .pyi files to sit alongside the just-built python module
set(Stubgen_Executable "${CMAKE_SOURCE_DIR}/venv/bin/stubgen")
if(WIN32)
    set(Stubgen_Executable "${CMAKE_SOURCE_DIR}/venv/Scripts/stubgen.exe")
endif()
add_custom_command(TARGET ${PYMODULE_NAME} POST_BUILD
    COMMAND ${Stubgen_Executable} -m ${PYMODULE_NAME} -o .
    WORKING_DIRECTORY ${CMAKE_CURRENT_BINARY_DIR}
    COMMENT "Use stubgen to create .pyi for statement completion"