Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

PyCharm, what is python_stubs?

Tags:

I have had numerous problems with PyCharm not recognising functions in libraries properly, so I've decided to look into the source code of some example functions that PyCharm recognises incorrectly. For example, PyCharm does not recognise pickle.load() correctly; it thinks that pickle.load() does not take any arguments, when in fact it does take one argument. I've asked that question here. So I've written the following short testing code.

import pickle   r = range(10) f = open("../temp/pickling_example.pkl", "wb") pickle.dump(r, f) f.close() f = open("../temp/pickling_example.pkl", "rb") pickle.load(f) print(r) 

I pressed Ctrl+B on pickle.load(f), the penultimate line. I was hoping that this would bring me to the source file containing the definition of pickle.load(), but instead it brought me to a file in the location C:\Users\ray\.PyCharm30\system\python_stubs\-1442825926\_pickle.py. This file contains the following abstract (see screenshot below).

Here you can see the source of the problem of PyCharm incorrectly recognising the signature of pickle.load(); the signature of pickle.load() according to this file has no arguments.

Could someone explain why I was brought here, and what this file is, and what the python_stubs folder (more precisely, C:\Users\ray\.PyCharm30\system\python_stubs\) is?

My guess is the following. I was brought to this file in this location because PyCharm can't find the actual source code where pickle.load() was defined, on my computer. In these situations, PyCharm just generates a dummy file with just the declarations (or just the signatures) of the functions used, in this case it's pickle.load(). This file is where I was brought to when I pressed Ctrl+B on pickle.load(). The purpose of this file is purely so that PyCharm's inspections works properly and can provide autocompletion, and PyCharm puts all of these files inf the python_stubs directory. The actual definition of the pickle.load() function is in a pyc or pyd file somewhere in my C:\Python34\ directory, and I don't have the actual py file containing the definition of pickle.load() because, when I installed Python, I didn't install the source codes.

Questions:

(1) Is my guess roughly correct? Could you provide and more correct and precise

(2) What can I do to prevent PyCharm from wrongly recognising or not recognising library functions? Shall I make sure I always install the source codes of Python and all 3rd party packages, in order to ensure that PyCharm can do its inspections properly?

(3) If I was right in that, PyCharm generates these files, how does PyCharm guess the functions' signatures?

like image 532
Ray Avatar asked Jun 17 '14 14:06

Ray


People also ask

What is a stubbed file?

A stub file is a computer file that appears to the user to be on disk and immediately available for use, but is actually held either in part or entirely on a different storage medium.

What is stub file in Python?

What is a stub file? It is a file with the . pyi extension, which describes the Python type information, both input and return for functions and omits the logical part. Using a stub file has the following advantages (see PEP 484). Example sample.py.

What type checker does PyCharm use?

Using Typeshed Typeshed stubs provide definitions for Python classes, functions, and modules defined with type hints. PyCharm uses this information for better code completion, inspections, and other code insight features. PyCharm is switching to Typeshed, the common repository for Python stubs.

How does PyCharm determine type of variable?

The only reliable way to know what type a given object is is to inspect it at runtime - which is usually done using a step debugger (either python's builtin pdb or PyCharm's own debugger). Note that you'll have to check the various code path that can lead to the portion of the code you want to inspect though.


1 Answers

(1) Yes, you are mostly correct.

The python_stubs file is an automatically generated file that contains dummy definitions for built-in functions. It is used by PyCharm to infer the types of the built-in functions in the case that they weren't hardcoded for the given version.

(3) It isn't always possible to correctly infer the type of a built-in functoin only from his docs. Some docstrings start with the "type signature":

>>> print(min.__doc__) min(iterable[, key=func]) -> value min(a, b, c, ...[, key=func]) -> value 

but pickle.load() doesn't.

Note that this will probably change in future python versions, because starting with python3.4 the Argument Clinic was introduced which allows better inspection for built-ins defined in C. I'm not sure whether PyCharm is already able to get that information.

(2) Try rebuilding the python skeletons. However, AFAIK, the only real option, if this doesn't work, is to open a ticket on PyCharm's issue tracker.

like image 200
Bakuriu Avatar answered Oct 03 '22 06:10

Bakuriu