Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

When are .pyc files refreshed?

I understand that ".pyc" files are compiled versions of the plain-text ".py" files, created at runtime to make programs run faster. However I have observed a few things:

  1. Upon modification of "py" files, program behavior changes. This indicates that the "py" files are compiled or at least go though some sort of hashing process or compare time stamps in order to tell whether or not they should be re-compiled.
  2. Upon deleting all ".pyc" files (rm *.pyc) sometimes program behavior will change. Which would indicate that they are not being compiled on update of ".py"s.

Questions:

  • How do they decide when to be compiled?
  • Is there a way to ensure that they have stricter checking during development?
like image 223
Aaron Schif Avatar asked Apr 05 '13 17:04

Aaron Schif


People also ask

When .PYC file is created?

When a Python source file (module) is imported during an execution for the first time, the appropriate . pyc file is created automatically. If the same module is imported again, then the already created . pyc file is used.

What is the difference between .py and .PYC files?

py files contain the source code of a program. Whereas, . pyc file contains the bytecode of your program.

Where are .PYC stored?

pyc files are placed in the same directory as the . py file. In Python 3.2, the compiled files are placed in a __pycache__ subdirectory, and are named differently depending on which Python interpreter created them. (This can be useful to people importing the same Python modules from multiple versions of Python.)

What is .PYC file and use of it?

pyc files are created by the Python interpreter when a . py file is imported. They contain the "compiled bytecode" of the imported module/program so that the "translation" from source code to bytecode (which only needs to be done once) can be skipped on subsequent imports if the . pyc is newer than the corresponding .


2 Answers

The .pyc files are created (and possibly overwritten) only when that python file is imported by some other script. If the import is called, Python checks to see if the .pyc file's internal timestamp is not older than the corresponding .py file. If it is, it loads the .pyc; if it isn't or if the .pyc does not yet exist, Python compiles the .py file into a .pyc and loads it.

What do you mean by "stricter checking"?

like image 82
DaveTheScientist Avatar answered Oct 21 '22 08:10

DaveTheScientist


.pyc files generated whenever the corresponding code elements are imported, and updated if the corresponding code files have been updated. If the .pyc files are deleted, they will be automatically regenerated. However, they are not automatically deleted when the corresponding code files are deleted.

This can cause some really fun bugs during file-level refactors.

First of all, you can end up pushing code that only works on your machine and on no one else's. If you have dangling references to files you deleted, these will still work locally if you don't manually delete the relevant .pyc files because .pyc files can be used in imports. This is compounded with the fact that a properly configured version control system will only push .py files to the central repository, not .pyc files, meaning that your code can pass the "import test" (does everything import okay) just fine and not work on anyone else's computer.

Second, you can have some pretty terrible bugs if you turn packages into modules. When you convert a package (a folder with an __init__.py file) into a module (a .py file), the .pyc files that once represented that package remain. In particular, the __init__.pyc remains. So, if you have the package foo with some code that doesn't matter, then later delete that package and create a file foo.py with some function def bar(): pass and run:

from foo import bar 

you get:

ImportError: cannot import name bar 

because python is still using the old .pyc files from the foo package, none of which define bar. This can be especially problematic on a web server, where totally functioning code can break because of .pyc files.

As a result of both of these reasons (and possibly others), your deployment code and testing code should delete .pyc files, such as with the following line of bash:

find . -name '*.pyc' -delete 

Also, as of python 2.6, you can run python with the -B flag to not use .pyc files. See How to avoid .pyc files? for more details.

See also: How do I remove all .pyc files from a project?

like image 22
Zags Avatar answered Oct 21 '22 09:10

Zags