pip install . creates only the dist-info not the package

Question

I am trying to make a python package which I want to install using pip install . locally. The package name is listed in pip freeze but import <package> results in an error No module named <package>. Also the site-packages folder does only contain a dist-info folder. find_packages() is able to find packages. What am I missing?

import io import os import sys from shutil import rmtree  from setuptools import find_packages, setup, Command  # Package meta-data. NAME = '<package>' DESCRIPTION = 'description' URL = '' EMAIL = 'email' AUTHOR = 'name'  # What packages are required for this module to be executed? REQUIRED = [     # 'requests', 'maya', 'records', ]  # The rest you shouldn't have to touch too much :) # ------------------------------------------------ # Except, perhaps the License and Trove Classifiers! # If you do change the License, remember to change the Trove Classifier for that!  here = os.path.abspath(os.path.dirname(__file__))    # Where the magic happens: setup(     name=NAME,     #version=about['__version__'],     description=DESCRIPTION,     # long_description=long_description,     author=AUTHOR,     author_email=EMAIL,     url=URL,     packages=find_packages(),     # If your package is a single module, use this instead of 'packages':     # py_modules=['mypackage'],      # entry_points={     #     'console_scripts': ['mycli=mymodule:cli'],     # },     install_requires=REQUIRED,     include_package_data=True,     license='MIT',     classifiers=[         # Trove classifiers         # Full list: https://pypi.python.org/pypi?%3Aaction=list_classifiers         'License :: OSI Approved :: MIT License',         'Programming Language :: Python',         'Programming Language :: Python :: 2.6',         'Programming Language :: Python :: 2.7',         'Programming Language :: Python :: 3',         'Programming Language :: Python :: 3.3',         'Programming Language :: Python :: 3.4',         'Programming Language :: Python :: 3.5',         'Programming Language :: Python :: 3.6',         'Programming Language :: Python :: Implementation :: CPython',         'Programming Language :: Python :: Implementation :: PyPy'     ],  ) 

Answer 1

Since the question has become quite popular, here are the diagnosis steps to go through when you're missing files after installation. Imagine having an example project with the following structure:

root ├── spam │   ├── __init__.py │   ├── data.txt │   ├── eggs.py │   └── fizz │       ├── __init__.py │       └── buzz.py ├── bacon.py └── setup.py 

Now I run pip install ., check that the package is installed:

$ pip list Package    Version ---------- ------- mypkg      0.1     pip        19.0.1  setuptools 40.6.3  wheel      0.32.3  

but see neither spam, nor spam/eggs.py nor bacon.py nor spam/fizz/buzz.py in the list of files belonging to the installed package:

$ pip show -f mypkg Name: mypkg Version: 0.1 ... Files:   mypkg-0.1.dist-info/DESCRIPTION.rst   mypkg-0.1.dist-info/INSTALLER   mypkg-0.1.dist-info/METADATA   mypkg-0.1.dist-info/RECORD   mypkg-0.1.dist-info/WHEEL   mypkg-0.1.dist-info/metadata.json   mypkg-0.1.dist-info/top_level.txt 

So what to do now?

Diagnose by inspecting the wheel build log

Unless told not to do so, pip will always try to build a wheel file and install your package from it. We can inspect the log for the wheel build process if reinstalling in the verbose mode. First step is to uninstall the package:

$ pip uninstall -y mypkg ... 

then install it again, but now with an additional argument:

$ pip install . -vvv ... 

Now if I inspect the log:

$ pip install . -vvv | grep 'adding'   adding 'mypkg-0.1.dist-info/METADATA'   adding 'mypkg-0.1.dist-info/WHEEL'   adding 'mypkg-0.1.dist-info/top_level.txt'   adding 'mypkg-0.1.dist-info/RECORD' 

I notice that no files from the spam directory or bacon.py are mentioned anywhere. This means they were simply not included in the wheel file and hence not installed by pip. The most common error sources are:

Missing packages: check the packages argument

Verify you have passed the packages argument to the setup function. Check that you have mentioned all of the packages that should be installed. Subpackages will not be collected automatically if only the parent package is mentioned! For example, in the setup script

from setuptools import setup  setup(     name='mypkg',     version='0.1',     packages=['spam'] ) 

spam will be installed, but not spam.fizz because it is a package itself and must be mentioned explicitly. Fixing it:

from setuptools import setup  setup(     name='mypkg',     version='0.1',     packages=['spam', 'spam.fizz'] ) 

If you have lots of packages, use setuptools.find_packages to automate the process:

from setuptools import find_packages, setup  setup(     name='mypkg',     version='0.1',     packages=find_packages()  # will return a list ['spam', 'spam.fizz'] ) 

In case you are missing a module:

Missing modules: check the py_modules argument

In the above examples, I will be missing bacon.py after installation since it doesn't belong to any package. I have to provide its module name in the separate argument py_modules:

from setuptools import find_packages, setup  setup(     name='mypkg',     version='0.1',     packages=find_packages(),     py_modules=['bacon'] ) 

Missing data files: check the package_data argument

I have all the source code files in place now, but the data.txt file is still not installed. Data files located under package directories should be added via the package_data argument. Fixing the above setup script:

from setuptools import find_packages, setup  setup(     name='mypkg',     version='0.1',     packages=find_packages(),     package_data={'spam': ['data.txt']},     py_modules=['bacon'] ) 

Don't be tempted to use the data_files argument. Place the data files under a package and configure package_data instead.

After fixing the setup script, verify the package files are in place after installation

If I now reinstall the package, I will notice all of the files are added to the wheel:

$ pip install . -vvv | grep 'adding'   adding 'bacon.py'   adding 'spam/__init__.py'   adding 'spam/data.txt'   adding 'spam/eggs.py'   adding 'spam/fizz/__init__.py'   adding 'spam/fizz/buzz.py'   adding 'mypkg-0.1.dist-info/METADATA'   adding 'mypkg-0.1.dist-info/WHEEL'   adding 'mypkg-0.1.dist-info/top_level.txt'   adding 'mypkg-0.1.dist-info/RECORD' 

They will also be visible in the list of files belonging to mypkg:

$ pip show -f mypkg Name: mypkg Version: 0.1 ... Files:   __pycache__/bacon.cpython-36.pyc   bacon.py   mypkg-0.1.dist-info/INSTALLER   mypkg-0.1.dist-info/METADATA   mypkg-0.1.dist-info/RECORD   mypkg-0.1.dist-info/WHEEL   mypkg-0.1.dist-info/top_level.txt   spam/__init__.py   spam/__pycache__/__init__.cpython-36.pyc   spam/__pycache__/eggs.cpython-36.pyc   spam/data.txt   spam/eggs.py   spam/fizz/__init__.py   spam/fizz/__pycache__/__init__.cpython-36.pyc   spam/fizz/__pycache__/buzz.cpython-36.pyc   spam/fizz/buzz.py