Build and install local package with pip and pipenv

Question

I want to be able, in any file of B, to do: import APackage. A and B are two different projects / packages and they are local.

How to build, install, and upgrade A so that B will be able to call A modules? How to do this with pip? And with pipenv?

Answer 1

Tested on python 3.6.9 with pip 20.2.2 and pipenv 2020.6.2

TL;DR

pipenv install path/to/your/package

Prelude

Pip and pipenv work quite the same. To be sure you understand, pip is the package manager of your global python interpreter. You can do pip --version to check to which python, pip is liked to. Pipenv is a venv tool liked to a specific project / folder.

Do I need an __init__.py file for A packages?

Yes. pip and pipenv need an __init__.py file (can be empty) in A if you want to be able to import A in B. (Even if recent versions of python doesn't need __init__.py, it seems pip and pipenv still need it.)

Do I need a setup.py file in A?

Yes. pip and pipenv will raise an error if there is no setup.py file at the root of you project A (even if you don't build your project).

Do I need to use the option -e?

It depends on what you want. See here for more insights.

Structure

As example I will use this project architecture for A:

A/
 | myPackage/
      | __init__.py
      | myfile.py (contains a hello() function) 
 | setup.py
 | MANIFEST.in

The setup.py file contains this (setuptools page for more details):

from setuptools import setup, find_packages
setup(
    name="myName",
    version="0.1",
    packages=find_packages(),
)

Import A without building it

pip

pip install [local path to my project folder (here A/)]

Then you can check by opening your python shell and do:

import myPackage

pipenv

Be sure your pipenv is the right python version. You can setup a new env with the good version with: pipenv --python 3.6 command.

pipenv install [local path to my project folder (here A/)]

You can verify by doing:

pipenv shell
python
import myPackage

Build your project before importing it

That's the more pythonic way.

To build your project, go in your project folder (here A/) and run this command:

 python3 setup.py sdist

It will create a dist/ folder and inside you will find a zip file like A/dist/myName-0.1.0.tar.gz. Then you can install it (and verify the same way as before).

Pip

pip install [path to the zip file (here A/dist/myName-0.1.0.tar.gz)]

pipenv

 pipenv install [path to the zip file (here A/dist/myName-0.1.0.tar.gz)]

Versions handling

By changing the version of your project in setup.py and building it again you will find one more zip file in your dist folder (the previous version and the new build version). To change the version available from pip or pipenv you can do this:

For pip

Simply uninstall the package and reinstall the new version (give it the new zip file created).

For pipenv

Change the path by the new zip file in you Pipfile. To avoir concurrency errors you can simple delete you Pipfile.lock. Then simply run:

pipenv upgrade

Uninstall

pip uninstall [value of field 'name' in setup.py (here myName)]

Manage module visibility ?

We want to specify which function / class is accessible from B and which isn't.

One way is to use a leading underscore at the beginning of you object name which means: inner object, not suppposed to be imported) as explained in PEP 8.

Another way is simply to use your __init__.py file. In our case if we do in __init__.py this: from myPackage.myfile import hello. Then, in B project, you can do: from myPackage import hello. With an empty __init__.py file you need to write: from myPackage.myfile import hello, which suppose you know the inner structure of project A.

Exclude files and folders from my build

You should use MANIFEST.in file like explained here.