What are the use cases for a Python distribution?

Question

I'm developing a distribution for the Python package I'm writing so I can post it on PyPI. It's my first time working with distutils, setuptools, distribute, pip, setup.py and all that and I'm struggling a bit with a learning curve that's quite a bit steeper than I anticipated :)

I was having a little trouble getting some of my test data files to be included in the tarball by specifying them in the data_files parameter in setup.py until I came across a different post here that pointed me toward the MANIFEST.in file. Just then I snapped to the notion that what you include in the tarball/zip (using MANIFEST.in) and what gets installed in a user's Python environment when they do easy_install or whatever (based on what you specify in setup.py) are two very different things; in general there being a lot more in the tarball than actually gets installed.

This immediately triggered a code-smell for me and the realization that there must be more than one use case for a distribution; I had been fixated on the only one I've really participated in, using easy_install or pip to install a library. And then I realized I was developing work product where I had only a partial understanding of the end-users I was developing for.

So my question is this: "What are the use cases for a Python distribution other than installing it in one's Python environment? Who else am I serving with this distribution and what do they care most about?"

Here are some of the working issues I haven't figured out yet that bear on the answer:

• Is it a sensible thing to include everything that's under source control (git) in the source distribution? In the age of github, does anyone download a source distribution to get access to the full project source? Or should I just post a link to my github repo? Won't including everything bloat the distribution and make it take longer to download for folks who just want to install it?

• I'm going to host the documentation on readthedocs.org. Does it make any sense for me to include HTML versions of the docs in the source distribution?

• Does anyone use python setup.py test to run tests on a source distribution? If so, what role are they in and what situation are they in? I don't know if I should bother with making that work and if I do, who to make it work for.

Answer 1

Some things that you might want to include in the source distribution but maybe not install include:

• the package's license
• a test suite
• the documentation (possibly a processed form like HTML in addition to the source)
• possibly any additional scripts used to build the source distribution

Quite often this will be the majority or all of what you are managing in version control and possibly a few generated files.

The main reason why you would do this when those files are available online or through version control is so that people know they have the version of the docs or tests that matches the code they're running.

If you only host the most recent version of the docs online, then they might not be useful to someone who has to use an older version for some reason. And the test suite on the tip in version control may not be compatible with the version of the code in the source distribution (e.g. if it tests features added since then). To get the right version of the docs or tests, they would need to comb through version control looking for a tag that corresponds to the source distribution (assuming the developers bothered tagging the tree). Having the files available in the source distribution avoids this problem.

As for people wanting to run the test suite, I have a number of my Python modules packaged in various Linux distributions and occasionally get bug reports related to test failures in their environments. I've also used the test suites of other people's modules when I encounter a bug and want to check whether the external code is behaving as the author expects in my environment.