Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

How to structure a simple command line based python project

Tags:

python

I wrote a command line app in a single file app_name.py and it works. Now I am breaking it into different .py files for easier management and readability. I have placed all these py files in a src/ folder as I see that a lot on github.

app_name/
    src/
        file1.py
        file2.py
        cli.py
        __init__.py

I have placed all the imports in my __init__.py The relative imports like from .file1 import function1 are not in __init__ and are placed within the individual files where there are needed. For example,

#!usr/bin/env python
#__init__.py

import os
import argparse
#etc...

run_cli()

In cli.py, I have

from .file1 import function1

def run_cli(): pass

if __name__ == '__main__':
    run_cli()

This is because when I use, on the actual command line app_name <arguments> then the __name__ isnt main, so I call run_cli() in __init__.py. Although this doesn't look right, as I will have to call src not app_name

like image 420
yayu Avatar asked Oct 24 '14 21:10

yayu


1 Answers

I think you may be confusing two things. Having a src directory in your source distribution is a reasonably common, idiomatic thing to do; having that in the installed package or app, not so much.

There are two basic ways to structure this.


First, you can build a Python distribution that installs a package into site-packages, and installs a script into wherever Python scripts go on the PATH. The Python Packaging User Guide covers this in its tutorial on building and distributing packages. See the Layout and Scripts sections, and the samples linked from there.

This will usually give you an installed layout something like this:

<...somewhere system/user/venv .../lib/pythonX.Y/site-packages>
    app_name/
        file1.py
        file2.py
        cli.py
        __init__.py

<...somewhere.../bin/>
    app_name

But, depending on how the user has chosen to install things, it could instead be an egg, or a zipped package, or a wheel, or anything else. You don't care, as long as your code works. In particular, your code can assume that app_name is an importable package.

Ideally, the app_name script on your path is an "entry-point" script (pip itself may be a good example on your system), ideally one built on the fly at install time. With setuptools, You can just specify which package it should import and which function it should call in that package, it will do everything else—making sure to shebang the Python actually used at install time, figuring out how to pkg_resources up the package and add it to the sys.path (not needed by default, but if you don't want it to be importable, you can make that work), and so on.

As mentioned in a comment from the original asker, python-commandline-bootstrap might help you put this solution together more quickly.


The alternative is to keep everything out of site-packages, and make the package specific to your app. In this case, what you basically want to do is:

  • Install a directory that contains both the package (either as a directory, or zipped up) and the wrapper script.
  • In the wrapper script, add dirname(abspath(argv[0])) to sys.path before the import.
  • Symlink the wrapper script into some place on the user's PATH.

In this case, you have to write the wrapper script manually, but then you don't need anything fancy this way.

But often, you don't really want an application to depend on the user having some specific version and setup of Python. You may want to use a tool like pyInstaller, py2exe, py2app, cx_Freeze, zc.buildout, etc. that does all of the above and more. They all do different things, all the way up to the extreme of building a Mac .app bundle directory with a custom, standalone, stripped Python framework and stdlib and a wrapper executable that embeds the framework.


Either way, you really don't want to call the package directory src. That means the package itself will be named src, which is not a great name. If your app is called spamifier, you want to see spamifier in your tracebacks, not src, right?

like image 172
abarnert Avatar answered Sep 21 '22 02:09

abarnert