How to dynamically add and load entry points?

Question

I am developing a slack bot with plugins using entry points. I want to dynamically add a plugin during runtime.

I have a project with this structure:

+ ~/my_project_dir/
    + my_projects_python_code/
    + plugins/
        - plugin1.py
        - plugin2.py
        - ...
        - pluginN.py
    - setup.py
    - venv/
    - install.sh

My setup.py file looks like this:

from setuptools import setup, find_packages

setup(
    name="My_Project_plugins",
    version="1.0",
    packages=['plugins'],
    entry_points="""
        [my_project.plugins]
        plugin1 = plugins.plugin1:plugin1_class
        plugin2 = plugins.plugin2:plugin2_class
        ...
        pluginN = plugins.pluginN:pluginN_class
    """
        )

Running sudo install.sh does the following:

1. Copies the needed files to /usr/share/my_project_dir/

2. Activate virtualenv at /usr/share/my_project_dir/venv/bin/activate

3. Run: python setup.py develop

This works as expected and sets up my entry points correctly so that I can use them through the bot.

But I want to be able to add a plugin to setup.py and be able to use it while the bot is running. So I want to add a line: pluginN+1 = plugins.pluginN+1:pluginN+1_class and have pluginN+1 available to use.

What I've tried/learned:

• After /usr/share/my_project_dir/venv/bin/activate I open a Python interactive shell and iterate through pkg_resources.iter_entry_points(), which lists everything that was loaded from the initial state of setup.py (i.e. plugin1 through pluginN)

• If I add a line to setup.py and run sudo python setup.py develop and iterate again with the same Python shell, it doesn't pick up the new plugin but if I exit the shell and reopen it, the new plugin gets picked up.

• I noticed that when I install the bot, part of the output says:

  • Copying My_Project_plugins-1.0-py2.7.egg to /usr/share/my_project-dir/venv/lib/python2.7/site-packages

• When I cd /usr/share/my_project_dir/, activate my virtualenv, and run setup.py from the shell it says:

  • Creating /usr/local/lib/python2.7/dist-packages/My_Project-plugins.egg-link (link to .) My_Project-plugins 1.0 is already the active version in easy-install.pth

Answer 1

I needed to do something similar to load a dummy plugin for test purposes. This differs slightly from your use-case in that I was specifically trying to avoid needing to define the entry points in the package (as it is just test code).

I found I could dynamically insert entries into the pkg_resources data structures as follows:

import pkg_resources
# Create the fake entry point definition
ep = pkg_resources.EntryPoint.parse('dummy = dummy_module:DummyPlugin')

# Create a fake distribution to insert into the global working_set
d = pkg_resources.Distribution()

# Add the mapping to the fake EntryPoint
d._ep_map = {'namespace': {'dummy': ep}}

# Add the fake distribution to the global working_set
pkg_resources.working_set.add(d, 'dummy')

This, at run time, added an entry point called 'dummy' to 'namespace', which would be the class 'DummyPlugin' in 'dummy_module.py'.

This was determined through use of the setuptools docs, and dir() on the objects to get more info as needed.

Docs are here: http://setuptools.readthedocs.io/en/latest/pkg_resources.html

You might especially look at http://setuptools.readthedocs.io/en/latest/pkg_resources.html#locating-plugins if all you need to do is load a plugin that you have just stored to your local filesystem.