python load zip with modules from memory

Question

So lets say I have a zip file with modules/classes inside. I then read this file - read binary ("rb") to store it into memory. How would I take this zip file in memory and load a module from it. Would I need to write an import hook for this? One cannot simply run exec on binary zip data from memory, can they?

I know its simple just to load a module from a plain zip file on disk as this is done automatically by python2.7. I , however; want to know if this is possible via memory.

Update: A lot of people are mentioning about importing the zip from disk. The issue is specifically that I want to import the zip from memory NOT disk. I obviously will read it from disk and into memory byte by byte. I want to take all these bytes from within memory that make up the zip file and use it as a regular import.

Answer 1

EDIT: Fixed the ZipImporter to work for everything (I think)

Test Data:

mkdir mypkg
vim mypkg/__init__.py
vim mypkg/test_submodule.py

__init__.py Contents:

def test():
    print("Test")

test_submodule.py Contents:

def test_submodule_func():
    print("This is a function")

Create Test Zip (on mac):

zip -r mypkg.zip mypkg
rm -r mypkg # don't want to accidentally load the directory

Special zip import in inmem_zip_importer.py:

import os
import imp
import zipfile

class ZipImporter(object):
    def __init__(self, zip_file):
        self.z = zip_file
        self.zfile = zipfile.ZipFile(self.z)
        self._paths = [x.filename for x in self.zfile.filelist]

    def _mod_to_paths(self, fullname):
        # get the python module name
        py_filename = fullname.replace(".", os.sep) + ".py"
        # get the filename if it is a package/subpackage
        py_package = fullname.replace(".", os.sep, fullname.count(".") - 1) + "/__init__.py"
        if py_filename in self._paths:
            return py_filename
        elif py_package in self._paths:
            return py_package
        else:
            return None

    def find_module(self, fullname, path):
        if self._mod_to_paths(fullname) is not None:
            return self
        return None

    def load_module(self, fullname):
        filename = self._mod_to_paths(fullname)
        if not filename in self._paths:
            raise ImportError(fullname)
        new_module = imp.new_module(fullname)
        exec self.zfile.open(filename, 'r').read() in new_module.__dict__
        new_module.__file__ = filename
        new_module.__loader__ = self
        if filename.endswith("__init__.py"):
            new_module.__path__ = [] 
            new_module.__package__ = fullname
        else:
            new_module.__package__ = fullname.rpartition('.')[0]
        return new_module

Use:

In [1]: from inmem_zip_importer import ZipImporter
In [2]: sys.meta_path.append(ZipImporter(open("mypkg.zip", "rb")))
In [3]: from mypkg import test
In [4]: test()
Test function
In [5]: from mypkg.test_submodule import test_submodule_func
In [6]: test_submodule_func()
This is a function

---

(from efel) one more thing... :

To read directly from memory one would need to do this :

f = open("mypkg.zip", "rb")

# read binary data we are now in memory
data = f.read()

f.close()  #important! close the file! we are now in memory

# at this point we can essentially delete the actual on disk zip file

# convert in memory bytes to file like object
zipbytes = io.BytesIO(data)

zipfile.ZipFile(zipbytes)