Python libraries on Web Job

Question

My goal is to run a Python script that uses Anaconda libraries (such as Pandas) on Azure WebJob but can't seem to figure out how to load the libraries.

I start out just by testing a simple Azure blob to blob file copy which works when run locally but hit into an error "ImportError: No module named 'azure'" when ran in WebJob.

example code:

    from azure.storage.blob import BlockBlobService
    blobAccountName = <name>
    blobStorageKey = <key>  
    containerName = <containername>
    blobService = BlockBlobService(account_name=blobAccountName,
    account_key=blobStorageKey)
    blobService.set_container_acl(containerName)   
    b = blobService.get_blob_to_bytes(containerName, 'file.csv')
    blobService.create_blob_from_bytes(containerName, 'file.csv', b.content)

I can't even get Azure SDK libraries to run. let alone Anaconda's

How do I run a python script that requires external libraries such as Anaconda (and even Azure SDK). How do I "pip install" these stuff for a WebJob?

Answer 1

It seems like you've kown about deployment of Azure WebJobs, I offer the below steps for you to show how to load external libraries in python scripts.

Step 1 : Use the virtualenv component to create an independent python runtime environment in your system.Please install it first with command pip install virtualenv if you don't have it.

If you installed it successfully ,you could see it in your python/Scripts file.

Step2 : Run the commad to create independent python runtime environment.

Step 3: Then go into the created directory's Scripts folder and activate it (this step is important , don't miss it)

Please don't close this command window and use pip install <your libraryname> to download external libraries in this command window.

Step 4：Keep the Sample.py uniformly compressed into a folder with the libs packages in the Libs/site-packages folder that you rely on.

Step 5: Create webjob in Web app service and upload the zip file,then you could execute your Web Job and check the log

You could also refer to the SO thread :Options for running Python scripts in Azure

In addition, if you want to use the modules in Anaconda, please download them separately. There is no need to download the entire library.

Hope it helps you.

Answer 2

You can point your Azure WebJob to your main WebApp environment (and thus its real site packages). This allows you to use the newest fastest version of Python supported by the WebApp (right now mine is 364x64), much better than 3.4 or 2.7 in x86. Another huge benefit is then you don't have to maintain an additional set of packages that are statically held in a file somewhere (this gave me a lot of trouble with dynamic libraries with crazy dependencies such as psycopg2 and pandas).

HOW: In your WebJobs files, set up a .cmd file that runs your run.py, and in that .cmd file, you can just have one line of code like this:

D:\home\python364x64\python.exe run.py

That's it!

Azure WebJobs looks at .cmd files first, then run.py and others. See this link for an official MS post on this method: https://blogs.msdn.microsoft.com/azureossds/2016/12/09/running-python-webjob-on-azure-app-services-using-non-default-python-version/