Deploy Python executable to Azure Service Fabric

Question

I am looking for helpful solutions to deploy Python executable to Azure Service Fabric

There is documentation for node js here but I could not find anything related to python

Answer 1

A tip: Save yourself some time and try to get this running locally first. My experience is that plenty of things can go wrong, one of the hardest ones to determine was execution rights on the nodes.

Setup

Steps needed to deploy Python to Service Fabric:

1. Make sure that Python is installed on the SF Nodes (with the packages you need). There are two ways to do this:
   • Use Remote Desktop to navigate to the nodes using your endpoint "sf_cluster.westeurope.cloudapp.azure.com:node_port" where node_port is (for me) 3389-3393. And then install python manually by downloading a installer from Python.org.
   • Create a python install script that you call in the Guest Executable SetupEntryPoint, more on this further down.
   In any case you need to get Python unto the Nodes and Make sure to add Python and Python/Scripts to the node paths.
2. Create a Service Fabric Application with a Guest Executable Service.
   1. Point it to your python code folder
   2. Select the startup file to run - leave this blank
   3. Select CodePackage as your Working Folder
   4. Press Ok.
   You now have a service fabric application that will copy your code to your service fabric cluster but do nothing. (actually it will probably yell at you for not defining a Endpoint, but that's next)

Now we get to some hacking. Service Fabric will not run Python.exe in the same way that it runs node.exe in the examples from Microsoft. So what you need to do is create a run.cmd script that basically does this:

python.exe main.py

use that as your EntryPoint.

Now your EntryPoint in ServiceManifest.xml looks like this:

...
    <EntryPoint>
        <ExeHost>
            <Program>run.cmd</Program>
            <Arguments></Arguments>
            <WorkingFolder>CodePackage</WorkingFolder>
            <ConsoleRedirection FileRetentionCount="5" FileMaxSizeInKb="2048"/> //use this to get logging (very helpful ;)
        </ExeHost>
    </EntryPoint>
</CodePackage>

if you want a SetupEntryPoint to install python for you you could do this:

<SetupEntryPoint>
  <ExeHost>
    <Program>Setup\setup.bat</Program>
    <WorkingFolder>CodePackage</WorkingFolder>
  </ExeHost>
</SetupEntryPoint>

My setup.bat looks like this (had some issues with the path):

C:\Windows\System32\WindowsPowerShell\v1.0\powershell.exe -ExecutionPolicy Bypass -Command ".\Setup\pythonInstall.ps1"

The code in my solution is copied into this structure:

MyGuestExecutablePkg\
    |
    - Code\
        - run.cmd
        - main.py
        - Setup\
            - setup.bat
        - ....
    - Config\
        - Settings.xml
    - ServiceManifest.xml
ApplicationManifest.xml

I hope this helps someone but do read the Guest Executable documentation for anything I've missed. 1

Policies

Some things need elivated priviledges on the Service Fabric Nodes and this is how you elivate your service in order for it to run .cmd, .bat, Python.exe etc. The following edits are from ApplicationManifest.xml

....
<ServiceManifestImport>
    <ServiceManifestRef ServiceManifestName="MyGuestExecutablePkg" ServiceManifestVersion="1.0.0" />
    <ConfigOverrides />
    <Policies>
      <RunAsPolicy CodePackageRef="Code" UserRef="SetupAdminUser" EntryPointType="Setup" />
      <RunAsPolicy CodePackageRef="Code" UserRef="SetupAdminUser" EntryPointType="Main" />
    </Policies>
  </ServiceManifestImport>
....

The first policy is to elevate the SetupEntryPoint while the second is to elicate the EntryPoint, aka. your service. Make sure you define the Principals at the bottom of ApplicationManifest.xml.

</DefaultServices>
  <Principals>
    <Users>
      <User Name="SetupAdminUser">
        <MemberOf>
          <SystemGroup Name="Administrators" />
        </MemberOf>
      </User>
    </Users>
  </Principals>
</ApplicationManifest>

Final thoughts

I've run my service as a Flask Api as this was ridiculously easy. So "all" Service Fabric does is start the Flask api and then make sure it is restarted if it crashes. This makes it easy to test and verify which part of my deployment was crashing and if it was up and running by performing

netstat -na | find "5000"

on the nodes to verify that the api was indeed up and running.

After all this you, of course, need to open up the ports in the load balanser for your service fabric cluster in the Azure Portal.

Answer 2

This shouldn't be any different for Python, as long as you either install the Python requirements directly on the nodes through a script extension (for Azure). Or call the Python installer as the SetupEntryPoint.