Independent python subprocess from AWS Lambda function

Question

I have successfully created a Lambda function (app1) that reads and writes to RDS.

My Lambda function is written in python2.7 and uploaded as a zipped package.

I created and tested the zipped package on an EC2 instance in the same VPC as my RDS and Lambda function.

Next, I added the functionality to my Lambda function to popen an independent subprocess (app2) using subprocess.popen and had app1 return while the app2 subprocess continued on its own. I tested that app1 would successfully return its handler's output while app2 continued by putting a 60 second sleep in app2 and tailed the output file of app2.

I successfully tested app1 and app2 functionality in the EC2 instance.

After uploading the new package, my app1 appears to behave exactly as expected, and returns its handler's output immediately, but the app2 functionality doesn't "appear" to instantiated, but there is no logs, errors, or output to capture from app2.

In app1, I tested that subprocess worked by performing a subprocess.check_output(['ls','-la']) prior to and after the independent subproccess.popen, and the local folder is displayed with my files. Except there isn't a app2output file created as expected.

Two questions

1. Is there something special that I am missing in AWS-Lambda concepts that is causing app2 to "fail"? By "fail" I mean not writing creating the new file and writing to it, nor creating any logs in Cloudwatch the same way app1 successfully does, nor printing out to the Lambda console like app1 does.
2. How do I catch any output (logging info and errors) for app2 in an AWS-Lambda environment?

app1.py

import subprocess
import sys
import logging
import rds_config
import pymysql
#rds settings
rds_host  = "rdshost"
name = rds_config.db_username
password = rds_config.db_password
db_name = rds_config.db_name
port = 3306

logger = logging.getLogger()
logger.setLevel(logging.INFO)

server_address = (rds_host, port)
try:
    conn = pymysql.connect(rds_host, user=name, passwd=password, db=db_name, connect_timeout=5)
except:
    logger.error("ERROR: Unexpected error: Could not connect to MySql instance.")
    sys.exit()

def handler(event, context):

    cur = conn.cursor()
    isql = "INSERT ..."
    cur.execute(isql)
    conn.commit()
    newid = cur.lastrowid
    cur.close()

    args = [str(newid),str(event['name'])]

    logger.info('ARGS: '+str(args))
    print 'pwd: '
    output = subprocess.check_output(['pwd'])
    print output
    print 'ls -la'
    output = subprocess.check_output(['ls','-l'])
    print output

    pid = subprocess.Popen([sys.executable, "app2.py"]+args, stdout=subprocess.PIPE, stderr=subprocess.PIPE, stdin=subprocess.PIPE)

    logger.info('PID: '+str(pid))
    output = subprocess.check_output(['ls','-l'])
    print output

    return "{'status':'success','newid':'"+str(newid)+"'}";

The output from "logger.info('PID: '+str(pid))" in app1.py

is like: "PID: <subprocess.Popen object at 0x7f51aba2a550>"

app2

import sys
import logging
from datetime import datetime
import time

fo = open('app2output','a+')
fo.write("starting with: "+str(sys.argv)+"\n")

logger = logging.getLogger()
logger.setLevel(logging.INFO)
logger.info("Starting with: "+str(sys.argv)+"\n")

#log accumulated processing time
t1 = datetime.now();
sleep(60)
t2 = datetime.now();
tstring = "{'t1':'"+str(t1)+"','t2':'"+str(t2)+"','args':'"+str(sys.argv[1])+"'}"
logger.info(tstring+"\n")
fo.write(tstring+"\n")
fo.close()
sys.exit()

Answer 1

The AWS Lambda environment will be terminated as soon as the handler function returns. You can't run subprocesses in the background in an AWS Lambda environment after your handler function is complete. You would need to code your Lambda function to wait for the subprocess to complete.