Inform browser clients when Lambda function is done using Amazon SQS

Question

In my scenario I'm trying to implement server less backend that runs pretty long time consuming calculations. This calculations is managed by Lambda that refers to some external API.

In oder to request this I'm using Amazon API Gateway which has 10 seconds execution limitation. However Lambda runs about 100 seconds.

To avoid this limitation I'm using 2nd Lambda function to execute this time consuming calculation & report that calculation is started.

I looks very similar to this:

var AWS = require('aws-sdk');
var colors = require('colors');

var functionName = 'really-long'

var lambda = new AWS.Lambda({apiVersion: '2015-03-31'});

var params = {
  FunctionName: functionName, 
  InvocationType: 'Event'
};


lambda.invoke(params, function(err, data) {
  if (err) console.log(err, err.stack); // an error occurred
  else     console.log(functionName.green + " was successfully executed and returned:\n" + JSON.stringify(data, null, 2).gray);           // successful response
});

console.log("All done!".rainbow);

This code is executed over AWS API Gateway by thousands of clients browsers independently.

To inform each particular client that his Lambda function execution was successfully done I'v planed to use AWS SQS (because of long polling and some other useful functionalities out of the box).

So my question is:

How can I determine on the client which message in the queue belongs to this particular client? Or should I iterate over all queue to find proper messages by some request ID parameter in every client browser? I guess that this method will be inefficient when 1000 client will be simultaneously waiting for their results.

---

I do understand that I can write results to DynamoDB for example and periodically poll DB for the result via some homemade API. But is there any elegant solution to notify browser based client about completion of execution of time consuming Lambda function based on some Amazon PaaS solution?

Answer 1

Honestly the DynamoDB route is probably your best bet. You can generate a uuid in the first Lambda function executed by the API Gateway. Pass that uuid to the long-running Lambda function. Before the second function completes have it write to a DynamoDB table with two columns: uuid and result.

The API Gateway responds to the client with the uuid it generated. The client then long-polls with a getItem request against your DynamoDB table (either via the aws-sdk directly or through another API Gateway request). Once it responds successfully, remove said item from the DynamoDB table.

Answer 2

The context object of the lambda function will have the AWS request ID returned to the client that invoked the Lambda function.

So, client will have the lambda request ID of Lambda 1, Lambda 1 Context object will have the same request Id (irrespective of lambda retries, request ID remains same). So pass this request ID to Lambda 2 there by actual request ID is chained till the end.

Polling using the request id from client is fairly easy on any data store like dynamodb.