Is it possible (or efficient) to run a complete backend with AWS Lambda (vs say, Elastic Beanstalk)

Question

I'm relatively new to the server world, so forgive me if some of this is basic (and the first bit of text will be me explaining my logic to make sure that's not flawed). All my questions will be bolded, to make your help easier :).

I've been researching and teaching myself some of the AWS technologies, and I noticed in their Mobile Hub, if you want cloud logic, they only allow "automatic" setup of Lambda functions. After reading and researching, I've found a few resources that point towards "serverless" architecture (which the introduction of Lambda supports). In the past, it's my understanding that Elastic Beanstalk was introduced to help make server management (especially for mobile) significantly simpler.

For mobile dev, there's 2 options (obviously more, but for simplicity sake, we'll agree):

• Set up an Elastic Beanstalk that will have at least 1 instance running 24/7 and has multiple endpoints for each url
• With API Gateway, we can easily route urls to a specific Lambda functions. With this, we can handle any requests (much like setting up an Elastic Beanstalk application).

All of this leads me to believe that a complete Lambda backend would be completely possible and easy to create at a fraction of the cost of having a server running 24/7. Is that correct?

Now, assuming the above is correct, we need to determine if using Lambda really is beneficial over Elastic Beanstalk.

For simple servers, we could setup a few Lambda functions and call it a day (and it's probably much simpler and cheaper (at least for small projects) than using Elastic Beanstalk).

However, for more complex servers with more urls and database connections, things get more interesting.

These are the issues I see with using Lambda in the above situation

• Each url would have it's own API Gateway with it's own Lambda function. If any code or modules are used in multiple functions, we'd have to copy and paste that into each function.
• Managing multiple Lambda functions (and API Gateways) is simply more work than managing a single project/repo/whatever-you-wanna-call-your-code-base.
• Each function that requires a DB connection, has to connect within the function (vs say, having a constant connection within a Node.js app).

The only way (I could think of) to avoid the first 2 issues is to make one robust function that acts as a dispatch (main function takes a param from the API Gateway and determines which file to run within the Lambda function).

Are there any major points that I'm missing to determine if using Lambda over Elastic Beanstalk is worthwhile?

Answer 1

It sounds like you have it figured out already. You are correct that using Lambda instead of having a server running 24/7 can be a significant cost savings. This article makes the claim:

And it's saving some of Amazon's customers big bucks, with at least one happy Lambda customer saving 80% off their cloud bills.

You might want to look at the Serverless Framework which manages some of the pain points.

I think in time many of the pain points will go away, as Amazon adds more features to Lambda and more third party tools are built for it. I'm constantly discovering new uses for Lambda, but I'm also constantly discovering uses that seem to be a good fit at first but don't really work on Lambda, at least not yet. For example I really need some way to limit the number of instances of a Lambda function that can be running concurrently to prevent maxing out available database connections or hitting usage limits on third party APIs. I also really need Lambda functions to run inside my VPC, but that is supposed to be coming very soon.

Answer 2

As already pointed out by others there are some benefits in running Lambda vs Elastic Beanstalk or your self managed EC2 instances.

Costs

While AWS supports Auto-Scaling for Elastic Beanstalk and EC2. One always should probably run at least two instances for failover behaviour. Running two "nano" instances as a minimum for failover each instance costs you (without reserved instances discounts): $0.0059 * 24 * 30.5 = $4.31 for the VM and $0.05 * 8 GB = $0.40. So one instance is $4.81 and two instances are $9.62. However, for the AutoScaling to work you need a Load-Balancer setting you back at least $0.0225 * 24 * 30.5 = $16.47 on top of that (ignoring LCU charges). The Load-Balancer can be shared by multiple services. For this calculation I just split it up artificially by 10 and come the conclusion that one microservice using Eleastic Beanstalk or EC2 will cost you $9.62 + $1.65 = $11.27.

So how does that compare to Lambda? With Lambda you pay per call and per Gigabyte second. I'm ignoring the call costs as it's $0.20 per 1 million requests. 1 million requests are 0.4 requests per second in a month. If you have higher loads you would have to pay for the Load Balancer LCU costs as well. Lambda is priced as $0.00001667 per GB-second. Elastic Beanstalk and EC2 will both consume parts of the nanos 512 MB memory for the operating system and the container. I just assume that 256 MB are actually usable. Having two instances this would be 2 * 256 MB/1024MB * 60 * 60 * 24 * 30.5 = 1317600 GB-seconds. This amount of GB-seconds would cost you 1317600 * $0.00001667 = $21.96 dollars. While this sounds more expensive bear in mind your traffic probably doesn't distribute evenly so you would probably need more instances hence increasing the costs. With Lambda you only pay what you actually use.

Scaling

Lambda scales on demand and as already said you only pay what you actually need instead of underutilized baseline.

Unpredictable Performance

A pitfall of Lambda is the unpredictable performance. While containers will be reused they need some warm up with each new instance. The first requests will usually be slow especially when using Java. Node.js is supposed to be lighter during startup, but slower during execution. For example when a new 128 MB low memory Java instance is created and your Lambda has some libraries in it the first call can take 30 seconds or longer. The instances are freezed between requests. If the instance isn't used for a while it takes longer to wake up again. Increasing memory can reduce the warm up and wakeup time. However, a main problem can however be the access to external data-sources. As the instances are freezed between requests proper connection pooling isn't supported. If you do connection pooling anyways you can end up getting stale connections. Depending on database and driver opening and closing a connection can be quite expensive.

Other limits

As mentioned above connection pooling isn't directly supported which can be a problem for database access or access to other systems in general. If you are using frameworks for your speeding up development they might be unsuitable for usage within Lambda.

The limits Mark B mentioned are now gone. Nowadays Lambdas can run within a VPC. Even though I'm not aware of an official way to limit the number of concurrent instances, if you use an VPC you can restrict the subnet of available IPs and each Lambda will require an IP so you can limit the number of Lambda instances indirectly.

Good use-cases

If one doesn't care too much about consistent performance Lambda is cheap and scalable. A very good fit is processing of small batches.