Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

Mount S3 File System in AWS Lambda?

Do you know if it possible mounting a File System over AWS S3 to increase AWS Lambda storage?

like image 402
Adrian Avatar asked Jun 22 '18 07:06

Adrian


People also ask

Can I use S3 as a file system?

Advantages of Mounting Amazon S3 as a File System Mounting an Amazon S3 bucket as a file system means that you can use all your existing tools and applications to interact with the Amazon S3 bucket to perform read/write operations on files and folders.

Which AWS service acts as a file system mount on S3?

Which native AWS service will act as a file system mounted on an S3 bucket? AWS Storage Gateway. The Storage Gateway service is primarily used for attaching infrastructure located in a Data centre to the AWS Storage infrastructure.

Can AWS Lambda read from S3?

S3 Object Lambda works with your existing applications and uses AWS Lambda functions to automatically process and transform your data as it is being retrieved from S3. The Lambda function is invoked inline with a standard S3 GET request, so you don't need to change your application code.


2 Answers

You cannot mount Amazon S3 as a filesystem for use with AWS Lambda.

AWS Lambda functions are designed to be short-running scripts that respond to an event in your environment (eg data coming into a Kinesis stream, a file being created in Amazon S3). They typically run only for a few seconds.

While you haven't described your use-case, something that requires heavy use of a filesystem is not a typically good use of a Lambda function.

You can, of course, retrieve and create objects in Amazon S3 via SDK calls from the Lambda function. This is a very fast and efficient way to load/store data from Lambda.

like image 136
John Rotenstein Avatar answered Sep 24 '22 12:09

John Rotenstein


The storage space provided with lambda is actually Ephemeral disk space that can be used for temporary storage during lambda execution only. It is limited to 512 Mb and can't be increased.

If your use-case needs more space for lambda than you can store and retrieve objects directly from s3 . It's highly fast and efficient. Moreover , s3 doesn't provide a file system , it is only object based storage and every folder , subfolder, and every file represents an object in s3 which virtually seems like a hierarchical structure but in actual it's not.

While the temporary disk space in lambda has a hierarchical file structure

like image 30
Mausam Sharma Avatar answered Sep 25 '22 12:09

Mausam Sharma