How stable is s3fs to mount an Amazon S3 bucket as a local directory in linux? Is it recommended/stable for high demand production environments?
Are there any better/similar solutions?
Update: Would it be better to use EBS and to mount it via NFS to all other AMIs?
s3fs is stable and is being used in number of production environments, e.g., rsync backup to s3.
A S3 bucket can be mounted in a AWS instance as a file system known as S3fs.
There's a good article on s3fs here, which after reading I resorted to an EBS Share.
It highlights a few important considerations when using s3fs, namely related to the inherent limitations of S3:
It therefore depends on what you are storing whether s3fs is a feasible option. If you're storing say, photos, where you want to write an entire file or read an entire file never incrementally change a file, then its fine, although one may ask, if you're doing this, then why not just use S3's API Directly?
If you're talking about appliation data, (say database files, logging files) where you want to make small incremental change then its a definite no - S3 Just doesn't work that way you can't incrementally change a file.
The article mentioned above does talk about a similar application - s3backer - which gets around the performance issues by implementing a virtual filesystem over S3. This gets around the performance issues but itself has a few issues of its own:
I resorted to EBS Mounted Drived shared from an EC2 instance. But you should know that although the most performant option it has one big problem An EBS Mounted NFS Share has its own problems - a single point of failure; if the machine that's sharing the EBS Volume goes down then you lose access on all machines which access the share.
This is a risk I was able to live with and was the option I chose in the end. I hope this helps.
If you love us? You can donate to us via Paypal or buy me a coffee so we can maintain and grow! Thank you!
Donate Us With