Goal
I'm trying to write a zip file to the /tmp folder in a python aws lambda, so I can extract manipulate before zipping, and placing it in s3 bucket.
Problem
Os Errno30 Read Only FileSystem
This code was tested locally on my computer to make sure the file would write to my working directory before I uploaded it to aws. This is the code i'm trying to use.
file = downloadFile() #This is api call that returns binary zip object
newFile = open('/tmp/myZip.zip','wb')
newFile.write(file)
extractAll('/tmp/myZip.zip')
here is the code that is trying to extract the zip file
def extractAll(self,source):
with zipfile.ZipFile(source, 'r') as archive:
archive.extractall()
here is the trace
[Errno 30] Read-only file system: '/var/task/test-deploy': OSError
Traceback (most recent call last):
File "/var/task/web.py", line 31, in web
performAction(bb, eventBody)
File "/var/task/src/api/web.py", line 42, in performAction
zipHelper.extractAll('/tmp/myZip.zip')
File "/var/task/src/shared/utils/zipfilehelper.py", line 24, in extractAll
archive.extractall()
File "/var/lang/lib/python3.6/zipfile.py", line 1491, in extractall
self.extract(zipinfo, path, pwd)
File "/var/lang/lib/python3.6/zipfile.py", line 1479, in extract
return self._extract_member(member, path, pwd)
File "/var/lang/lib/python3.6/zipfile.py", line 1538, in _extract_member
os.mkdir(targetpath)
OSError: [Errno 30] Read-only file system: '/var/task/test-deploy'
The /tmp directory provides a transient cache of data between invocations on the same Lambda worker instance. It's useful in cases where you need to unzip files or clone a Git repository as part of a Lambda invocation.
Just open the file as you would do normally: with open('/tmp/'+ filename, 'rb') as file: ... This works all the time for me. Have you tried this and experienced any issues?
During the runtime of your lambda function, you may want to save some temporary files, to be accessed later during the function execution. Luckily, AWS provides 512 MB of file system ('/tmp' folder) that can be used by us for this purpose. It is equivalent to your local hard disk, providing fast throughput.
The /tmp directory is backed by an Amazon EC2 instance store and is encrypted at-rest. Customers that want to persist data to the file system outside of the execution environment should consider using Lambda's integration with Amazon Elastic File System (Amazon EFS).
extractAll()
will extract files in the current directory, which is /var/task/test-deploy
in your case.
You need to use os.chdir()
to change the current directory:
import os, zipfile
os.chdir('/tmp')
with zipfile.ZipFile(source, 'r') as archive:
archive.extractall()
Even better, you can create a temporary directory and extract the files there, to avoid polluting /tmp
:
import os, tempfile, zipfile
with tempfile.TemporaryDirectory() as tmpdir:
os.chdir(tmpdir)
with zipfile.ZipFile(source, 'r') as archive:
archive.extractall()
If you want to restore the current working directory after extracting the file, consider using this decorator:
import os, tempfile, zipfile, contextlib
@contextlib.context_manager
def temporary_work_dir():
old_work_dir = os.getcwd()
with tempfile.TemporaryDirectory() as tmp_dir:
os.chdir(tmp_dir)
try:
yield
finally:
os.chdir(old_work_dir)
with temporary_work_dir():
with zipfile.ZipFile(source, 'r') as archive:
archive.extractall()
If you love us? You can donate to us via Paypal or buy me a coffee so we can maintain and grow! Thank you!
Donate Us With