Django: How to allow a Suspicious File Operation / copy a file

Question

I want to do a SuspiciousFileOperation which django disallows by default.

I am writing a command (to run via manage.py importfiles) to import a given directory structure on the real file system in my self written filestorage in Django.

I think, this is my relevant code:

def _handle_directory(self, directory_path, directory):
    for root, subFolders, files in os.walk(directory_path):
        for filename in files:
            self.cnt_files += 1
            new_file = File(directory=directory, filename=filename, file=os.path.join(root, filename),
                 uploader=self.uploader)
            new_file.save()

The backtrace is:

Traceback (most recent call last):
  File ".\manage.py", line 10, in <module>
    execute_from_command_line(sys.argv)
  File "C:\Python27\lib\site-packages\django\core\management\__init__.py", line 399, in execute_from_command_line
    utility.execute()
  File "C:\Python27\lib\site-packages\django\core\management\__init__.py", line 392, in execute
    self.fetch_command(subcommand).run_from_argv(self.argv)
  File "C:\Python27\lib\site-packages\django\core\management\base.py", line 242, in run_from_argv
    self.execute(*args, **options.__dict__)
  File "C:\Python27\lib\site-packages\django\core\management\base.py", line 285, in execute
    output = self.handle(*args, **options)
  File "D:\Development\github\Palco\engine\filestorage\management\commands\importfiles.py", line 53, in handle
    self._handle_directory(args[0], root)
  File "D:\Development\github\Palco\engine\filestorage\management\commands\importfiles.py", line 63, in _handle_directory
    new_file.save()
  File "D:\Development\github\Palco\engine\filestorage\models.py", line 157, in save
    self.sha512 = hashlib.sha512(self.file.read()).hexdigest()
  File "C:\Python27\lib\site-packages\django\core\files\utils.py", line 16, in <lambda>
    read = property(lambda self: self.file.read)
  File "C:\Python27\lib\site-packages\django\db\models\fields\files.py", line 46, in _get_file
    self._file = self.storage.open(self.name, 'rb')
  File "C:\Python27\lib\site-packages\django\core\files\storage.py", line 33, in open
    return self._open(name, mode)
  File "C:\Python27\lib\site-packages\django\core\files\storage.py", line 160, in _open
    return File(open(self.path(name), mode))
  File "C:\Python27\lib\site-packages\django\core\files\storage.py", line 261, in path
    raise SuspiciousFileOperation("Attempted access to '%s' denied." % name)
django.core.exceptions.SuspiciousFileOperation: Attempted access to 'D:\Temp\importme\readme.html' denied.

The full model can be found at GitHub. The full command is currently on gist.github.com available.

If you do not want to check the model: the attribute file of my File class is a FileField.

I assume, this problem happens, because I am just "linking" to the file found. But I need to copy it, huh? How can I copy the file into the file?

Answer 1

In Django, SuspiciousFileOperation can be avoid by read the file from external dir and make a tmp file within the project media then save in the appropriate file filed as below

import tempfile

file_name="file_name.pdf"
EXT_FILE_PATH = "/home/somepath/"
file_path = EXT_FILE_PATH + file_name
if exists(file_path):
    #create a named temporary file within the project base , here in media

    lf = tempfile.NamedTemporaryFile(dir='media')
    f = open(file_path, 'rb')
    lf.write(f.read())
    #doc object with file FileField.

    doc.file.save(file_name, File(lf), save=True)
    lf.close()

Answer 2

I haven't faced similar problem but related issue. I have recently upgraded Django 1.8 to 1.11.

Now I am getting the following error if try to save a file in a model having FileField field:

SuspiciousFileOperation at /api/send_report/ The joined path (/vagrant/tmp/test_file.pdf) is located outside of the base path component (/vagrant/media)

My model where I want to save the file:

class Report(BaseModel):
    file = models.FileField(max_length=200, upload_to=os.path.join(settings.REPORTS_URL, '%Y/week_%W/'))
    type = models.CharField(max_length=20, verbose_name='Type', blank=False, default='', db_index=True)

I am trying following codes to save the file from tmp folder which is not located in MEDIA_ROOT:

from django.core.files import File

filepath = "/vagrant/tmp/test_file.pdf"
file = File(open(filepath, "rb"))
report_type = "My_report_type"
report = Report.objects.create(
    file=file,
    type=report_type,
)

What I have done to solve the issue:

import os
from django.core.files import File

filepath = "/vagrant/tmp/test_file.pdf"
file = File(open(filepath, "rb"))
file_name = os.path.basename(file.name)
report_type = "My_report_type"
report = Report.objects.create(
    type=report_type,
)
report.file.save(file_name, file, save=True)

Hope it will help someone.

Answer 3

Analyzing this part of stacktrace:

File "C:\Python27\lib\site-packages\django\core\files\storage.py", line 261, in path
    raise SuspiciousFileOperation("Attempted access to '%s' denied." % name)

leads to the standard Django FileSystemStorage. It expects files to be within your MEDIA_ROOT. Your files can be anywhere in the file system, therefore this problem occurs.

You should pass file-like object instead of a path to your File model. The easiest way to achieve that would be to use Django File class, which is a wrapper around python file-like objects. See File object documentation for more details.

Update:

Ok, I am suggesting here a route taken from the docs:

from django.core.files import File as FileWrapper

def _handle_directory(self, directory_path, directory):
    for root, subFolders, files in os.walk(directory_path):
        for filename in files:
            self.cnt_files += 1
            new_file = File(
                 directory=directory, filename=filename,
                 file=os.path.join(root, filename),
                 uploader=self.uploader)
            with open(os.path.join(root, filename), 'r') as f:
                file_wrapper = FileWrapper(f)
                new_file = File(
                    directory=directory, filename=filename,
                    file=file_wrapper,
                    uploader=self.uploader)
                new_file.save()

If it works it should copy the file to the location provided by your secure_storage callable.