Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

Django import export - Unable to import model with BinaryField

I'm trying to import data from a model which has nullable BinaryField. The data doesn't contain the field and I want it to be imported with a null value in the field. If the field already exists in the database for a given id, it should keep the value as it is.

I removed the field from the fields whitelist in the corresponding Resource object and added it to the exclude blacklist. However, I'm getting this error while importing - can't pickle memoryview objects.

Traceback:

Traceback (most recent call last):
File "/lib/python3.5/site-packages/import_export/resources.py", line 451, in import_row
original = deepcopy(instance)
File "/lib/python3.5/copy.py", line 182, in deepcopy
y = _reconstruct(x, rv, 1, memo)
File "/lib/python3.5/copy.py", line 297, in _reconstruct
state = deepcopy(state, memo)
File "/lib/python3.5/copy.py", line 155, in deepcopy
y = copier(x, memo)
File "/lib/python3.5/copy.py", line 243, in _deepcopy_dict
y[deepcopy(key, memo)] = deepcopy(value, memo)
File "/lib/python3.5/copy.py", line 174, in deepcopy
rv = reductor(4)
TypeError: can't pickle memoryview objects

Package versions - django==1.11, django-import-export==0.6

EDIT:

class ABC(models.Model):
    name = models.CharField('Name', max_length=128, blank=False, null=False)
    binary_field = models.BinaryField('Some name', null=True, blank=True)

class ABCResource(resources.ModelResource):
    class Meta:
        model = ABC
        fields = (
            'id',
            'name',
        )
        import_id_fields = ('id',)
        exclude = ('binary_field',)

class ABCAdmin(ImportExportModelAdmin):
    form = ABCModelForm
    list_display = (
        'id',
        'name',
    )
    exclude = ('binary_field',)
    resource_class = ABCResource

class ABCModelForm(forms.ModelForm):
    class Meta:
        model = ABC
        exclude = ['binary_field']
like image 454
Roshan Avatar asked Dec 19 '17 09:12

Roshan


1 Answers

Sadly this seems to be a problem with django-import-export module.

In Python 2.7+ and 3.5.2+, pickling memoryview objects, is permitted. Try the following code just to confirm that:

import pickle

test = memoryview(b'abc')
test.__reduce__()

The above will raise the following error:

Traceback (most recent call last):
    File "<stdin>", line 1, in <module>
    File "/usr/lib/python2.7/copy_reg.py", line 70, in _reduce_ex
    raise TypeError, "can't pickle %s objects" % base.__name__
TypeError: can't pickle memoryview objects

Apparently, in line 451 of django-import-export (version 0.6), the BinaryField is passed as a memoryview object to

original = deepcopy(instance)

which causes the error (instance contains the BinaryField as a memoryview object).
In the current version (1.0.0) this happens in line 446.

The instance retrieved/generated by the instance_loader of the module doesn't take into consideration the BinaryField.

You should probably open a ticket at django-import-export about this.

like image 162
John Moutafis Avatar answered Sep 17 '22 15:09

John Moutafis