unsupported operand type(s) for <<: 'str' and 'int' while reading file

Question

I am writing unit test cases. I create a file objects and reading it, but I am getting the following error:

unsupported operand type(s) for <<: 'str' and 'int'

Method:

def uploadExamineeDetails(request, exam_id):
    try:
        upload_file = request.FILES['upload-file']
    except Exception:
        return [_('Uploaded file is required.')]
    try:
        exam = get_object_or_404_from_admin(CourseExam, request, exam_id)
        book = xlrd.open_workbook(file_contents=upload_file.read())
        # further code

My testing code:

def test_uploadExamineeDetails(self):

    self.examinee_result = os.path.join(os.path.dirname(settings.BASE_DIR),\
                                        'var/sample_files_for_testing/examinees_result_upload.xls')
    file = File(open(self.file, errors='ignore'))
    uploaded_file = InMemoryUploadedFile(file=file, field_name='upload-file', name='examinee_result.xls',
                                        content_type = 'application/vnd.ms-excel', size = file.size, charset = None)
    self.request.FILES['upload-file'] = uploaded_file
    xlrd.open_workbook(file_contents=uploaded_file.read())
    response = uploadExamineeDetails(self.request, 1)
    assert isinstance(response, tuple), 'should upload the examinee details'

data in excel file:

[{'Enrollment Number': '', 'Username': 'exam_course_manager',
 'row_num': 1, 'Obtained Score': 60.0, 'GR Number': ''},
 {'Enrollment Number': '', 'Username': 'instructor',
 'row_num': 2, 'Obtained Score': 20.0, 'GR Number': ''}]

Traceback:

       (py_3.5_dj_1.9) dikshaj@PTU16SPSD79:~/Projects/DROANA_3.0/droana/droana$ py.test  droana/apps/course_planner/tests/test_methods.py 
============================= test session starts ==============================
platform linux -- Python 3.5.2, pytest-3.0.6, py-1.4.32, pluggy-0.4.0
Django settings: droana.test_settings (from command line option)
rootdir: /home/dikshaj/Projects/DROANA_3.0/droana/droana, inifile: pytest.ini
plugins: django-3.1.2, cov-2.4.0, ipdb-0.1.dev2
collected 1 items 

droana/apps/course_planner/tests/test_methods.py F

----------- coverage: platform linux, python 3.5.2-final-0 -----------
Coverage HTML written to dir htmlcov


=================================== FAILURES ===================================
____________________ TestMethods.test_uploadExamineeDetails ____________________

self = <droana.apps.course_planner.tests.test_methods.TestMethods testMethod=test_uploadExamineeDetails>

    def test_uploadExamineeDetails(self):
        """
            Test uploadExamineeDetails method
            """
        self.examinee_result = os.path.join(os.path.dirname(settings.BASE_DIR),\
                                            'var/sample_files_for_testing/examinees_result_upload.xls')
        file = File(open(self.file, errors='ignore'))
        uploaded_file = InMemoryUploadedFile(file=file, field_name='upload-file', name='examinee_result.xls',
                                            content_type = 'application/vnd.ms-excel', size = file.size, charset = None)
        self.request.FILES['upload-file'] = uploaded_file
>       xlrd.open_workbook(file_contents=uploaded_file.read())

droana/apps/course_planner/tests/test_methods.py:100: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
/opt/myvirtual/py_3.5_dj_1.9/lib/python3.5/site-packages/xlrd/__init__.py:435: in open_workbook
    ragged_rows=ragged_rows,
/opt/myvirtual/py_3.5_dj_1.9/lib/python3.5/site-packages/xlrd/book.py:91: in open_workbook_xls
    biff_version = bk.getbof(XL_WORKBOOK_GLOBALS)
/opt/myvirtual/py_3.5_dj_1.9/lib/python3.5/site-packages/xlrd/book.py:1226: in getbof
    opcode = self.get2bytes()
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 

self = <xlrd.book.Book object at 0x7fb57b458c88>

    def get2bytes(self):
        pos = self._position
        buff_two = self.mem[pos:pos+2]
        lenbuff = len(buff_two)
        self._position += lenbuff
        if lenbuff < 2:
            return MY_EOF
        lo, hi = buff_two
>       return (BYTES_ORD(hi) << 8) | BYTES_ORD(lo)
E       TypeError: unsupported operand type(s) for <<: 'str' and 'int'

/opt/myvirtual/py_3.5_dj_1.9/lib/python3.5/site-packages/xlrd/book.py:631: TypeError

The code throwing an exception while reading this line:

xlrd.open_workbook(file_contents=upload_file.read())

This is what I have done. I create a file, stored it in directory, then open it and made an in memory object, as shown in code. According to error I get it that this error occur when you try to compare a string to int. but I am not getting why this occurring in reading file, and where.

Does anybody know what the problem is?

Answer 1

Open the file in binary mode:

file = File(open(self.file, 'rb'))

By default, files are opened with mode 'r', which reads in the file as a stream of Unicode characters (type str on Python 3). That get2bytes method in xlrd expects a stream of bytes, which is returned when opening the file in 'rb' mode.

This requirement seems to be misdocumented by xlrd, which says any "string or an mmap.mmap object or some other behave-alike object" will do for file_contents. Peering into the code, BYTES_ORD is defined as the identity function on Python 3, so when file_contents is passed into get2bytes, a str blob will result in 'c'[0] << 8, which fails, but a bytes blob will result in b'c'[0] << 8, which succeeds (because each element in a bytes blob is an int).