Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

Catch error in a for loop python

I have a for loop on an avro data reader object

for i in reader:
    print i

then I got a unicode decode error in the for statement so I wanted to ignore that particular record. So I did this

try:
    for i in reader:
        print i
except:
    pass

but it does not continue further. How can I overcome this problem

Edit: Error trace added

    Traceback (most recent call last):
  File "modify.py", line 22, in <module>
    for record in reader:
  File "/usr/lib/python2.6/site-packages/avro-1.7.7-py2.6.egg/avro/datafile.py", line 362, in next
    datum = self.datum_reader.read(self.datum_decoder) 
  File "/usr/lib/python2.6/site-packages/avro-1.7.7-py2.6.egg/avro/io.py", line 445, in read
    return self.read_data(self.writers_schema, self.readers_schema, decoder)
  File "/usr/lib/python2.6/site-packages/avro-1.7.7-py2.6.egg/avro/io.py", line 490, in read_data
    return self.read_record(writers_schema, readers_schema, decoder)
  File "/usr/lib/python2.6/site-packages/avro-1.7.7-py2.6.egg/avro/io.py", line 690, in read_record
    field_val = self.read_data(field.type, readers_field.type, decoder)
  File "/usr/lib/python2.6/site-packages/avro-1.7.7-py2.6.egg/avro/io.py", line 468, in read_data
    return decoder.read_utf8()
  File "/usr/lib/python2.6/site-packages/avro-1.7.7-py2.6.egg/avro/io.py", line 233, in read_utf8
    return unicode(self.read_bytes(), "utf-8")
UnicodeDecodeError: 'utf8' codec can't decode byte 0xb4 in position 14: invalid start byte

could it be due to the fact that the file was corrupted?

Edit2: As per suggestion in answers to go through iterobject I modified code and got this error

    Traceback (most recent call last):
  File "modify.py", line 28, in <module>
    print next(iterobject)["filepath"]
  File "/usr/lib/python2.6/site-packages/avro-1.7.7-py2.6.egg/avro/datafile.py", line 362, in next
    datum = self.datum_reader.read(self.datum_decoder) 
  File "/usr/lib/python2.6/site-packages/avro-1.7.7-py2.6.egg/avro/io.py", line 445, in read
    return self.read_data(self.writers_schema, self.readers_schema, decoder)
  File "/usr/lib/python2.6/site-packages/avro-1.7.7-py2.6.egg/avro/io.py", line 490, in read_data
    return self.read_record(writers_schema, readers_schema, decoder)
  File "/usr/lib/python2.6/site-packages/avro-1.7.7-py2.6.egg/avro/io.py", line 690, in read_record
    field_val = self.read_data(field.type, readers_field.type, decoder)
  File "/usr/lib/python2.6/site-packages/avro-1.7.7-py2.6.egg/avro/io.py", line 468, in read_data
    return decoder.read_utf8()
  File "/usr/lib/python2.6/site-packages/avro-1.7.7-py2.6.egg/avro/io.py", line 233, in read_utf8
    return unicode(self.read_bytes(), "utf-8")
  File "/usr/lib/python2.6/site-packages/avro-1.7.7-py2.6.egg/avro/io.py", line 226, in read_bytes
    return self.read(self.read_long())
  File "/usr/lib/python2.6/site-packages/avro-1.7.7-py2.6.egg/avro/io.py", line 184, in read_long
    b = ord(self.read(1))
TypeError: ord() expected a character, but string of length 0 found
like image 232
iLoveCamelCase Avatar asked Nov 02 '25 21:11

iLoveCamelCase


1 Answers

If your error is in for i in. Then try this, it will skip element in iterator if UnicodeDecodeError occurs.

iterobject = iter(reader)
while iterobject:
    try:
        print(next(iterobject))
    except StopIteration:
        break
    except UnicodeDecodeError:
        pass
like image 158
itzMEonTV Avatar answered Nov 04 '25 10:11

itzMEonTV



Donate For Us

If you love us? You can donate to us via Paypal or buy me a coffee so we can maintain and grow! Thank you!