Python http.client.Incomplete Read(0 bytes read) error

Question

I have seen this error on the forum and read through the responses yet I still don't understand what it is or how to address it. I'm scraping data from the internet from 16k links, my script scrapes similar information from each link and writes it to a .csv some of the date gets written before this error.

Traceback (most recent call last):
 File "/usr/local/Cellar/python3/3.5.2_3/Frameworks/Python.framework/Versions/3.5/lib/python3.5/http/client.py", line 541, in _get_chunk_left
   chunk_left = self._read_next_chunk_size()
 File "/usr/local/Cellar/python3/3.5.2_3/Frameworks/Python.framework/Versions/3.5/lib/python3.5/http/client.py", line 508, in _read_next_chunk_size
   return int(line, 16)
ValueError: invalid literal for int() with base 16: b''

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
 File "/usr/local/Cellar/python3/3.5.2_3/Frameworks/Python.framework/Versions/3.5/lib/python3.5/http/client.py", line 558, in _readall_chunked
   chunk_left = self._get_chunk_left()
File "/usr/local/Cellar/python3/3.5.2_3/Frameworks/Python.framework/Versions/3.5/lib/python3.5/http/client.py", line 543, in _get_chunk_left
   raise IncompleteRead(b'')
http.client.IncompleteRead: IncompleteRead(0 bytes read)

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
File "MoviesToDb.py", line 91, in <module>
html = r.read()
File "/usr/local/Cellar/python3/3.5.2_3/Frameworks/Python.framework/Versions/3.5/lib/python3.5/http/client.py", line 455, in read
   return self._readall_chunked()
File "/usr/local/Cellar/python3/3.5.2_3/Frameworks/Python.framework/Versions/3.5/lib/python3.5/http/client.py", line 565, in _readall_chunked
   raise IncompleteRead(b''.join(value))
http.client.IncompleteRead: IncompleteRead(17891 bytes read)

I would like to know:
1) What does this error mean?
2) How do I prevent it?

Answer 1

try to import :

from http.client import IncompleteRead

and add this in your script :

except IncompleteRead:
    # Oh well, reconnect and keep trucking
        continue