Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

How to fix "unexpected keyword argument 'useChardet'" in html5lib

I'm using html5lib and after updating it to the latest version, I keep getting this error:

Traceback (most recent call last):
  File "/home/travis/build/freelawproject/juriscraper/tests/test_everything.py", line 119, in test_scrape_all_example_files
    site.parse()
  File "/home/travis/build/freelawproject/juriscraper/juriscraper/AbstractSite.py", line 95, in parse
    self.html = self._download()
  File "/home/travis/build/freelawproject/juriscraper/juriscraper/AbstractSite.py", line 384, in _download
    html_tree = self._make_html_tree(text)
  File "/home/travis/build/freelawproject/juriscraper/juriscraper/opinions/united_states/federal_appellate/ca11_u.py", line 26, in _make_html_tree
    e = html5parser.document_fromstring(text)
  File "/home/travis/virtualenv/python2.7.9/lib/python2.7/site-packages/lxml/html/html5parser.py", line 64, in document_fromstring
    return parser.parse(html, useChardet=guess_charset).getroot()
  File "/home/travis/virtualenv/python2.7.9/lib/python2.7/site-packages/html5lib/html5parser.py", line 235, in parse
    self._parse(stream, False, None, *args, **kwargs)
  File "/home/travis/virtualenv/python2.7.9/lib/python2.7/site-packages/html5lib/html5parser.py", line 85, in _parse
    self.tokenizer = _tokenizer.HTMLTokenizer(stream, parser=self, **kwargs)
  File "/home/travis/virtualenv/python2.7.9/lib/python2.7/site-packages/html5lib/_tokenizer.py", line 36, in __init__
    self.stream = HTMLInputStream(stream, **kwargs)
  File "/home/travis/virtualenv/python2.7.9/lib/python2.7/site-packages/html5lib/_inputstream.py", line 149, in HTMLInputStream
    return HTMLUnicodeInputStream(source, **kwargs)
TypeError: __init__() got an unexpected keyword argument 'useChardet'

The code I'm using is very simple:

from lxml.html import html5parser
html5parser.document_fromstring(u'<html></html')

Any ideas?

like image 339
mlissner Avatar asked Jul 25 '16 20:07

mlissner


1 Answers

Turns out that if you feed a unicode object to the document_fromstring method, it barfs. It didn't used to because this only happened when I updated my dependencies.

Anyway, the fix is easy:

html5parser.document_fromstring(u'<html></html'.encode('utf-8'))
like image 58
mlissner Avatar answered Nov 02 '22 23:11

mlissner