How to "normalize" python 3 unicode string

Question

I need to compare two strings. aa is extracted from a PDF file (using pdfminer/chardet) and bb is a keyboard input. How can I normalize first string to make a comparison?

>>> aa = "ā"
>>> bb = "ā"
>>> aa == bb
False
>>> 
>>> aa.encode('utf-8')
b'\xc4\x81'
>>> bb.encode('utf-8')
b'a\xcc\x84'

Answer 1

You normalize with unicodedata.normalize:

>>> aa = b'\xc4\x81'.decode('utf8')   # composed form
>>> bb = b'a\xcc\x84'.decode('utf8')  # decomposed form
>>> aa
'ā'
>>> bb
'ā'
>>> aa == bb
False
>>> import unicodedata as ud
>>> aa == ud.normalize('NFC',bb)  # compare composed
True
>>> ud.normalize('NFD',aa) == bb  # compare decomposed
True