shlex: Escaping quotes in Python 3

Question

I want to split the ascii text

1 'K\^o, Suk\'e'

which is printed as the Python string

line = "1 'K\\^o, Suk\\'e'\n"

into

['1', 'K\\^o, Suk\\'e']

shlex.split(line) doesn't work, giving a ValueError("No closing quotation").

I tried adding to wordchars as follows:

>>> lexer = shlex.shlex(line)
>>> lexer.wordchars += "\\'"
>>> list(lexer)
['1', "'K\\", '^', 'o', ',', "Suk\\'e'"]

but again this is not my intention.

Without the quote, everything works as expected:

>>> line2= "1 'K\\^o, Suk\\e'\n"
>>> shlex.split(line2)
['1', 'K\\^o, Suk\\e']

How can this be fixed?

Edit: The closest I've come is:

>>> lexer = shlex.shlex(line)
>>> lexer.wordchars += "\\'"
>>> lexer.wordchars += " "
>>> lexer.wordchars += "\\^"
>>> lexer.wordchars += ","
>>> list(lexer)
['1', "'K\\^o,", "Suk\\'e'"]

which is still not right.

Edit: The example string is somewhat misleading. It should also be able to split, say

>>> l1 = "C12 H2 'Hello friend'"

shlex gets l1 no problem.

Answer 1

You can use posix mode and escapedquotes to '" instead of just "

>>> line = "1 'K\\^o, Suk\\'e'\n"
>>> lexer = shlex.shlex(line, posix=True)
>>> lexer.escapedquotes = "'\""
>>> list(lexer)
['1', "K\\^o, Suk'e"]