Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

Escape unescaped characters in XML with Python

I need to escape special characters in an invalid XML file which is about 5000 lines long. Here's an example of the XML that I have to deal with:

<root>
 <element>
  <name>name & surname</name>
  <mail>[email protected]</mail>
 </element>
</root>

Here the problem is the character "&" in the name. How would you escape special characters like this with a Python library? I didn't find a way to do it with BeautifulSoup.

like image 833
Jérôme Pigeot Avatar asked Feb 11 '11 17:02

Jérôme Pigeot


2 Answers

If you don't care about invalid characters in the xml you could use XML parser's recover option (see Parsing broken XML with lxml.etree.iterparse):

from lxml import etree

parser = etree.XMLParser(recover=True) # recover from bad characters.
root = etree.fromstring(broken_xml, parser=parser)
print etree.tostring(root)

Output

<root>
<element>
<name>name  surname</name>
<mail>[email protected]</mail>
</element>
</root>
like image 139
jfs Avatar answered Sep 23 '22 00:09

jfs


You're probably just wanting to do some simple regexp-ery on the HTML before throwing it into BeautifulSoup.

Even simpler, if there aren't any SGML entities (&...;) in the code, html=html.replace('&','&amp;') will do the trick.

Otherwise, try this:

x ="<html><h1>Fish & Chips & Gravy</h1><p>Fish &amp; Chips &#x0026; Gravy</p>"
import re
q=re.sub(r'&([^a-zA-Z#])',r'&amp;\1',x)
print q

Essentially the regex looks for & not followed by alpha-numeric or # characters. It won't deal with ampersands at the end of lines, but that's probably fixable.

like image 44
Dragon Avatar answered Sep 23 '22 00:09

Dragon