Fixing bad XML file (eg. unescaped & etc.) [duplicate]

Question

I got an XML file from 3rd party that I must import in my app, and XML had elements with unescaped & in inner text, and they don't wont to fix that ! So my question is what is the best way to deal with this problem ?

This XML is pretty big and that fix has to be fast, my first solution is just replace & character with ampersand but really I don't like this "solution" for obvious reasons. I don't know how to use XmlStringReader with such XML because is throws exception on such lines, so I can't use HtmlEncode on inner text. I tried to set XmlTextReader Settings.CheckCharacters to false but no result.

Here is the sample, & is in element, and in that field can be anything that can be in some company name, so my replace fix maybe don't work for some other company name, I would like to use HtmlEncode somehow, but only on inner text of course.

<komitent ID="001398">
  <sifra>001398</sifra>
  <redni_broj>001398</redni_broj>
  <naziv>LJUBICA & ŽARKO</naziv>
  <adresa1>Odvrtnica 27</adresa1>
  <adresa2></adresa2>
  <drzava>HRVATSKA</drzava>
  <grad>Zagreb</grad>
</komitent>

Answer 1

The key message below is that unless you know the exact format of the input file, and have guarantees that any deviation from XML is consistent, you can't programmatically fix without risking that your fixes will be incorrect.

Fixing it by replacing & with & is an acceptable solution if and only if:

1. There is no acceptable well-formed source of these data.

   • As @Darin Dimitrov comments, try to find a better provider, or get this provider to fix it.
   • JSON (for example) is preferable to poorly formed XML, even if you aren't using javascript.

2. This is a one off (or at least extremely infrequent) import.

   • If you have to fetch this in at runtime, then this solution will not work.

3. You can keep iterating through, devising new fixes for it, adding a solution to each problem as you come across it.

   • You will probably find that once you have "fixed" it by escaping & characters, there will be other errors.

4. You have the resources to manually check the integrity of the "fixed" data.

   • The errors you "fix" may be more subtle than you realise.

5. There are no correctly formatted entities in the document -

   • Simply replacing & with & will erroneously change " to ". You may be able to get around this, but don't be naive about how tricky it might be (entities may be defined in a DTD, may refer to a unicode code-point ...)

   • If it is a particular element that misbehaves, you could consider wrapping the content of the element with <![CDATA ]]>, but that still relies on you being able to find the start and end tags reliably.

Answer 2

Start by changing your mindset. The input is not XML, so don't call it XML. Don't even use "xml" to tag your questions about it. The fact that it isn't XML means that you can't use any XML tools with it, and you can't get any of the benefits of XML data interchange. You're dealing with a proprietary format that comes without a specification and without any tools. Treat it as you would any other proprietary format - try to discover a specification for what you are getting, and write a parser for it.