Python 3.3: Convert XML to YAML

Question

I'm trying to convert XML files to YAML using Python 3.3. This is my code:

#! /etc/python3

test_filename_input = './reference-conversions/wikipedia-example.xml'
test_filename_output = 'wikipedia-example_xml_read-as-binary.yaml'

file_object = open( test_filename_input, 'rb')
data_in = file_object.read()
file_object.close()

from xml.dom.minidom import parseString
document_object = parseString( data_in)

import yaml
stream = open( test_filename_output, 'w')
yaml.dump( document_object, stream)
stream.close()

As a reference I used the XML-file from here:

<person>
  <firstName>John</firstName>
  <lastName>Smith</lastName>
  <age>25</age>
  <address>
    <streetAddress>21 2nd Street</streetAddress>
    <city>New York</city>
    <state>NY</state>
    <postalCode>10021</postalCode>
  </address>
  <phoneNumbers>
    <phoneNumber type="home">212 555-1234</phoneNumber>
    <phoneNumber type="fax">646 555-4567</phoneNumber>
  </phoneNumbers>
  <gender>
    <type>male</type>
  </gender>
</person>

which should result in something like this:

---
  firstName: John
  lastName: Smith
  age: 25
  address: 
        streetAddress: 21 2nd Street
        city: New York
        state: NY
        postalCode: 10021

  phoneNumber: 
        -  
            type: home
            number: 212 555-1234
        -  
            type: fax
            number: 646 555-4567
  gender: 
        type: male

However, the result is:

&id001 !!python/object/new:xml.dom.minidom.Document
state: !!python/tuple
- implementation: !!python/object:xml.dom.minidom.DOMImplementation {}
- _elem_info: {}
  _id_cache: {}
  _id_search_stack: null
  childNodes: !!python/object/new:xml.dom.minicompat.NodeList
    listitems:
    - &id039 !!python/object/new:xml.dom.minidom.Element
      state: !!python/tuple
      - null
      - _attrs: null
        _attrsNS: null
        childNodes: !!python/object/new:xml.dom.minicompat.NodeList
          listitems:
          - &id045 !!python/object/new:xml.dom.minidom.Text
            state: !!python/tuple
            - null
            - _data: "\n  "
              nextSibling: &id002 !!python/object/new:xml.dom.minidom.Element
                state: !!python/tuple
                - null
                - _attrs: null
                  _attrsNS: null
                  childNodes: !!python/object/new:xml.dom.minicompat.NodeList
                    listitems:
[...]

Any idea, how to get PyYAML filter out the object-stuff from xml.dom.minidom or any alternative to using xml.dom.minidom?

Thanks!

Answer 1

Here's an approach that addresses the problems with xml.dom and provides an approach to map the ambiguous situation where a node has both content and attributes or children. For the example input above it yields:

$ python3 yamlout.py person.xml
---
person:
  firstName: John
  lastName: Smith
  age: 25
  address:
    streetAddress: 21 2nd Street
    city: New York
    state: NY
    postalCode: 10021
  phoneNumbers:
    phoneNumber:
      _xml_node_content: 212 555-1234 
      type: home # Attribute
    phoneNumber:
      _xml_node_content: 646 555-4567 
      type: fax # Attribute
  gender:
    type: male

The implemention, yamlout.py:

import sys
import json
import xml.etree.ElementTree as ET

if len(sys.argv) != 2:
    sys.stderr.write("Usage: {0} <file>.xml".format(sys.argv[0]))

XML_NODE_CONTENT = '_xml_node_content'
ATTR_COMMENT = '# Attribute'
def yamlout(node, depth=0):
    if not depth:
        sys.stdout.write('---\n')
    # Nodes with both content AND nested nodes or attributes
    # have no valid yaml mapping. Add  'content' node for that case
    nodeattrs = node.attrib
    children = list(node)
    content = node.text.strip() if node.text else ''
    if content:
        if not (nodeattrs or children):
            # Write as just a name value, nothing else nested
            sys.stdout.write(
                '{indent}{tag}: {text}\n'.format(
                    indent=depth*'  ', tag=node.tag, text=content or ''))
            return
        else:
            # json.dumps for basic handling of multiline content
            nodeattrs[XML_NODE_CONTENT] = json.dumps(node.text)

    sys.stdout.write('{indent}{tag}:\n'.format(
        indent=depth*'  ', tag=node.tag))

    # Indicate difference node attributes and nested nodes
    depth += 1
    for n,v in nodeattrs.items():
        sys.stdout.write(
            '{indent}{n}: {v} {c}\n'.format(
                indent=depth*'  ', n=n, v=v,
                c=ATTR_COMMENT if n!=XML_NODE_CONTENT else ''))
    # Write nested nodes
    for child in children:
        yamlout(child, depth)

with open(sys.argv[1]) as xmlf:
    tree = ET.parse(xmlf)
    yamlout(tree.getroot())