F# XML Parsing using XMLReader

Question

I am trying to process a particularly large XML document using F#. Since loading whole document is ruled out, I am trying to use XmlReader to serve my purpose. My first step is to define XML document as sequence of nodes.

// Read XML as a lazy sequence
let Read (s:string) = 
    let r = XmlReader.Create s
    let src = seq {
                while r.Read()
                    do
                        if XmlNodeType.Element = r.NodeType then
                            yield CreateNodeData r
                            while r.MoveToNextAttribute() 
                                do
                                    yield CreateNodeData r
                                done
                        else
                            yield CreateNodeData r
                    done
                }
    LazyList.ofSeq src

This builds XML document as a sequence of NodeData (which is created by function CreateNodeData, and not given here for simplicity). The lazy list is used for using active pattern matching.

Now the parser for the schema is constructed by defining grammar like FParsec. For example

type NodeSeq = NS of LazyList<NodeData>

(* 
Define a generic parser that takes an XML Reader and returns a singleton
list containing parsed element and unparsed parser. Failure is denoted by 
an empty list 
*)

type 'a Parser = P of ( NodeSeq -> list<'a * NodeSeq > )

And adding monadic constructs to create a monadic parser such that following code parses NodeData that matches given criteria.

let item = P ( fun inp ->
    match inp with
    | NS(LazyList.Nil)          -> [] 
    | NS(LazyList.Cons(a,b))    -> [(a,NS(b))]
    )

let nodeFilter (f: NodeData -> bool) = 
    parser {
        let! c = item
        if (f c) then
            return c
        }

Also, the choice operator (+++) is added such that p +++ q represents alternative parsers.

The problem that I am facing is parsing XML with element such as

<Node Color="Red" Transparency="90%" Material="Wood"/>

Here the attributes Color, Transparency and Material are required attributes, however, their sequence is immaterial. In addition, there can be other optional attributes. How can I create a combinatorial parser to represent

• sequence independent attribute handling
• optional attributes

This is equivalent to matching any one of following strings

xabc,xacb,xbac,xbca,xcab,xcba

How can I simplify it?

Answer 1

If you like XElement from LINQ to XML but don't want to load the entire document into memory, you can stream individual XElement instances from an XmlReader:

type XmlReader with
    /// Returns a lazy sequence of XElements matching a given name.
    member reader.StreamElements(name, ?namespaceURI) =
        let readOp =
            match namespaceURI with
            | None    -> fun () -> reader.ReadToFollowing(name)
            | Some ns -> fun () -> reader.ReadToFollowing(name, ns)
        seq {
            while readOp() do
                match XElement.ReadFrom reader with
                | :? XElement as el -> yield el
                | _ -> ()
        }

You can then query the attributes of each element, and the source order of the attributes won't matter, but you're still streaming the document rather than loading the entire thing into memory.

Answer 2

Check out the following...maybe you will find it useful http://fssnip.net/bd