Suds generates empty elements; how to remove them?

Question

[Major Edit based on experience since 1st post two days ago.]

I am building a Python SOAP/XML script using Suds, but am struggling to get the code to generate SOAP/XML that is acceptable to the server. I had thought that the issue was that Suds was not generating prefixes for inner elements, but subsequently it turns out that the lack of prefixes (see Sh-Data and inner elements) is not an issue, as the Sh-Data and MetaSwitchData elements declare appropriate namespaces (see below).

<SOAP-ENV:Envelope xmlns:ns3="http://www.metaswitch.com/ems/soap/sh" xmlns:ns0="http://www.metaswitch.com/ems/soap/sh/userdata" xmlns:ns1="http://www.metaswitch.com/ems/soap/sh/servicedata" xmlns:ns2="http://schemas.xmlsoap.org/soap/envelope/" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xmlns:SOAP-ENV="http://schemas.xmlsoap.org/soap/envelope/">
   <SOAP-ENV:Header/>
   <ns2:Body>
      <ns3:ShUpdate>
         <ns3:UserIdentity>Meribel/TD Test Sub Gateway 3</ns3:UserIdentity>
         <ns3:DataReference>0</ns3:DataReference>
         <ns3:UserData>
            <Sh-Data xmlns="http://www.metaswitch.com/ems/soap/sh/userdata">
               <RepositoryData>
                  <ServiceIndication>Meta_SubG_BaseInformation</ServiceIndication>
                  <SequenceNumber>0</SequenceNumber>
                  <ServiceData>
                     <MetaSwitchData xmlns="http://www.metaswitch.com/ems/soap/sh/servicedata" IgnoreSequenceNumber="False" MetaSwitchVersion="?">
                        <Meta_SubG_BaseInformation Action="apply">
                           <NetworkElementName>Meribel</NetworkElementName>
                           <Description>TD Test Sub Gateway 3</Description>
                           <DomainName>test.datcon.co.uk</DomainName>
                           <MediaGatewayModel>Cisco ATA</MediaGatewayModel>
                           <CallFeatureServerControlStatus/>
                           <CallAgentControlStatus/>
                           <UseStaticNATMapping/>
                           <AuthenticationRequired/>
                           <ProviderStatus/>
                           <DeactivationMode/>
                        </Meta_SubG_BaseInformation>
                     </MetaSwitchData>
                  </ServiceData>
               </RepositoryData>
            </Sh-Data>
         </ns3:UserData>
         <ns3:OriginHost>[email protected]?clientVersion=7.3</ns3:OriginHost>
      </ns3:ShUpdate>
   </ns2:Body>
</SOAP-ENV:Envelope>

But this still fails. The issue is that Suds generates empty elements for optional elements (marked as Mandatory = No in the WSDL). But the server requires that an optional element is either present with a sensible value or absent, and I get the following error (because the <CallFeatureServerControlStatus/> element is not one of the allowable values.

The user data provided did not validate against the MetaSwitch XML Schema for user data.
Details: cvc-enumeration-valid: Value '' is not facet-valid with respect to enumeration '[Controlling, Abandoned, Cautiously controlling]'. It must be a value from the enumeration.

If I take the generated SOAP/XML into SOAPUI and delete the empty elements, the request works just fine.

Is there a way to get Suds to either not generate empty elements for optional fields, or for me to remove them in code afterwards?

Major Update

I have solved this problem (which I've seen elsewhere) but in a pretty inelegant way. So I am posting my current solution in the hope that a) it helps others and/or b) someone can suggest a better work-around.

It turns out that the problem was not that Suds generates empty elements for optional elements (marked as Mandatory = No in the WSDL). But rather that that Suds generates empty elements for optional complex elements. For example the following Meta_SubG_BaseInformation elements are simple elements and Suds does not generate anything for them in the SOAP/XML.

<xs:element name="CMTS" type="xs:string" minOccurs="0">
    <xs:annotation>
        <xs:documentation>
            <d:DisplayName firstVersion="5.0" lastVersion="7.4">CMTS</d:DisplayName>
            <d:ValidFrom>5.0</d:ValidFrom>
            <d:ValidTo>7.4</d:ValidTo>
            <d:Type firstVersion="5.0" lastVersion="7.4">String</d:Type>
            <d:BaseAccess firstVersion="5.0" lastVersion="7.4">RWRWRW</d:BaseAccess>
            <d:Mandatory firstVersion="5.0" lastVersion="7.4">No</d:Mandatory>
            <d:MaxLength firstVersion="5.0" lastVersion="7.4">1024</d:MaxLength>
        </xs:documentation>
    </xs:annotation>
</xs:element>

<xs:element name="TAGLocation" type="xs:string" minOccurs="0">
    <xs:annotation>
        <xs:documentation>
            <d:DisplayName>Preferred location of Trunk Gateway</d:DisplayName>
            <d:Type>String</d:Type>
            <d:BaseAccess>RWRWRW</d:BaseAccess>
            <d:Mandatory>No</d:Mandatory>
            <d:DefaultValue>None</d:DefaultValue>
            <d:MaxLength>1024</d:MaxLength>
        </xs:documentation>
    </xs:annotation>
</xs:element>

In contrast the following Meta_SubG_BaseInformation element is a complex element, and even when it is optional and my code does not assign a value to it, it ends up in the generated SOAP/XML.

<xs:element name="ProviderStatus" type="tMeta_SubG_BaseInformation_ProviderStatus" minOccurs="0">
    <xs:annotation>
        <xs:documentation>
            <d:DisplayName>Provider status</d:DisplayName>
            <d:Type>Choice of values</d:Type>
            <d:BaseAccess>R-R-R-</d:BaseAccess>
            <d:Mandatory>No</d:Mandatory>
            <d:Values>
                <d:Value>Unavailable</d:Value>
                <d:Value>Available</d:Value>
                <d:Value>Inactive</d:Value>
                <d:Value>Active</d:Value>
                <d:Value>Out of service</d:Value>
                <d:Value>Quiescing</d:Value>
                <d:Value>Unconfigured</d:Value>
                <d:Value>Pending available</d:Value>
            </d:Values>
        </xs:documentation>
    </xs:annotation>
</xs:element>

Suds generates the following for ProviderStatus which (as stated above) upsets my server.

<ProviderStatus/>

The work-around is to set all Meta_SubG_BaseInformation elements to None after creating the parent element, and before assigning values, as in the following. This is superfluous for the simple elements, but does ensure that non-assigned complex elements do not result in generated SOAP/XML.

subGatewayBaseInformation = client.factory.create('ns1:Meta_SubG_BaseInformation')
for (el) in subGatewayBaseInformation:
  subGatewayBaseInformation.__setitem__(el[0], None)
subGatewayBaseInformation._Action            = 'apply'
subGatewayBaseInformation.NetworkElementName = 'Meribel'
etc...

This results in Suds generating SOAP/XML without empty elements, which is acceptable to my server.

But does anyone know of a cleaner way to achieve the same effect?

Solution below is based on answers / comments from both dusan and Roland Smith below.

This solution uses a Suds MessagePlugin to prune "empty" XML of the form <SubscriberType/> before Suds puts the request on the wire. We only need to prune on ShUpdates (where we are updating data on the server), and the logic (especially the indexing down into the children to get the service indication element list) is very specific to the WSDL. It would not work for different WSDL.

class MyPlugin(MessagePlugin):
  def marshalled(self, context):
    pruned = []
    req = context.envelope.children[1].children[0]
    if (req.name == 'ShUpdate'):
      si = req.children[2].children[0].children[0].children[2].children[0].children[0]
      for el in si.children:
        if re.match('<[a-zA-Z0-9]*/>', Element.plain(el)):
          pruned.append(el)
      for p in pruned:
        si.children.remove(p)

And then we just need to reference the plugin when we create the client.

client = Client(url, plugins=[MyPlugin()])

Answer 1

You can use a plugin to modify the XML before is sent to the server (my answer is based on Ronald Smith's solution):

from suds.plugin import MessagePlugin
from suds.client import Client
import re

class MyPlugin(MessagePlugin):
    def sending(self, context):
        context.envelope = re.sub('\s+<.*?/>', '', context.envelope)


client = Client(URL_WSDL, plugins=[MyPlugin()])

Citing the documentation:

The MessagePlugin currently has (5) hooks ::
(...)
sending()
Provides the plugin with the opportunity to inspect/modify the message text before it is sent.

Basically Suds will call sending before the XML is sent, so you can modify the generated XML (contained in context.envelope). You have to pass the plugin class MyPlugin to the Client constructor for this to work.

Edit

Another way is to use marshalled to modify the XML structure, removing the empty elements (untested code):

class MyPlugin(MessagePlugin):
    def marshalled(self, context):
        #remove empty tags inside the Body element
        #context.envelope[0] is the SOAP-ENV:Header element
        context.envelope[1].prune()

Answer 2

There's an even easier way - no need for any Reg Ex or exciting iterators ;)

First, define the plugin:

class PrunePlugin(MessagePlugin):
    def marshalled(self, context):
        context.envelope = context.envelope.prune()

Then use it when creating the client:

client = Client(url, plugins=[PrunePlugin()])

The prune() method will remove any empty nodes, as documented here: http://jortel.fedorapeople.org/suds/doc/suds.sax.element.Element-class.html