Nested complex-type elements never get populated when calling AXIS-based web service from .NET

Question

I've been trying for a couple of days now to get a .NET client working fully with a Web Server provided by my Coldfusion-based web app. I'm not a .NET developer, per se, but I happen to have a copy of VS 2003, which seems like it should do the trick.

I can use a simple multiplier() method in my web service that takes two numbers and returns a number, so simple types are working fine. It's the complex types that are killing me. I'm basically trying to return an associative array from a method called get_struct(). I get an object of type Map back, but the property (called item), which is supposed to be an array with two elements (of type mapItem) always has an "undefined value".

Here's the WSDL that is generated by ColdFusion:

<?xml version="1.0" encoding="UTF-8"?>
<wsdl:definitions targetNamespace="http://trunk.v.pfapi.remote_api" xmlns:apachesoap="http://xml.apache.org/xml-soap" xmlns:impl="http://trunk.v.pfapi.remote_api" xmlns:intf="http://trunk.v.pfapi.remote_api" xmlns:soapenc="http://schemas.xmlsoap.org/soap/encoding/" xmlns:tns1="http://rpc.xml.coldfusion" xmlns:wsdl="http://schemas.xmlsoap.org/wsdl/" xmlns:wsdlsoap="http://schemas.xmlsoap.org/wsdl/soap/" xmlns:xsd="http://www.w3.org/2001/XMLSchema">
<!--WSDL created by Macromedia ColdFusion MX version 7,0,2,142559-->
 <wsdl:types>
  <schema targetNamespace="http://rpc.xml.coldfusion" xmlns="http://www.w3.org/2001/XMLSchema">
   <import namespace="http://trunk.v.pfapi.remote_api"/>
   <import namespace="http://xml.apache.org/xml-soap"/>
   <import namespace="http://schemas.xmlsoap.org/soap/encoding/"/>
   <complexType name="CFCInvocationException">
    <sequence/>
   </complexType>
   <complexType name="QueryBean">
    <sequence>
     <element name="columnList" nillable="true" type="impl:ArrayOf_xsd_string"/>
     <element name="data" nillable="true" type="impl:ArrayOfArrayOf_xsd_anyType"/>
    </sequence>
   </complexType>
  </schema>
  <schema targetNamespace="http://xml.apache.org/xml-soap" xmlns="http://www.w3.org/2001/XMLSchema">
   <import namespace="http://trunk.v.pfapi.remote_api"/>
   <import namespace="http://rpc.xml.coldfusion"/>
   <import namespace="http://schemas.xmlsoap.org/soap/encoding/"/>
   <complexType name="mapItem">
    <sequence>
     <element name="key" nillable="true" type="xsd:anyType"/>
     <element name="value" nillable="true" type="xsd:anyType"/>
    </sequence>
   </complexType>
   <complexType name="Map">
    <sequence>
     <element maxOccurs="unbounded" minOccurs="0" name="item" type="apachesoap:mapItem"/>
    </sequence>
   </complexType>
  </schema>
  <schema targetNamespace="http://trunk.v.pfapi.remote_api" xmlns="http://www.w3.org/2001/XMLSchema">
   <import namespace="http://rpc.xml.coldfusion"/>
   <import namespace="http://xml.apache.org/xml-soap"/>
   <import namespace="http://schemas.xmlsoap.org/soap/encoding/"/>
   <complexType name="ArrayOf_xsd_string">
    <complexContent>
     <restriction base="soapenc:Array">
      <attribute ref="soapenc:arrayType" wsdl:arrayType="xsd:string[]"/>
     </restriction>
    </complexContent>
   </complexType>
   <complexType name="ArrayOfArrayOf_xsd_anyType">
    <complexContent>
     <restriction base="soapenc:Array">
      <attribute ref="soapenc:arrayType" wsdl:arrayType="xsd:anyType[][]"/>
     </restriction>
    </complexContent>
   </complexType>
  </schema>
 </wsdl:types>
   <wsdl:message name="CFCInvocationException">
      <wsdl:part name="fault" type="tns1:CFCInvocationException"/>
   </wsdl:message>
   <wsdl:message name="multiplierResponse">
      <wsdl:part name="multiplierReturn" type="xsd:double"/>
   </wsdl:message>
   <wsdl:message name="get_structResponse">
      <wsdl:part name="get_structReturn" type="apachesoap:Map"/>
   </wsdl:message>
   <wsdl:message name="struct_keycountResponse">
      <wsdl:part name="struct_keycountReturn" type="xsd:double"/>
   </wsdl:message>
   <wsdl:message name="get_structRequest">
   </wsdl:message>
   <wsdl:message name="multiplierRequest">
      <wsdl:part name="factor1" type="xsd:double"/>
      <wsdl:part name="factor2" type="xsd:double"/>   </wsdl:message>
   <wsdl:message name="struct_keycountRequest">
      <wsdl:part name="theStruct" type="apachesoap:Map"/>
   </wsdl:message>
   <wsdl:portType name="remote_io_test">
      <wsdl:operation name="multiplier" parameterOrder="factor1 factor2">
         <wsdl:input message="impl:multiplierRequest" name="multiplierRequest"/>
         <wsdl:output message="impl:multiplierResponse" name="multiplierResponse"/>
         <wsdl:fault message="impl:CFCInvocationException" name="CFCInvocationException"/>
      </wsdl:operation>
      <wsdl:operation name="get_struct">
         <wsdl:input message="impl:get_structRequest" name="get_structRequest"/>
         <wsdl:output message="impl:get_structResponse" name="get_structResponse"/>
         <wsdl:fault message="impl:CFCInvocationException" name="CFCInvocationException"/>
      </wsdl:operation>
      <wsdl:operation name="struct_keycount" parameterOrder="theStruct">
         <wsdl:input message="impl:struct_keycountRequest" name="struct_keycountRequest"/>
         <wsdl:output message="impl:struct_keycountResponse" name="struct_keycountResponse"/>
         <wsdl:fault message="impl:CFCInvocationException" name="CFCInvocationException"/>
      </wsdl:operation>
   </wsdl:portType>
   <wsdl:binding name="remote_io_test.cfcSoapBinding" type="impl:remote_io_test">
      <wsdlsoap:binding style="rpc" transport="http://schemas.xmlsoap.org/soap/http"/>
      <wsdl:operation name="multiplier">
         <wsdlsoap:operation soapAction=""/>
         <wsdl:input name="multiplierRequest">
            <wsdlsoap:body encodingStyle="http://schemas.xmlsoap.org/soap/encoding/" namespace="http://trunk.v.pfapi.remote_api" use="encoded"/>
         </wsdl:input>
         <wsdl:output name="multiplierResponse">
            <wsdlsoap:body encodingStyle="http://schemas.xmlsoap.org/soap/encoding/" namespace="http://trunk.v.pfapi.remote_api" use="encoded"/>
         </wsdl:output>
         <wsdl:fault name="CFCInvocationException">
            <wsdlsoap:fault encodingStyle="http://schemas.xmlsoap.org/soap/encoding/" name="CFCInvocationException" namespace="http://trunk.v.pfapi.remote_api" use="encoded"/>
         </wsdl:fault>
      </wsdl:operation>
      <wsdl:operation name="get_struct">
         <wsdlsoap:operation soapAction=""/>
         <wsdl:input name="get_structRequest">
            <wsdlsoap:body encodingStyle="http://schemas.xmlsoap.org/soap/encoding/" namespace="http://trunk.v.pfapi.remote_api" use="encoded"/>
         </wsdl:input>
         <wsdl:output name="get_structResponse">
            <wsdlsoap:body encodingStyle="http://schemas.xmlsoap.org/soap/encoding/" namespace="http://trunk.v.pfapi.remote_api" use="encoded"/>
         </wsdl:output>
         <wsdl:fault name="CFCInvocationException">
            <wsdlsoap:fault encodingStyle="http://schemas.xmlsoap.org/soap/encoding/" name="CFCInvocationException" namespace="http://trunk.v.pfapi.remote_api" use="encoded"/>
         </wsdl:fault>
      </wsdl:operation>
      <wsdl:operation name="struct_keycount">
         <wsdlsoap:operation soapAction=""/>
         <wsdl:input name="struct_keycountRequest">
            <wsdlsoap:body encodingStyle="http://schemas.xmlsoap.org/soap/encoding/" namespace="http://trunk.v.pfapi.remote_api" use="encoded"/>
         </wsdl:input>
         <wsdl:output name="struct_keycountResponse">
            <wsdlsoap:body encodingStyle="http://schemas.xmlsoap.org/soap/encoding/" namespace="http://trunk.v.pfapi.remote_api" use="encoded"/>
         </wsdl:output>
         <wsdl:fault name="CFCInvocationException">
            <wsdlsoap:fault encodingStyle="http://schemas.xmlsoap.org/soap/encoding/" name="CFCInvocationException" namespace="http://trunk.v.pfapi.remote_api" use="encoded"/>
         </wsdl:fault>
      </wsdl:operation>
   </wsdl:binding>
   <wsdl:service name="remote_io_testService">
  <wsdl:documentation xmlns:wsdl="http://schemas.xmlsoap.org/wsdl/">
This is a collection of test methods to allow remote developers
   to evaluate datatype support, etc in their programming environment.
   The WSDL endpoint for this web service is [YOUR PEERFOCUS SITE]/remote_api/pfapi/v/trunk/remote_io_test.cfc?wsdl  </wsdl:documentation>
      <wsdl:port binding="impl:remote_io_test.cfcSoapBinding" name="remote_io_test.cfc">
         <wsdlsoap:address location="http://leon.cupahr.tafkan.localhost/remote_api/pfapi/v/trunk/remote_io_test.cfc"/>
      </wsdl:port>
   </wsdl:service>
</wsdl:definitions>

and here's the web service stub generated when I add a web reference in VS:

//------------------------------------------------------------------------------
// <autogenerated>
//     This code was generated by a tool.
//     Runtime Version: 1.1.4322.2443
//
//     Changes to this file may cause incorrect behavior and will be lost if 
//     the code is regenerated.
// </autogenerated>
//------------------------------------------------------------------------------

// 
// This source code was auto-generated by Microsoft.VSDesigner, Version 1.1.4322.2443.
// 
namespace pfapi_test.remote_io_test {
    using System.Diagnostics;
    using System.Xml.Serialization;
    using System;
    using System.Web.Services.Protocols;
    using System.ComponentModel;
    using System.Web.Services;


    /// <remarks/>
    [System.Diagnostics.DebuggerStepThroughAttribute()]
    [System.ComponentModel.DesignerCategoryAttribute("code")]
    [System.Web.Services.WebServiceBindingAttribute(Name="remote_io_test.cfcSoapBinding", Namespace="http://trunk.v.pfapi.remote_api")]
    [System.Xml.Serialization.SoapIncludeAttribute(typeof(QueryBean))]
    [System.Xml.Serialization.SoapIncludeAttribute(typeof(CFCInvocationException))]
    public class remote_io_testService : System.Web.Services.Protocols.SoapHttpClientProtocol {

        /// <remarks/>
        public remote_io_testService() {
            this.Url = "http://leon.cupahr.tafkan.nooch/remote_api/pfapi/v/trunk/remote_io_test.cfc";
        }

        /// <remarks/>
        [System.Web.Services.Protocols.SoapRpcMethodAttribute("", RequestNamespace="http://trunk.v.pfapi.remote_api", ResponseNamespace="http://trunk.v.pfapi.remote_api")]
        [return: System.Xml.Serialization.SoapElementAttribute("multiplierReturn")]
        public System.Double multiplier(System.Double factor1, System.Double factor2) {
            object[] results = this.Invoke("multiplier", new object[] {
                        factor1,
                        factor2});
            return ((System.Double)(results[0]));
        }

        /// <remarks/>
        public System.IAsyncResult Beginmultiplier(System.Double factor1, System.Double factor2, System.AsyncCallback callback, object asyncState) {
            return this.BeginInvoke("multiplier", new object[] {
                        factor1,
                        factor2}, callback, asyncState);
        }

        /// <remarks/>
        public System.Double Endmultiplier(System.IAsyncResult asyncResult) {
            object[] results = this.EndInvoke(asyncResult);
            return ((System.Double)(results[0]));
        }

        /// <remarks/>
        [System.Web.Services.Protocols.SoapRpcMethodAttribute("", RequestNamespace="http://trunk.v.pfapi.remote_api", ResponseNamespace="http://trunk.v.pfapi.remote_api")]
        [return: System.Xml.Serialization.SoapElementAttribute("get_structReturn")]
        public Map get_struct() {
            object[] results = this.Invoke("get_struct", new object[0]);
            return ((Map)(results[0]));
        }

        /// <remarks/>
        public System.IAsyncResult Beginget_struct(System.AsyncCallback callback, object asyncState) {
            return this.BeginInvoke("get_struct", new object[0], callback, asyncState);
        }

        /// <remarks/>
        public Map Endget_struct(System.IAsyncResult asyncResult) {
            object[] results = this.EndInvoke(asyncResult);
            return ((Map)(results[0]));
        }

        /// <remarks/>
        [System.Web.Services.Protocols.SoapRpcMethodAttribute("", RequestNamespace="http://trunk.v.pfapi.remote_api", ResponseNamespace="http://trunk.v.pfapi.remote_api")]
        [return: System.Xml.Serialization.SoapElementAttribute("struct_keycountReturn")]
        public System.Double struct_keycount(Map theStruct) {
            object[] results = this.Invoke("struct_keycount", new object[] {
                        theStruct});
            return ((System.Double)(results[0]));
        }

        /// <remarks/>
        public System.IAsyncResult Beginstruct_keycount(Map theStruct, System.AsyncCallback callback, object asyncState) {
            return this.BeginInvoke("struct_keycount", new object[] {
                        theStruct}, callback, asyncState);
        }

        /// <remarks/>
        public System.Double Endstruct_keycount(System.IAsyncResult asyncResult) {
            object[] results = this.EndInvoke(asyncResult);
            return ((System.Double)(results[0]));
        }
    }

    /// <remarks/>
    [System.Xml.Serialization.SoapTypeAttribute("Map", "http://xml.apache.org/xml-soap")]
    public class Map {

        /// <remarks/>
        public mapItem[] item;
    }

    /// <remarks/>
    [System.Xml.Serialization.SoapTypeAttribute("mapItem", "http://xml.apache.org/xml-soap")]
    public class mapItem {

        /// <remarks/>
        public object key;

        /// <remarks/>
        public object value;
    }

    /// <remarks/>
    [System.Xml.Serialization.SoapTypeAttribute("QueryBean", "http://rpc.xml.coldfusion")]
    public class QueryBean {

        /// <remarks/>
        public string[] columnList;

        /// <remarks/>
        public object[] data;
    }

    /// <remarks/>
    [System.Xml.Serialization.SoapTypeAttribute("CFCInvocationException", "http://rpc.xml.coldfusion")]
    public class CFCInvocationException {
    }
}

And finally, my CLI app that tests the service:

using System;

namespace pfapi_test
{
 /// <summary>
 /// Summary description for Class1.
 /// </summary>
 class Class1
 {
  /// <summary>
  /// The main entry point for the application.
  /// </summary>
  [STAThread]
  static void Main(string[] args)
  {
   //
   // TODO: Add code to start application here
   //

   Console.WriteLine("Instantiating WS");

   remote_io_test.remote_io_testService testWS = new remote_io_test.remote_io_testService();

   Console.WriteLine("Calling multiplier(3,15)");
   Console.WriteLine(testWS.multiplier(3,15));

   Console.WriteLine("Calling get_struct()");
   remote_io_test.Map theStruct = testWS.get_struct();
   Console.Write("result: ");
   Console.WriteLine(theStruct);
   Console.Write("result.item: ");
   Console.WriteLine(theStruct.item);

   Console.WriteLine("Press Enter to exit...");
   Console.ReadLine();
  }
 }
}

No matter what I try, theStruct.item is always "undefined value" according to the debugger. The printed output looks like:

Instantiating WS
Calling multiplier(3,15)
45
Calling get_struct()
result: pfapi_test.remote_io_test.Map
result.item:
Press Enter to exit...

I've tried using ColdFusion 8, and there's no difference. I've tried returning a real custom database with two properties instead of my ad-hoc associative array, and that works fine, but rewriting my API to avoid associative arrays isn't really an option at this time. The API works fine with ColdFusion, PHP/NuSOAP, and Ruby on Rails, so it seems like it should be possible to get it working with .NET as well.

I hope someone can provide some insight. I have the suspicion that there's some kind of namespace issue here, but I don't know SOAP and XML well enough to figure out what it is. I've also searched like mad for a solution online, but I haven't found a single person who's solved this problem, which is very disheartening!

Request:

POST /remote_api/pfapi/v/trunk/remote_io_test.cfc HTTP/1.1
VsDebuggerCausalityData: [snip]
User-Agent: Mozilla/4.0 (compatible; MSIE 6.0; MS Web Services Client Protocol 1.1.4322.2443)
Content-Type: text/xml; charset=utf-8
SOAPAction: ""
Content-Length: 488
Expect: 100-continue
Host: leon.cupahr.tafkan.nooch

<?xml version="1.0" encoding="utf-8"?>
<soap:Envelope xmlns:soap="http://schemas.xmlsoap.org/soap/envelope/" xmlns:soapenc="http://schemas.xmlsoap.org/soap/encoding/" xmlns:tns="http://trunk.v.pfapi.remote_api" xmlns:types="http://trunk.v.pfapi.remote_api/encodedTypes" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xmlns:xsd="http://www.w3.org/2001/XMLSchema">
  <soap:Body soap:encodingStyle="http://schemas.xmlsoap.org/soap/encoding/">
    <tns:get_struct />
  </soap:Body>
</soap:Envelope>

Response:

HTTP/1.1 200 OK
Date: Thu, 17 Dec 2009 15:14:33 GMT
Server: Apache/2.2.11 (Unix) mod_ssl/2.2.11 OpenSSL/0.9.7l DAV/2 PHP/5.2.8 JRun/4.0 Phusion_Passenger/2.2.7
Set-Cookie: CFID=21543;expires=Sat, 10-Dec-2039 15:14:33 GMT;path=/
Set-Cookie: CFTOKEN=479cc311ca4875db-9D346355-ED36-6183-C8895635E4EE1252;expires=Sat, 10-Dec-2039 15:14:33 GMT;path=/
Transfer-Encoding: chunked
Content-Type: text/xml; charset=utf-8

<?xml version="1.0" encoding="utf-8"?>
<soapenv:Envelope xmlns:soapenv="http://schemas.xmlsoap.org/soap/envelope/" xmlns:xsd="http://www.w3.org/2001/XMLSchema" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance">
 <soapenv:Body>
  <ns1:get_structResponse soapenv:encodingStyle="http://schemas.xmlsoap.org/soap/encoding/" xmlns:ns1="http://trunk.v.pfapi.remote_api">
   <get_structReturn xsi:type="ns2:Map" xmlns:ns2="http://xml.apache.org/xml-soap">
    <item xmlns:soapenc="http://schemas.xmlsoap.org/soap/encoding/">
     <key xsi:type="soapenc:string">FOO</key>
     <value xsi:type="soapenc:string">bar</value>
    </item>
    <item>
     <key xsi:type="soapenc:string" xmlns:soapenc="http://schemas.xmlsoap.org/soap/encoding/">ANOTHERKEY</key>
     <value xsi:type="soapenc:string" xmlns:soapenc="http://schemas.xmlsoap.org/soap/encoding/">another value</value>
    </item>
   </get_structReturn>
  </ns1:get_structResponse>
 </soapenv:Body>
</soapenv:Envelope>

Update: I heard from the .NET developer on the other end, and he's tried all of the following to no avail:

• using WCF instead of ASMX
• using .NET 3.5
• using VB.NET to communicate with the web service instead of C#

He did send me a screenshot of the "Data Type" section of the Service Reference Settings dialog. Is it possible that changing the settings for Collection type and Dictionary collection type might resolve this issue?

Update 2: here's the ColdFusion code for remote_io_test.cfc

<cfcomponent name="remote_io_test"
    hint="This is a collection of test methods to allow remote developers
        to evaluate datatype support, etc in their programming environment.">

<cffunction name="get_struct" returntype="struct" access="remote" output="no"
        hint="Returns an associative array with two keys, 'foo' and 'anotherKey'.
            Allows you to test your implementation's support for WDDX encoding.
            ColdFusion and PHP (w/ NuSOAP) will automatically decode the result
            into an associative array. Feedback on other languages is appreciated.">
    <cfset var stFoo = structNew()>
    <cfset stFoo.foo = "bar">
    <cfset stFoo.anotherKey = "another value">
    <cfreturn duplicate(stFoo)>
</cffunction> <!--- get_struct --->


<cffunction name="multiplier" returntype="numeric" access="remote" output="no"
        hint="Multiplies two factors and returns the result. Allows you to test
            passing simple variables to a remote method.">
    <cfargument name="factor1" type="numeric" required="yes">
    <cfargument name="factor2" type="numeric" required="yes">

    <cfreturn factor1 * factor2>
</cffunction> <!--- multiplier --->


<cffunction name="struct_keycount" returntype="numeric" access="remote" output="no"
        hint="Returns the number of keys in an upload associative array. Allows
            you to test passing complex variables to a remote method.">
    <cfargument name="theStruct" type="struct" required="yes">

    <cfreturn structCount(theStruct)>
</cffunction> <!--- struct_keycount --->

Thanks for reading, and thanks in advance for your replies!

Cross-posted at link text

Answer 1

I've been struggling with this item myself from the .NET client side of things - I'm trying to consume a web service written in ColdFusion. From the documentation I found, the struct data type in ColdFusion does not map directly onto any Web Services types, and it would seem to me that it is a fault with Axis.

You've probably read the same web pages as I have when trying to solve this problem, but it would seem if you want to create a web service using ColdFusion, the recommendation is not to use the struct type. If your web service is not live yet, and you can get away with using something different, I would suggest going down this route. I tried returning objects from ColdFusion, which worked fine for me from a .NET client.

I actually gave up on this problem and wrote a custom client for my .NET program which would just read the Xml returned by the ColdFusion web service and convert this into a dictionary, as I have no influence over the service I am trying to call.

One thing I did find during my testing - the service I am trying to call is running on an older version of ColdFusion (7 I believe). Whenever I call that service, the .NET client returns null. I installed ColdFusion on my own machine and wrote a simple service, and found that the .NET client returned an object of type Map (which had come from the WSDL), but the properties of the Map object were all null.

I'd be interested to see if Chris Haas's suggestion in the comments to your question of setting dotNetSoapEncFix solves the problem.

Answer 2

The answer is: don't use Coldfusion structs in your web service.

Like everyone else before us, we gave up and rewrote our API to not use any associative arrays in the request or response. We're now using scalars, arrays, and complexTypes that are automatically created from CFCs. This time around, we tested our proof-of-concept with PHP, Ruby, Coldfusion, Java, and .NET to make sure it was actually interoperable.

It makes a lot of sense now that a statically-typed language would be unable to handle a totally arbitrary remote datatype in any sensible way.

Thanks for all of your feedback!