CURLOPT_WRITEFUNCTION to get xml content

Question

I have used

curl_easy_setopt(curl, CURLOPT_WRITEFUNCTION, write_data);

and

size_t write_data(void *ptr, size_t size, size_t count, void *stream)
{
    printf("%*.*s", size * count, size * count, ptr);
}

to get the below output but i need to get only body content

* About to connect() to 10.10.1.112 port 8081 (#0)
*   Trying 10.10.1.112... * connected
* Connected to 10.10.1.112 (10.10.1.112) port 8081 (#0)
> POST /iit_mobile/services/application?wsdl HTTP/1.1
Host: 10.10.1.112:8081
Accept: */*
Content-type:application/soap+xml; charset=utf-8; action="http://wsdlclass.wsdlcreat.sims.test.com/userloginMethod"
Content-Length: 629

< HTTP/1.1 200 OK
< Server: Apache-Coyote/1.1
< Content-Type: application/soap+xml;charset=utf-8
< Transfer-Encoding: chunked
< Date: Tue, 11 Jun 2013 13:22:35 GMT
< 
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100   338    0   338    0     0     36      0 --:--:--  0:00:09 --:--:--     0* Connection #0 to host 10.10.1.112 left intact

* Closing connection #0
<?xml version="1.0" encoding="utf-8"?><soapenv:Envelope xmlns:soapenv="http://www.w3.org/2003/05/soap-envelope" xmlns:xsd="http://www.w3.org/2001/XMLSchema" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"><soapenv:Body><userloginMethodResponse xmlns="http://wsdlclass.wsdlcreat.sims.test.com"/></soapenv:Body></soapenv:Envelope>

Code:

curl = curl_easy_init();
if(curl)
{

    out_fd = fopen (filename, "w");
    curl_easy_setopt(curl, CURLOPT_FILE, out_fd);

    headers = curl_slist_append(headers, "Content-type:application/soap+xml; charset=utf-8; action=\"http://wsdlclass.wsdlcreat.sims.test.com/login\"");
    curl_easy_setopt(curl, CURLOPT_URL, tmpbuff);
    curl_easy_setopt(curl, CURLOPT_NOPROGRESS, 0);
    curl_easy_setopt(curl, CURLOPT_VERBOSE, 1);

    curl_easy_setopt(curl, CURLOPT_HTTPHEADER, headers);

    curl_easy_setopt(curl, CURLOPT_WRITEFUNCTION, write_data);


    curl_easy_setopt(curl, CURLOPT_POSTFIELDSIZE, buffer_size);
    curl_easy_setopt(curl, CURLOPT_POSTFIELDS, buffer);

    curl_easy_setopt(curl, CURLOPT_VERBOSE, 1L);
    curl_easy_setopt(curl, CURLOPT_TIMEOUT, Timeout);
    curl_easy_setopt(curl, CURLOPT_ERRORBUFFER,errmsg);

    printf("The Server%s:Performing Transaction.....\n",__func__);
    res = curl_easy_perform(curl);
    printf("res=after culreasey perform%d\n",res);
    curl_slist_free_all(headers);
    curl_easy_cleanup(curl);

    fclose(out_fd);
}

Answer 1

By default the headers should not be written out with the body data. If your are receiving headers in your write_data callback, that probably means you've set either the CURLOPT_HEADER option or the CURLOPT_WRITEHEADER option. You could try resetting both of those to be safe:

curl_easy_setopt(curl, CURLOPT_HEADER, 0L);
curl_easy_setopt(curl, CURLOPT_WRITEHEADER, 0L);

If you do still want to retrieve the headers, but just not in the write_data callback, you can set a separate callback for your header data like this:

curl_easy_setopt(curl, CURLOPT_HEADERFUNCTION, write_header);

Where write_header is a callback function, much like your write_data function.

size_t write_header(void *ptr, size_t size, size_t count, void *stream)
{
    printf("HEADER: %*.*s", size * count, size * count, ptr);
    return count*size;
}

Note that it's important you return the number of bytes written from these callback functions otherwise curl would treat that as an error.

If this is still not working for you, the only other explanation I can think of is that your server is returning a blank line before the header data, making it seem as if the headers are actually part of the body.

You easily see if that is the case by testing against a known valid url (e.g. http://www.example.com/). If it works correctly there, then you know the fault is in your server code and not the client code.

Having looked at your full code, though, all of that extra stuff you're seeing in your output is coming from the CURLOPT_VERBOSE options (which you've set twice), and CURLOPT_NOPROGRESS option. Just comment those out and you should get a nice clean response with nothing but the content body.

I should also mention that there's no point in setting CURLOPT_FILE unless you're going to be using the stream parameter in the write_data callback. When you set the CURLOPT_WRITEFUNCTION callback, it replaces the default output function, so nothing will get written to the CURLOPT_FILE stream unless you do so yourself.