C++ 'GET' request or how do you download files to work with in C++?

Question

Alright I've spent a good three days trying this, here's the scenario:

I want to download a '.csv' file from Google and then do stuff with the data from the file. It's for a Win32 Console Application. I have the latter down, I just cannot for the life of me figure out how to download the file. I've heard of libcurl, curlpp, ptypes, rolling my own, just using the .NET api, and been told a bunch of times:

...it's just a GET request

Well that's all well and good, but I must be missing something because it seems like everyone was just born knowing how to do this. I have been combing through books looking to figure this out and even had a huge problem with LNKerrors after traveling down the road with "The Art of C++" for a while.

All that being said, I have learned a LOT from this, but at this point I just want to know how to do it. The API for C++ is seriously lacking, no example code to be found. Tutorials online are almost non-existent. And no book out there seems to think this is important.

Can someone please throw me a life raft? I'm a man on the edge here.

edit

By "from Google" I mean that I want to download a .csv file that they host. An example can be found here.

Answer 1

You should be able to bend this to your will.

Now that I have kinda answered your question. Why C++? Nothing against the language, but pick the best language for the job. Perl, PHP, and Python(and I am sure more) all have great documentation and support on this kind of operation.

In perl(the one I am familiar with) it's just about 3-5 lines of code.

---

Here is the code snippet previously available in (from WayBackMachine):

/*
 * This is a very simple example of how to use libcurl from within 
 * a C++  program. The basic idea is that you want to retrieve the 
 * contents of a web page as a string. Obviously, you can replace 
 * the buffer object with anything you want and adjust elsewhere 
 * accordingly.
 * 
 * Hope you find it useful..
 * 
 * Todd Papaioannou
 */

#include <string>
#include <iostream>
#include "curl/curl.h"

using namespace std;

// Write any errors in here
static char errorBuffer[CURL_ERROR_SIZE];

// Write all expected data in here
static string buffer;

// This is the writer call back function used by curl
static int writer(char *data, size_t size, size_t nmemb,
                  std::string *buffer)
{
  // What we will return
  int result = 0;

  // Is there anything in the buffer?
  if (buffer != NULL)
  {
    // Append the data to the buffer
    buffer->append(data, size * nmemb);

    // How much did we write?
    result = size * nmemb;
  }

  return result;
}

// You know what this does..
void usage()
{
  cout < < "curltest: \n" << endl;
  cout << "  Usage:  curltest url\n" << endl;
} 

/*
 * The old favorite
 */
int main(int argc, char* argv[])
{
  if (argc > 1) 
  {
    string url(argv[1]);

    cout < < "Retrieving " << url << endl;

    // Our curl objects
    CURL *curl;
    CURLcode result;

    // Create our curl handle
    curl = curl_easy_init();

    if (curl)
    {
      // Now set up all of the curl options
      curl_easy_setopt(curl, CURLOPT_ERRORBUFFER, errorBuffer);
      curl_easy_setopt(curl, CURLOPT_URL, argv[1]);
      curl_easy_setopt(curl, CURLOPT_HEADER, 0);
      curl_easy_setopt(curl, CURLOPT_FOLLOWLOCATION, 1);
      curl_easy_setopt(curl, CURLOPT_WRITEFUNCTION, writer);
      curl_easy_setopt(curl, CURLOPT_WRITEDATA, &buffer);

      // Attempt to retrieve the remote page
      result = curl_easy_perform(curl);

      // Always cleanup
      curl_easy_cleanup(curl);

      // Did we succeed?
      if (result == CURLE_OK)
      {
        cout << buffer << "\n";
        exit(0);
      }
      else
      {
        cout << "Error: [" << result << "] - " << errorBuffer;
        exit(-1);
      }
    }
  }
}

Answer 2

Why not just using what's already there?

UrlDownloadToFile()