Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

LibCurl CURLOPT_URL not accepting string? C++

So basically what I want to do is use libcurl to fetch slightly different urls, e.g.:

http://foo.com/foo.asp?name=*NAMEHERE*

What I would like to do is iterate through a vector of names and get each one, e.g.:

http://foo.com/foo.asp?name=James

Then

http://foo.com/foo.asp?name=Andrew

And so on.

However, when I try doing this:

int foo (){
    CURL *curl;
    CURLcode success;
    char errbuf[CURL_ERROR_SIZE];
    int m_timeout = 15;

    if ((curl = curl_easy_init()) == NULL) {
        perror("curl_easy_init");
        return 1;
    }

    std::vector<std::string> names;

    names.push_back("James");
    names.push_back("Andrew");

    for (std::vector<std::string>::const_iterator i = names.begin(); i != names.end(); ++i){
        curl_easy_setopt(curl, CURLOPT_ERRORBUFFER, errbuf);
        curl_easy_setopt(curl, CURLOPT_VERBOSE, 1L);

        curl_easy_setopt(curl, CURLOPT_TIMEOUT, long(m_timeout));
        curl_easy_setopt(curl, CURLOPT_URL, "http://foo.com/foo.asp?name=" + *i);
        curl_easy_setopt(curl, CURLOPT_COOKIEFILE, "cookies.txt");
        curl_easy_setopt(curl, CURLOPT_COOKIEJAR, "cookies.txt");
        curl_easy_setopt(curl, CURLOPT_FOLLOWLOCATION, 1L);
        curl_easy_setopt(curl, CURLOPT_AUTOREFERER, 1L);
    }

    if ((success = curl_easy_perform(curl)) != 0) {
        fprintf(stderr, "%s: %s\n", "curl_easy_perform", errbuf);
        return 1;
    }

    curl_easy_cleanup(curl);

    return 0;
}

It gives me an error:

Cannot pass object of non-trivial type 'std::__1::basic_string<char>' through variadic function; call will abort at runtime

On this line:

curl_easy_setopt(curl, CURLOPT_URL, "http://foo.com/foo.asp?name=" + *i);

because of the + *i.

Is what I want to do possible? Is there a solution?

EDIT: Thanks for the answer, but for some reason, when I run this it only gets the website with the last string in the vector, and it ignores the other ones. In my case, it skips James and goes straight to Andrew. Why does that happen?

like image 966
SomeRandomGuy Avatar asked May 18 '14 11:05

SomeRandomGuy


2 Answers

The parameter passed to curl_easy_setopt for CURLOPT_URL needs to be a char * instead of a std::string. You can get a const char * from a std::string by calling its c_str member function:

std::string url = "http://foo.com/foo.asp?name=" + *i;
curl_easy_setopt(curl, CURLOPT_URL, url.c_str());

For anyone who ends up here in the future, take a look at the curl manual if you haven't already. There's also an online book.

The documentation for curl_easy_setopt says you need to read about a specific option to know what parameter type to use.

All options are set with an option followed by a parameter. That parameter can be a long, a function pointer, an object pointer or a curl_off_t, depending on what the specific option expects. Read this manual carefully as bad input values may cause libcurl to behave badly!

The documentation for CURLOPT_URL says exactly what the parameter type needs to be.

Pass in a pointer to the URL to work with. The parameter should be a char * to a zero terminated string which must be URL-encoded in the following format:

scheme://host:port/path

like image 126
jhoffman0x Avatar answered Nov 15 '22 05:11

jhoffman0x


for some reason, when I run this it only gets the website with the last string in the vector, and it ignores the other ones. In my case, it skips James and goes straight to Andrew. Why does that happen?

You are looping through all of the names configuring the curl object with a different name on each loop iteration, THEN you are calling curl_easy_perform() only after the loop has finished, so it will retrieve only the last name that was configured. You need to move your call to curl_easy_perform() inside of the loop instead:

for (std::vector<std::string>::const_iterator i = names.begin(); i != names.end(); ++i)
{
    curl_easy_setopt(curl, CURLOPT_ERRORBUFFER, errbuf);
    curl_easy_setopt(curl, CURLOPT_VERBOSE, 1L);

    curl_easy_setopt(curl, CURLOPT_TIMEOUT, long(m_timeout));

    string url = "http://foo.com/foo.asp?name=" + *i;
    curl_easy_setopt(curl, CURLOPT_URL, url.c_str());

    curl_easy_setopt(curl, CURLOPT_COOKIEFILE, "cookies.txt");
    curl_easy_setopt(curl, CURLOPT_COOKIEJAR, "cookies.txt");
    curl_easy_setopt(curl, CURLOPT_FOLLOWLOCATION, 1L);
    curl_easy_setopt(curl, CURLOPT_AUTOREFERER, 1L);

    success = curl_easy_perform(curl);
    if (success != 0) {
        fprintf(stderr, "%s: %s\n", "curl_easy_perform", errbuf);
        // handle the error as needed...
        continue;
    }

    // use the downloaded content as needed...
}
like image 1
Remy Lebeau Avatar answered Nov 15 '22 05:11

Remy Lebeau