So basically what I want to do is use libcurl to fetch slightly different urls, e.g.:
http://foo.com/foo.asp?name=*NAMEHERE*
What I would like to do is iterate through a vector of names and get each one, e.g.:
http://foo.com/foo.asp?name=James
Then
http://foo.com/foo.asp?name=Andrew
And so on.
However, when I try doing this:
int foo (){
CURL *curl;
CURLcode success;
char errbuf[CURL_ERROR_SIZE];
int m_timeout = 15;
if ((curl = curl_easy_init()) == NULL) {
perror("curl_easy_init");
return 1;
}
std::vector<std::string> names;
names.push_back("James");
names.push_back("Andrew");
for (std::vector<std::string>::const_iterator i = names.begin(); i != names.end(); ++i){
curl_easy_setopt(curl, CURLOPT_ERRORBUFFER, errbuf);
curl_easy_setopt(curl, CURLOPT_VERBOSE, 1L);
curl_easy_setopt(curl, CURLOPT_TIMEOUT, long(m_timeout));
curl_easy_setopt(curl, CURLOPT_URL, "http://foo.com/foo.asp?name=" + *i);
curl_easy_setopt(curl, CURLOPT_COOKIEFILE, "cookies.txt");
curl_easy_setopt(curl, CURLOPT_COOKIEJAR, "cookies.txt");
curl_easy_setopt(curl, CURLOPT_FOLLOWLOCATION, 1L);
curl_easy_setopt(curl, CURLOPT_AUTOREFERER, 1L);
}
if ((success = curl_easy_perform(curl)) != 0) {
fprintf(stderr, "%s: %s\n", "curl_easy_perform", errbuf);
return 1;
}
curl_easy_cleanup(curl);
return 0;
}
It gives me an error:
Cannot pass object of non-trivial type 'std::__1::basic_string<char>' through variadic function; call will abort at runtime
On this line:
curl_easy_setopt(curl, CURLOPT_URL, "http://foo.com/foo.asp?name=" + *i);
because of the + *i
.
Is what I want to do possible? Is there a solution?
EDIT: Thanks for the answer, but for some reason, when I run this it only gets the website with the last string in the vector, and it ignores the other ones. In my case, it skips James
and goes straight to Andrew
. Why does that happen?
The parameter passed to curl_easy_setopt
for CURLOPT_URL
needs to be a char *
instead of a std::string
. You can get a const char *
from a std::string
by calling its c_str
member function:
std::string url = "http://foo.com/foo.asp?name=" + *i;
curl_easy_setopt(curl, CURLOPT_URL, url.c_str());
For anyone who ends up here in the future, take a look at the curl manual if you haven't already. There's also an online book.
The documentation for curl_easy_setopt says you need to read about a specific option to know what parameter type to use.
All options are set with an option followed by a parameter. That parameter can be a long, a function pointer, an object pointer or a curl_off_t, depending on what the specific option expects. Read this manual carefully as bad input values may cause libcurl to behave badly!
The documentation for CURLOPT_URL says exactly what the parameter type needs to be.
Pass in a pointer to the URL to work with. The parameter should be a char * to a zero terminated string which must be URL-encoded in the following format:
scheme://host:port/path
for some reason, when I run this it only gets the website with the last string in the vector, and it ignores the other ones. In my case, it skips
James
and goes straight toAndrew
. Why does that happen?
You are looping through all of the names configuring the curl
object with a different name on each loop iteration, THEN you are calling curl_easy_perform()
only after the loop has finished, so it will retrieve only the last name that was configured. You need to move your call to curl_easy_perform()
inside of the loop instead:
for (std::vector<std::string>::const_iterator i = names.begin(); i != names.end(); ++i)
{
curl_easy_setopt(curl, CURLOPT_ERRORBUFFER, errbuf);
curl_easy_setopt(curl, CURLOPT_VERBOSE, 1L);
curl_easy_setopt(curl, CURLOPT_TIMEOUT, long(m_timeout));
string url = "http://foo.com/foo.asp?name=" + *i;
curl_easy_setopt(curl, CURLOPT_URL, url.c_str());
curl_easy_setopt(curl, CURLOPT_COOKIEFILE, "cookies.txt");
curl_easy_setopt(curl, CURLOPT_COOKIEJAR, "cookies.txt");
curl_easy_setopt(curl, CURLOPT_FOLLOWLOCATION, 1L);
curl_easy_setopt(curl, CURLOPT_AUTOREFERER, 1L);
success = curl_easy_perform(curl);
if (success != 0) {
fprintf(stderr, "%s: %s\n", "curl_easy_perform", errbuf);
// handle the error as needed...
continue;
}
// use the downloaded content as needed...
}
If you love us? You can donate to us via Paypal or buy me a coffee so we can maintain and grow! Thank you!
Donate Us With