how to properly reuse a curl handle

Question

I want to properly reuse a curl handle, so that it won't give me errors and function normally.

Suppose I have this piece of code:

    CURL *curl;      curl_global_init(CURL_GLOBAL_ALL);     curl = curl_easy_init();      curl_easy_setopt(curl, CURLOPT_USERAGENT, "Mozilla/5.0...");     curl_easy_setopt(curl, CURLOPT_URL, "http://www.google.com");     curl_easy_perform(curl);      curl_easy_setopt(curl, CURLOPT_URL, "http://www.bbc.com");     curl_easy_perform(curl);      curl_easy_cleanup(curl);     curl_global_cleanup(); 

Would this be a good or correct way of reusing a curl handle? Or do I need to use curl_easy_reset() on that handle?

I would also appreciate if anyone suggested what you should avoid doing in curl. Maybe someone could give me a link to an already existing source of information?

Answer 1

If I understand the question correctly you would like to know if you can make a call to curl_easy_perform() and then only change the url through curl_easy_setopt() and then make a second call? This should work without any errors since the function does not change any previously set options for the handle. This is a short working example:

size_t writeCallback(char* contents, size_t size, size_t nmemb, std::string* buffer) {   size_t realsize = size * nmemb;   if(buffer == NULL) {     return 0;   }   buffer->append(contents, realsize);   return realsize;   }  int main(int argc, char** argv) {   std::string buffer;    // Initialize global.   curl_global_init(CURL_GLOBAL_ALL);    // Start a libcurl easy session.   CURL* ch = curl_easy_init();   if (ch) {     // Something went wrong     curl_global_cleanup();     return -1;   }    // These options will only be set once.   curl_easy_setopt(ch, CURLOPT_VERBOSE, 0);   curl_easy_setopt(ch, CURLOPT_FOLLOWLOCATION, 1);   curl_easy_setopt(ch, CURLOPT_USERAGENT, "Crawler");   curl_easy_setopt(ch, CURLOPT_WRITEFUNCTION, &writeCallback);   curl_easy_setopt(ch, CURLOPT_WRITEDATA, &buffer);    // Push a couple of URLs onto queue.   std::vector<const char*> queue;   queue.push_back("http://www.google.com");   queue.push_back("http://www.stackoverflow.com");    const char* url;   CURLcode code;    do {       // Grab an URL from the queue.       url = queue.back();       queue.pop_back();        // Only change the CURLOPT_URL option for the handle       // the rest will stay intact.       curl_easy_setopt(ch, CURLOPT_URL, url);        // Perform transfer.       code = curl_easy_perform(ch);        // Check if everything went fine.       if(code != CURLE_OK) {         // Handle any errors.       }        // Clear the buffer.       buffer.clear();   } while(queue.size() > 0);    // Cleanup.   curl_easy_cleanup(ch);   curl_global_cleanup();    return 0; } 

Or do I need to use curl_easy_reset() on that handle?

The answer is no since curl_easy_perform() not will reset any options your code should be fine and you can stick with only changing the url like curl_easy_setoption(curl, CURLOPT_URL, <newurl>);.

Answer 2

When you use the environment libcurl on the easy interface, you first have to call :

• curl_easy_init(), which init the easy handle,
• curl_global_init(), most of the case the flag option has to be CURL_GLOBAL_ALL

Each of those two functions is called just once at the beginning and need their opposite clean up :

• curl_easy_cleanup() when you've finished handles you've declare,
• curl_global_cleanup() when you're done with libcurl,

For better results check errors as much as you can. Libcurl provides curl_easy_strerror() function for that. It returns a string describing the CURLcode error. Also, some functions return the value CURL_OK or a specific integer if everything is OK.

For instance, here's the proper way to use CURLOPT_URL option :

#include <curl.h>  int main(void) {     /* declaration of an object CURL */     CURL *handle;                         /* result of the whole process */     CURLcode result;                    /* the first functions */     /* set up the program environment that libcurl needs */     curl_global_init(CURL_GLOBAL_ALL);     /* curl_easy_init() returns a CURL easy handle that you're gonna reuse in other easy functions*/     handle = curl_easy_init();        /* if everything's all right with the easy handle... */     if(handle)      {             /* ...you can list the easy functions */             /* here we just gonna try to get the source code of http://example.com */             curl_easy_setopt(handle, CURLOPT_URL, "http://example.com");              /* but in that case we also tell libcurl to follow redirection */             curl_easy_setopt(handle, CURLOPT_FOLLOWLOCATION, 1L);              /* perform, then store the expected code in 'result'*/              result = curl_easy_perform(handle);              /* Check for errors */              if(result != CURLE_OK)             {                     /* if errors have occured, tell us wath's wrong with 'result'*/                     fprintf(stderr, "curl_easy_perform() failed: %s\n", curl_easy_strerror(result));                      return 1;             }     }      /* if something's gone wrong with curl at the beginning, we'll appriciate that piece of code */       else      {             fprintf(stderr, "Curl init failed!\n");              return 1;     }      /* cleanup since you've used curl_easy_init */      curl_easy_cleanup(handle);      /* this function releases resources acquired by curl_global_init() */     curl_global_cleanup();      /* make the programme stopping for avoiding the console closing befor you can see anything */     system("PAUSE");      return 0; } 

If you want to reuse that handle for a totally different purpose you'd better use different CURL easy handles. Still your code should work fine but i would use different handles because it's obviously two seperate operations.

However sometimes you'll need to work with the same handle and if you don't want to do reset it automatically, use the appropriate function :

void curl_easy_reset(CURL *handle);  

Note that it does not change live connections, the Session ID cache, the DNS cache, the cookies and shares from the handle.

I haven't tried it but with your code it should give us something like that :

#include <curl.h>  int main(void) {     CURL *handle;                        CURLcode result;       int error = 0;     int error2 = 0;                   curl_global_init(CURL_GLOBAL_ALL);     handle = curl_easy_init();      if(handle)      {             curl_easy_setopt(handle, CURLOPT_USERAGENT, "Mozilla/5.0 (Windows; U; Windows NT 6.1; fr; rv:1.9.2) Gecko/20100115 Firefox/3.6");             curl_easy_setopt(handle, CURLOPT_URL, "http://www.google.com");             result = curl_easy_perform(handle);              if(result != CURLE_OK)             {                     fprintf(stderr, "curl_easy_perform() failed: %s\n", curl_easy_strerror(result));                      error++;             }              Sleep(5000);         // make a pause if you working on console application              curl_easy_reset(handle);              curl_easy_setopt(handle, CURLOPT_USERAGENT, "Mozilla/5.0 (Windows; U; Windows NT 6.1; fr; rv:1.9.2) Gecko/20100115 Firefox/3.6");      // have to write it again             curl_easy_setopt(handle, CURLOPT_URL, "http://www.bbc.com");             result = curl_easy_perform(handle);              if(result != CURLE_OK)             {                     fprintf(stderr, "curl_easy_perform() failed: %s\n", curl_easy_strerror(result));                      error2++;             }              if(error == 1 || error2 == 1)             {                     return 1;             }     }     else      {             fprintf(stderr, "Curl init failed!\n");              return 1;     }      curl_easy_cleanup(handle);     curl_global_cleanup();      system("PAUSE");      return 0; } 

If you have any problem with Sleep, try to replace it by sleep or _sleep or replace 5000 by 5.