unnexpected increment of count in multithreaded program

Question

I am trying to run given number of curls from each thread. pull_one_url() is the function which is called in pthread_create() from main(). cnt_limit is a global variable which is set in main and read only in threads.

The unexpected behavior I am seeing sometimes is that the local count variable get incremented to high values.

For given run number of threads were 10 and cnt_limit (number of curls from each thread) set to 10.

Command line run: ./a.out 10 10 10.140.71.12

The unexpected output is:

10 2682
10 2858
10 2804
10 2988
10 2871
10 2940
10 2864
10 2609
10 2816
10 2893

expected output: for given input each 10 threads need to perform 10 curl requests.

number of lines = number of threads

first number in each line = number of curl requests each thread require to perform.

second number in each line = count of curl requests each thread performed.

10 10
10 10
10 10
10 10
10 10
10 10
10 10
10 10
10 10
10 10

The source code:

#include <stdio.h>
#include <pthread.h>
#include <curl/curl.h>
#include <stdlib.h>

size_t write_data(void *buffer, size_t size, size_t nmemb, void *userp)
{
//    printf("got response\n");
    return size * nmemb;
}
int cnt_limit;
static void *pull_one_url(void *url)
{
    CURL *curl;
    int cnt;
    int count = 0;
    for (cnt=0; cnt < cnt_limit; cnt++) {
        CURLcode res;
        curl = curl_easy_init();
        curl_easy_setopt(curl, CURLOPT_URL, url);
        curl_easy_setopt(curl, CURLOPT_SSL_VERIFYHOST, 1L);
        curl_easy_setopt(curl, CURLOPT_SSL_VERIFYPEER, 1L);
        curl_easy_setopt(curl, CURLOPT_WRITEFUNCTION, write_data);
        res = curl_easy_perform(curl);
        int response_code;
        if (res == CURLE_OK) {
            curl_easy_getinfo(curl, CURLINFO_RESPONSE_CODE, &response_code);
            //printf("%d\n", response_code);
        }
        else {
            //printf("400\n");
        }
        curl_easy_cleanup(curl);
        count++;
        //printf("%d\n", cnt);
    }
    printf("%d %d\n", cnt_limit, count);
    return NULL;
}

int main(int argc, char **argv)
{
    int n = atoi(argv[1]);
    cnt_limit=atoi(argv[2]);
    pthread_t tid[n];
    int i;

    curl_global_init(CURL_GLOBAL_ALL);

    for(i = 0; i< n; i++) {
        int error = pthread_create(&tid[i],
                               NULL,
                               pull_one_url,
                               (void *) argv[3]);
        if(0 != error)
            fprintf(stderr, "Couldn't run thread number %d, errno %d\n", i, error);
        //else
            //fprintf(stderr, "Thread %d, gets %s\n", i, argv[2]);
    }

    /* now wait for all threads to terminate */
    for(i = 0; i< n; i++) {
        pthread_join(tid[i], NULL);
        //fprintf(stderr, "Thread %d terminated\n", i);
    }

    return 0;
}

compile: gcc source_code.c -lpthread -lcurl

run : ./a.out 10 10 <url>

Edited: After commenting out libcurl code present inside loop, i don't observe this behavior yet. But previously observed unexpected behavior was also not very consistent. question still remain how this libcurl code affecting the simple counter in loop.

After spending a lot of time trying to understand this I am posting it here. I think i am making some silly mistake. Any help will be appreciated.

Answer 1

Based on the results of the discussion in the comments section, it seems the problem was the following:

Part of the code that the OP posted was the following:

int response_code;
[...]
curl_easy_getinfo(curl, CURLINFO_RESPONSE_CODE, &response_code);

According to the curl documentation on CURLINFO_RESPONSE_CODE, the last parameter is of type long *. However, the OP was passing a parameter of type int *. Since the OP is using a platform on which sizeof(long) == 8 whereas sizeof(int) == 4, the curl library wrote 8 bytes to the address of response_code, although only 4 bytes had been allocated on the stack for this variable. This likely caused a neighboring variable on the stack to be overwritten.

To fix this problem, the line

int response_code;

should be changed to:

long response_code;