How to store extra data into multi curl info handler?

Question

I'm having trouble finding more info about a particular curl handle in a multi curl scenario. Here is the code.

$job_count = 5;
 while ( $eachPr = $prList->fetch () ) {


            for ( $job_number = 1 ;
                        $job_number <= $job_count ;
                        $job_number ++ , $index ++ ) {


                $url = $this->getURL ( $eachPr[ "name" ] ,
                                       $eachPr[ "category" ] ) ;

                $this->log ( $url ) ;



                $curl_handle = curl_init () ;

                curl_setopt ( $curl_handle ,
                              CURLOPT_USERAGENT ,
                              $userAgent ) ;
                curl_setopt ( $curl_handle ,
                              CURLOPT_URL ,
                              $url ) ;
                curl_setopt ( $curl_handle ,
                              CURLOPT_FAILONERROR ,
                              TRUE ) ;
                curl_setopt ( $curl_handle ,
                              CURLOPT_FOLLOWLOCATION ,
                              TRUE ) ;
                curl_setopt ( $curl_handle ,
                              CURLOPT_AUTOREFERER ,
                              TRUE ) ;
                curl_setopt ( $curl_handle ,
                              CURLOPT_RETURNTRANSFER ,
                              TRUE ) ;

                curl_setopt ( $curl_handle ,
                              CURLOPT_COOKIE ,
                              $cookie ) ;
                var_dump($curl_handle);

                /* add a request to the multiple handle */
                curl_multi_add_handle ( $multi_handler ,
                                        $curl_handle ) ;
                $eachPr = $prList->fetch () ;
            }


            do {
                while ( ($execrun = curl_multi_exec ( $multi_handler ,
                                                      $running )) == CURLM_CALL_MULTI_PERFORM )  ;
                if ( $execrun != CURLM_OK ) {
                    break ;
                }
                /* a request was just completed -- find out which one */
                while ( $done = curl_multi_info_read ( $multi_handler ) ) {

                    /* get the info and content returned on the request */
                    $info   = curl_getinfo ( $done[ 'handle' ] ) ;
                    $output = curl_multi_getcontent ( $done[ 'handle' ] ) ;
                    var_dump($info);
                    /* send the return values to the thread waiting to process the data . 
                    $this->work_pool[] = $this->submit ( new PrStacker ( $eachPr[ "name" ] ,
                                                                                        $eachPr[ "id" ] ,
                                                                                        $output ) ) ;

                    $this->work_pool[ count ( $this->work_pool ) - 1 ]->wait () ;


                    /* remove the curl handle that just completed */
                    curl_multi_remove_handle ( $multi_handler ,
                                               $done[ 'handle' ] ) ;


                }

                /* Block for data in / output; error handling is done by curl_multi_exec */
                if ( $running ) {
                    curl_multi_select ( $multi_handler ,
                                        30 ) ;
                }
            } while ( $running ) ;


            /* write the current index to the file */
            file_put_contents ( $symbols_index_file ,
                                $index ) ;

            $sleep_interval = rand ( 5 ,
                                     10 ) ;

            $this->log ( " Sleeping Now For " . $sleep_interval . "  seconds" ) ;

            sleep ( $sleep_interval ) ;

            $index ++ ;
        }
        curl_multi_close ( $multi_handler ) ;

So in here im looping through a list of 11K product using while ( $eachPr = $prList->fetch () ). Then taking 5 product at a time i'm initializing curl handles which i add to the curl multi handle.

The handles are executed at the do while loop. Here comes the trouble after selecting the request which was just done using $done = curl_multi_info_read ( $multi_handler ) . Each response is passed to another thread which handles other task. And each thread requires the product name, the product id and the raw html response. Here this is how each of the stackers are initialized

$this->work_pool[] = $this->submit ( new PrStacker ( $eachPr[ "name" ] ,
                                                                                            $eachPr[ "id" ] ,
                                                                                            $output ) ) ;

But after each of the curl requests gets done i can't find a way to send the correct product name and id which corresponds to the request that was finished. When in the above code i pass the name, id and output to the PrStacker thread i realized that it was not the correct product that corresponds to the request which was done. Its a different and wrong product that gets passed to the threads.

So is there any way in which i can include the product name and id along with each of the curl handles/requests before it gets executed so that the program may recognize which response corresponds to which product. I hope my explanation can be understood.

Please let me know if there is any way in which this can be done.

Answer 1

Store private data inside the cURL easy handle, e.g. the product id:

curl_setopt($curl_handle, CURLOPT_PRIVATE, $this->getId());
// then later
$id = curl_getinfo($done['handle'], CURLINFO_PRIVATE);

This "private data" feature was not documented in the PHP manual until early 2015. It was introduced already in PHP 5.2.4. It allows you to store and retrieve a string of your choice inside the cURL handle. Use it for a key/ID that uniquely identifies the product, and that you can use to lookup the product in your own data structure.

See: curl_getinfo and curl's Predefined Constants.