Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

Concatenating fields of mongo db using php

Tags:

php

mongodb

I am trying to concatenate two fields in mongo db using php for a huge data set. Below is the sample solution that I have found and tried so far but this doesn't works in php. Can anyone help with the necessary corrections if any in the below code.

var pipeline = [
        {
           "$project": {
            "data1": 1,
            "data2": { $concat: [ "$data1", " ", "$data2" ] }
            }
        }
],
cursor = db.collection.aggregate(pipeline),
bulkUpdateOps = cursor.map(function (doc) { 
    return { 
            "updateOne": {
                "filter": { "_id": doc._id },
                "update": { "$set": { "data2": doc.data2 } } 
             }
          };
});         
db.collection.bulkWrite(bulkUpdateOps);

I am installing mongo-php-library as you suggested in comments by using the composer command :

$composer require "mongodb/mongodb=^1.0.0" 

but it didnt work. php mongo driver is 1.6.12. and mongo db version is 3.2.3 . and the error is as follows :

 Your version of PHP, 5.4.16, is affected by CVE-2013-6420 and cannot safely perform certificate validation, we strongly suggest you upgrade.
  ./composer.json has been created
 Loading composer repositories with package information
 Updating dependencies (including require-dev)
 Your requirements could not be resolved to an installable set of packages.

  Problem 1

     - mongodb/mongodb 1.0.1 requires ext-mongodb ^1.1.0 -> the requested PHP extension mongodb is missing from your system.

     - mongodb/mongodb 1.0.0 requires ext-mongodb ^1.1.0 -> the requested PHP extension mongodb is missing from your system.

- Installation request for mongodb/mongodb ^1.0.0 -> satisfiable by mongodb/mongodb[1.0.0, 1.0.1].
  To enable extensions, verify that they are enabled in those .ini files:

- /etc/php.ini

- /etc/php.d/curl.ini

- /etc/php.d/dom.ini

- /etc/php.d/fileinfo.ini

- /etc/php.d/json.ini

- /etc/php.d/mbstring.ini

- /etc/php.d/mysql.ini

- /etc/php.d/mysqli.ini

- /etc/php.d/pdo.ini

- /etc/php.d/pdo_mysql.ini

- /etc/php.d/pdo_sqlite.ini

- /etc/php.d/phar.ini

- /etc/php.d/posix.ini

- /etc/php.d/sqlite3.ini

- /etc/php.d/sysvmsg.ini

- /etc/php.d/sysvsem.ini

- /etc/php.d/sysvshm.ini

- /etc/php.d/wddx.ini

- /etc/php.d/xmlreader.ini

- /etc/php.d/xmlwriter.ini

- /etc/php.d/xsl.ini

- /etc/php.d/zip.ini

  You can also run `php --ini` inside terminal to see which files are used by PHP in CLI mode.

   Installation failed, deleting ./composer.json.

I need a help in installing this library. Am I installing the wrong version?

like image 383
Varsha Avatar asked Mar 08 '16 13:03

Varsha


2 Answers

Thank you all for the help but I could not install the mongo php library so the above solution didn't work for me. I found another way that I would like to share to merge two fields which used the shell commands. In php file :

$concat_Command = "mongo < concat.sql";
shell_exec($concat_Command);

concat.sql

 use dbname;
 db.collection.find().forEach(function(doc){
    db.collection.update(
        { "_id": doc._id },
        { "$set": { "description": doc.title+" "+doc.description } }
    )
});
like image 115
Varsha Avatar answered Sep 28 '22 01:09

Varsha


Without seeing your attempts at solving this in PHP, much of what follows is based on assumptions and may therefore not offer the best solution but will nonetheless serve as a guideline to sway you in the right direction to solve your issue.

Using the PHP driver with the complimentary mongo-php-library, which implements a more full-featured API on top of the bare-bones driver, construct the aggregation pipeline and the subsequent bulk update operation as:

$pipeline = [["$project" => ["data1" => 1, "data2" => ["$concat" => [ "$data1", " ", "$data2" ]]]]];
$operation = new Aggregate($databaseName, $collectionName, $pipeline, ['typeMap' => $typeMap]);
$cursor = $operation->execute($primaryServer);
$results = iterator_to_array($cursor);

function mapper($doc) {
    return [
        "updateOne" => [
            ["_id" => $doc["_id"]], 
            [
                "$set" => [
                    "data2" => $doc["_id"]
                ]
            ]
        ]
    ]
};

$ops = array_map("mapper", $results);
$bulkUpdateOps = new BulkWrite($databaseName, $collectionName, $ops);
$writeResult = $bulkUpdateOps->execute($primaryServer);

In the above, $primaryServer is a parameter for the BulkWrite execute() method that should contain an instance of the MongoDB\Driver\Server object i.e. your primary MongoDB server connection details. The $typeMap variable is an optional parameter in the Aggregate() constructor to denote Type map for BSON deserialization. This will be applied to the returned cursor.


If you're using the legacy driver then consider running the following operation which uses the Bulk Operations API which is available from MongoDB >= 2.6 <= 3.0:

$client = new MongoClient();
$collection = $client->selectCollection("database", "collection");

$pipeline = array(         
    array(
        "$project" => array(
            'data1' => 1,
            "data2" => array("$concat" => array( "$data1", " ", "$data2" ))
        )
    )
);

$batch = new MongoUpdateBatch($collection);
$counter = 0;

foreach ($collection->aggregate($pipeline) as $doc ) {
    $batch->add(
        array(
            "q" => array( '_id' => $doc['_id'] ),
            "u" => array( 
                '$set' => array(
                    "data2" => $doc["data2"]
                )
            )
        )
    );
    $counter++;

    if ($counter % 500 === 0) {
        $retval = $batch->execute(array( 'w' => 1));
        $counter++;
        $batch = new MongoUpdateBatch($collection);        
    }
}

if ($counter > 0) {
    $retval = $batch->execute(array( 'w' => 1 ));
}
like image 44
chridam Avatar answered Sep 28 '22 02:09

chridam