Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

MongoDB change stream replica-set limitation

What is the reasoning behind only making change streams available on replica sets?

like image 893
Daniel F Avatar asked Jan 07 '18 16:01

Daniel F


People also ask

What is the maximum number of nodes in a MongoDB replica set?

What kind of replication does MongoDB support? MongoDB supports replica sets, which can have up to 50 nodes.

Is MongoDB change stream reliable?

Security: Change streams are safe because users can only establish change streams on collections to which they have been allowed read access. Ease of use: Change streams are well-known – the API syntax uses the established MongoDB drivers and query language and is independent of any specific oplog format.

How do MongoDB change streams work?

Change streams transform a MongoDB database into a real-time database by taking advantage of MongoDB's replication process. They monitor replication in MongoDB, providing an API for external applications that require real-time data without the risk involved in tailing the oplog or the overhead that comes with polling.

Is MongoDB replication synchronous?

In order to maintain up-to-date copies of the shared data set, secondary members of a replica set sync or replicate data from other members. MongoDB uses two forms of data synchronization: initial sync to populate new members with the full data set, and replication to apply ongoing changes to the entire data set.


2 Answers

Change streams implementation is based on the oplog, which is only available on replica sets. Another reason is that a replica set contains a superset of the features of a standalone node, and is the recommended MongoDB deployment for production applications. Hence, it makes sense to implement the change stream feature based on the recommended production deployment topology.

Another major reason is that change streams will output documents that will not be rolled back in a replica set setting (see Rollbacks During Replica Set Failover), so the use of majority read concern is a requirement.

Note that change streams are also available in a sharded cluster, and also a single-node replica set (i.e. a replica set with only one member, although this setup is generally not recommended).

The high-level description of change streams is available in the Change Streams page

The design of change streams is outlined in SERVER-13932.

like image 87
kevinadi Avatar answered Sep 21 '22 22:09

kevinadi


My environment was Windows and the following steps helped me:

  1. Find "C:\Program Files\MongoDB\Server\4.2\bin\mongod.cfg"
  2. Add the following code to the mongod.cfg and options, that you need. More information: Mongo replication Options

    replication:
     replSetName: "rs0"
    
  3. Restart MongoDB process:

    Windows->Task Manager->Services->MongoDB [run restart]
    
  4. Run in the console mongo 127.0.0.1:27017

  5. Then run rs.initiate()

As a result you are lucky to have something like the following:

> rs.initiate()                                                                               

        "info2" : "no configuration specified. Using a default configuration for the set",    
        "me" : "127.0.0.1:27017",                                                             
        "ok" : 1,                                                                             
        "$clusterTime" : {                                                                    
                "clusterTime" : Timestamp(1584218777, 1),                                     
                "signature" : {                                                               
                        "hash" : BinData(0,"AAAAAAAAAAAAAAAAAAAAAAAAAAA="),                   
                        "keyId" : NumberLong(0)                                               
                }                                                                             
        },                                                                                    
        "operationTime" : Timestamp(1584218777, 1)                                            
}                                                                                             
rs0:SECONDARY>                                                                                
like image 31
Roman Avatar answered Sep 22 '22 22:09

Roman