How do you set-up Mongo replica set on Kubernetes?

Question

I'd like to set-up a Mongo replica set on Kubernetes. I'd like to have three replicas. This means I'd need to start 3 instances.

Should I start three pods, with Mongo in each one, and use the service the point to the primary? Or should I used a replication controller somehow?

Answer 1

This answer is out of date. I wrote a detailed step-by-step tutorial here using more up to date methods. I highly recommend reading it all.

In a nutshell, you run a sidecar app to configure the replica set for you, and either use a service per instance or ping the K8s API for the pod IP addresses.

Example: This will only work in Google Cloud. You will need to make modifications for other platforms, particularly around the volumes:

1. Follow the example in https://github.com/leportlabs/mongo-k8s-sidecar.git
   • git clone https://github.com/leportlabs/mongo-k8s-sidecar.git
   • cd mongo-k8s-sidecar/example/
   • make add-replica ENV=GoogleCloudPlatform (do this three times)
2. Connect to the replica set via services.
   • mongodb://mongo-1,mongo-2,mongo-3:27017/dbname_?
3. You can also use the raw pod IP addresses instead of creating a service per pod
   • Use this https://github.com/thesandlord/kubernetes-pod-ip-finder.git

Answer 2

Typically, to set up a clustered set of nodes like mongo with replicas sets, you would create a Service that tracks the pods under the service name (so for example, create a MongoDB replication controller with a tag mongodb, and a Service tracking those instances) The Service can then be queried for its members (using the API server, you can look up the nodes with

curl -H "Authorization: Bearer $(cat /var/run/secrets/kubernetes.io/serviceaccount/token)" --cacert /var/run/secrets/kubernetes.io/serviceaccount/ca.crt https://kubernetes/api/v1/namespaces/default/endpoints/mongodb

where mongodb is your selector on the name of the service.

that returns a JSON object with a bunch of fields, so a good way to parse these easily is to use jq https://stedolan.github.io/jq/

piping the curl command into a jq query like

jq '.subsets[].addresses[]' | jq '{ip: .ip, host:.targetRef.name}' will return the IP and hostnames of the mongodb instances in your cluster.

So now you know who is in the cluster and you can create the replica set in your init script. Obviously here that means you need to start the Service first, your startup script needs to wait for all the nodes to be up and registered with the service, and then you can proceed. If you use one image, with one script, it will run n each node, so you need to check that the replica set does not exists already or handle errors. The first pod to register should do the work. Another option is to run all nodes as single nodes, then run a separate bootstrapping script that will create the replica set.

Finally, then you call the mongodb cluster, you will need to make sure you specify the url with replica set name as an option:

mongodb://mongodb:27017/database?replicaSet=replicaSetName

Since you don't know the IP of the master, you would call it through the service mongodb which will load balance the requests to one of the nodes, and if you don't specify the replica set name, you will end up with connection errors as only the master can get write requests.

Obviously this is not a step by step tutorial, but i hope that gets you started.