How to run an etcd cluster among pod replicas?

Question

I have a pod/service running an application that consumes etcd as a synchronization system and datastore. I want to run etcd within the pod, such that all of the replicas form a coherent cluster. In other words, so the application in replica #1 can write "foo" to localhost:4001/v2/keys/my_key and then replica #2 can then read localhost:4001/v2/keys/my_key and get "foo" as a result.

It's not clear how this can be done, since pod replicas are not individually addressable. I could in theory create an "etcd" service exposing the cluster ports, but any requests would round-robin to all the replicas so the individual etcd nodes would not be able to find each other.

Am I approaching this problem the correct way?

Answer 1

You can deploy etcd on kubernetes using an Operator (from the extensions/v1beta1) and the quay.io/coreos/etcd-operator image.

An example deployment with a cluster size of 3 looks like this:

apiVersion: extensions/v1beta1
kind: Deployment
metadata:
  name: etcd-operator
spec:
  replicas: 1
  template:
    metadata:
      name: etcd-operator
      labels:
        app: etcd
        component: operator
    spec:
      containers:
      - name: etcd-operator
        image: quay.io/coreos/etcd-operator:v0.3.0
        env:
        - name: MY_POD_NAMESPACE
          valueFrom: { fieldRef: { fieldPath: metadata.namespace } }
        - name: MY_POD_NAME
          valueFrom: { fieldRef: { fieldPath: metadata.name } }

---

apiVersion: etcd.coreos.com/v1beta1
kind: Cluster
metadata:
  name: etcd-cluster
  labels:
    app: etcd
    component: cluster
spec:
  size: 3
  version: "3.1.8"

Please be aware of the beta status of this project. However according to the maintainers the operator is now stable. I have deployed the configuration above successfully but I didn't run any of this in production.

The operator code is available on github. You can find additional documentation there.