How to setMasterUrl in Ignite XML config for Kubernetes IPFinder

Question

Using test config with Ignite 2.4 and k8s 1.9:

<?xml version="1.0" encoding="UTF-8"?>
    <beans xmlns="http://www.springframework.org/schema/beans"
           xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
           xmlns:util="http://www.springframework.org/schema/util"
           xsi:schemaLocation="
            http://www.springframework.org/schema/beans
            http://www.springframework.org/schema/beans/spring-beans.xsd
            http://www.springframework.org/schema/util
            http://www.springframework.org/schema/util/spring-util.xsd">

    <bean class="org.apache.ignite.configuration.IgniteConfiguration">
      <property name="discoverySpi">
        <bean class="org.apache.ignite.spi.discovery.tcp.TcpDiscoverySpi">
          <property name="ipFinder">
            <bean class="org.apache.ignite.spi.discovery.tcp.ipfinder.kubernetes.TcpDiscoveryKubernetesIpFinder"/>
          </property>
        </bean>
      </property>
    </bean>
</beans>

Unable to find Kubernetes API Server at https://kubernetes.default.svc.cluster.local:443 Can I set the API Server URL in the XML config file? How?

Answer 1

@Denis was right.

Kubernetes using RBAC access controlling system and you need to authorize your pod to access to API.

For that, you need to add a Service Account to your pod.

So, for do that you need:

1. Create a service account and set role for it:

   apiVersion: v1
   kind: ServiceAccount
   metadata:
     name: ignite
     namespace: <Your namespace>
   

2. I am not sure that permissions to access only pods will be enough for Ignite, but if not - you can add as more permissions as you want. Here is example of different kind of roles with large list of permissions. So, now we create Cluster Role for your app:

   apiVersion: rbac.authorization.k8s.io/v1beta1
   kind: ClusterRole
   metadata:
     name: ignite
     namespace: <Your namespace>
   rules:
   - apiGroups:
     - ""
     resources:
     - pods # Here is resources you can access
     verbs: # That is what you can do with them
     - get
     - list
     - watch
   

3. Create binding for that role:

   kind: ClusterRoleBinding
   apiVersion: rbac.authorization.k8s.io/v1beta1
   metadata:
     name: ignite
   roleRef:
     kind: ClusterRole
     name: ignite
     apiGroup: rbac.authorization.k8s.io
   subjects:
   - kind: ServiceAccount
     name: ignite
     namespace: <Your namespace>
   

4. Now, you need to associate ServiceAccount to pods with your application:

   apiVersion: extensions/v1beta1
   kind: DaemonSet
   metadata:
     ....
   spec:
     template:
       spec:
         serviceAccountName: ignite
   

After that, your application will have an access to K8s API. P.S. Do not forget to change <Your namespace> to namespace where you running Ignition.

Answer 2

Platform versions

• Kubernetes: v1.8
• Ignite: v2.4

@Anton Kostenko design is mostly right, but here's a refined suggestion that works and grants least access privileges to Ignite.

1. If you're using a Deployment to manage Ignite, then all of your Pods will launch within a single namespace. Therefore, you should really use a Role and a RoleBinding to grant API access to the service account associated with your deployment.

2. The TcpDiscoveryKubernetesIpFinder only needs access to the endpoints for the headless service that selects your Ignite pods. The following 2 manifests will grant that access.

   apiVersion: rbac.authorization.k8s.io/v1
   kind: Role
   metadata:
     name: ignite-endpoint-access
     namespace: <your-ns>
     labels:
       app: ignite
   rules:
     - apiGroups: [""]
       resources: ["endpoints"]
       resourceNames: ["<your-headless-svc>"]
       verbs: ["get"]
   
   ---
   apiVersion: rbac.authorization.k8s.io/v1
   kind: RoleBinding
   metadata:
     name: ignite-role-binding
     labels:
       app: ignite
   subjects:
     - kind: ServiceAccount
       name: <your-svc-account>
   roleRef:
     kind: Role
     name: ignite-endpoint-access
     apiGroup: rbac.authorization.k8s.io
   

Answer 3

Take a look at this thread: http://apache-ignite-users.70518.x6.nabble.com/Unable-to-connect-ignite-pods-in-Kubernetes-using-Ip-finder-td18009.html

The problem of 403 error can be solved by granting more permissions to the service account.