Kubernetes API

The Kubernetes API can be used to discover peers and form an Akka Cluster. The kubernetes-api mechanism queries the Kubernetes API server to find pods with a given label. A Kubernetes service isn’t required for the cluster bootstrap but may be used for external access to the application.

The following Kubernetes resources are created:

  • Deployment: For creating the Akka pods
  • Role and RoleBinding to give the pods access to the API server

An example deployment (used for integration testing):

apiVersion: extensions/v1beta1
kind: Deployment
metadata:
  labels:
    app: akka-bootstrap-demo
  name: akka-bootstrap-demo
spec:
  replicas: 3
  selector:
    matchLabels:
     app: akka-bootstrap-demo
  strategy:
    rollingUpdate:
      maxSurge: 1
      maxUnavailable: 0
    type: RollingUpdate

  template:
    metadata:
      labels:
        app: akka-bootstrap-demo
        actorSystemName: akka-bootstrap-demo
    spec:
      containers:
      - name: akka-boostrap-demo
        image: integration-test-kubernetes-api:1.3.3.7
        # Remove for a real project, the image is picked up locally for the integratio test
        imagePullPolicy: Never
        livenessProbe:
          httpGet:
            path: /alive
            port: management
        readinessProbe:
          httpGet:
            path: /ready
            port: management
        ports:
        # akka remoting
        - name: remoting
          containerPort: 2552
          protocol: TCP
        # akka-management bootstrap
        - containerPort: 8558
          protocol: TCP
          # when contact-point-discovery.port-name is set for cluster bootstrap,
          # the management port must be named accordingly:
          # name: management
        env:
        # The Kubernetes API discovery will use this service name to look for
        # nodes with this value in the 'app' label.
        # This can be customized with the 'pod-label-selector' setting.
        - name: AKKA_CLUSTER_BOOTSTRAP_SERVICE_NAME
          valueFrom:
            fieldRef:
              apiVersion: v1
              fieldPath: "metadata.labels['app']"
Full source at GitHub

An example role and rolebinding to allow the nodes to query the Kubernetes API server:

kind: Role
apiVersion: rbac.authorization.k8s.io/v1
metadata:
  name: pod-reader
rules:
- apiGroups: [""] # "" indicates the core API group
  resources: ["pods"]
  verbs: ["get", "watch", "list"]
---
kind: RoleBinding
apiVersion: rbac.authorization.k8s.io/v1
metadata:
  name: read-pods
subjects:
  # Create the default user for the 'akka-bootstrap-demo-ns' namespace, update for the namespace you are running in
- kind: User
  name: system:serviceaccount:akka-bootstrap-demo-ns:default
roleRef:
  kind: Role
  name: pod-reader
  apiGroup: rbac.authorization.k8s.io
Full source at GitHub

The User name includes the namespace, this will need updated for your namespace.

The following configuration is required:

  • Set akka.management.cluster.bootstrap.contact-point-discovery.discovery-method to akka.discovery.kubernetes-api
  • Set akka.discovery.kubernetes-api.pod-label-selector to a label selector that will match the Akka pods e.g. app=%s
akka.discovery {
  kubernetes-api {
    # in fact, this is already the default:
    pod-label-selector = "app=%s"
  }
}
Full source at GitHub

The lookup needs to know which namespace to look in. By default, this will be detected by reading the namespace from the service account secret, in /var/run/secrets/kubernetes.io/serviceaccount/namespace, but can be overridden by setting akka.discovery.kubernetes-api.pod-namespace.

For more details on how to configure the Kubernetes deployment see recipes.

The source code for this page can be found here.