Installation on Google Kubernetes Engine (GKE) Quick Start

To install Akka Cloud Platform on Google Kubernetes Engine (GKE), you must have a Google account and a subscription to the Akka Cloud Platform new tab.

Expected Time to Install

A new installation will take approximately 20 minutes, and by default, the following steps will be executed:

  1. Create an GKE cluster

  2. Install the Akka Cloud Platform Helm chart

  3. Install the Grafana Helm chart

  4. Install the Prometheus Helm Chart

  5. Set up a PostgreSQL Database Instance

1. Verify Prerequisites

Before installation, you must subscribe to the Akka Cloud Platform new tab on GCP Marketplace by clicking the Purchase button on the product page and accepting the terms and conditions.

To install and use the Akka Cloud Platform, you must have the following tools installed. We recommend using the latest versions of these tools. If you do not have them yet or need to update, we provide links to installation instructions:

  • The git new tab command-line tool to clone the Pulumi Playbook.

  • The npm new tab command-line tool to download and setup the NPM dependencies of the Pulumi Playbook.

  • The pulumi new tab cloud engineering (provisioning) tool to run the Playbook.

  • The Google Cloud SDK new tab

In addition, in order to operate the cluster, you need to set up:

  • The Kubernetes command-line tool, kubectl, allows you to run commands against Kubernetes clusters. Follow the instructions in the Kubernetes documentation new tab to install kubectl.

2. Setup your Google Cloud Platform (GCP)

The Pulumi Google Cloud Platform Provider new tab needs to be configured with Google credentials before it can be used to create resources. Ensure the following steps are complete before proceeding with the creation of the cluster:

gcloud config set project <YOUR_GCP_PROJECT_ID>
  • You are required to set up default application credentials in order to interact with your Google Cloud resources.

gcloud auth application-default login
  • Optionally you can set the following environment variables to provide preset default configurations for the Pulumi Google Cloud Platform provider:

    1. GOOGLE_PROJECT - The default project for new resources, if one is not specified when creating a resource

    2. GOOGLE_REGION - The default region for new resources, if one is not specified when creating a resource

    3. GOOGLE_ZONE - The default zone for new resources, if one is not specified when creating a resource.

Alternatively these can also be set through pulumi config to make them defaults for newly created resources:

pulumi config set gcp:project <your-project-here>
pulumi config set gcp:region <your-region-here>
pulumi config set gcp:zone <your-zone-here>

3. Quick Start with Pulumi Playbook

Pulumi is a cloud provisioning tool used to set up cloud infrastructure and Kubernetes cluster resources. The fastest way to get started with Akka Cloud Platform on GCP is to run the Akka Cloud Platform Deployment Pulumi playbook. The playbook contains a set of instructions that, when run, will provision the cluster and all the appropriate artifacts. Here are the steps to follow to have your playbook up and running:

git clone https://github.com/lightbend/akka-cloud-platform-deploy
  • Navigate to the base directory of the cloned repository in a terminal.

cd akka-cloud-platform-deploy
  • The repository contains scripts for both AWS and GCP deployment. Navigate to the gcp folder.

cd gcp
  • Install npm dependencies:

npm install
  • Initialize a Pulumi Stack new tab. A stack is the cluster and its artifacts with associated state, as provisioned by a Pulumi playbook. For example, you might use different stacks to represent different environments such as dev, testing, and staging. State is managed by default using Pulumi’s state management service (a remote service), but you can choose a self-managed state backend new tab if you like. Using Pulumi’s service is the fastest way to get started and provides several other benefits such as sharing state with others, view progress of a deployment, and view output variables and the resulting cloud resources that were created. To learn more about Pulumi Stack new tab and Pulumi State Management new tab see the Pulumi documentation.

pulumi stack init
  • Choose the Google Project Id, zone and region to use:

pulumi config set gcp:project <google project id>
pulumi config set gcp:zone europe-west1-b
pulumi config set gcp:region europe-west1

Alternately they can also be set up using the environment variables as mentioned above in an earlier section.

The setting of project id and zone are mandatory for a GCP installation. In case these are not set, the installation will be aborted with appropriate error messages.
  • Set up the license key file path. This is required to be set before you run the playbook. It will be passed to the operator deployment process. If you are deploying from your local environment you need to download the license file once from GCP Marketplace following the instructions in the installation guide. In the Akka Cloud Platform new tab for GCP page, click on Configure and then go to the tab Deploy via command line and click on Generate license key. A license.yaml will be generated on your local system.

pulumi config set akka-cloud-platform-gcp-deploy:license-file-path "/<path>/<to>/license.yaml"
  • Set the version of the operator.

pulumi config set akka-cloud-platform-gcp-deploy:operator-version 1.1.22
If you forget to set the license key file path and operator version up, the playbook will give you a gentle reminder to set the values appropriately.

When you run a pulumi up (deploy) all the resources defined in the playbook will be provisioned to your target environment. In almost all cases, resources will start with the prefix acp-<STACK NAME>, where acp stands for "Akka Cloud Platform" and <STACK NAME> is the name you chose for the Pulumi stack. Most resource names will end with a randomly generated alphanumeric string.

When you run pulumi up you will be prompted with a list of resources that the playbook will create. Select yes to deploy the stack to GCP.

  • Preview and provision the playbook.

pulumi up

While the cluster is provisioning you will be informed, in the terminal, about which resources are in the process of being created or have been created. If you are using the Pulumi service for the state, then the pulumi up command will include a link to "View Live" the progress of the deployment. The pulumi up command will exit when all resources are fully deployed and online.

If a failure occurs, the command will exit early and inform you of the reason. When this occurs, and it is possible to correct the issue yourself, you can then rerun pulumi up to resume updating where you left off. If the error can’t be resolved, you can use the pulumi destroy command to roll back any partially deployed resources.

If the deployment is successful, all of the pertinent details of the cluster will be output to the console (except for passwords).

After executing pulumi up, you will see something like the following:

$ pulumi up
Previewing update (<STACK NAME>)

View Live: https://app.pulumi.com/<USER NAME>/akka-cloud-platform-gcp-deploy/<STACK NAME>/previews/f7a820f3-0fd8-4f2e-903a-fecdd3c72749

     Type                                                                  Name                                                                                       Plan
 +   pulumi:pulumi:Stack                                                   akka-cloud-platform-gcp-deploy-<STACK NAME>                                                create
...

Resources:
    + 66 to create

Do you want to perform this update? yes

     Type                                                                 Name                                                                                        Status
 +   pulumi:pulumi:Stack                                                  akka-cloud-platform-gcp-deploy-<STACK NAME>                                                 created

...

When the stack is deployed successfully you will have a list of outputs presented. Example outputs for a stack named dev:

Outputs:
    clusterName                       : "projects/bubbly-observer-178213/locations/europe-west1-b/clusters/acp-dev-gke-111241b"
    kubeconfig                        : "[secret]"
    network                           : "projects/bubbly-observer-178213/global/networks/default"
    operatorNamespace                 : "lightbend"
    postgreSQLEndpoint                : "10.10.0.3"
    postgreSQLInstanceConnectionName  : "bubbly-observer-178213:europe-west1:instance-b787e55"
    postgreSQLInstanceName            : "instance-b787e55"

Resources:
    + 66 created

Duration: 22m39s

These output variables are required to access your cluster and associated resources. You can reference variables manually, or access their values at the command line with pulumi stack output <OUTPUT KEY>. Later on this page you can find the full list of output variables.

For example, to return the Kubernetes namespace that the Akka Cloud Platform Operator is installed in:

$ pulumi stack output operatorNamespace
lightbend
Pulumi has an issue with tracking ready status for containers with sidecars. In GCP implementation of the Akka Platform Operator, Usage-based Billing Agent (ubbagent) is implemented as a sidecar. Hence you may see the following failure in pulumi up where the update of the stack fails for akka-operator:
Diagnostics:
  kubernetes:apps/v1:Deployment (lightbend/akka-operator):
    error: 4 errors occurred:
    	* resource lightbend/akka-operator was successfully created, but the Kubernetes API server reported that it failed to fully initialize or become live: 'akka-operator' timed out waiting to be Ready
    	* [MinimumReplicasUnavailable] Deployment does not have minimum availability.
    	* Minimum number of live Pods was not attained
    	* [Pod lightbend/akka-operator-9b6f567cc-psqhh]: containers with unready status: [akka-operator ubbagent]

  pulumi:pulumi:Stack (akka-cloud-platform-gcp-deploy-dev):
    error: update failed

Just run pulumi up again, which is basically a noop, but the stack gets updated fully.

Another documented issue in Pulumi is within concurrency control. In this case, an operation fails with the following message. This is also documented in Pulumi troubleshooting section new tab
gcp:sql:DatabaseInstance (pg12instance):
  error: 1 error occurred:
  	* updating urn:pulumi:dev::akka-cloud-platform-gcp-deploy::gcp:sql/databaseInstance:DatabaseInstance::pg12instance: 1 error occurred:
  	* Error, failed to update instance settings for : googleapi: Error 409: Operation failed because another operation was already in progress., operationInProgress

 pulumi:pulumi:Stack (akka-cloud-platform-gcp-deploy-dev):
  error: update failed

Entering pulumi cancel will cancel the running update. However, in some cases, the update might have a status of 'OK', contrary to the message mentioned above. In that case, pulumi cancel gives the following message: The Update has already completed.

pulumi cancel

This will irreversibly cancel the currently running update for 'dev'!
Please confirm that this is what you'd like to do by typing ("dev"): dev
error: [409] Conflict: The Update has already completed

To destroy the cluster and all its dependencies:

pulumi destroy
You may need to run pulumi destroy a couple of times to destroy all resources effectively. You can run with a --refresh flag to refresh the state of the stack’s resources before destroying resources. For example, pulumi destroy --refresh.

3.1. Connect to the GKE Cluster

To set up your local kubectl configuration, you can obtain the Pulumi kubeconfig output by exporting it to a file, and defining the KUBECONFIG environment variable:

pulumi stack output kubeconfig --show-secrets > kubeconfig.yml
export KUBECONFIG=$PWD/kubeconfig.yml

3.2. Connect to the Cloud SQL database

To open a Postgres interactive shell with the Cloud SQL database instance:

First, you may want to set up a password for the Postgres user that you will be using. Here’s how to do that for the default user postgres

gcloud sql users set-password postgres --instance=$(pulumi stack output postgreSQLInstanceName) --prompt-for-password

Once the password is set, use kubectl run to set up an interactive psql session on a temporary pod:

kubectl run -i --tty rds-mgmt --image=postgres --restart=Never --rm \
  --env=PGPASSWORD=<password> -- \
  psql -h $(pulumi stack output postgreSQLEndpoint) -U postgres

3.3. Connect to the Grafana UI

The following commands will help you to access the Grafana UI, but they do not expose the Grafana service to the internet.

First you need to get the user and password that you use to login to the Grafana UI:

kubectl get secret --namespace default grafana -o jsonpath="{.data.admin-user}" | base64 --decode ; echo
kubectl get secret --namespace default grafana -o jsonpath="{.data.admin-password}" | base64 --decode ; echo

After that, you need to start a port forwarder to expose the Grafana UI service:

kubectl port-forward --namespace default service/grafana 3000:80

You can then open your browser and access the Grafana UI at http://localhost:3000. Log in with the user and password that you got above. If you navigate to http://localhost:3000/datasources you will see the "Cinnamon Prometheus" data source that was created automatically when running pulumi up. If you navigate to http://localhost:3000/dashboards you will see the Grafana dashboards from Lightbend Telemetry were also installed.

4. Output Variables

To print all output variables for a deployed stack, you can run the following command:

pulumi stack output

To output the value of only one variable, use the same command and include the variable. For example, to return the clusterName.

pulumi stack output clusterName

The following output variables are available:

GCP GKE

  • clusterName - Cluster name

  • kubeconfig - Complete contents of a KUBECONFIG configuration for the provisioned cluster as a secret. Need to use pulumi stack output kubeconfig --show-secrets to get the list.

  • network - The name or self_link of the Google Compute Engine network to which the cluster is connected.

  • operatorNamespace - Kubernetes namespace where the Akka Cloud Platform operator is running

Cloud SQL Instance

  • postgreSQLEndpoint - The private IP through which direct connectivity between GKE and Cloud SQL instance is set up

  • postgreSQLInstanceConnectionName - The Postgresql instance connection name

  • postgreSQLInstanceName - The Postgresql instance name

5. Pulumi Playbook configuration

You can configure the playbook with the following syntax at the command line:

pulumi config set <CONFIG KEY> <CONFIG VALUE>`

Or by populating the stack’s Pulumi.<STACK NAME>.yaml.

Table 1. Table Configuration
Configuration Key Description Required Default value

gcp:region

The AWS region for the Google GKE cluster and other services.

No

-

gcp:zone

The AWS region for the Google GKE cluster and other services.

Yes (if not set at the provider level)

-

akka.operator.namespace

The Kubernetes namespace to install the Akka Cloud Platform operator into.

No

"lightbend"

akka.operator.version

The version of the Akka Cloud Platform operator to install.

No

"1.1.22"

akka.operator.installTelemetryBackends

Whether to install Lightbend Telemetry backends (Prometheus and Grafana) for monitoring.

No

true

gke.nodePool.initialNodeCount

The initial number of nodes in the GKE node pool.

No

3

gke.nodePool.autoscaling.maxNodeCount

The maximum number of nodes in the GKE node pool.

No

7

gke.nodePool.autoscaling.minNodeCount

The minimum number of nodes in the GKE node pool.

No

1

gke.nodePool.nodeConfig.machineType

The machine type for the GKE node pool.

No

n1-standard-4

cloudSql.databaseVersion

The version of the Cloud SQL database to use.

No

POSTGRES_13

cloudSql.settings.tier

The Cloud SQL instance tier.

No

db-f1-micro