Develop and test easily on AKS with NetApp® Astra Control Service® and Azure NetApp Files

Published Monday, September 12, 2022

Table of Contents

Installation of sample application

Manage WordPress application with ACS

Enable application consistent snapshots

Upgrade test

Summary

Additional Information

Abstract

In this article, we describe how NetApp Astra Control® can simplify development & testing (dev & test) use cases for applications running on Azure Kubernetes Service (AKS), using Azure NetApp Files as the storage service. We demonstrate how Astra Control enables application teams to easily test production applications running on AKS for compatibility with newer AKS versions before upgrading the production clusters.

Co-authors: Patric Uebele, Sayan Saha

Introduction

NetApp Astra Control is a solution that makes it easier for our customers to manage, protect, and move their data-rich containerized workloads running on Kubernetes within and across public clouds and on-premises. Astra Control provides persistent container storage that leverages NetApp’s proven and expansive storage portfolio in the public cloud and on premises. It also offers a rich set of advanced application-aware data management functionality (like snapshot and revert, backup and restore, activity logs, and active cloning) for local data protection, disaster recovery, data audit, and migration use cases for your modern apps. Astra Control provides a complete protection of stateful Kubernetes applications by saving data stored in persistent storage, but also the metadata, Kubernetes objects like deployments, config maps, services, secrets... that constitute an application in Kubernetes.

Astra Control can either be managed via its user interface, accessed by any web browser, or via its powerful REST API.

Astra Control is available in two variants:

Astra Control Service (ACS) – A fully managed application-aware data management service that supports Azure Kubernetes Service (AKS), Azure Disk Storage, and Azure NetApp Files (ANF).

Astra Control Center (ACC) – application-aware data management for on-premises Kubernetes clusters, delivered as a customer-managed Kubernetes application from NetApp.

Astra Control can support, simplify, and help automating various test/dev use cases in Kubernetes, like:

Application mobility
- Astra Control provides the ability to move an application across namespaces, clusters, accounts, regions, and clouds. This enables a variety of use cases including Disaster Recovery (DR), Test/Dev with realistic data sets, and performance testing in isolated environments.
Compliance audit environment
- Comply with audit requests that require reproducing an application environment configuration and state, including underlying data from a past given period.
Cluster upgrade testing
- Understand the impact of Kubernetes and application upgrades by migrating applications in a predictable manner to an upgraded cluster. Catch and address upgrade related issues before it is too late.

Scenario

To demonstrate how Astra Control Service makes it easy to test Kubernetes upgrades, we deployed a small test environment of two AKS clusters running with different Kubernetes versions. The cluster with the older K8s version hosts a WordPress application, and with the help of Astra Control’s application cloning feature, we will test the WordPress application with its current data set on the newer K8s version. The persistent data of the WordPress application are stored in two persistent volumes, backed by Azure NetApp Files volumes.

Preparation

In our test scenario, we have prepared two AKS clusters:

Cluster aks, running K8s v1.21.9 and hosting the WordPress application we want to test on a higher K8s version.
Cluster aks-new, running K8s v1.23.5 to test application compatibility with this K8s version.

Both clusters are managed by Astra Control Service already, the Cluster view in ACS shows them as managed compute and displays their K8s version:

Figure 1: ACS Cluster view with managed compute.

Both clusters have persistent storage backed by Azure NetApp Files, persistent volumes will be provisioned via the Astra Trident dynamic provisioner.

Installation of sample application

We install WordPress as our sample application on cluster aks (K8s v1.21.9) in namespace wordpress1 using the latest bitnami helm chart:

~# helm install wordpress1 bitnami/wordpress --namespace wordpress1 --set wordpressUsername=astra,wordpressPassword=NetApp1! --create-namespace
NAME: wordpress1
LAST DEPLOYED: Mon Aug  8 10:07:34 2022
NAMESPACE: wordpress1
STATUS: deployed
REVISION: 1
TEST SUITE: None
NOTES:
CHART NAME: wordpress
CHART VERSION: 15.0.14
APP VERSION: 6.0.1

** Please be patient while the chart is being deployed **

Your WordPress site can be accessed through the following DNS name from within your cluster:

    wordpress1.wordpress1.svc.cluster.local (port 80)

To access your WordPress site from outside the cluster follow the steps below:
1. Get the WordPress URL by running these commands:

  NOTE: It may take a few minutes for the LoadBalancer IP to be available.
        Watch the status with: 'kubectl get svc --namespace wordpress1 -w wordpress1'

   export SERVICE_IP=$(kubectl get svc --namespace wordpress1 wordpress1 --include "{{ range (index .status.loadBalancer.ingress 0) }}{{ . }}{{ end }}")
   echo "WordPress URL: http://$SERVICE_IP/"
   echo "WordPress Admin URL: http://$SERVICE_IP/admin"

2. Open a browser and access WordPress using the obtained URL.

3. Login with the following credentials below to see your blog:

  echo Username: astra
  echo Password: $(kubectl g h^Cecret --namespace wordpress1 wordpress1 -o jsonpath="{.data.wordpress-password}" | base64 -d)

The persistent volumes of the WordPress application are backed by Azure NetApp Files volumes in storage class netapp-anf-perf-standard:

~# kubectl get all,pvc -n wordpress1
NAME                              READY   STATUS    RESTARTS   AGE
pod/wordpress1-595c9db864-tftjs   1/1     Running   1          11m
pod/wordpress1-mariadb-0          1/1     Running   0          11m

NAME                         TYPE           CLUSTER-IP    EXTERNAL-IP     PORT(S)                      AGE
service/wordpress1           LoadBalancer   10.0.45.148   51.124.229.33   80:31523/TCP,443:30023/TCP   11m
service/wordpress1-mariadb   ClusterIP      10.0.208.55   <none>          3306/TCP                     11m

NAME                         READY   UP-TO-DATE   AVAILABLE   AGE
deployment.apps/wordpress1   1/1     1            1           11m

NAME                                    DESIRED   CURRENT   READY   AGE
replicaset.apps/wordpress1-595c9db864   1         1         1       11m

NAME                                  READY   AGE
statefulset.apps/wordpress1-mariadb   1/1     11m

NAME                                              STATUS   VOLUME                                     CAPACITY   ACCESS MODES   STORAGECLASS               AGE
persistentvolumeclaim/data-wordpress1-mariadb-0   Bound    pvc-0e2b9075-84ce-4dd6-afb4-23e24d63fbe8   100Gi      RWO            netapp-anf-perf-standard   11m
persistentvolumeclaim/wordpress1                  Bound    pvc-b7f12752-d820-417f-aa5d-dcb67c569fdb   100Gi      RWO            netapp-anf-perf-standard   11m

Checking the Kubernetes resources in the wordpress1 namespace, we can find out the external IP address of the WordPress service: 51.124.229.33 and connect to WordPress in a browser window:

Figure 2: WordPress blog post on K8s v1.21.9.

We’ve added a blog post entry to WordPress, running on K8s v1.21.9.

Manage WordPress application with ACS

Now that WordPress is running on cluster aks, we can proceed to protecting the application with Astra Control Service. To manage the application with ACS, we first need to define it as a managed application in ACS. Astra Control regularly discovers the namespaces on the managed clusters, and we can define applications we want to manage based on the discovered namespaces. We can choose to manage an entire namespace as a single application or manage one or more apps in the namespace individually.

With the relatively simple WordPress application, it’s easiest to manage its whole namespace. We can do that for example in the details view of the cluster aks. In the Namespaces tab, we see all discovered namespaces on the cluster and can see that the namespace wordpress1 does not yet have an application associated. To do so, we select the wordpress1 namespace and define it as an application via the Actions menu:

Figure 3: Defining wordpress1 namespace as application wordpress1.

This will immediately define the contents of the wordpress1 namespace as managed application wordpress1, so we can see a link to the associated application in the list of namespaces of cluster aks now:

Figure 4: Application wordpress1 is now associated to namespace wordpress1.

Enable application consistent snapshots

To create application consistent snapshots and backups, we can leverage Astra Controls capability to include execution hooks into data protection operations. An execution hook is a custom action that you can configure to run in conjunction with a data protection operation of a managed app. For example, if you have a database app, you can use execution hooks to pause all database transactions before a snapshot, and resume transactions after the snapshot is complete to ensure application-consistent snapshots.

The NetApp-provided default pre- and post-snapshot execution hooks for specific applications have been removed in the latest release of Astra Control. These hooks, plus a variety of further sample execution hooks, are now available in the NetApp Verda GitHub repository and can be added to your Astra Control account for use as execution hooks. The scripts can also be modified to fit your environment and requirements.

The add a pre-and post-snapshot execution hook for MariaDB to our ACS account, we download the MariaDB/MySQL hook example script from the NetApp Verda GitHub repository:

Figure 5: NetApp Verda GitHub repo for execution hook example scripts.

The readme of the MariaDB/MySQL hook script details the required arguments (pre and post) and the supported operations (pre- and post-snapshot). In the Scripts tab of the Astra Control Account view, we can add scripts to the library of execution hooks of our Astra Control account:

Click Add
Figure 6: Adding scripts to the Astra Control script library.
Select script to upload from your local PC:

Figure 7: Upload script file to the library.

Figure 8: Upload the mariadb_mysql.sh hooks script from the Verda GitHub repo.

Confirm or change the suggested script name and save the uploaded script to the hook script library of your Astra Control Account:

Figure 9: Save the uploaded file to the script library.

The uploaded mariadb_mysql.sh script is now available in our account’s script library. It’s not used by any execution hooks yet.

Figure 10: Astra Control script library.

To configure the uploaded mariadb_mysql.sh script as pre-and post-snapshot execution hook for the MariaDB instance in our wordpress1 app, we navigate to the Scripts tab in the app details view of the wordpress1 app and click Add to add an execution hook:

Figure 11: Adding execution hooks to wordpress1 application.

Now we can enter the details to define the pre-snapshot hook for MariaDB:

As operation, select pre-snapshot from the drop-down list.
As hook argument, enter pre (see the hook script or its readme for supported and required hook arguments).
Enter a hook name – must be unique across all your managed applications.
Enter a regular expression to determine the containers where the hook script shall be executed – based on container image matches. To find out the container image name, you can use the kubectl describe command:

# kubectl describe po wordpress1-mariadb-0 -n wordpress1 | grep Image
    Image:          docker.io/bitnami/mariadb:10.6.8-debian-11-r22
    Image ID:       docker.io/bitnami/mariadb@sha256:7ada50ba1a77a0543ab88d7fb74cc2eb914ddd4d18328a4d45e1afa1c1cb0618

Select the mariadb_mysql.sh script from the list of available scripts in the script library.
Click Add hook.

Figure 12: Define the pre-snapshot hook for MariaDB.

Repeating the steps above, we also add a post-snapshot hook for MariaDB:

Figure 13: Configuration of the post-snapshot hook for MariaDB.

Both hooks are now shown in the list of execution hooks for the wordpress1 app, including the container image matches:

Figure 14: Execution hooks for wordpress1.

By taking a first on-demand snapshot of wordpress1 from its data protection tab, we test the proper execution of the pre- and post-snapshot hooks:

Figure 15: On-demand snapshot creation of wordpress1.

Astra Control’s activity log does log the successful start and completion of the pre- and post-snapshot hooks during the snapshot operation:

Figure 16: Details of hooks execution in activity log.

Upgrade test

Now, we’re all set to test the WordPress application on K8s v1.23.5 before upgrading the “production” cluster, by cloning the live WordPress application to our AKS cluster aks-new, which is running on K8s v1.23.5 already.

We initiate the clone process from the application view in the ACS UI:

Figure 17: Initiate clone of live WordPress app on K8s v1.21.9

After specifying wordpress1-clone as name for the cloned WordPress application in ACS, and wordpress1-clone as namespace in which the cloned app will reside, we can define the destination cluster. Here we select the cluster aks-new from the drop-down menu, which runs on K8s v1.23.5 already. As we want to test with up-to-date application data, we don’t select an existing snapshot or backup as clone source, but let Astra Control create a fresh snapshot and backup:

Figure 18: Select cluster aks-new running K8s v1.23.5 as destination cluster.

After reviewing the clone details, we start the clone operation:

Figure 19: Review the clone information.

ACS will create an application-consistent snapshot of the WordPress application, and then take a backup (to object storage) from the snapshot. Azure NetApp Files snapshots are based on the proven ONTAP snapshot technology, so they’re fast and have no performance impact on the running application.

The backup progress can be watched in the ACS UI in the Data Protection tab of the wordpress1 application view:

Figure 20: Backup is running as part of the clone process.

After some minutes, the backup process finishes, and the restore operation on the destination cluster aks-new begins. ACS first creates the namespace wordpress1-clone on the destination cluster and then starts restoring the persistent volumes from the backup and creating the K8s resources of the application. We can watch the progress of the clone operation on the destination cluster aks-new:

~# kubectl config use-context aks-new
Switched to context "aks-new".
~#
~# kubectl get all,pvc -n wordpress1-clone
NAME                                    READY   STATUS      RESTARTS   AGE
pod/r-data-wordpress1-mariadb-0-rg57q   0/1     Completed   0          3m25s
pod/r-wordpress1-sdf67                  1/1     Running     0          3m26s

NAME                                    COMPLETIONS   DURATION   AGE
job.batch/r-data-wordpress1-mariadb-0   1/1           2m50s      3m25s
job.batch/r-wordpress1                  0/1           3m26s      3m26s

NAME                                              STATUS   VOLUME                                     CAPACITY   ACCESS MODES   STORAGECLASS               AGE
persistentvolumeclaim/data-wordpress1-mariadb-0   Bound    pvc-e7ee741e-363f-4d0c-ab78-93ebf8f2ff7a   100Gi      RWO            netapp-anf-perf-standard   3m27s
persistentvolumeclaim/wordpress1                  Bound    pvc-6998f7e4-e54f-4250-ab01-12f87c9a3111   100Gi      RWO            netapp-anf-perf-standard   3m27s

Once the restore operation finishes, all the resources will come up and we can find the external IP address of the cloned WordPress app wordpress1-clone on cluster aks-new:

~# kubectl get all,pvc -n wordpress1-clone
NAME                              READY   STATUS    RESTARTS   AGE
pod/wordpress1-686775fff8-47mrb   1/1     Running   0          85s
pod/wordpress1-mariadb-0          1/1     Running   0          83s

NAME                         TYPE           CLUSTER-IP     EXTERNAL-IP   PORT(S)                      AGE
service/wordpress1           LoadBalancer   10.0.181.20    20.4.106.46   80:32390/TCP,443:30838/TCP   83s
service/wordpress1-mariadb   ClusterIP      10.0.251.162   <none>        3306/TCP                     83s

NAME                         READY   UP-TO-DATE   AVAILABLE   AGE
deployment.apps/wordpress1   1/1     1            1           85s

NAME                                    DESIRED   CURRENT   READY   AGE
replicaset.apps/wordpress1-686775fff8   1         1         1       85s

NAME                                  READY   AGE
statefulset.apps/wordpress1-mariadb   1/1     83s

NAME                                              STATUS   VOLUME                                     CAPACITY   ACCESS MODES   STORAGECLASS               AGE
persistentvolumeclaim/data-wordpress1-mariadb-0   Bound    pvc-e7ee741e-363f-4d0c-ab78-93ebf8f2ff7a   100Gi      RWO            netapp-anf-perf-standard   5m16s
persistentvolumeclaim/wordpress1                  Bound    pvc-6998f7e4-e54f-4250-ab01-12f87c9a3111   100Gi      RWO            netapp-anf-perf-standard   5m16s

We can now connect to the cloned WordPress application running on K8s v1.23.5 on cluster aks-new and see that it came up with the latest blog post entry:

Figure 21: Cloned WordPress application running on K8s v1.23.5.

To test for write access to WordPress, let’s create another blog post...:

Figure 22: Adding a blog post in the cloned WordPress app on K8s v1.23.5.

...which works without problems:

Figure 23: Blog post added successfully in the cloned WP app.

We can conclude that the WordPress application is functioning properly on K8s v1.23.5.

Summary

This article demonstrated how the application clone functionality of Astra Control Service can be utilized to test Kubernetes applications quickly and easily on new Kubernetes versions before upgrading a Kubernetes cluster.