You are looking at the documentation of a prior release. To read the documentation of the latest release, please
visit here.
Hooks in Batch Backup
Stash 0.9.0+ supports taking backup of multiple co-related stateful workloads using a single configuration named BackupBatch
. Combining with backup hooks, this can be very powerful. For example, you can prepare your application stack before backup to ensure backup integrity or you can send a notification to a webhook (i.e. in a Slack channel via slack incoming webhook) before or after the backup of all the resources of your application stack.
Here, we are going to demonstrate how to send a notification to a Slack channel before and after the backup of a WordPress application. WordPress application consists of two different workloads. One is for WordPress itself and another for a MySQL database.
Note that, this is an advanced concept. If you haven’t tried the normal backup restore processes yet, we will recommend to try them first.
Before You Begin
- At first, you need to have a Kubernetes cluster, and the
kubectl
command-line tool must be configured to communicate with your cluster. If you do not already have a cluster, you can create one by using kind. - Install Stash in your cluster following the steps here.
- Install KubeDB in your cluster following the steps here. This step is optional. You can deploy your database using any method you want. We are using KubeDB because KubeDB simplifies many of the difficult or tedious management tasks of running production-grade databases on private and public clouds.
- If you are not familiar with how Stash backup and restore MySQL databases, please check the following guide here.
- Also, if you haven’t read about how hooks work in Stash, please check it from here.
You should be familiar with the following Stash
concepts:
To keep everything isolated, we are going to use a separate namespace called demo
throughout this tutorial.
$ kubectl create ns demo
namespace/demo created
Prepare Application
At first, let’s deploy a WordPress application. Here, we are going to deploy WordPress 5.3.2
with MySQL 8.0.14
. We are going to deploy the database first. Then, we are going to deploy the Deployment for WordPress once the database is ready.
Deploy Database
Here, we are using KubeDB to deploy MySQL 8.0.14
. You can deploy the database without KubeDB but you have to create some resources manually to ensure it works with Stash.
Create MySQL:
Below is the YAML of the MySQL
CR that we are going to create.
apiVersion: kubedb.com/v1alpha2
kind: MySQL
metadata:
name: wordpress-mysql
namespace: demo
spec:
version: 8.0.27
replicas: 1
storageType: Durable
storage:
accessModes:
- ReadWriteOnce
resources:
requests:
storage: 1Gi
podTemplate:
spec:
args:
- --default-authentication-plugin=mysql_native_password # without this flag wordpress will not connect with newer version of MySQL
terminationPolicy: WipeOut
Notice the
spec.podTemplate
part. We are providing--default-authentication-plugin=mysql_native_password
flag tomysqd
. Otherwise, WordPress won’t be able to connect with this database.
Let’s create the above database,
$ kubectl apply -f https://github.com/stashed/docs/raw/v2024.2.9-rc.0/docs/guides/hooks/batch-backup/examples/wordpress-mysql.yaml
mysql.kubedb.com/wordpress-mysql created
KubeDB will deploy a MySQL database according to the above specification. It will also create the necessary Secrets and Services to access the database.
Wait for the database to go into Running
state,
$ kubectl get mysql -n demo -w
NAME VERSION STATUS AGE
wordpress-mysql 8.0.14 Creating 17s
wordpress-mysql 8.0.14 Running 3m10s
Verify Database Secret:
Verify that KubeDB has created a Secret for the database.
$ kubectl get secret -n demo -l=app.kubernetes.io/instance=wordpress-mysql
NAME TYPE DATA AGE
wordpress-mysql-auth Opaque 2 4m1s
Verify AppBinding:
KubeDB creates an AppBinding
CR that holds the necessary information to connect with the database. Verify that the AppBinding
has been created for the above database:
$ kubectl get appbindings -n demo -l=app.kubernetes.io/instance=wordpress-mysql
NAME TYPE VERSION AGE
wordpress-mysql kubedb.com/mysql 8.0.14 2m10s
Deploy Wordpress
Now, let’s deploy WordPress itself. Below is the YAML of the respective resources that we are going to create for WordPress.
apiVersion: v1
kind: PersistentVolumeClaim
metadata:
name: wordpress-pvc
namespace: demo
labels:
app: wordpress
spec:
accessModes:
- ReadWriteOnce
resources:
requests:
storage: 1Gi
---
apiVersion: apps/v1
kind: Deployment
metadata:
name: wordpress-deployment
namespace: demo
labels:
app: wordpress
spec:
selector:
matchLabels:
app: wordpress
tier: frontend
strategy:
type: Recreate
template:
metadata:
labels:
app: wordpress
tier: frontend
spec:
containers:
- image: wordpress:5.3.2-apache
name: wordpress
env:
- name: WORDPRESS_DB_HOST
value: wordpress-mysql
- name: WORDPRESS_DB_PASSWORD
valueFrom:
secretKeyRef:
name: wordpress-mysql-auth
key: password
- name: WORDPRESS_DB_USER
valueFrom:
secretKeyRef:
name: wordpress-mysql-auth
key: username
ports:
- containerPort: 80
name: wordpress
volumeMounts:
- name: web
mountPath: /var/www/html
volumes:
- name: web
persistentVolumeClaim:
claimName: wordpress-pvc
Let’s create the above resources,
$ kubectl apply -f https://github.com/stashed/docs/raw/v2024.2.9-rc.0/docs/guides/hooks/batch-backup/examples/wordpress-deployment.yaml
persistentvolumeclaim/wordpress-pvc created
deployment.apps/wordpress-deployment created
Verify that WordPress pod ready
$ kubectl get pod -n demo -l=app=wordpress,tier=frontend
NAME READY STATUS RESTARTS AGE
wordpress-deployment-586f94487c-nm8p5 1/1 Running 0 2m26s
Prepare for Backup
Now, let’s prepare for the backup. Here, we are going to prepare the backend where we will store our backed up data and we are going to setup a Slack Incoming Webhook where we will send the notifications for backup. Configure a Slack incoming webhook following the guides from here.
Prepare Backend
We are going to store our backed up data into a GCS bucket. At first, we need to create a secret with GCS credentials then we need to create a Repository
CR. If you want to use a different backend, please read the respective backend configuration doc from here.
Create Storage Secret:
Let’s create a secret called gcs-secret
with access credentials to our desired GCS bucket,
$ echo -n 'changeit' > RESTIC_PASSWORD
$ echo -n '<your-project-id>' > GOOGLE_PROJECT_ID
$ cat /path/to/downloaded-sa-key.json > GOOGLE_SERVICE_ACCOUNT_JSON_KEY
$ kubectl create secret generic -n demo gcs-secret \
--from-file=./RESTIC_PASSWORD \
--from-file=./GOOGLE_PROJECT_ID \
--from-file=./GOOGLE_SERVICE_ACCOUNT_JSON_KEY
secret/gcs-secret created
Create Repository:
Now, create a Repository
using this secret. Below is the YAML of Repository
CR we are going to create,
apiVersion: stash.appscode.com/v1alpha1
kind: Repository
metadata:
name: gcs-repo
namespace: demo
spec:
backend:
gcs:
bucket: appscode-qa
prefix: /demo/batch-backup/hook-example
storageSecretName: gcs-secret
Let’s create the Repository
we have shown above,
$ kubectl apply -f https://github.com/stashed/docs/raw/v2024.2.9-rc.0/docs/guides/hooks/batch-backup/examples/repository.yaml
repository.stash.appscode.com/gcs-repo created
Backup
Let’s schedule a backup for our WordPress application stack. Here, we are going to configure a preBackup
hook of a BatchBackup
CR to send a notification when the backup starts and a postBackup
hook to send another notification when the backup ends.
Create BackupBatch:
Below is the YAML of the BackupBatch
CR with preBackup
and postBackup
hooks configured to send a notification into a slack channel before and after a backup respectively.
apiVersion: stash.appscode.com/v1beta1
kind: BackupBatch
metadata:
name: wordpress-backup
namespace: demo
spec:
repository:
name: gcs-repo
schedule: "*/3 * * * *"
members:
- target:
alias: db
ref:
apiVersion: apps/v1
kind: AppBinding
name: wordpress-mysql
task:
name: mysql-backup-8.0.14
- target:
alias: app
ref:
apiVersion: apps/v1
kind: Deployment
name: wordpress-deployment
volumeMounts:
- name: web
mountPath: /var/www/html
paths:
- /var/www/html
hooks:
preBackup:
httpPost:
host: hooks.slack.com
path: /services/TXXXXX/BXXXXXXX/XXXXXXXXXXXXXXXXXXXXXXX
port: 443
scheme: HTTPS
httpHeaders:
- name: Content-Type
value: application/json
body: '{
"blocks": [
{
"type": "divider"
},
{
"type": "section",
"text": {
"type": "mrkdwn",
"text": "Backup started for *wordpress-backup*.\n *Targets:*\n- deployment/wordpress-deployment\n- mysql/wordpress-mysql"
}
}
]
}'
postBackup:
httpPost:
host: hooks.slack.com
path: /services/TXXXXX/BXXXXXXX/XXXXXXXXXXXXXXXXXXXXXXX
port: 443
scheme: HTTPS
httpHeaders:
- name: Content-Type
value: application/json
body: '{
"blocks": [
{
"type": "section",
"text": {
"type": "mrkdwn",
"text": "Backup has been completed for *wordpress-backup*."
}
}
]
}'
retentionPolicy:
name: 'keep-last-5'
keepLast: 5
prune: true
Here, we have used hooks.slack.com
part of the webhook URL in host
field and the rest of the path in path
field.
You can customize the body
section of httpPost
hook to change the visual representation of your notification. You can use slack’s official Block Kit Builder to build a beautiful notification layout.
Let’s create the above BackupBatch
object,
$ kubectl apply -f https://github.com/stashed/docs/raw/v2024.2.9-rc.0/docs/guides/hooks/batch-backup/examples/wordpress-backup.yaml
backupbatch.stash.appscode.com/wordpress-backup created
Verify CronJob
If everything goes well, Stash will create a CronJob with the schedule specified in spec.schedule
field of the BackupBatch
CR.
$ kubectl get cronjob -n demo
NAME SCHEDULE SUSPEND ACTIVE LAST SCHEDULE AGE
stash-backup-wordperss-backup */3 * * * * False 0 <none> 27s
Wait for BackupSession
The stash-backup-wordpress-backup
CronJob will trigger a backup on each scheduled slot by creating a BackupSession
CR.
Wait for a schedule to appear. Run the following command to watch BackupSession
CR,
$ kubectl get backupsession -n demo -w
NAME INVOKER-TYPE INVOKER-NAME PHASE AGE
wordpress-backup-1579526461 BackupBatch wordpress-backup Running 0s
wordpress-backup-1579526461 BackupBatch wordpress-backup Running 24s
wordpress-backup-1579526461 BackupBatch wordpress-backup Running 37s
wordpress-backup-1579526461 BackupBatch wordpress-backup Succeeded 38s
Here, the phase Succeeded
means that the backup process has been completed successfully.
Verify Backup
Once a backup is completed, Stash will update the respective Repository
CR to reflect the backup completion. Check that the repository gcs-repo
has been updated by the following command,
$ kubectl get repository -n demo gcs-repo
NAME INTEGRITY SIZE SNAPSHOT-COUNT LAST-SUCCESSFUL-BACKUP AGE
gcs-repo true 2 72s 18m
Here, SNAPSHOT-COUNT
2 indicates that one snapshot for each target has been taken successfully.
Verify Backup Hooks Executed
Now, go to your slack channel. You should see that Stash has sent notification before and after the backup of WordPress application.
![Backup Notification](images/notification.png)
Cleanup
To cleanup the Kubernetes resources created by this tutorial, run:
kubectl -n demo delete backupbatch wordpress-backup
kubectl -n demo delete repository gcs-repo
kubectl -n demo delete secret gcs-secret
kubectl -n demo delete deployment wordpress-deployment
kubectl -n demo delete mysql wordpress-mysql