The following describes how to migrate resources from TKE cluster A to EKS cluster B.
For detailed directions, see Using COS as Velero Storage to Implement Backup and Restoration of Cluster Resources.
Download the latest version of Velero to the cluster environment. Velero v1.8.1 is used as an example in this document.
Run the following command to decompress the installation package, which contains Velero command lines and some sample files.
tar -xvf velero-v1.8.1-linux-amd64.tar.gz
Run the following command to migrate the Velero executable file from the decompressed directory to the system environment variable directory, that is,
/usr/bin in this document, as shown below:
cp velero-v1.8.1-linux-amd64/velero /usr/bin/
Configure the Velero client and enable CSI.
velero client config set features=EnableCSI
Run the following command to install Velero in clusters A and B and create Velero workloads as well as other necessary resource objects.
velero install --provider aws \
EKS doesn't support DaemonSet deployment, so none of the samples in this document support the restic add-on.
- If you don't need to back up the PVC, see the following installation sample:
./velero install --provider aws --use-volume-snapshots=false --bucket gtest-1251707795 --plugins velero/velero-plugin-for-aws:v1.1.0 --secret-file ./credentials-velero --backup-location-config region=ap-guangzhou,s3ForcePathStyle="true",s3Url=https://cos.ap-guangzhou.myqcloud.com
For installation parameters, see Using COS as Velero Storage to Implement Backup and Restoration of Cluster Resources or run the
velero install --help command.
Other installation parameters are as described below:
|--plugins||Use the AWS S3 API-compatible add-on `velero-plugin-for-aws`; use the CSI add-on velero-plugin-for-csi to back up `csi-pv`. We recommend you enable it.|
|--features||Enable optional features:Enable the API group version feature. This feature is used for compatibility with different API group versions and we recommend you enable it.Enable the CSI snapshot feature. This feature is used to back up the CSI-supported PVC, so we recommend you enable it.|
|--use-restic||Velero supports the restic open-source tool to back up and restore Kubernetes storage volume data (
|--use-volume-snapshots=false||Disable the default snapshot backup of storage volumes.|
velero backup-location get NAME PROVIDER BUCKET/PREFIX PHASE LAST VALIDATED ACCESS MODE DEFAULT default aws <BucketName> Available 2022-03-24 21:00:05 +0800 CST ReadWrite true
At this point, you have completed the Velero installation. For more information, see Velero Documentation.
VolumeSnapshotClassin clusters A and B
- Skip this step if you don't need to back up the PVC.
- For more information on storage snapshot, see Backing up and Restoring PVC via CBS-CSI Add-on.
Make sure you have installed the CBS-CSI add-on.
Use the following YAML to create a
VolumeSnapshotClass object as shown below:
apiVersion: snapshot.storage.k8s.io/v1beta1 kind: VolumeSnapshotClass metadata: labels: velero.io/csi-volumesnapshot-class: "true" name: cbs-snapclass driver: com.tencent.cloud.csi.cbs deletionPolicy: Delete
Run the following command to check whether the
VolumeSnapshotClass has been created successfully as shown below:
$ kubectl get volumesnapshotclass NAME DRIVER DELETIONPOLICY AGE cbs-snapclass com.tencent.cloud.csi.cbs Delete 17m
Skip this step if you don't need to back up the PVC.
Deploy a MinIO workload with the PVC in a Velero instance in cluster A. Here, the
cbs-csi dynamic storage class is used to create the PVC and PV.
provisionerin the cluster to dynamically create the PV for the
com.tencent.cloud.csi.cbsstorage class. A sample PVC is as follows:
apiVersion: v1 kind: PersistentVolumeClaim metadata: annotations: volume.beta.kubernetes.io/storage-provisioner: com.tencent.cloud.csi.cbs name: minio spec: accessModes:
To create a backup in cluster A, see Creating a backup in cluster A in the Cluster Migration directions.
To perform a restoration in cluster B, see Performing a restoration in cluster B in the Cluster Migration directions.
Verify the migration result:
Run the following command to verify the resources in cluster B after migration. You can see that the Pods, PVC, and Service have been successfully migrated as shown below:
Log in to the MinIO service in cluster B. You can see that the images in the MinIO service are not lost, indicating that the persistent volume data has been successfully migrated as expected.
At this point, the migration of resources between TKE and EKS clusters has been completed.
After the migration is complete, run the following command to restore the backup storage locations of clusters A and B to read/write mode as shown below, so that the next backup task can be performed normally:
kubectl patch backupstoragelocation default --namespace velero \
Timeout to ensure pod sandboxerror is reported: The add-ons in EKS Pods communicate with the control plane for health checks. If the network remains disconnected for six minutes after Pod creation, the control plane will initiate the termination and recreation. In this case, you need to check whether the security group associated with the Pod has allowed access to the 169.254 route.
kubectl logscommand, adversely affecting debugging. You can dump the business logs by delaying the termination or setting the
terminationMessagefield as instructed in How to set container's termination message?.
ImageGCFailederror is reported: An EKS Pod has 20 GiB disk size by default. If the disk usage reaches 80%, the EKS control plane will trigger the container image repossession process to try to repossess the unused images and free up the space. If it fails to free up any space,
ImageGCFailed: failed to garbage collect required amount of imageswill be reported to remind you that the disk space is insufficient. Common causes of insufficient disk space include: