Kubevirt-Image-Service (KIS) has 3 custom resources to manage images and volumes.
VirtualMachineImage
: imports a qcow2 image from external sources like HTTP, S3, and local path to K8s cluster. Imported image is saved as read-only PVC in K8s cluster and will be used to create volume for VMs. Currently HTTP is only supported import option.VirtualMachineVolume
: creates a volume which will be used by VM from an image. Different from read-only image, created volume is able to write data. Only changed data between each volume is stored by utilizing snapshot and restore feature of CSI (Container Storage Interface). By this way, user can manage storage capacity efficiently.VirtualMachineExport
: converts a volume to a qcow2 file and exports it to external destinations. Currently export to local destination is only supported import option.
- deploy K8s cluster through minikube, kubeadm, kubespray, or other methods
- If K8s version is below
1.17
make sure volume snapshotting feature is enabled. Set the following flag on the API server binary:--feature-gates=VolumeSnapshotDataSource=true
. Check this article for more details.
- If K8s version is below
- deploy kubevirt
A CSI plugin that can provision volume snapshot is needed. StorageClass
and VolumeSnapshotClass
are needed to deploy as well.
For example, you can use rook-ceph rbd plugin. In this case, make sure you are using ceph-csi version above 2.0.0
. A bugfix for volume snapshot is applied on this version.
- deploy rook-ceph
- deploy StorageClass
- deploy VolumeSnapshotClass
# Download kubevirt-image-service project
$ git clone https://github.com/tmax-cloud/kubevirt-image-service.git
# Select release version that you want to install
$ cd kubevirt-image-service/
$ git checkout v1.0.0
# Deploy CRD
$ kubectl apply -f deploy/crds/hypercloud.tmaxanc.com_virtualmachineimages_crd.yaml
$ kubectl apply -f deploy/crds/hypercloud.tmaxanc.com_virtualmachinevolumes_crd.yaml
$ kubectl apply -f deploy/crds/hypercloud.tmaxanc.com_virtualmachinevolumeexports_crd.yaml
# Deploy operator
$ kubectl apply -f deploy/namespace.yaml
$ kubectl apply -f deploy/role.yaml
$ kubectl apply -f deploy/role_binding.yaml
$ kubectl apply -f deploy/service_account.yaml
$ kubectl apply -f deploy/operator.yaml
# Check operator status
$ kubectl get deploy -n kis
NAME READY UP-TO-DATE AVAILABLE AGE
kubevirt-image-service 3/3 3 3 23s
vmim is the shortname for VirtualMachineImage
.
# Deploy image CR. See the following yaml file for more information about each CR fields
$ kubectl apply -f deploy/crds/hypercloud.tmaxanc.com_v1alpha1_virtualmachineimage_http_cr.yaml
# Wait until image state is ready to use
$ kubectl get vmim
NAME STATE
myubuntu Available
# Create a qcow2 image file with the name `disk.img` in the desired path (/mnt/data).
# Deploy host path image CR using the path that created the qcow2 file.
$ kubectl apply -f deploy/crds/hypercloud.tmaxanc.com_v1alpha1_virtualmachineimage_hostpath_cr.yaml
# Wait until image state is ready to use
$ kubectl get vmim
NAME STATE
localvmim Available
# When the status of vmim becomes Availalbe, user can delete the qcow2 file.
vmv is the shortname for VirtualMachineVolume
.
# Deploy volume CR
$ kubectl apply -f deploy/crds/hypercloud.tmaxanc.com_v1alpha1_virtualmachinevolume_cr.yaml
# Wait until volume state is ready to use
$ kubectl get vmv
NAME STATE
myrootdisk Available
On VM yaml file, add disks
and volumes
section with PVC and disk information
apiVersion: kubevirt.io/v1alpha3
kind: VirtualMachine
metadata:
name: vm
spec:
template:
spec:
domain:
devices:
disks:
- name: disk0
disk:
bus: virtio
volumes:
- name: disk0
persistentVolumeClaim:
claimName: myubuntu-pvc # name of pvc created by volume, {$VmvName}-vmv-pvc
vmve is the shortname for VirtualMachineExport
.
# Deploy export CR
$ kubectl apply -f deploy/crds/hypercloud.tmaxanc.com_v1alpha1_virtualmachinevolumeexport_cr.yaml
# Wait until export is completed
$ kubectl get vmve
NAME STATE
disk-export Completed
# Check exported pod
$ kubectl get pod
NAME READY STATUS RESTARTS AGE
disk-export-exporter-local 1/1 Running 6 67m
# Copy exported volume to local path
# export/disk.img is in qcow2 format
# kubectl cp {vmve exporter pod name}:export/disk.img {local path to download}
$ kubectl cp disk-export-exporter-local:export/disk.img localpath.img
# Create k8s secret with your accessKeyId and secretAccessKey of the external endpoint
$ kubectl apply -f deploy/example/endpoint-secret.yaml
# Deploy export CR
# Your s3 endpoint must be written in virtual hosted style, https://bucket-name.endpoint/object-key-name
# You need to specify secretRef as well with the secret you created in the previous step
$ kubectl apply -f deploy/crds/hypercloud.tmaxanc.com_v1alpha1_virtualmachinevolumeexport_s3_cr.yaml
# Wait until export is completed
$ kubectl get vmve
NAME STATE
s3-export Completed