9. Managing Images¶
In a cloud environment, Virtual Machines are instantiated from images. These images are registered in an Image Management Service, in our case provided by the Glance OpenStack component.
There are several types of images:
These are images provided by the CloudVeneto administrators. They are visible to all users.
A private image is owned by a specific project and cannot be viewed or used by other projects.
A shared image is a private image that can be viewed/used by specific other projects that the image owner adds as “members” to the image.
A community image is an image uploaded by a project, and such project wants other projects use such image, but isn’t interested in maintaining a relationship with these tenants by making them image members.
9.1. Public Images¶
Public images are provided by the CloudVeneto administrators. These images are visible to all users. They appear with Visibility equal to Public in the Compute → Images menu.
9.1.1. Public Images for INFN Padova users¶
The SL6x-INFNPadova-x86-64-<date> and CentOS7x-INFNPadova-x86-64-<date> images are basic SL6.x / CentOS 7.x images which also include cloud-init to perform contextualization based on the user data specified when the VM are instantiated. They also configure CVMFS and the relevant squid servers.
Such images also configure the Padova LDAP server for user authentication. This means that it is just necessary to “enable” the relevant accounts on the VM adding in the /etc/passwd file:
+name1:::::: +name2:::::: ...
and creating their home directories (and changing their ownership). E.g.
sudo mkdir /home/pippo sudo id pippo
The last command will return a string “<userid>(pippo) gid=<groupid>(utenti) groups=….” with the userid and gid to be used in the chown command:
sudo chown -R <userid>.<groupid> /home/pippo
You might also want to set a different home directory wrt the one specified in LDAP, e.g.:
Changes done in /etc/passwd could not be applied immediately by the system. In this case a:
nscd -i passwd
You can access instances created using these images, through ssh using the Cloud keypair, considering the ‘root’ account, e.g.:
ssh -i ~/private/my_key email@example.com
The SL6x-INFNPadova-x86-64-<date> and CentOS7x-INFNPadova-x86-64-<date> images also allow INFN-Padova system administrators to log (with admin privileges) on the instance.
These images for INFN-Padova users are supposed to be used only by INFN Padova users and only for projects using a INFN (i.e. 10.64.x.0/24 network). Using these images on Unipd projects (i.e. the one using 10.67.x.0/24 networks) can cause problems.
9.2. Private Images¶
Users can provide their own images and upload them to the Cloud Image service: these images are private, meaning that they are only available to the users of the project they are uploaded for.
Users are not allowed to publish public (i.e. available to all projects) images.
Many open source projects such as Ubuntu and Fedora produce pre-built images which can be used for certain clouds. If these are available, it is much easier to use them compared to building your own.
Using an Ubuntu image as an example, after you downloaded the image from the relevant web site, to upload such image proceed as follows:
- Be sure you have selected the right Project from the dropdown menu on the top.
- Go to Compute → Images on the left hand menu. This will display the list of public and private images currently available in your project.
Click on +Create Image. The following form will appear:
Fill the needed fields. In particular specify a name for this image (to be specified in the Image Name field). Select the downloaded file in the Image Source field, select the Format. Select Private as Visibilility
When done, click on Create Image.
Once loaded, the image can then be used to create virtual machines.
Images are usually provided in several formats. The most used ones in Cloud environments are ‘qcow2’ and ‘raw’.
When considering the format to choose, you should take into account how the instantiation of a virtual machine on a Cloud compute node works. When a virtual machine is instantiated, first of all the relevant image is staged to the target hypervisor, if needed (i.e. if this is the first instance created on that compute node using this image). Then the image is converted to raw, if it is not already in that format.
A qcow2 image is usually much smaller with respect to the same image provided in raw format. This means that:
- the registration to the OpenStack image service for images in qcow2 format is faster with respect to images in raw format
- the staging of images in qcow2 format to the target compute node is faster with respect to images in raw format
However qcow2 images, as reported above, need to be translated into raw format, and this can take a non negligible time, in particular for images with a large virtual disk size.
In CloudVeneto image size is limited to 25 GB.
9.4. Community images¶
Let’s suppose that a project wants to make a prepared VM image available so that in order to use the project’s software, all you have to do is boot an instance from their image. However they worry about the hassle of maintaining a list of “customers” (i.e. they don’t want to use the image sharing stuff described in the previous section).
The vendor can make the image a community’ image. Such image won’t appear in user’s default image lists of other projects (so they won’t know about it unless they are motivated to seek it out).
The process to create a community image is the one described to create “ref”private images<userprovidedimages>: the only difference is that the Visibility field must be set to Community.
A consumer can find a community image uploaded by another project by doing an image list and filtering with the ‘Community’ value for the ‘visibility’ field.
9.5. Building Images¶
Users can also build custom images, that can then been uploaded in the Cloud Image service as described in User Provided Images.
There are several tools providing support for image creation. Some of them are described in the Openstack documentation.
One example is virt-builder, which is briefly described in the next subsection.
Please consider these guidelines when creating an image:
- Data inside images can be accessed by the other members of the project. So please don’t store sensitive information (e.g. password) in the image.
- Please make sure that the cloud-init and cloud-utils packages are installed. This is needed for contextualisation (see Contextualisations), which allows to customize the image after installation.
- If you use Fedora, CentOS, or RHEL, the cloud-utils-growpart and gdisk packages must be installed, since they are needed for extending partitions. if you use Ubuntu or Debian, the cloud-initramfs-growroot package, which supports resizing root partition on the first boot, must be installed.
Another possible way to create an image is:
- instantiating a virtual machine using an already registered image
- applying your own customization on this instance (e.g. installing new packages)
- creating a snapshot of this image
- using the snapshot to instantiate new virtual machines
However this usually means creating very large and quite inefficient images. Therefore we strongly suggest instead to use the provided public images and applying your customizations using contextualization (see Contextualisations) or to create new images using one of the tools mentioned above.
Virt-builder is a command line tool for quickly creating customized images.
It takes cleanly prepared, digitally signed OS templates and customizes them.
Documentation how to use virt-builder is provided in the relevant man pages.
Here we provide some examples.
To see the operating system available to install, please use the following command:
The following command:virt-builder -v -x centos-7.7 \ --uninstall "chrony" \ --install "ntp,cloud-init,cloud-utils,cloud-utils-growpart,gdisk" \ --timezone Europe/Rome" \ --write '/etc/ntp.conf:server ntp.pd.infn.it' \ --run-command 'systemctl enable ntpd' \ --edit '/etc/resolv.conf:s/nameserver 10.0.2.3//' \ --output c7.7-test-grow.qcow2 --format qcow2 --selinux-relabel
creates a centos 7.7 image, called centos-7.7.qcow2 in qcow2 format, where the packages ntp, cloud-init, cloud-utils, cloud-utils-growpart and gdisk are installed.
The above command removes from the image the package chrony.
The option –timezone Europe/Rome sets the timezone.
The string ‘server ntp.pd.infn.it’ is written into the file /etc/ntp.conf.
The command ‘systemctl enable ntpd’ is also issued (to start the ntpd service at boot).
The option –edit ‘/etc/resolv.conf:s/nameserver 10.0.2.3//’ is used as a workaround to address a problem in the centos7 default image (a wrong nameserver is defined in /etc/resolv.conf).
The following command will instead create an ubuntu 18.04 image, called k8s-ubuntu-18.04.raw, in raw format:
virt-builder -v -x ubuntu-18.04 \ --install "net-tools,vim,apt-transport-https,ca-certificates,curl,software-properties-common,docker.io,sudo,wget,rsync,cloud-init,cloud-utils" \ --timezone Europe/Rome \ --output k8s-ubuntu-18.04.raw --format raw \ --run install.sh \ --firstboot-command 'apt-get update' \ --firstboot-command 'apt-get upgrade -y' \ --selinux-relabel
Besides specifying a set of packages to be installed, the above command specifies that the script install.sh must be run on the disk image. The script can be used e.g. to install packages from an extra repository:
$ cat install.sh #!/bin/bash curl -s https://packages.cloud.google.com/apt/doc/apt-key.gpg | apt-key add - cat <<EOF >/etc/apt/sources.list.d/kubernetes.list deb https://apt.kubernetes.io/ kubernetes-xenial main EOF apt-get update apt-get install -y kubelet kubeadm kubectl
The command also specifies that the commands apt-get update and apt-get upgrade -y must be executed at the first boot.
virt-builder is very easy to install (it is provided by the libguestfs-tools-c package) and doesn’t require root privileges. If needed, access to an instance where virt-builder is installed can be provided: in such case please contact firstname.lastname@example.org.
9.6. Deleting Images¶
Images that are not used anymore can be deleted. Deletion of images is permanent and cannot be reversed.
To delete an image, log in to the dashboard and select the appropriate project from the drop down menu at the top left. On the Compute → Images page select the images you want to delete and click Delete Images. Confirm your action pressing the Delete Images again on the confirmation dialog box.
Don’t delete an image if there are virtual machines created using this image, otherwise these VMs won’t be able to start if rebooted.
9.7. Migrating an image to another cloud¶
Sometimes it becomes necessary to migrate an image (or snapshot) from one OpenStack cloud to another one.
In short the procedure is the following:
In the source cloud:
- Download the image/snapshot as a file
In the destination cloud:
- Upload the downloaded image file to the new environment
To download the image from the source cloud, you need to use the OpenStack CLI. After having sourced the environment script of the source cloud (as explained in Accessing the Cloud with command line tools), run the openstack image list to find the id of the image (photo-slave in the following example):
$ openstack image list | grep photo-slave | 753e8fa1-3f1b-407d-9294-d22b89ec3184 | photo-slave | active |
Then use the openstack image save command, using the relevant id, to download the image file:
$ openstack image save --file photo-slave.qcow2 753e8fa1-3f1b-407d-9294-d22b89ec3184
To upload the image to the destination environment, first of all source the environment script of the target cloud.
Then run the openstack image create command:
$ openstack image create --private --disk-format=qcow2 \ --container-format=bare \ --file photo-slave.qcow2 photo-slave