10. Container-as-a-Service (CaaS) Platform (EXPERIMENTAL)
Container-as-a-Service provides an easy way to run containerized software packages in the cloud. Unlike the more widely recognized Kubernetes-as-a-Service (KaaS) model, where users are responsible for creating and managing their Kubernetes clusters, with CaaS we offer a fully managed orchestration platform as a cloud service. This means you don’t need an in-depth understanding of Kubernetes infrastructure management. Instead, you can effortlessly deploy your containers to our Kubernetes-based platform using the straightforward kubectl command line interface.
Please be aware that our CaaS service is currently in an experimental phase, so some issues may arise. If you encounter any problems, we kindly ask you to report them to email@example.com. Your feedback is invaluable for enhancing the quality of our service.
Before accessing the platform, it is important to grasp some key concepts about the architecture, usability, security, and a few limitations. We assume that you already have a basic understanding of Kubernetes.
10.2. OSNodes vs Nodes
Please note that in this guide, we use the terms OSNode and Node. They both refer to the same concept, which is the Kubernetes node, but with some distinctions:
OSNode stands for OpenStackNode and pertains to our API for creating Kubernetes nodes as Virtual Machines within an OpenStack Cloud Infrastructure, as provided by CloudVeneto;
On the other hand, when we mention Node, we are referring to a Kubernetes node where the Pods run. These nodes can be instantiated either through the OSNode API or by using different tools and methods.
Kubernetes efficiently manages your workloads by deploying containers within pods, which are then scheduled to operate on nodes. In our CaaS, each node essentially represents a virtual machine provided within the CloudVeneto infrastructure. We refer to these nodes as OSNodes, which are within your administrative control but are configured by our platform. In contrast, the control plane, responsible for orchestrating container deployments and managing the cluster, is completely managed by CloudVeneto.
This separation of responsibilities ensures a streamlined user experience while providing the following benefits:
Flexibility: You have the freedom to manage your nodes according to your specific requirements in terms of CPU, RAM, and storage (flavor) without to worry about their setup;
Isolation: Your pods run on dedicated nodes, effectively creating a virtual cluster tailored to your needs;
Resource Sharing: You can share one or more nodes with users who belong to your CloudVeneto project;
Service Deployment: You can either use pre-deployed services (e.g., nginx) or deploy new ones in your own namespace.
Please note that the OSNodes you create utilize the quota assigned to your CloudVeneto project. Therefore, the size of your virtual cluster is constrained by the available resources at any given moment. Since node creation typically takes just a few minutes (usually less than 5 minutes), we encourage you to create new nodes as needed but also to promptly remove them when they are no longer necessary in order to conserve cloud resources.
Running pods on your own OSNdes ensures a high level of isolation. However, Kubernetes doesn’t provide complete isolation for users within the same namespace. To address this specific limitation, we have introduced integrated add-ons for Kubernetes. These enhancements include authentication with Keystone, along with refined authorization procedures to ensure comprehensive user and resource isolation.
Kindly be aware that pods running on shared nodes do not achieve full isolation as they share the same computing resources (virtual machine) and rely on the security capabilities of the container runtime, such as Docker or Containerd.
10.5. Accessing the CaaS
Since our CaaS is built on Kubernetes, accessing the platform requires the correct configuration of kubectl, the Kubernetes management client. For simplifying this configuration process, CloudVeneto has developed a new plugin for kubectl: kubectl-openstack. This plugin set up the kubeconfig file with the appropriate authentication method for Keystone (OpenStack).
10.6. Configuring kubectl with the kubectl-openstack plugin
This section explains how to install and configure our plugin.
10.6.2. Plugin installation
Download kubectl-openstack (linux version), then make the file executable using the command ‘chmod 755 kubectl-openstack’, and finally, copy it to the directory ‘/usr/local/bin/.
To view the syntax and the list of parameters use the help:
$ kubectl-openstack --help Usage: kubectl-openstack [FLAG] -user=<USERNAME> -password=<PASSOWRD> -project=<PROJECT> Options: -force overwrite the existing configuration -password string your CloudVeneto password -project string your CloudVeneto project -user string your CloudVeneto username
The following example configures kubectl selecting CMS as project:
$ kubectl-openstack -firstname.lastname@example.org -password=******** -project=CMS kubectl configured correctly
After configuring kubectl with the kubectl-openstack plugin, the kubeconfig file (/home/<username>/.kube/config) is either created or updated if it already exists. This file contains the Keystone token and various parameters essential for kubectl to manage authentication.
In scenarios where you belong to multiple CloudVeneto projects, you can utilize the kubectl-openstack command to configure kubectl for all your projects seamlessly.
Now you can access the CaaS:
$ kubectl get pods No resources found in cms namespace.
10.7. How to Create and Manage OSNodes
In this section we explain how to create and manage your OSNodes.
Before you can create and manage OSNodes in Kubernetes, make sure you have completed the following prerequisites:
Ensure that your kubectl is correctly configured with the kubectl-openstack plugin.
Verify the existence of the ‘K8S’ security group in the CloudVeneto project with the following rules, or create it if missing:
10.7.2. Creating a new OSNode
To create a new OSNodes in your Kubernetes cluster, you’ll use kubectl, the standard Kubernetes command-line interface. Specifically, you will utilize the kubectl apply command, which takes a YAML file as input.
The YAML file required to create a new OSNode should follow this structure:
--- apiVersion: osnode.infn.it/v1 kind: OpenStackNode metadata: name: NODE_NAME spec: flavor: FLAVOR_NAME keyPair: KEYPAIR_NAME policy: [shared | private ]
NODE_NAME: <Unique node name>
FLAVOR_NAME: <CloudVeneto flavor name>
KEYPAIR_NAME: <User-defined SSH keypair name>
shared | private: <Choose one: shared or private>
Kindly be aware that pods running on shared nodes do not achieve full isolation as they share the same computing resources (virtual machine) and rely on the security capabilities of the container runtime, such as Docker or Containerd. Pods running on shared nodes could be accessed by the node’s owner.
In the following example, we request the creation of two OSNodes (osn-01 and osn-02), the first being shared and the second private, with different flavors (cloudveneto.medium and cloudveneto.large). Both nodes use the same SSH keypair (my-key):
$ cat osnode.yml --- apiVersion: osnode.infn.it/v1 kind: OpenStackNode metadata: name: osn-01 spec: flavor: cloudveneto.medium keyPair: my-key policy: shared --- apiVersion: osnode.infn.it/v1 kind: OpenStackNode metadata: name: osn-02 spec: flavor: cloudveneto.large keyPair: my-key policy: private
Copy and paste the above yaml code into an empty (osnode.yml) file, then execute the following command:
$ kubectl apply -f osnode.yml openstacknode.osnode.infn.it/osn-01 created openstacknode.osnode.infn.it/osn-02 created
10.7.3. Verifying the OSNode status
To check the status of one or more OSNodes in your Kubernetes cluster, you can use the following commands:
To list all OSNodes and their basic information:
$ kubectl get osn NAME PHASE OWNER NODE ID POLICY PROVIDER VM IPV4 AGE osn-01 Running zangrand-at-infn.it osn-01-1696949601872 shared CloudVeneto 10.64.53.91 13d osn-02 Running zangrand-at-infn.it osn-02-1696949605128 private CloudVeneto 10.64.53.251 13d
Please note that each OSNode has a Kubernetes Node associated to it (e.g. osn-01 -> osn-01-1696949601872).
To list all OSNodes with additional details, including flavor, status, and IP address:
$ kubectl get osn -o wide NAME PHASE OWNER NODE ID POLICY PROVIDER VM FLAVOR VM STATUS VM IPV4 AGE osn-01 Running zangrand-at-infn.it osn-01-1696949601872 shared CloudVeneto cloudveneto.medium ACTIVE 10.64.53.91 13d osn-02 Running zangrand-at-infn.it osn-02-1696949605128 private CloudVeneto cloudveneto.medium ACTIVE 10.64.53.251 13d
To view detailed information about a specific OSNodes (replace osn-01 with the desired OSNodes name):
$ kubectl get osn -o wide osn-01 NAME PHASE OWNER NODE ID POLICY PROVIDER VM FLAVOR VM STATUS VM IPV4 AGE osn-01 Running zangrand-at-infn.it osn-01-1696949601872 shared CloudVeneto cloudveneto.medium ACTIVE 10.64.53.91 13d
10.7.4. Removing OSNodes
To remove one or more OSNodes and their associated Virtual Machine from CloudVeneto, use the following command:
$ kubectl delete osn <node_name_1> <node_name_2> ...
For example, to remove osn-01 and osn-02, you would run:
$ kubectl delete osn osn-01 osn-02 openstacknode.osnode.infn.it "osn-01" deleted openstacknode.osnode.infn.it "osn-02" deleted
10.7.5. Geeting details about your OSNode
For more detailed information about your OSNodes, you can use the following command:
$ kubectl describe osn qst-gpu-01 # kubectl -n qst describe osn qst-gpu-01 Name: qst-gpu-01 Namespace: qst Labels: SECRET=bootstrap-token-mqkldv osn.infn.it/projectid=55158de200964f7c8aa5ca486e6cb7ea osn.infn.it/projectname=QST osn.infn.it/userid=2ddd446e119b417791492e950553a055 osn.infn.it/username=zangrand-at-infn.it Annotations: <none> API Version: osnode.infn.it/v1 Kind: OpenStackNode Metadata: Creation Timestamp: 2023-09-27T16:22:29Z Finalizers: openstacknode/finalizer Generation: 2 Resource Version: 3897894 UID: e29c8c4c-4f50-49ad-91bc-b6c491bcc42a Spec: Availability Zone: nova Flavor: cloudveneto.50cores249GB25GB+500GB1A Image: Id: Name: almalinux9-k8s-node-26-09-2023 Key Pair: Lisa Policy: shared Provider: cloudveneto Region: regionOne Security Groups: K8S User Data: Status: Created: 2023-09-27T16:29:51Z Description: The node is running Nodeid: qst-gpu-01-1695831749722 Phase: Running Server: Created: 2023-09-27T16:22:32Z Id: ac4a4470-4469-4ed7-8520-db9ea8c0b56d ipv4: 10.64.51.42 Name: qst-gpu-01 Status: ACTIVE Updated: 2023-10-09T11:35:37Z Updated: 2023-10-09T11:35:37Z Events: <none>
This can be particularly useful for troubleshooting purposes.