Reference

This chapter contains extra reference information (more details) about the socok8s GitHub repository.

For information on how to deploy SUSE Containerized OpenStack, refer to Deployment Guide.

For information on how to manage and operate SUSE Containerized OpenStack, refer to Administration and Operations Guide.

For information on how to contribute to SUSE Containerized OpenStack, refer to Developer Documentation.

Project history

This project started as a way to build and test the OpenStack-Helm charts for SUSE, on SUSE products: The Container as a Service Platform (CaaSP) and the SUSE Enterprise Storage (SES).

It started as a series of shell scripts and Ansible playbooks, choosing the simplest and fastest way to bring a test infrastructure for the upstream charts. It was easier to start with a shell script than writing a CLI in <insert language here>, mostly because the shell script organically grew out of its usage and CI needs.

The mechanism of deployment was flexible from the beginning to allow developers to test their changes independently. It would allow them to override specific parts of the deployment, like other users or customers would want to do.

Project goals

  • Simplicity

  • Stability

  • Use the latest stable products from SUSE

  • Carry the minimum amount of code to support upstream work on SUSE products

  • Be packagable/installable offline

  • Leverage upstream first

Design considerations

Workspace

In order to not pollute the developer/CI machine (called localhost), all the data relevant for a deployment (like any eventual override) is stored in a user-space ~/socok8s-workspace folder, with unprivileged access.

This also supports the use case of running behind a corporate firewall. The localhost can connect to a bastion host with the “deployment” actions happening behind the firewall.

Why…

… Ansible?

Using Ansible is more robust than having written socok8s fully on shell scripts. Its ecosystem allows a nice interface to track deployment progress with ARA, run in a CI/CD like Zuul or Tower/AWX.

… OpenStack on top of Kubernetes on top of OpenStack by default in run.sh?

We have a cloud for our Engineers, and that cloud is used for CI. From that point, creating a node for testing is as simple as doing an API call, and creating a stack of nodes is simple as re-using an existing Heat stack.

The run.sh was mainly used for developers and CI. This is why the run.sh script still points to openstack as the default DEPLOYMENT_MECHANISM.

… OpenStack on top of Kubernetes?

Robust structure

… Splitting run.sh in so many steps?

The current interface of run.sh is flexible enough to work for many different cases. It is semantically close to the actions that deploy OpenStack. run.sh itself is just an interface. Behind the scenes, it runs a DEPLOYMENT_MECHANISM-dependent script, starting the appropriate Ansible playbooks for the step called.

… A shell script for this interface?

It was easier to start with a shell script rather than writing a CLI in <insert your favorite language here>, mostly because the shell script grew organically out of actual use and CI needs.

… Installing from sources?

Neither the socok8s repo nor the OpenStack-Helm project’s repositories have been packaged for Leap/SLE 15 yet.

Image building process

Upstream process

The OpenStack-Helm project tries to be neutral about the images by providing the ability for deployers to override any image used in the charts.

However, the OpenStack-Helm project has a repository, openstack-helm-images <https://github.com/openstack/openstack-helm-images>_, containing a reference implementation for the images. That repository holds the images used for the OpenStack-Helm project charts. All its images are built with Docker.

The openstack-helm-images repository provides Dockerfiles directly for all the non-OpenStack images.

For the OpenStack images, openstack-helm-images contains shell scripts, situated in openstack/loci/. The build.sh script is a thin wrapper around LOCI. LOCI is the official OpenStack project to build lightweight Open Container Initiative (OCI) compliant images of OpenStack projects. It uses docker build to construct images from OpenStack sources. Their requirements are expressed in bindep files (bindep.txt for rpm/apt packages, pydep.txt for Python3 packages). The build.sh script runs LOCI for the master branch. Other branches can be built using build-{branchname}.sh where branchname is the name of the OpenStack release (for example, rocky). See also Build LOCI images.

In the future, openstack-helm-images could add images for OpenStack that would be based on packages by simply providing the appropriate Dockerfiles. There is no announced plan to offer such a resource.

Additionally, some images are not built in openstack-helm-images, and they are directly consumed/fetched from upstream projects official dockerfiles, like xrally.

socok8s process

socok8s leverages the existing OSH-images code.

When running the build_images step, the localhost asks the deployer to build images based on the code that was checked in on the deployer node using the vars/manifest.yml file.

For the non-LOCI images, the suse-build-images role invoked in the build_images step is running a docker build command.

For the LOCI images, the suse-build-images role runs the command available in openstack-helm-images calling the LOCI build.

OpenStack-Helm chart overrides

Helm chart values overriding principle

A Helm chart installation (See https://helm.sh/docs/using_helm/#customizing-the-chart-before-installing ) accepts an argument named --values or -f.

This argument expects the filename of a YAML file to be present on the Helm client machine. It can be specified multiple times, and the rightmost file will take precedence.

In the following example, the different values of socok8s-glance.yaml overrides would win over the existing values in /tmp/glance.yaml:

helm upgrade --install glance ./glance --namespace=openstack \
  --values=/tmp/glance.yaml --values=/tmp/socok8s-glance.yaml

OpenStack-Helm scripts

The OpenStack-Helm project provides shell scripts to deploy the Helm charts, with overrides per context (for example, multinode).

Those shell scripts calling the Helm installation include an environment variable to allow users to pass extra arguments.

See this example from the openstack-helm repository.

Customizing OSH charts for SUSE when deploying in OSH only mode

socok8s uses the previously explained environment variable to pass an extra values file, a SUSE-specific YAML file. All the SUSE-specific files are present in playbooks/roles/deploy-osh/templates/ (for example socok8s-glance.yml), if they are not part of upstream yet.

How deployers can extend a custom SUSE OSH chart in OSH-only mode

Deployers can pass their own YAML overrides in user space by using extravars to extend Helm chart behavior beyond the SUSE customizations.

These overrides are in playbooks/roles/deploy-osh/defaults/main.yml.

Customizing OSH charts for SUSE when deploying with Airship

How deployers can extend a custom SUSE OSH chart with Airship

Summary “deploy on OpenStack” diagrams

Simplified network diagram

OSH deploy on OpenStack process

Setup hosts

This is the sequence of steps that generates, in OpenStack, the environment for deploying OSH later.

Setup OpenStack

This is the sequence of steps that ends up with your OpenStack-Helm deployment. The solid lines represent Ansible plays and their connections.

The dotted lines represent extra connections happening on the Ansible targets.

Environment variables

In socok8s

run.sh behavior can be modified with environment variables.

DEPLOYMENT_MECHANISM contains the target destination of the deploy tooling. Currently set to openstack by default, but will later include a baremetal and kvm.

SOCOK8S_DEVELOPER_MODE determines if you want to enter developer mode or not. This adds a step for patching upstream code, builds images and then continues the deployment.

SOCOK8S_USE_VIRTUALENV determines if the script should set up and use a virtualenv for Python3 and Ansible requirements. Without this it is expected that ansible and the requirements are installed via system packages. When SOCOK8S_DEVELOPER_MODE is set to True, this defaults to True, otherwise this defaults to False.

USE_ARA determines if you want to store records in ARA. Set its value to ‘True’ for using ARA.

Ansible environment variables

You can use Ansible environment variables to alter Ansible behavior, for example by being more verbose.

OpenStack-Helm environment variables

OpenStack Helm deployment scripts accepts environment variables to alter their behavior. Read each of the scripts to know more about their override mechanisms.