The Assisted Installer
is a project to help simplify OpenShift Container Platform (OCP)
installation for a number of different platforms, but focuses on bare
metal deployments. The service provides validation and discovery of
targeted hardware and greatly improves success rates of installations. It can be accessed via Red Hat’s provided SaaS portal to deploy an OCP cluster either on baremetal or virtual machines.
In this article however I want to demonstrate a Single Node OpenShift deployment without using the UI web interface and instead rely on the underlying REST API that drives the Assisted Installer. This can be useful for automating the deployment of clusters without user intervention.
The first step to achieve this will be to obtain an OpenShift Cluster Manager API Token. This token provides the ability to authenticate against your Red Hat OpenShift Cluster Manager account without the need of a username or password.
Place this token into a file called ocm-token:
$ echo "Token String From OCM API Token Link Above" > ~/ocm-token
Next lets set some variables that we will refer to throughout this deployment process:
export OFFLINE_ACCESS_TOKEN=$(cat ~/ocm-token) # Loading my token into a variable
export ASSISTED_SERVICE_API="api.openshift.com" # Setting the Assisted Installer API endpoint
export CLUSTER_VERSION="4.8" # OpenShift version
export CLUSTER_IMAGE="quay.io/openshift-release-dev/ocp-release:4.8.2-x86_64" # OpenShift Quay image version
export CLUSTER_NAME="kni1" # OpenShift cluster name
export CLUSTER_DOMAIN="schmaustech.com" # Domain name where my cluster will be deployed
export CLUSTER_NET_TYPE="OVNKubernetes" # Network type to deploy with OpenShift
export MACHINE_CIDR_NET="192.168.0.0/24" # Machine CIDR network
export SNO_STATICIP_NODE_NAME="master-0" # Node name of my SNO node
export PULL_SECRET=$(cat ~/pull-secret.json | jq -R .) # Loading my pull-secret into variable
export CLUSTER_SSHKEY=$(cat ~/.ssh/id_rsa.pub) # Loading the public key into variable
With the primary variables set lets go ahead and create a deployment.json file. This file will reference some of the variables we set previously and also have a few that are statically set. The key one to notice in this deployment is the high_availability_mode. Having that variable set to None ensures we are doing a Single Node OpenShift (SNO) deployment:
cat << EOF > ~/deployment.json
{
"kind": "Cluster",
"name": "$CLUSTER_NAME",
"openshift_version": "$CLUSTER_VERSION",
"ocp_release_image": "$CLUSTER_IMAGE",
"base_dns_domain": "$CLUSTER_DOMAIN",
"hyperthreading": "all",
"user_managed_networking": true,
"vip_dhcp_allocation": false,
"high_availability_mode": "None",
"hosts": [],
"ssh_public_key": "$CLUSTER_SSHKEY",
"pull_secret": $PULL_SECRET,
"network_type": "OVNKubernetes"
}
EOF
Now that we have the deployment.json file created lets refresh our bearer token:
$ export TOKEN=$(curl \
--silent \
--data-urlencode "grant_type=refresh_token" \
--data-urlencode "client_id=cloud-services" \
--data-urlencode "refresh_token=${OFFLINE_ACCESS_TOKEN}" \
https://sso.redhat.com/auth/realms/redhat-external/protocol/openid-connect/token | \
jq -r .access_token)
With the token refereshed lets go ahead and create our deployment via the assisted installer REST API using curl and a post command. When the command completes the output will only be a cluster id with some quotes on it. I used sed to clean off the quotes so we end up with just the UUID number. Note that the cluster configuration has only been created at this point but not installed.
$ export CLUSTER_ID=$( curl -s -X POST "https://$ASSISTED_SERVICE_API/api/assisted-install/v1/clusters" \
-d @./deployment.json \
-H "Content-Type: application/json" \
-H "Authorization: Bearer $TOKEN" \
| jq '.id' )
$ export CLUSTER_ID=$( sed -e 's/^"//' -e 's/"$//' <<<"$CLUSTER_ID")
$ echo $CLUSTER_ID
e85fc7d5-f274-4359-acc5-48044fc67132
At this point we need to generate a discovery iso for the SNO node to be booted from. However before we do that I wanted to make sure that my SNO node was using a static IP address instead of the default of DHCP. To do this we need to create a data file that contains the information on how the static IP should be set. NMState will take this information when applied to the OCP node during the installation. Below we have defined some arguments that provide a mac interface map and a NMState yaml file. All of this information gets pushed into the DATA variable which is just pointing to a temp file.
$ DATA=$(mktemp)
$ jq -n --arg SSH_KEY "$CLUSTER_SSHKEY" --arg NMSTATE_YAML1 "$(cat ~/sno-server.yaml)" \
'{
"ssh_public_key": $SSH_KEY,
"image_type": "full-iso",
"static_network_config": [
{
"network_yaml": $NMSTATE_YAML1,
"mac_interface_map": [{"mac_address": "52:54:00:82:23:e2", "logical_nic_name": "ens9"}]
}
]
}' >> $DATA
The sno-server.yaml used in the NMState argument looks like the following below. It contains the IP address, mask, interface and route information.
$ cat ~/sno-server.yaml
dns-resolver:
config:
server:
- 192.168.0.10
interfaces:
- ipv4:
address:
- ip: 192.168.0.204
prefix-length: 24
dhcp: false
enabled: true
name: ens9
state: up
type: ethernet
routes:
config:
- destination: 0.0.0.0/0
next-hop-address: 192.168.0.1
next-hop-interface: ens9
table-id: 254
We can confirm that the DATA was set appropriately by looking at the DATA variable and then cat out the tmp file it points to:
$ echo $DATA
/tmp/tmp.3Jqw7lU6Qf
$ cat /tmp/tmp.3Jqw7lU6Qf
{
"ssh_public_key": "SSHKEY REDACTED",
"image_type": "full-iso",
"static_network_config": [
{
"network_yaml": "dns-resolver:\n config:\n server:\n - 192.168.0.10\ninterfaces:\n- ipv4:\n address:\n - ip: 192.168.0.204\n prefix-length: 24\n dhcp: false\n enabled: true\n name: ens9\n state: up\n type: ethernet\nroutes:\n config:\n - destination: 0.0.0.0/0\n next-hop-address: 192.168.0.1\n next-hop-interface: ens9\n table-id: 254",
"mac_interface_map": [
{
"mac_address": "52:54:00:82:23:e2",
"logical_nic_name": "ens9"
}
]
}
]
}
With the static IP configuration set we can go ahead and generate our discovery ISO with another curl post command. The command will generate quite a bit of output but our main concern is visually seeing the section where the static network configuration gets defined:
$ curl -X POST \
"https://$ASSISTED_SERVICE_API/api/assisted-install/v1/clusters/$CLUSTER_ID/downloads/image" \
-H "Content-Type: application/json" \
-H "Authorization: Bearer $TOKEN" \
-d @$DATA
(...)
"static_network_config":"dns-resolver:\n config:\n server:\n - 192.168.0.10\ninterfaces:\n- ipv4:\n address:\n - ip: 192.168.0.204\n prefix-length: 24\n dhcp: false\n enabled: true\n name: ens9\n state: up\n type: ethernet\nroutes:\n config:\n - destination: 0.0.0.0/0\n next-hop-address: 192.168.0.1\n next-hop-interface: ens9\n table-id: 254HHHHH52:54:00:82:23:e2=ens9","type":"full-iso"}
(...)
Now that the discovery image has been created lets go ahead and download that image:
$ curl -L \
"http://$ASSISTED_SERVICE_API/api/assisted-install/v1/clusters/$CLUSTER_ID/downloads/image" \
-o ~/discovery-image-$CLUSTER_NAME.iso \
-H "Authorization: Bearer $TOKEN"
% Total % Received % Xferd Average Speed Time Time Time Current
Dload Upload Total Spent Left Speed
0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0
100 984M 100 984M 0 0 10.4M 0 0:01:34 0:01:34 --:--:-- 10.5M
Now that the image is downloaded we can move it to where we need to boot the SNO node machine. This node could be baremetal or in my case a virtual machine. If it was baremetal for example, like a Dell, we might use racadm to do a virtual media mount and then ipmitool to power on the server. In my case since I am using a virtual machine, I need to do a couple things. First I copy the image over to my KVM hypervisor host. Next I ensure the power is off on my virtual machine. I can use ipmitool here because I am leveraging virtual BMC. Next I use the virsh command to change the media to my ISO that I moved over. I format the disk image on my virtual machine that way I do not have to mess around with boot order as the primary disk will be skipped because its empty and the CDROM will boot. And finally I power on the host to initiate the discover phase. At this point we have to wait for the node to boot up and report in what was discovered from an introspection perspective. I usually wait 5 minutes before proceeding hence why I have the sleep command.
$ scp ~/discovery-image-kni1.iso root@192.168.0.5:/slowdata/images/
$ /usr/bin/ipmitool -I lanplus -H192.168.0.10 -p6252 -Uadmin -Ppassword chassis power off
$ ssh root@192.168.0.5 "virsh change-media rhacm-master-0 hda /slowdata/images/discovery-image-kni1.iso"
$ ssh root@192.168.0.5 "virt-format --format=raw --partition=none -a /fastdata2/images/master-0.img"
$ /usr/bin/ipmitool -I lanplus -H192.168.0.10 -p6252 -Uadmin -Ppassword chassis power on
$ sleep 300
After 5 minutes the node should have reporting in to the Assisted Installer portal. And inventory of the hardware of the machine and capabilities is provided in the portal. We can now proceed with the deployment.
First though we need to ensure the hostname is set correctly. With DHCP it was automatically being set but since we used a static IP I found I needed to set it manually. To do this we will patch the installation and set the requested_hostname:
$ curl -X PATCH \
"https://$ASSISTED_SERVICE_API/api/assisted-install/v1/clusters/$CLUSTER_ID" \
-H "accept: application/json" \
-H "Content-Type: application/json" \
-H "Authorization: Bearer $TOKEN" \
-d "{ \"requested_hostname\": \"$SNO_STATICIP_NODE_NAME.$CLUSTER_NAME.$CLUSTER_DOMAIN\"}" | jq
(...)
"requested_hostname": "master-0.kni1.schmaustech.com",
(...)
We also need to patch the machine network to the appropriate network:
$ curl -X PATCH \
"https://$ASSISTED_SERVICE_API/api/assisted-install/v1/clusters/$CLUSTER_ID" \
-H "accept: application/json" \
-H "Content-Type: application/json" \
-H "Authorization: Bearer $TOKEN" \
-d "{ \"machine_network_cidr\": \"$MACHINE_CIDR_NET\"}" | jq
(...)
"machine_network_cidr": "192.168.0.0/24",
(...)
Finally after all of the preparation we can finally run the curl post command that actually starts the installation process:
$ curl -X POST \
"https://$ASSISTED_SERVICE_API/api/assisted-install/v1/clusters/$CLUSTER_ID/actions/install" \
-H "accept: application/json" \
-H "Content-Type: application/json" \
-H "Authorization: Bearer $TOKEN" | jq
(...)
"status": "preparing-for-installation",
"status_info": "Preparing cluster for installation",
"status_updated_at": "2021-08-06T20:56:17.565Z",
(...)
The installation process does take about 60 minutes or so minutes to complete so go grab lunch or a cup of coffee.
After 60 minutes or so we can check and see if the cluster is installed or still in progress. The first thing we should do though is refresh our token again:
$ export TOKEN=$(curl \
--silent \
--data-urlencode "grant_type=refresh_token" \
--data-urlencode "client_id=cloud-services" \
--data-urlencode "refresh_token=${OFFLINE_ACCESS_TOKEN}" \
https://sso.redhat.com/auth/realms/redhat-external/protocol/openid-connect/token | \
jq -r .access_token)
After we have refreshed our token lets go ahead and confirm if indeed the cluster has finished installing. We can achieve this by doing a curl get against the cluster ID. There will be a lot of output but we are specifically looking for the status and status_info lines:
$ curl -s -X GET \
-H "Content-Type: application/json" \
-H "Authorization: Bearer $TOKEN" \
"https://$ASSISTED_SERVICE_API/api/assisted-install/v1/clusters/$CLUSTER_ID" | jq .
(...)
"status": "installed",
"status_info": "Cluster is installed",
"status_updated_at": "2021-08-06T21:45:04.375Z",
(...)
From the output above my cluster has completed so now I can pull my kubeconfig down and redirect it to a file:
$ curl -s -X GET \
"https://$ASSISTED_SERVICE_API/api/assisted-install/v1/clusters/$CLUSTER_ID/downloads/kubeconfig" > kubeconfig-kni1 \
-H "Content-Type: application/json" \
-H "Authorization: Bearer $TOKEN"
Now lets export the kubeconfig variable and look at the cluster with some oc commands:
$ export KUBECONFIG=~/kubeconfig-kni1
$ oc get nodes -o wide
NAME STATUS ROLES AGE VERSION INTERNAL-IP EXTERNAL-IP OS-IMAGE KERNEL-VERSION CONTAINER-RUNTIME
master-0.kni1.schmaustech.com Ready master,worker 156m v1.21.1+051ac4f 192.168.0.204 none Red Hat Enterprise Linux CoreOS 48.84.202107202156-0 (Ootpa) 4.18.0-305.10.2.el8_4.x86_64 cri-o://1.21.2-5.rhaos4.8.gitb27d974.el8
$ oc get co
NAME VERSION AVAILABLE PROGRESSING DEGRADED SINCE
authentication 4.8.2 True False False 134m
baremetal 4.8.2 True False False 144m
cloud-credential 4.8.2 True False False 147m
cluster-autoscaler 4.8.2 True False False 146m
config-operator 4.8.2 True False False 151m
console 4.8.2 True False False 135m
csi-snapshot-controller 4.8.2 True False False 147m
dns 4.8.2 True False False 144m
etcd 4.8.2 True False False 146m
image-registry 4.8.2 True False False 141m
ingress 4.8.2 True False False 141m
insights 4.8.2 True False False 135m
kube-apiserver 4.8.2 True False False 143m
kube-controller-manager 4.8.2 True False False 143m
kube-scheduler 4.8.2 True False False 143m
kube-storage-version-migrator 4.8.2 True False False 151m
machine-api 4.8.2 True False False 146m
machine-approver 4.8.2 True False False 147m
machine-config 4.8.2 True False False 143m
marketplace 4.8.2 True False False 144m
monitoring 4.8.2 True False False 138m
network 4.8.2 True False False 152m
node-tuning 4.8.2 True False False 146m
openshift-apiserver 4.8.2 True False False 143m
openshift-controller-manager 4.8.2 True False False 146m
openshift-samples 4.8.2 True False False 142m
operator-lifecycle-manager 4.8.2 True False False 144m
operator-lifecycle-manager-catalog 4.8.2 True False False 147m
operator-lifecycle-manager-packageserver 4.8.2 True False False 144m
service-ca 4.8.2 True False False 151m
storage 4.8.2 True False False 146m
Everything looks good with this example Single Node OpenShift installation! If one is interested in pursuing more complex examples it might be worth looking at what is available with the Assisted Installer REST API. To do that take a look at this
swagger.yaml file and use it with the online
Swagger Editor.
