Sometimes complexity can sting but this blog on using Hive to deploy OpenShift 4.3 on baremetal will make it sweeter than honey.
For those not familiar with Hive it is a API driven OpenShift 4 cluster and provisioning tool. Hive itself is an operator that runs in an existing OpenShift or Kubernetes cluster and allow for the provisioning and initial configuration of one or more OpenShift clusters.
This blog will walk through the current process of how to use Hive to deploy OpenShift 4.3 on baremetal nodes. Since I do not have enough resources to have two fully functional OpenShift clusters I will be deploying Hive in Minishift and then deploying OpenShift 4.3 on bare metal.
Minishift Installation & Configuration:
Minishift and Hive have a few requirements in order for them to work. The first thing we need to do on this fresh install of Red Hat Enterprise Linux 8 is install some package dependencies:
$ sudo yum -y install qemu-kvm* libvirt go git make
Once libvirt is installed the next step is to enable the service and configure the user we are using to be part of the libvirt group:
$ sudo systemctl enable --now libvirtd $ sudo usermod -a -G libvirt bschmaus $ newgrp - libvirt $ groups libvirt wheel bschmaus
Next lets confirm the host supports virtualization using virt-host-validate. All of the checks should come back as a PASS except IOMMU support. If they do not ensure the virtualized extensions in the BIOS are enabled. If the system is a virtual machine ensure nested virtualization is enabled on the hypervisor host and that the virtual machine has the CPU set to host-passthrough.
$ sudo virt-host-validate QEMU: Checking for hardware virtualization : PASS QEMU: Checking if device /dev/kvm exists : PASS QEMU: Checking if device /dev/kvm is accessible : PASS QEMU: Checking if device /dev/vhost-net exists : PASS QEMU: Checking if device /dev/net/tun exists : PASS QEMU: Checking for cgroup 'cpu' controller support : PASS QEMU: Checking for cgroup 'cpuacct' controller support : PASS QEMU: Checking for cgroup 'cpuset' controller support : PASS QEMU: Checking for cgroup 'memory' controller support : PASS QEMU: Checking for cgroup 'devices' controller support : PASS QEMU: Checking for cgroup 'blkio' controller support : PASS QEMU: Checking for device assignment IOMMU support : WARN (No ACPI DMAR table found, IOMMU either disabled in BIOS or not supported by this hardware platform)
Next lets grab the docker-machine binary:
$ curl --location https://github.com/docker/machine/releases/download/v0.16.1/docker-machine-Linux-`uname -i` > docker-machine $ chmod +x docker-machine $ sudo mv docker-machine /usr/local/bin/
We will also need the docker-machine-driver-kvm binary as well:
$ curl --location https://github.com/dhiltgen/docker-machine-kvm/releases/download/v0.10.0/docker-machine-driver-kvm-centos7 > docker-machine-driver-kvm $ chmod +x docker-machine-driver-kvm $ sudo mv docker-machine-driver-kvm /usr/local/bin/
Finally we can pull down the Minishift binary and set a soft link to minishift:
$ pwd /home/bschmaus $ mkdir bin $ wget -qO- https://github.com/minishift/minishift/releases/download/v1.34.2/minishift-1.34.2-linux-amd64.tgz | tar --extract --gzip --verbose -C ~/bin/ minishift-1.34.2-linux-amd64/ minishift-1.34.2-linux-amd64/LICENSE minishift-1.34.2-linux-amd64/README.ado $ ln -s ~/bin/minishift-1.34.2-linux-amd64/minishift ~/bin/minishift
Before we start Minishift we need to enable the admissions webhook addon that is required for the Hive deployment:
$ minishift addons enable admissions-webhook
At this point we are ready to launch Minishift. Many documents will show just executing minishift start but in my case I did not want to use the defaults for cpu/memory/disk resources and therefore I have specified how I want the Minishift virtual environment to be configured. The process of starting will take some time.
$ minishift start --cpus 6 --memory 12GB --disk-size 30GB
-- Starting profile 'minishift'
-- Check if deprecated options are used ... OK
-- Checking if https://github.com is reachable ... OK
-- Checking if requested OpenShift version 'v3.11.0' is valid ... OK
-- Checking if requested OpenShift version 'v3.11.0' is supported ... OK
-- Checking if requested hypervisor 'kvm' is supported on this platform ... OK
-- Checking if KVM driver is installed ...
Driver is available at /usr/local/bin/docker-machine-driver-kvm ...
Checking driver binary is executable ... OK
-- Checking if Libvirt is installed ... OK
-- Checking if Libvirt default network is present ... OK
-- Checking if Libvirt default network is active ... OK
-- Checking the ISO URL ... OK
-- Downloading OpenShift binary 'oc' version 'v3.11.0'
-- Downloading OpenShift v3.11.0 checksums ... OK
-- Checking if provided oc flags are supported ... OK
-- Starting the OpenShift cluster using 'kvm' hypervisor ...
-- Minishift VM will be configured with ...
Memory: 12 GB
vCPUs : 6
Disk size: 30 GB
-- Downloading ISO 'https://github.com/minishift/minishift-centos-iso/releases/download/v1.16.0/minishift-centos7.iso'
-- Starting Minishift VM ................. OK
-- Checking for IP address ... OK
-- Checking for nameservers ... OK
-- Checking if external host is reachable from the Minishift VM ...
Pinging 8.8.8.8 ... OK
-- Checking HTTP connectivity from the VM ...
Retrieving http://minishift.io/index.html ... OK
-- Checking if persistent storage volume is mounted ... OK
-- Checking available disk space ... 1% used OK
-- Writing current configuration for static assignment of IP address ... WARN
Importing 'openshift/origin-control-plane:v3.11.0' CACHE MISS
Importing 'openshift/origin-docker-registry:v3.11.0' CACHE MISS
Importing 'openshift/origin-haproxy-router:v3.11.0' CACHE MISS
-- OpenShift cluster will be configured with ...
Version: v3.11.0
-- Pulling the OpenShift Container Image ................. OK
-- Copying oc binary from the OpenShift container image to VM ... OK
-- Starting OpenShift cluster .........................................................
Getting a Docker client ...
Checking if image openshift/origin-control-plane:v3.11.0 is available ...
Pulling image openshift/origin-cli:v3.11.0
E0124 01:04:01.549540 2696 helper.go:173] Reading docker config from /home/docker/.docker/config.json failed: open /home/docker/.docker/config.json: no such file or directory, will attempt to pull image docker.io/openshift/origin-cli:v3.11.0 anonymously
Image pull complete
Pulling image openshift/origin-node:v3.11.0
E0124 01:04:02.757778 2696 helper.go:173] Reading docker config from /home/docker/.docker/config.json failed: open /home/docker/.docker/config.json: no such file or directory, will attempt to pull image docker.io/openshift/origin-node:v3.11.0 anonymously
Pulled 5/6 layers, 85% complete
Pulled 6/6 layers, 100% complete
Extracting
Image pull complete
Checking type of volume mount ...
Determining server IP ...
Using public hostname IP 192.168.42.173 as the host IP
Checking if OpenShift is already running ...
Checking for supported Docker version (=>1.22) ...
Checking if insecured registry is configured properly in Docker ...
Checking if required ports are available ...
Checking if OpenShift client is configured properly ...
Checking if image openshift/origin-control-plane:v3.11.0 is available ...
Starting OpenShift using openshift/origin-control-plane:v3.11.0 ...
I0124 01:04:20.142573 2696 config.go:40] Running "create-master-config"
I0124 01:04:23.992407 2696 config.go:46] Running "create-node-config"
I0124 01:04:25.078851 2696 flags.go:30] Running "create-kubelet-flags"
I0124 01:04:25.640019 2696 run_kubelet.go:49] Running "start-kubelet"
I0124 01:04:25.908558 2696 run_self_hosted.go:181] Waiting for the kube-apiserver to be ready ...
I0124 01:04:53.925511 2696 interface.go:26] Installing "kube-proxy" ...
I0124 01:04:53.925549 2696 interface.go:26] Installing "kube-dns" ...
I0124 01:04:53.925566 2696 interface.go:26] Installing "openshift-service-cert-signer-operator" ...
I0124 01:04:53.925581 2696 interface.go:26] Installing "openshift-apiserver" ...
I0124 01:04:53.925647 2696 apply_template.go:81] Installing "kube-dns"
I0124 01:04:53.925666 2696 apply_template.go:81] Installing "openshift-service-cert-signer-operator"
I0124 01:04:53.925651 2696 apply_template.go:81] Installing "openshift-apiserver"
I0124 01:04:53.925651 2696 apply_template.go:81] Installing "kube-proxy"
I0124 01:04:57.836630 2696 interface.go:41] Finished installing "kube-proxy" "kube-dns" "openshift-service-cert-signer-operator" "openshift-apiserver"
I0124 01:07:10.862452 2696 run_self_hosted.go:242] openshift-apiserver available
I0124 01:07:10.863431 2696 interface.go:26] Installing "openshift-controller-manager" ...
I0124 01:07:10.863453 2696 apply_template.go:81] Installing "openshift-controller-manager"
I0124 01:07:13.147594 2696 interface.go:41] Finished installing "openshift-controller-manager"
Adding default OAuthClient redirect URIs ...
Adding centos-imagestreams ...
Adding router ...
Adding persistent-volumes ...
Adding web-console ...
Adding registry ...
Adding sample-templates ...
I0124 01:07:13.167941 2696 interface.go:26] Installing "centos-imagestreams" ...
I0124 01:07:13.167974 2696 interface.go:26] Installing "openshift-router" ...
I0124 01:07:13.167988 2696 interface.go:26] Installing "persistent-volumes" ...
I0124 01:07:13.168012 2696 interface.go:26] Installing "openshift-web-console-operator" ...
I0124 01:07:13.168024 2696 interface.go:26] Installing "openshift-image-registry" ...
I0124 01:07:13.168035 2696 interface.go:26] Installing "sample-templates" ...
I0124 01:07:13.168129 2696 apply_list.go:67] Installing "centos-imagestreams"
I0124 01:07:13.168532 2696 interface.go:26] Installing "sample-templates/rails quickstart" ...
I0124 01:07:13.168557 2696 interface.go:26] Installing "sample-templates/sample pipeline" ...
I0124 01:07:13.168573 2696 interface.go:26] Installing "sample-templates/mongodb" ...
I0124 01:07:13.168586 2696 interface.go:26] Installing "sample-templates/postgresql" ...
I0124 01:07:13.168599 2696 interface.go:26] Installing "sample-templates/cakephp quickstart" ...
I0124 01:07:13.168614 2696 interface.go:26] Installing "sample-templates/django quickstart" ...
I0124 01:07:13.168628 2696 interface.go:26] Installing "sample-templates/nodejs quickstart" ...
I0124 01:07:13.168643 2696 interface.go:26] Installing "sample-templates/jenkins pipeline ephemeral" ...
I0124 01:07:13.168653 2696 apply_list.go:67] Installing "sample-templates/rails quickstart"
I0124 01:07:13.168656 2696 interface.go:26] Installing "sample-templates/mariadb" ...
I0124 01:07:13.168935 2696 interface.go:26] Installing "sample-templates/mysql" ...
I0124 01:07:13.168948 2696 interface.go:26] Installing "sample-templates/dancer quickstart" ...
I0124 01:07:13.169015 2696 apply_list.go:67] Installing "sample-templates/dancer quickstart"
I0124 01:07:13.169327 2696 apply_list.go:67] Installing "sample-templates/mariadb"
I0124 01:07:13.169432 2696 apply_template.go:81] Installing "openshift-web-console-operator"
I0124 01:07:13.169635 2696 apply_list.go:67] Installing "sample-templates/postgresql"
I0124 01:07:13.169931 2696 apply_list.go:67] Installing "sample-templates/mongodb"
I0124 01:07:13.170038 2696 apply_list.go:67] Installing "sample-templates/mysql"
I0124 01:07:13.170098 2696 apply_list.go:67] Installing "sample-templates/cakephp quickstart"
I0124 01:07:13.170197 2696 apply_list.go:67] Installing "sample-templates/sample pipeline"
I0124 01:07:13.170038 2696 apply_list.go:67] Installing "sample-templates/jenkins pipeline ephemeral"
I0124 01:07:13.170332 2696 apply_list.go:67] Installing "sample-templates/django quickstart"
I0124 01:07:13.170537 2696 apply_list.go:67] Installing "sample-templates/nodejs quickstart"
I0124 01:07:18.883279 2696 interface.go:41] Finished installing "sample-templates/rails quickstart" "sample-templates/sample pipeline" "sample-templates/mongodb" "sample-templates/postgresql" "sample-templates/cakephp quickstart" "sample-templates/django quickstart" "sample-templates/nodejs quickstart" "sample-templates/jenkins pipeline ephemeral" "sample-templates/mariadb" "sample-templates/mysql" "sample-templates/dancer quickstart"
I0124 01:08:08.797919 2696 interface.go:41] Finished installing "centos-imagestreams" "openshift-router" "persistent-volumes" "openshift-web-console-operator" "openshift-image-registry" "sample-templates"
Login to server ...
Creating initial project "myproject" ...
Server Information ...
OpenShift server started.
The server is accessible via web console at:
https://192.168.42.173:8443/console
You are logged in as:
User: developer
Password:
To login as administrator:
oc login -u system:admin
-- Applying addon 'admissions-webhook':..
-- Restarting Openshift API server ...
-- Restarting Kube API server .....
-- Successfully installed addon admission-webhook ... OK
-- Exporting of OpenShift images is occuring in background process with pid 3668.
Once Minishift is up and running we can now set the minishift oc environment up and then login to the OpenShift cluster Minishift is providing:
$ minishift oc-env export PATH="/home/bschmaus/.minishift/cache/oc/v3.11.0/linux:$PATH" # Run this command to configure your shell: # eval $(minishift oc-env) $ eval `minishift oc-env` $ oc login -u system:admin Logged into "https://192.168.42.173:8443" as "system:admin" using existing credentials. You have access to the following projects and can switch between them with 'oc project': default kube-dns kube-proxy kube-public kube-system * myproject openshift openshift-apiserver openshift-controller-manager openshift-core-operators openshift-infra openshift-node openshift-service-cert-signer openshift-web-console Using project "myproject".
At this point we are now ready to move on to the Hive portion of installing and configuring.
Hive Installation & Configuration:
Now that we have Minishift up and running we can deploy Hive. Hive requires a couple of additional components: Mockgen and Kustomize. MockGen generates mock implementations of Go interfaces and Kustomize traverses a Kubernetes/OpenShift manifest to add, remove or update configuration options without forking. Below we will setup the Go path, gather Mockgen, Kustomize and Hive:
$ mkdir ~/go/
$ export PATH=~/go/bin:$PATH
$ export GOROOT=/usr/lib/golang
$ export GOPATH=~/go
$ go get -v github.com/openshift/hive
github.com/openshift/hive (download)
package NaN,NaN github.com/openshift/hive: no Go files in /home/bschmaus/go/src/github.com/openshift/hive
$ go get -v github.com/golang/mock/mockgen
github.com/golang/mock (download)
Fetching https://golang.org/x/tools/go/packages?go-get=1
Parsing meta tags from https://golang.org/x/tools/go/packages?go-get=1 (status code 200)
get "golang.org/x/tools/go/packages": found meta tag get.metaImport{Prefix:"golang.org/x/tools", VCS:"git", RepoRoot:"https://go.googlesource.com/tools"} at https://golang.org/x/tools/go/packages?go-get=1
get "golang.org/x/tools/go/packages": verifying non-authoritative meta tag
Fetching https://golang.org/x/tools?go-get=1
Parsing meta tags from https://golang.org/x/tools?go-get=1 (status code 200)
golang.org/x/tools (download)
golang.org/x/tools/internal/packagesinternal
github.com/golang/mock/mockgen/model
golang.org/x/tools/go/internal/packagesdriver
golang.org/x/tools/go/internal/gcimporter
golang.org/x/tools/go/gcexportdata
golang.org/x/tools/go/packages
github.com/golang/mock/mockgen
$ cd $GOPATH/src/github.com/openshift/hive/
$ export PATH=$PATH:$GOPATH/bin
$ curl -sL https://github.com/kubernetes-sigs/kustomize/releases/download/kustomize%2Fv3.3.0/kustomize_v3.3.0_linux_amd64.tar.gz | sudo tar xvz -C /usr/local/bin/
kustomize
$ git fetch -a -v
From https://github.com/openshift/hive
= [up to date] v1 -> origin/v1
= [up to date] master -> origin/master
$ git checkout v1 Already on 'v1' Your branch is up to date with 'origin/v1'.
At this point we are ready to make and deploy Hive on our Minishift cluster:
$ DEPLOY_IMAGE=registry.svc.ci.openshift.org/openshift/hivev1:hive make deploy # The apis-path is explicitly specified so that CRDs are not created for v1alpha1 go run tools/vendor/sigs.k8s.io/controller-tools/cmd/controller-gen/main.go crd --apis-path=pkg/apis/hive/v1 CRD files generated, files can be found under path /home/bschmaus/go/src/github.com/openshift/hive/config/crds. oc apply -f config/crds customresourcedefinition.apiextensions.k8s.io/checkpoints.hive.openshift.io created customresourcedefinition.apiextensions.k8s.io/clusterdeployments.hive.openshift.io created customresourcedefinition.apiextensions.k8s.io/clusterdeprovisions.hive.openshift.io created customresourcedefinition.apiextensions.k8s.io/clusterimagesets.hive.openshift.io created customresourcedefinition.apiextensions.k8s.io/clusterprovisions.hive.openshift.io created customresourcedefinition.apiextensions.k8s.io/clusterstates.hive.openshift.io created customresourcedefinition.apiextensions.k8s.io/dnszones.hive.openshift.io created customresourcedefinition.apiextensions.k8s.io/hiveconfigs.hive.openshift.io created customresourcedefinition.apiextensions.k8s.io/machinepools.hive.openshift.io created customresourcedefinition.apiextensions.k8s.io/selectorsyncidentityproviders.hive.openshift.io created customresourcedefinition.apiextensions.k8s.io/selectorsyncsets.hive.openshift.io created customresourcedefinition.apiextensions.k8s.io/syncidentityproviders.hive.openshift.io created customresourcedefinition.apiextensions.k8s.io/syncsets.hive.openshift.io created customresourcedefinition.apiextensions.k8s.io/syncsetinstances.hive.openshift.io created go generate ./pkg/... ./cmd/... hack/update-bindata.sh # Deploy the operator manifests: mkdir -p overlays/deploy cp overlays/template/kustomization.yaml overlays/deploy cd overlays/deploy && kustomize edit set image registry.svc.ci.openshift.org/openshift/hive-v4.0:hive=registry.svc.ci.openshift.org/openshift/hivev1:hive kustomize build overlays/deploy | oc apply -f - namespace/hive created serviceaccount/hive-operator created clusterrole.rbac.authorization.k8s.io/hive-frontend created clusterrole.rbac.authorization.k8s.io/hive-operator-role created clusterrole.rbac.authorization.k8s.io/manager-role created clusterrole.rbac.authorization.k8s.io/system:openshift:hive:hiveadmission created rolebinding.rbac.authorization.k8s.io/extension-server-authentication-reader-hiveadmission created clusterrolebinding.rbac.authorization.k8s.io/auth-delegator-hiveadmission created clusterrolebinding.rbac.authorization.k8s.io/hive-frontend created clusterrolebinding.rbac.authorization.k8s.io/hive-operator-rolebinding created clusterrolebinding.rbac.authorization.k8s.io/hiveadmission-hive-hiveadmission created clusterrolebinding.rbac.authorization.k8s.io/manager-rolebinding created deployment.apps/hive-operator created rm -rf overlays/deploy
After a few minutes we can use the oc get pods command to see if our Hive deployment is running:
$ oc get pods -n hive NAME READY STATUS RESTARTS AGE hive-controllers-84889bb864-74qc2 1/1 Running 0 45s hive-operator-86bf7c785f-qw74r 1/1 Running 0 1m hiveadmission-56c86f475d-2nbj7 1/1 Running 0 41s hiveadmission-56c86f475d-bf268 1/1 Running 0 41s
At this point we can move onto the steps required to configure and deploy our OpenShift 4.3 baremetal cluster.
OpenShift 4.3 Baremetal Deployment with Hive:
Currently we have a running Minishift cluster with Hive deployed as a operator inside. Now we can begin the process of setting up the baremetal host that will be used to launch the bootstrap deployment virtual machine for the OpenShift 4.3 that will then deploy the Openshift master nodes on baremetal hosts.
Many of the steps needed to prepare the deployment host are documented in a previous blog of mine here: Deploying OpenShift IPI Baremetal Disconnected (this will be referred as "the blog above" in later context). Follow the requirements in the blog above up to the point of actually deploying the cluster.
One item the blog above does not include is ensuring that the deploy user being used to do the deployment can ssh into the provisioning node without a password from the provisioning node. To achieve this login to the provisioning node and run the following commands:
$ ssh-copy-id bschmaus@192.168.0.246 /usr/bin/ssh-copy-id: INFO: Source of key(s) to be installed: "/home/bschmaus/.ssh/id_rsa.pub" The authenticity of host '192.168.0.246 (192.168.0.246)' can't be established. ECDSA key fingerprint is SHA256:xyfms8ClEh+s4n+Vwbgsz9JGzXSN6X8sZDfiCVjpaOU. Are you sure you want to continue connecting (yes/no/[fingerprint])? yes /usr/bin/ssh-copy-id: INFO: attempting to log in with the new key(s), to filter out any that are already installed
Lets return back to the RHEL 8 Minishift host command prompt where we had setup our oc environment.
Here we will take the install-config.yaml that was generated from the blog above and add the following line adjusting the user and ipaddress for the environment:
libvirtURI: qemu+ssh://bschmaus@192.168.0.246/system
The install-config.yaml should then look something like the following (adjusted based on environment of course) and be saved on the Minishift host:
apiVersion: v1
baseDomain: schmaustech.com
metadata:
name: kni5
networking:
networkType: OpenShiftSDN
machineCIDR: 192.168.0.0/24
compute:
- name: worker
replicas: 0
controlPlane:
name: master
replicas: 3
platform:
baremetal: {}
platform:
baremetal:
libvirtURI: qemu+ssh://bschmaus@192.168.0.246/system
apiVIP: 192.168.0.199
ingressVIP: 192.168.0.197
dnsVIP: 192.168.0.198
bootstrapOSImage: http://172.22.0.1/images/rhcos-43.81.202001142154.0-qemu.x86_64.qcow2.gz?sha256=6dbd3513a824cfe6de157e5d86972c2005b23d71bd8b3a61548f7a1f0a516b68
clusterOSImage: http://172.22.0.1/images/rhcos-43.81.202001142154.0-openstack.x86_64.qcow2.gz?sha256=a1bda656fa0892f7b936fdc6b6a6086bddaed5dafacedcd7a1e811abb78fe3b0
hosts:
- name: master-0
role: master
bmc:
address: ipmi://192.168.0.11:6241
username: admin
password: password
bootMACAddress: 52:54:00:3d:04:ae
hardwareProfile: default
- name: master-1
role: master
bmc:
address: ipmi://192.168.0.11:6261
username: admin
password: password
bootMACAddress: 52:54:00:37:80:89
hardwareProfile: default
- name: master-2
role: master
bmc:
address: ipmi://192.168.0.11:6231
username: admin
password: password
bootMACAddress: 52:54:00:4f:e4:7c
hardwareProfile: default
sshKey: 'INSERT YOUR SSH KEY HERE'
imageContentSources:
- mirrors:
- rhel8-ocp-auto.schmaustech.com:5000/ocp4/openshift4
source: quay.io/openshift-release-dev/ocp-v4.0-art-dev
- mirrors:
- rhel8-ocp-auto.schmaustech.com:5000/ocp4/openshift4
source: registry.svc.ci.openshift.org/ocp/release
pullSecret: ' INSERT YOUR PULL SECRET HERE'
additionalTrustBundle: |
-----BEGIN CERTIFICATE-----
MIIF9zCCA9+gAwIBAgIUeFymZ6Et3peUhTizIH5aeHXh7ikwDQYJKoZIhvcNAQEL
BQAwgYoxCzAJBgNVBAYTAlVTMRIwEAYDVQQIDAlNaW5uZXNvdGExFjAUBgNVBAcM
DUJyb29rbHluIFBhcmsxEDAOBgNVBAoMB1JlZCBIYXQxFDASBgNVBAsMC0VuZ2lu
ZWVyaW5nMScwJQYDVQQDDB5yaGVsOC1vY3AtYXV0by5zY2htYXVzdGVjaC5jb20w
HhcNMjAwMTI1MTQzOTQ0WhcNMjEwMTI0MTQzOTQ0WjCBijELMAkGA1UEBhMCVVMx
EjAQBgNVBAgMCU1pbm5lc290YTEWMBQGA1UEBwwNQnJvb2tseW4gUGFyazEQMA4G
A1UECgwHUmVkIEhhdDEUMBIGA1UECwwLRW5naW5lZXJpbmcxJzAlBgNVBAMMHnJo
ZWw4LW9jcC1hdXRvLnNjaG1hdXN0ZWNoLmNvbTCCAiIwDQYJKoZIhvcNAQEBBQAD
ggIPADCCAgoCggIBAL28gRfNtpezIE7JpuMoSreaCSnvAf0+9s92WVgn0Q5Ke3Iu
OV997lsSJP4oHfSbR2J0ApsAsoPbp2gzOwHvrjcDff4xsBQBY3Juy6PKgBKPbgUY
jp4b+BEnoI8/OCRzvVHCHWPRus+N4FF2nfv0T2qjiNEQ6vITbZ/kJu/WofVDVKzO
Ro5nbppTuZ5psMa6SR8/5KO2IT+ULa6+Pwpwj7Vw5fj1NpBNlgJtGZUm/07t1BH/
U6VNtRfoJbGFvBC8IJoy7hIn74RI7xFvICXvaJ6VE0OIwNsOUvwN/Gn9jp2I+tBg
K9tleq2boVahql3Or18bai8AqjuCsYOOaPsxjfnKju5/C13rTBkr9KTTrHlf+c9N
+kFkUqnqtEymSa2Fsn0HbJfp8f6mHtaszFv6lxBElkfeMLvvF+P7MWxTJOMTaB83
1lQjlCJNVg8KQphd31V5A4FwlLGvUeM3siph2YMQTTxF/maEjppfPWPu0eZiUbxE
11Zp2u87gtO9MCABjpt1xiMCvn2R2MxUUfD0tL1P6wws2bcq2Cbo0Z0I55DEjAG/
wRjVFa7s/qQYG9EP6HTIHKWhrAI2peOfDbn9ADb+YMnIik4Vsmk4omo9yhDqGbTO
Dl5FrfShLUzeWeou5CkPxxed0JKsvaejh4+45A42H4//V9dIfOPhzMJhUqSVAgMB
AAGjUzBRMB0GA1UdDgQWBBQdyoUTFGBK4OaPG+yxkRldGcNk3jAfBgNVHSMEGDAW
gBQdyoUTFGBK4OaPG+yxkRldGcNk3jAPBgNVHRMBAf8EBTADAQH/MA0GCSqGSIb3
DQEBCwUAA4ICAQCc9Fj3bm+wbHlXEPDNWBM1Oe2J3sN11N4z6DaozAeEjCnJb4SE
/dluktdEkF325qgSr+Wxwi0W/mdUXPXQv/bjlcH+SMMicp7gw6tYPze6RaEk1REk
oSuxAyHyiU081o+lbVQP/ec5CzMgHqcjwBD4eId1ijbAPGrXO+3mVGBsrsqbBNF/
HCDiKw3SnGC+z8Kcl20Qn08FvioNNdGCWQmH2e3gKK7mz4KtUtq2qoduGsw/12QC
6teSoyUxlXtPpB5mpNHzTSxkyn1yiX7edPdadnNtDGr6FSDFvGLtHusgITwQ4Dt3
FQyjCcRQ8/WllYJbE8fb+7wsx+xc7pTSHj7I73giYtig/tFmE6us/9TNCt9mSIzZ
c29BHmVtaCnmtTlPgFzJYXAYBc3XmA+Zfu6u4G5e7b+IkrOD2w4BK4D2zJaHnhzO
ol+rnN7Ff6hZSjgZg2fPDq3L6AMvqpbKcFe1FzCy1FsHcYoW1hKWySD6KOJ2GtfQ
lcTfxMR+c4bG5y6KFgQ9fJOJYLoCo3y8gaFDPQ1Xycie02zWqdH42dstyyPug/VZ
eIcrl3yOSdEzN5wx4Cu0Y8CrSCnf5ZnCuPU3DcD2JpSZRcKaJVX7ITuW2iNrlgkO
flzKt5T35OKENQAvs7GxbEEJvFMruHnHp73SivVBqBnuZv31Q7oLzgQnRw==
-----END CERTIFICATE-----
Now lets take the contents of the metal3-config.yaml created from the blog above and fashion them into a file called installer-manifests.yaml saving it on the Minishift host as well. The contents will look similar to the following (adjusted based on environment of course):
apiVersion: v1
data:
99_metal3-config.yaml: |
kind: ConfigMap
apiVersion: v1
metadata:
name: metal3-config
namespace: openshift-machine-api
data:
http_port: "6180"
provisioning_interface: "ens3"
provisioning_ip: "172.22.0.3/24"
dhcp_range: "172.22.0.10,172.22.0.100"
deploy_kernel_url: "http://172.22.0.3:6180/images/ironic-python-agent.kernel"
deploy_ramdisk_url: "http://172.22.0.3:6180/images/ironic-python-agent.initramfs"
ironic_endpoint: "http://172.22.0.3:6385/v1/"
ironic_inspector_endpoint: "http://172.22.0.3:5050/v1/"
cache_url: "http://192.168.0.246/images"
rhcos_image_url: "http://172.22.0.1/images/rhcos-43.81.202001142154.0-openstack.x86_64.qcow2.gz?sha256=2da72a1eaee005458014cfbab93f77c459b8c63ff110f0d4cbf4c13e7001de68"
kind: ConfigMap
metadata:
name: kni5-install-manifests
Finally lets construct the clusterdeploy.yaml that will be used to initiate the deployment via Hive of our cluster. Again make sure to adjust the pull secret, ssh private key and known host fields here to suit environment:
apiVersion: v1
items:
- apiVersion: v1
kind: Secret
metadata:
creationTimestamp: null
name: kni5-pull-secret
stringData:
.dockerconfigjson: 'PULL SECRETS HERE'
type: kubernetes.io/dockerconfigjson
- apiVersion: v1
kind: Secret
metadata:
creationTimestamp: null
name: kni5-ssh-private-key
stringData:
ssh-privatekey: |
-----BEGIN RSA PRIVATE KEY-----
-----END RSA PRIVATE KEY-----
type: Opaque
- apiVersion: hive.openshift.io/v1
kind: ClusterDeployment
metadata:
creationTimestamp: null
name: kni5
annotations:
hive.openshift.io/try-install-once: "true"
hive.openshift.io/ssh-known-host: "KNOWN HOST STRING HERE"
spec:
baseDomain: e2e.bos.redhat.com
clusterName: kni5
controlPlaneConfig:
servingCertificates: {}
images:
installerImagePullPolicy: Always
installed: false
platform:
bareMetal: {}
provisioning:
releaseImage: quay.io/openshift-release-dev/ocp-release-nightly@sha256:73007e97e00d526426c116b1a35d8103bd00a887656d065b7979ab2e8de44b17
installConfigSecretRef:
name: kni5-install-config
sshPrivateKeySecretRef:
name: kni5-ssh-private-key
manifestsConfigMapRef:
name: kni5-install-manifests
pullSecretRef:
name: kni5-pull-secret
status:
clusterVersionStatus:
availableUpdates: null
desired:
force: false
image: ""
version: ""
observedGeneration: 0
versionHash: ""
- apiVersion: hive.openshift.io/v1
kind: MachinePool
metadata:
creationTimestamp: null
name: kni5-worker
spec:
clusterDeploymentRef:
name: kni5
name: worker
platform:
bareMetal: {}
replicas: 0
status: {}
kind: List
metadata: {}
Now that we have our three configuration files: clusterdeploy.yaml, installer-manifests.yaml and install-config.yaml, we can begin to deploy OpensShift 4.3 on baremetal via Hive.
First we need to create a new project on the Minishift cluster:
$ oc new-project ocpbaredeploy
Now using project "ocpbaredeploy" on server "https://192.168.42.173:8443".
You can add applications to this project with the 'new-app' command. For example, try:
oc new-app centos/ruby-25-centos7~https://github.com/sclorg/ruby-ex.git
to build a new example application in Ruby.
Next we will configure a secret, in my case called kni5-install-config, from the install-config.yaml file:
$ oc create secret generic kni5-install-config --from-file=install-config.yaml secret/kni5-install-config created
In addition to the secret we need to also create the installer-manifests from file:
$ oc create -f installer-manifests.yaml configmap/kni5-install-manifests created
And finally we can create the cluster deployment itself which will actually launch the OCP 4.3 baremetal deployment:
$ oc create -f clusterdeploy.yaml secret/kni5-pull-secret created secret/kni5-ssh-private-key created clusterdeployment.hive.openshift.io/kni5 created
If we do a oc get pods we can see the status of our provision container:
$ oc get pods NAME READY STATUS RESTARTS AGE kni5-0-6d6wc-provision-rnxqf 0/3 ContainerCreating 0 1s kni5-imageset-7vh7c 0/2 Completed 0 30s
We can follow along with the deployment by watching the logs of the provision container with the following oc logs -f command:
$ oc logs -f kni5-0-6d6wc-provision-rnxqf -c hive time="2020-01-28T16:42:10Z" level=debug msg="checking for SSH private key" installID=gm4l6fj7 time="2020-01-28T16:42:10Z" level=debug msg="no SSH_AUTH_SOCK defined. starting ssh-agent" installID=gm4l6fj7 Identity added: /tmp/ssh-privatekey (ansible-generated on rhel8-ocp-auto) time="2020-01-28T16:42:10Z" level=info msg="waiting for files to be available: [/output/openshift-install /output/oc]" installID=gm4l6fj7 time="2020-01-28T16:42:10Z" level=info msg="found file" installID=gm4l6fj7 path=/output/openshift-install time="2020-01-28T16:42:10Z" level=info msg="found file" installID=gm4l6fj7 path=/output/oc time="2020-01-28T16:42:10Z" level=info msg="all files found, ready to proceed" installID=gm4l6fj7 time="2020-01-28T16:42:10Z" level=info msg="copying install-config.yaml" installID=gm4l6fj7 time="2020-01-28T16:42:10Z" level=info msg="copied /installconfig/install-config.yaml to /output/install-config.yaml" installID=gm4l6fj7 time="2020-01-28T16:42:10Z" level=info msg="Adding user ID to passwd file: 1000170000" installID=gm4l6fj7 time="2020-01-28T16:42:10Z" level=info msg="Wrote passwd line: default:x:1000170000:0:default user:/home/hive:/sbin/nologin\n" installID=gm4l6fj7 time="2020-01-28T16:42:10Z" level=info msg="Wrote known host to /home/hive/.ssh/known_hosts: rhel8-ocp-auto.schmaustech.com,192.168.0.246 ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBAvnpsB2rUlrxgNQGdqSB10bjPybWFdSX7jqZta5FxOZEYPEVwC5QD/FlSYEjM4GUteNRI1vcEFstBkQs0T4WJU=" installID=gm4l6fj7 time="2020-01-28T16:42:10Z" level=info msg="cleaning up from past install attempts" installID=gm4l6fj7 time="2020-01-28T16:42:10Z" level=debug msg="object does not exist" installID=gm4l6fj7 object=ocpbaredeploy/kni5-0-6d6wc-admin-kubeconfig time="2020-01-28T16:42:10Z" level=debug msg="object does not exist" installID=gm4l6fj7 object=ocpbaredeploy/kni5-0-6d6wc-admin-password time="2020-01-28T16:42:10Z" level=warning msg="skipping cleanup as no infra ID set" installID=gm4l6fj7 time="2020-01-28T16:42:10Z" level=info msg="generating assets" installID=gm4l6fj7 time="2020-01-28T16:42:10Z" level=info msg="running openshift-install create manifests" installID=gm4l6fj7 time="2020-01-28T16:42:10Z" level=info msg="running openshift-install binary" args="[create manifests]" installID=gm4l6fj7 time="2020-01-28T16:42:10Z" level=info msg="waiting for files to be available: [/output/.openshift_install.log]" installID=gm4l6fj7 time="2020-01-28T16:42:10Z" level=info msg="found file" installID=gm4l6fj7 path=/output/.openshift_install.log time="2020-01-28T16:42:10Z" level=info msg="all files found, ready to proceed" installID=gm4l6fj7 time="2020-01-28T16:42:10Z" level=debug msg="OpenShift Installer v4.3.0" time="2020-01-28T16:42:10Z" level=debug msg="Built from commit e39341c5c2b9e6fb2c7ddfa2d3220419aa0e2072" time="2020-01-28T16:42:10Z" level=debug msg="Fetching Master Machines..." time="2020-01-28T16:42:10Z" level=debug msg="Loading Master Machines..." time="2020-01-28T16:42:10Z" level=debug msg=" Loading Cluster ID..." time="2020-01-28T16:42:10Z" level=debug msg=" Loading Install Config..." time="2020-01-28T16:42:10Z" level=debug msg=" Loading SSH Key..." time="2020-01-28T16:42:10Z" level=debug msg=" Loading Base Domain..." time="2020-01-28T16:42:10Z" level=debug msg=" Loading Platform..." time="2020-01-28T16:42:10Z" level=debug msg=" Loading Cluster Name..." time="2020-01-28T16:42:10Z" level=debug msg=" Loading Base Domain..." time="2020-01-28T16:42:10Z" level=debug msg=" Loading Pull Secret..." time="2020-01-28T16:42:10Z" level=debug msg=" Loading Platform..." time="2020-01-28T16:42:10Z" level=debug msg=" Using Install Config loaded from target directory" time="2020-01-28T16:42:10Z" level=debug msg=" Loading Platform Credentials Check..." time="2020-01-28T16:42:10Z" level=debug msg=" Loading Install Config..." time="2020-01-28T16:42:10Z" level=debug msg=" Loading Install Config..." time="2020-01-28T16:42:10Z" level=debug msg=" Loading Image..." time="2020-01-28T16:42:10Z" level=debug msg=" Loading Install Config..." time="2020-01-28T16:42:10Z" level=debug msg=" Loading Master Ignition Config..." time="2020-01-28T16:42:10Z" level=debug msg=" Loading Install Config..." time="2020-01-28T16:42:10Z" level=debug msg=" Loading Root CA..." time="2020-01-28T16:42:10Z" level=debug msg=" Fetching Cluster ID..." time="2020-01-28T16:42:10Z" level=debug msg=" Fetching Install Config..." time="2020-01-28T16:42:10Z" level=debug msg=" Reusing previously-fetched Install Config" time="2020-01-28T16:42:10Z" level=debug msg=" Generating Cluster ID..." time="2020-01-28T16:42:10Z" level=debug msg=" Fetching Platform Credentials Check..." time="2020-01-28T16:42:10Z" level=debug msg=" Fetching Install Config..." time="2020-01-28T16:42:10Z" level=debug msg=" Reusing previously-fetched Install Config" time="2020-01-28T16:42:10Z" level=debug msg=" Generating Platform Credentials Check..." time="2020-01-28T16:42:10Z" level=debug msg=" Fetching Install Config..." time="2020-01-28T16:42:10Z" level=debug msg=" Reusing previously-fetched Install Config" time="2020-01-28T16:42:10Z" level=debug msg=" Fetching Image..." time="2020-01-28T16:42:10Z" level=debug msg=" Fetching Install Config..." time="2020-01-28T16:42:10Z" level=debug msg=" Reusing previously-fetched Install Config" time="2020-01-28T16:42:10Z" level=debug msg=" Generating Image..." time="2020-01-28T16:42:10Z" level=debug msg=" Fetching Master Ignition Config..." time="2020-01-28T16:42:10Z" level=debug msg=" Fetching Install Config..." time="2020-01-28T16:42:10Z" level=debug msg=" Reusing previously-fetched Install Config" time="2020-01-28T16:42:10Z" level=debug msg=" Fetching Root CA..." time="2020-01-28T16:42:10Z" level=debug msg=" Generating Root CA..." time="2020-01-28T16:42:10Z" level=debug msg=" Generating Master Ignition Config..." time="2020-01-28T16:42:10Z" level=debug msg="Generating Master Machines..." time="2020-01-28T16:42:10Z" level=info msg="Consuming Install Config from target directory"
Once the install completes we should see this at the end of the logging:
time="2020-01-28T17:42:52Z" level=debug msg="Still waiting for the cluster to initialize: Working towards 4.3.0-0.nightly-2019-12-06-094536: 100% complete, waiting on authentication" time="2020-01-28T17:42:52Z" level=debug msg="Still waiting for the cluster to initialize: Working towards 4.3.0-0.nightly-2019-12-06-094536: 100% complete, waiting on authentication" time="2020-01-28T17:42:52Z" level=debug msg="Still waiting for the cluster to initialize: Working towards 4.3.0-0.nightly-2019-12-06-094536: 100% complete, waiting on authentication" time="2020-01-28T17:44:50Z" level=debug msg="Cluster is initialized" time="2020-01-28T17:44:50Z" level=debug msg="Cluster is initialized" time="2020-01-28T17:44:50Z" level=info msg="Waiting up to 10m0s for the openshift-console route to be created..." time="2020-01-28T17:44:50Z" level=debug msg="Cluster is initialized" time="2020-01-28T17:44:50Z" level=info msg="Waiting up to 10m0s for the openshift-console route to be created..." time="2020-01-28T17:44:50Z" level=info msg="Waiting up to 10m0s for the openshift-console route to be created..." time="2020-01-28T17:44:50Z" level=debug msg="Route found in openshift-console namespace: console" time="2020-01-28T17:44:50Z" level=debug msg="Route found in openshift-console namespace: downloads" time="2020-01-28T17:44:50Z" level=debug msg="OpenShift console route is created" time="2020-01-28T17:44:50Z" level=debug msg="Route found in openshift-console namespace: console" time="2020-01-28T17:44:50Z" level=debug msg="Route found in openshift-console namespace: console" time="2020-01-28T17:44:50Z" level=debug msg="Route found in openshift-console namespace: downloads" time="2020-01-28T17:44:50Z" level=debug msg="OpenShift console route is created" time="2020-01-28T17:44:50Z" level=debug msg="Route found in openshift-console namespace: downloads" time="2020-01-28T17:44:50Z" level=debug msg="OpenShift console route is created" time="2020-01-28T17:44:50Z" level=info msg="Install complete!" time="2020-01-28T17:44:50Z" level=info msg="To access the cluster as the system:admin user when using 'oc', run 'export KUBECONFIG=/output/auth/kubeconfig'" time="2020-01-28T17:44:50Z" level=info msg="Access the OpenShift web-console here: https://console-openshift-console.apps.kni5.schmaustech.com" REDACTED LINE OF OUTPUT time="2020-01-28T17:44:50Z" level=info msg="Install complete!" time="2020-01-28T17:44:50Z" level=info msg="To access the cluster as the system:admin user when using 'oc', run 'export KUBECONFIG=/output/auth/kubeconfig'" time="2020-01-28T17:44:50Z" level=info msg="Access the OpenShift web-console here: https://console-openshift-console.apps.kni5.schmaustech.com" REDACTED LINE OF OUTPUT time="2020-01-28T17:44:50Z" level=info msg="Install complete!" time="2020-01-28T17:44:50Z" level=info msg="To access the cluster as the system:admin user when using 'oc', run 'export KUBECONFIG=/output/auth/kubeconfig'" time="2020-01-28T17:44:50Z" level=info msg="Access the OpenShift web-console here: https://console-openshift-console.apps.kni5.schmaustech.com" REDACTED LINE OF OUTPUT time="2020-01-28T17:44:51Z" level=info msg="command completed successfully" installID=gm4l6fj7 time="2020-01-28T17:44:51Z" level=info msg="saving installer output" installID=gm4l6fj7 time="2020-01-28T17:44:51Z" level=debug msg="installer console log: level=info msg=\"Obtaining RHCOS image file from 'https://releases-art-rhcos.svc.ci.openshift.org/art/storage/releases/rhcos-4.3/43.81.201911221453.0/x86_64/rhcos-43.81.201911221453.0-qemu.x86_64.qcow2.gz?sha256=073fb598762e56c7852e743f4e2f1187f7783f9bc741aa993e4cc36a70325ab2'\"\nlevel=info msg=\"Consuming Worker Ignition Config from target directory\"\nlevel=info msg=\"Consuming Bootstrap Ignition Config from target directory\"\nlevel=info msg=\"Consuming Master Ignition Config from target directory\"\nlevel=info msg=\"Creating infrastructure resources...\"\nlevel=info msg=\"Waiting up to 30m0s for the Kubernetes API at https://api.kni5.schmaustech.com:6443...\"\nlevel=info msg=\"API v1.16.2 up\"\nlevel=info msg=\"Waiting up to 30m0s for bootstrapping to complete...\"\nlevel=info msg=\"Destroying the bootstrap resources...\"\nlevel=info msg=\"Waiting up to 30m0s for the cluster at https://api.kni5.schmaustech.com:6443 to initialize...\"\nlevel=info msg=\"Waiting up to 10m0s for the openshift-console route to be created...\"\nlevel=info msg=\"Install complete!\"\nlevel=info msg=\"To access the cluster as the system:admin user when using 'oc', run 'export KUBECONFIG=/output/auth/kubeconfig'\"\nlevel=info msg=\"Access the OpenShift web-console here: https://console-openshift-console.apps.kni5.schmaustech.com\"\nREDACTED LINE OF OUTPUT\n" installID=gm4l6fj7 time="2020-01-28T17:44:51Z" level=info msg="install completed successfully" installID=gm4l6fj7
To validate we have a fully operational cluster we will need to obtain the kubeconfig. This is stored in the secrets of hive. Below we will grep for all secrets related to the cluster kni5:
$ oc get secrets|grep kni5 kni5-0-6d6wc-admin-kubeconfig Opaque 2 1h kni5-0-6d6wc-admin-password Opaque 2 1h kni5-install-config Opaque 1 1h kni5-merged-pull-secret kubernetes.io/dockerconfigjson 1 1h kni5-pull-secret kubernetes.io/dockerconfigjson 1 1h kni5-ssh-private-key Opaque 1 1h
We are specifically interested in the secret with kubeconfig in the naming convention and we can use the command below to extract out the kubeconfig and place it into a file:
oc get secret kni5-0-6d6wc-admin-kubeconfig -o jsonpath='{ .data.kubeconfig }' | base64 --decode > kni5.kubeconfig
After extraction we can export the kni5 kubeconfig to the KUBECONFIG variable:
export KUBECONFIG=./kni5.kubeconfig
And finally we can do an oc get nodes to see the nodes:
$ oc get nodes NAME STATUS ROLES AGE VERSION master-0.kni5.schmaustech.com Ready master,worker 74m v1.16.2 master-1.kni5.schmaustech.com Ready master,worker 74m v1.16.2 master-2.kni5.schmaustech.com Ready master,worker 74m v1.16.2
And a get all pods to see the running containers:
$ oc get pods --all-namespaces NAMESPACE NAME READY STATUS RESTARTS AGE openshift-apiserver-operator openshift-apiserver-operator-6467df49bb-k9scl 1/1 Running 2 75m openshift-apiserver apiserver-d8rbk 1/1 Running 0 67m openshift-apiserver apiserver-ngg7c 1/1 Running 0 68m openshift-apiserver apiserver-sqv5l 1/1 Running 0 68m openshift-authentication-operator authentication-operator-c4788d544-r2cns 1/1 Running 0 66m openshift-authentication oauth-openshift-664ff64575-n7hbf 1/1 Running 0 63m openshift-authentication oauth-openshift-664ff64575-qp9r9 1/1 Running 0 63m openshift-cloud-credential-operator cloud-credential-operator-cfccb976c-5h9hb 1/1 Running 0 75m openshift-cluster-machine-approver machine-approver-58bf678f78-tf75d 2/2 Running 0 75m openshift-cluster-node-tuning-operator cluster-node-tuning-operator-84dcb88fdb-rv6w6 1/1 Running 0 66m openshift-cluster-node-tuning-operator tuned-6pqvv 1/1 Running 0 66m openshift-cluster-node-tuning-operator tuned-gcpsl 1/1 Running 0 66m openshift-cluster-node-tuning-operator tuned-hlf5j 1/1 Running 0 66m openshift-cluster-samples-operator cluster-samples-operator-6b477c8f46-pl7x4 2/2 Running 0 62m openshift-cluster-storage-operator cluster-storage-operator-85cd5b4567-zqbdz 1/1 Running 0 64m openshift-cluster-version cluster-version-operator-6747896cf7-68mz6 1/1 Running 0 75m openshift-console-operator console-operator-7f6b8d755d-b4hk4 1/1 Running 0 64m openshift-console console-869c7d68fb-jbz7j 1/1 Running 0 60m openshift-console console-869c7d68fb-jfd8p 1/1 Running 0 63m openshift-console downloads-79cb8f7c7f-49xt4 1/1 Running 0 64m openshift-console downloads-79cb8f7c7f-kcfnh 1/1 Running 0 64m openshift-controller-manager-operator openshift-controller-manager-operator-6bd6b5c9b7-4djr8 1/1 Running 2 75m openshift-controller-manager controller-manager-hq5pw 1/1 Running 0 66m openshift-controller-manager controller-manager-vjdlc 1/1 Running 0 66m openshift-controller-manager controller-manager-wb8m2 1/1 Running 0 68m openshift-dns-operator dns-operator-768898c96f-cpb4m 2/2 Running 0 75m openshift-dns dns-default-8rs4t 2/2 Running 0 71m openshift-dns dns-default-bt8jr 2/2 Running 0 71m openshift-dns dns-default-t2bx7 2/2 Running 0 71m openshift-etcd etcd-member-master-0.kni5.schmaustech.com 2/2 Running 0 74m openshift-etcd etcd-member-master-1.kni5.schmaustech.com 2/2 Running 0 75m openshift-etcd etcd-member-master-2.kni5.schmaustech.com 2/2 Running 0 75m openshift-image-registry cluster-image-registry-operator-69cfd6b89-ldv6d 2/2 Running 0 64m openshift-ingress-operator ingress-operator-846944b6cd-442mh 2/2 Running 0 64m openshift-ingress router-default-668c8bc58c-5p65n 1/1 Running 0 63m openshift-ingress router-default-668c8bc58c-7lbl7 1/1 Running 0 62m openshift-insights insights-operator-5f8867cf5b-2b4c6 1/1 Running 3 75m openshift-kni-infra coredns-master-0.kni5.schmaustech.com 1/1 Running 0 74m openshift-kni-infra coredns-master-1.kni5.schmaustech.com 1/1 Running 0 75m openshift-kni-infra coredns-master-2.kni5.schmaustech.com 1/1 Running 0 74m openshift-kni-infra haproxy-master-0.kni5.schmaustech.com 2/2 Running 0 74m openshift-kni-infra haproxy-master-1.kni5.schmaustech.com 2/2 Running 0 75m openshift-kni-infra haproxy-master-2.kni5.schmaustech.com 2/2 Running 0 75m openshift-kni-infra keepalived-master-0.kni5.schmaustech.com 2/2 Running 0 74m openshift-kni-infra keepalived-master-1.kni5.schmaustech.com 2/2 Running 0 75m openshift-kni-infra keepalived-master-2.kni5.schmaustech.com 2/2 Running 0 73m openshift-kni-infra mdns-publisher-master-0.kni5.schmaustech.com 1/1 Running 0 74m openshift-kni-infra mdns-publisher-master-1.kni5.schmaustech.com 1/1 Running 0 75m openshift-kni-infra mdns-publisher-master-2.kni5.schmaustech.com 1/1 Running 0 74m openshift-kube-apiserver-operator kube-apiserver-operator-5866656944-6j7jd 1/1 Running 2 75m openshift-kube-apiserver installer-2-master-0.kni5.schmaustech.com 0/1 Completed 0 68m openshift-kube-apiserver installer-2-master-1.kni5.schmaustech.com 0/1 Completed 0 64m openshift-kube-apiserver installer-2-master-2.kni5.schmaustech.com 0/1 Completed 0 69m openshift-kube-apiserver installer-3-master-1.kni5.schmaustech.com 0/1 Completed 0 64m openshift-kube-apiserver installer-5-master-0.kni5.schmaustech.com 0/1 Completed 0 57m openshift-kube-apiserver installer-5-master-1.kni5.schmaustech.com 0/1 Completed 0 62m openshift-kube-apiserver installer-5-master-2.kni5.schmaustech.com 0/1 Completed 0 60m openshift-kube-apiserver kube-apiserver-master-0.kni5.schmaustech.com 3/3 Running 0 57m openshift-kube-apiserver kube-apiserver-master-1.kni5.schmaustech.com 3/3 Running 0 62m openshift-kube-apiserver kube-apiserver-master-2.kni5.schmaustech.com 3/3 Running 0 60m openshift-kube-apiserver revision-pruner-2-master-0.kni5.schmaustech.com 0/1 Completed 0 64m openshift-kube-apiserver revision-pruner-2-master-1.kni5.schmaustech.com 0/1 Completed 0 64m openshift-kube-apiserver revision-pruner-2-master-2.kni5.schmaustech.com 0/1 Completed 0 64m openshift-kube-apiserver revision-pruner-3-master-1.kni5.schmaustech.com 0/1 Completed 0 62m openshift-kube-apiserver revision-pruner-5-master-0.kni5.schmaustech.com 0/1 Completed 0 56m openshift-kube-apiserver revision-pruner-5-master-1.kni5.schmaustech.com 0/1 Completed 0 60m openshift-kube-apiserver revision-pruner-5-master-2.kni5.schmaustech.com 0/1 Completed 0 58m openshift-kube-controller-manager-operator kube-controller-manager-operator-7967d57b6c-c2wlk 1/1 Running 2 75m openshift-kube-controller-manager installer-4-master-2.kni5.schmaustech.com 0/1 Completed 0 69m openshift-kube-controller-manager installer-5-master-0.kni5.schmaustech.com 0/1 Completed 0 68m openshift-kube-controller-manager installer-5-master-1.kni5.schmaustech.com 0/1 Completed 0 67m openshift-kube-controller-manager installer-5-master-2.kni5.schmaustech.com 0/1 Completed 0 65m openshift-kube-controller-manager installer-6-master-0.kni5.schmaustech.com 0/1 Completed 0 61m openshift-kube-controller-manager installer-6-master-1.kni5.schmaustech.com 0/1 Completed 0 60m openshift-kube-controller-manager installer-6-master-2.kni5.schmaustech.com 0/1 OOMKilled 0 63m openshift-kube-controller-manager installer-7-master-0.kni5.schmaustech.com 0/1 Completed 0 57m openshift-kube-controller-manager installer-7-master-1.kni5.schmaustech.com 0/1 Completed 0 59m openshift-kube-controller-manager installer-7-master-2.kni5.schmaustech.com 0/1 Completed 0 59m openshift-kube-controller-manager kube-controller-manager-master-0.kni5.schmaustech.com 3/3 Running 0 57m openshift-kube-controller-manager kube-controller-manager-master-1.kni5.schmaustech.com 3/3 Running 0 59m openshift-kube-controller-manager kube-controller-manager-master-2.kni5.schmaustech.com 3/3 Running 0 58m openshift-kube-controller-manager revision-pruner-4-master-2.kni5.schmaustech.com 0/1 Completed 0 68m openshift-kube-controller-manager revision-pruner-5-master-0.kni5.schmaustech.com 0/1 Completed 0 68m openshift-kube-controller-manager revision-pruner-5-master-1.kni5.schmaustech.com 0/1 Completed 0 65m openshift-kube-controller-manager revision-pruner-5-master-2.kni5.schmaustech.com 0/1 Completed 0 64m openshift-kube-controller-manager revision-pruner-6-master-0.kni5.schmaustech.com 0/1 Completed 0 60m openshift-kube-controller-manager revision-pruner-6-master-1.kni5.schmaustech.com 0/1 Completed 0 59m openshift-kube-controller-manager revision-pruner-6-master-2.kni5.schmaustech.com 0/1 Completed 0 61m openshift-kube-controller-manager revision-pruner-7-master-0.kni5.schmaustech.com 0/1 Completed 0 56m openshift-kube-controller-manager revision-pruner-7-master-1.kni5.schmaustech.com 0/1 Completed 0 59m openshift-kube-controller-manager revision-pruner-7-master-2.kni5.schmaustech.com 0/1 Completed 0 58m openshift-kube-scheduler-operator openshift-kube-scheduler-operator-6bbb9c89c4-8gfql 1/1 Running 2 75m openshift-kube-scheduler installer-2-master-1.kni5.schmaustech.com 0/1 Completed 0 71m openshift-kube-scheduler installer-3-master-2.kni5.schmaustech.com 0/1 Completed 0 71m openshift-kube-scheduler installer-4-master-0.kni5.schmaustech.com 0/1 Completed 0 70m openshift-kube-scheduler installer-4-master-1.kni5.schmaustech.com 0/1 Completed 0 69m openshift-kube-scheduler installer-5-master-0.kni5.schmaustech.com 0/1 Completed 0 65m openshift-kube-scheduler installer-5-master-1.kni5.schmaustech.com 0/1 Completed 0 68m openshift-kube-scheduler installer-5-master-2.kni5.schmaustech.com 0/1 Completed 0 67m openshift-kube-scheduler installer-6-master-0.kni5.schmaustech.com 0/1 Completed 0 60m openshift-kube-scheduler installer-6-master-1.kni5.schmaustech.com 0/1 Completed 0 63m openshift-kube-scheduler installer-6-master-2.kni5.schmaustech.com 0/1 Completed 0 61m openshift-kube-scheduler openshift-kube-scheduler-master-0.kni5.schmaustech.com 1/1 Running 0 60m openshift-kube-scheduler openshift-kube-scheduler-master-1.kni5.schmaustech.com 1/1 Running 0 63m openshift-kube-scheduler openshift-kube-scheduler-master-2.kni5.schmaustech.com 1/1 Running 1 61m openshift-kube-scheduler revision-pruner-2-master-1.kni5.schmaustech.com 0/1 Completed 0 71m openshift-kube-scheduler revision-pruner-3-master-2.kni5.schmaustech.com 0/1 Completed 0 70m openshift-kube-scheduler revision-pruner-4-master-0.kni5.schmaustech.com 0/1 Completed 0 69m openshift-kube-scheduler revision-pruner-4-master-1.kni5.schmaustech.com 0/1 Completed 0 69m openshift-kube-scheduler revision-pruner-5-master-0.kni5.schmaustech.com 0/1 Completed 0 64m openshift-kube-scheduler revision-pruner-5-master-1.kni5.schmaustech.com 0/1 Completed 0 67m openshift-kube-scheduler revision-pruner-5-master-2.kni5.schmaustech.com 0/1 Completed 0 65m openshift-kube-scheduler revision-pruner-6-master-0.kni5.schmaustech.com 0/1 Completed 0 59m openshift-kube-scheduler revision-pruner-6-master-1.kni5.schmaustech.com 0/1 Completed 0 61m openshift-kube-scheduler revision-pruner-6-master-2.kni5.schmaustech.com 0/1 Completed 0 60m openshift-machine-api cluster-autoscaler-operator-5967b8b469-r8r7r 2/2 Running 0 62m openshift-machine-api machine-api-controllers-59876d48b9-5pc8n 4/4 Running 0 69m openshift-machine-api machine-api-operator-7dd8fbc5c-bd4h6 2/2 Running 0 74m openshift-machine-api metal3-6d87476688-x5chm 8/8 Running 0 69m openshift-machine-config-operator etcd-quorum-guard-d6ff6bd8f-247hv 1/1 Running 0 70m openshift-machine-config-operator etcd-quorum-guard-d6ff6bd8f-27vd6 1/1 Running 0 70m openshift-machine-config-operator etcd-quorum-guard-d6ff6bd8f-d6jql 1/1 Running 0 70m openshift-machine-config-operator machine-config-controller-98b8ff5d-m7764 1/1 Running 1 71m openshift-machine-config-operator machine-config-daemon-lkw5r 2/2 Running 0 72m openshift-machine-config-operator machine-config-daemon-nlbbn 2/2 Running 0 72m openshift-machine-config-operator machine-config-daemon-sxnpb 2/2 Running 0 72m openshift-machine-config-operator machine-config-operator-5c4c599bc7-fmgn2 1/1 Running 0 75m openshift-machine-config-operator machine-config-server-bx9q2 1/1 Running 0 70m openshift-machine-config-operator machine-config-server-pfhgn 1/1 Running 0 70m openshift-machine-config-operator machine-config-server-vztjh 1/1 Running 0 70m openshift-marketplace certified-operators-699d47c664-sp9z9 1/1 Running 0 63m openshift-marketplace community-operators-66d78d8cd-hs2td 1/1 Running 0 63m openshift-marketplace marketplace-operator-6487d7d4c7-lwzvd 1/1 Running 0 64m openshift-marketplace redhat-operators-6b6f87448f-dwbbr 1/1 Running 0 63m openshift-monitoring alertmanager-main-0 3/3 Running 0 61m openshift-monitoring alertmanager-main-1 3/3 Running 0 61m openshift-monitoring alertmanager-main-2 3/3 Running 0 61m openshift-monitoring cluster-monitoring-operator-6bd6f6485b-qgtwx 1/1 Running 0 64m openshift-monitoring grafana-7d5458c694-k2q4j 2/2 Running 0 61m openshift-monitoring kube-state-metrics-544fbcbfbb-nk2v7 3/3 Running 0 63m openshift-monitoring node-exporter-9qbz9 2/2 Running 0 63m openshift-monitoring node-exporter-gzd5b 2/2 Running 0 63m openshift-monitoring node-exporter-s8lm6 2/2 Running 0 63m openshift-monitoring openshift-state-metrics-cfbc56bbc-9zprs 3/3 Running 0 63m openshift-monitoring prometheus-adapter-5686f9c8bb-8tv9l 1/1 Running 0 61m openshift-monitoring prometheus-adapter-5686f9c8bb-d54hf 1/1 Running 0 61m openshift-monitoring prometheus-k8s-0 7/7 Running 1 58m openshift-monitoring prometheus-k8s-1 7/7 Running 1 59m openshift-monitoring prometheus-operator-d7c899b8c-qq7qw 1/1 Running 0 61m openshift-monitoring telemeter-client-778b4db874-d9c6t 3/3 Running 0 61m openshift-monitoring thanos-querier-5984bb45dc-6g2xs 4/4 Running 0 61m openshift-monitoring thanos-querier-5984bb45dc-vlttb 4/4 Running 0 61m openshift-multus multus-276kq 1/1 Running 0 74m openshift-multus multus-admission-controller-5k87f 1/1 Running 1 72m openshift-multus multus-admission-controller-m7tg8 1/1 Running 0 72m openshift-multus multus-admission-controller-mxj4j 1/1 Running 1 72m openshift-multus multus-qnfm2 1/1 Running 0 74m openshift-multus multus-xw572 1/1 Running 0 74m openshift-network-operator network-operator-69d75fd9b6-msztp 1/1 Running 0 75m openshift-operator-lifecycle-manager catalog-operator-767bc6d8c5-nlbt4 1/1 Running 0 75m openshift-operator-lifecycle-manager olm-operator-85b66cf96d-r4tqw 1/1 Running 0 75m openshift-operator-lifecycle-manager packageserver-69dcc957f8-p5rp6 1/1 Running 0 70m openshift-operator-lifecycle-manager packageserver-69dcc957f8-p9vhq 1/1 Running 1 70m openshift-sdn ovs-59mq4 1/1 Running 0 74m openshift-sdn ovs-k8gs8 1/1 Running 0 74m openshift-sdn ovs-tbl69 1/1 Running 0 74m openshift-sdn sdn-4s4pv 1/1 Running 0 74m openshift-sdn sdn-controller-l4g5k 1/1 Running 0 74m openshift-sdn sdn-controller-lkjxc 1/1 Running 0 74m openshift-sdn sdn-controller-skrkp 1/1 Running 0 74m openshift-sdn sdn-f4j6f 1/1 Running 0 74m openshift-sdn sdn-wpp5n 1/1 Running 0 74m openshift-service-ca-operator service-ca-operator-7556fc7b7-9hq5t 1/1 Running 1 75m openshift-service-ca apiservice-cabundle-injector-c865fd686-pp66j 1/1 Running 1 71m openshift-service-ca configmap-cabundle-injector-796f8b9994-ntnwp 1/1 Running 1 71m openshift-service-ca service-serving-cert-signer-5f65674db4-x7ltk 1/1 Running 1 71m openshift-service-catalog-apiserver-operator openshift-service-catalog-apiserver-operator-84689c4d85-45kcd 1/1 Running 0 66m openshift-service-catalog-controller-manager-operator openshift-service-catalog-controller-manager-operator-67fd4fkzk 1/1 Running 0 66m
If everything worked out you should now have a OpenShift 4.3 Baremetal deployment that was deployed via Hive. However I should note that Hive is rapidly evolving the method used here could be changing in the future.
