Sunday, November 06, 2022

Microshift, RHEL9 & Apple M1 Virtual Machine

I previously had written a blog around Microshift running on a virtual machine with Fedora 35 on a MacBook Pro with an M1 processor. However that blog was using Fedora and an older version of Microshift based on the 4.8 release of OpenShift. However in this blog I want to demonstrate running the virtual machine with Red Hat Enterprise Linux 9 and Microshift based on the future 4.12 OpenShift release.

Lab Environment

The following lab environment was created in order to provide this demonstration, which includes the following:

  • MacBook Pro
    • M1 Max Processor
    • 32GB of memory
    • 1.8TB SSD
  • MacOS Ventura
  • UTM Virtualization UI 4.1.0
  • 1 Virtual Machine using Apple Virtualization Framework
    • 4 cores of vCPU
    • 8GB memory
    • 256GB disk
    • Red Hat Enterprise Linux 9
    • Static ip address configured

I have already documented how to install Red Hat Enterprise Linux 9 on an M1 virtual machine and the video for it can be found here.

Microshift Enhancements:

  • OVN replaces Flannel as CNI
  • TopoLVM replaces HostPathProvisioning as CSI storage backend

Build->Deploy->Run Microshift

To get started lets ensure we have the right repositories enabled on our Red Hat Enterprise Linux 9 virtual machine. We will go ahead and register the system, disable all repositories and then enable the repositories we will need.

$ sudo subscription-manager register $ sudo subscription-manager repos --disable=* $ sudo subscription-manager repos --enable=rhel-9-for-aarch64-baseos-rpms --enable=rhel-9-for-aarch64-appstream-rpms --enable=rhel-9-for-aarch64-supplementary-rpms --enable=fast-datapath-for-rhel-9-aarch64-rpms Repository 'rhel-9-for-aarch64-baseos-rpms' is enabled for this system. Repository 'rhel-9-for-aarch64-appstream-rpms' is enabled for this system. Repository 'rhel-9-for-aarch64-supplementary-rpms' is enabled for this system. Repository 'fast-datapath-for-rhel-9-aarch64-rpms' is enabled for this system.

Now let's install some of the pre-requisite packages we will need. Notice we are not installing Golang here and that is because we need a more recent version then what is shipping with Red Hat Enterprise Linux 9.

$ sudo dnf install -y git cockpit make selinux-policy-devel rpm-build bash-completion jq gcc

Now let's fetch Golang with wget and then extract it into /usr/local. We can also make a soft link from /usr/bin/go to the actual binary for convenience.

$ cd ~/ $ wget $ sudo rm -rf /usr/local/go && sudo tar -C /usr/local -xzf go1.19.3.linux-arm64.tar.gz $ sudo ln -s /usr/local/go/bin/go /usr/bin/go

Next we can open up some firewall rules that are required for access when running Microshift.

$ sudo firewall-cmd --permanent --zone=trusted --add-source= $ sudo firewall-cmd --permanent --zone=trusted --add-source= $ sudo firewall-cmd --permanent --zone=public --add-port=80/tcp $ sudo firewall-cmd --permanent --zone=public --add-port=443/tcp $ sudo firewall-cmd --permanent --zone=public --add-port=5353/udp $ sudo firewall-cmd --permanent --zone=public --add-port=30000-32767/tcp $ sudo firewall-cmd --permanent --zone=public --add-port=30000-32767/udp $ sudo firewall-cmd --permanent --zone=public --add-port=6443/tcp $ sudo firewall-cmd --reload

We also need to manually grab the following packages from as we could not find the Red Hat Enterprise Linux 9 equivalents in our repositories.

$ ls -1 *.el8.aarch64* cri-o-1.24.3-5.rhaos4.11.gitc4567c0.el8.aarch64.rpm cri-tools-1.24.2-6.el8.aarch64.rpm

Once the packages finish downloading we can install them.

$ sudo dnf localinstall cri-tools-1.24.2-6.el8.aarch64.rpm cri-o-1.24.3-5.rhaos4.11.gitc4567c0.el8.aarch64.rpm

Next we can go ahead and clone the GitHub repository for Microshift.

$ git clone ~/microshift

Update the following release_arm64.go file to the following based off of this Github issue. Note these image location are not publicly accessible until the Arm version of Microshift images because readily available.

$ cp ~/microshift/pkg/release/release_arm64.go ~/microshift/pkg/release/release_arm64.go.bak $ cat << EOF > ~/microshift/pkg/release/release_arm64.go /* Copyright © 2021 MicroShift Contributors Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with the License. You may obtain a copy of the License at Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and limitations under the License. */ package release // For the amd64 architecture we use the existing and tested and // published OCP or other component upstream images func init() { Image = map[string]string{ "cli": "", "coredns": "", "haproxy_router": "", "kube_rbac_proxy": "", "openssl": "", "ovn_kubernetes_microshift": "", "pause": "", "service_ca_operator": "", "odf_topolvm": "", "ose_csi_ext_provisioner": "", "ose_csi_ext_resizer": "", "topolvm-csi-snapshotter": "", "ose_csi_livenessprobe": "", "ose_csi_node_registrar": "", } } EOF

Also since we are not using a packaged installed Golang we will comment out the Golang build requirements in the specification file.

$ sed -e '/golang/ s/^#*/#/' -i ~/microshift/packaging//rpm/microshift.spec

With the release file updated we can proceed to make the packages of Microshift.

$ cd ~/microshift $ make rpm

After the rpm packages have completed being created proceed to install them.

$ sudo dnf localinstall -y ~/microshift/_output/rpmbuild/RPMS/*/*.rpm

Note the above installation will pull in the following dependencies:

NetworkManager-ovs aarch64 1:1.36.0-5.el9_0 rhel-9-for-aarch64-appstream-rpms conntrack-tools aarch64 1.4.5-10.el9_0.1 rhel-9-for-aarch64-appstream-rpms libnetfilter_cthelper aarch64 1.0.0-22.el9 rhel-9-for-aarch64-appstream-rpms libnetfilter_cttimeout aarch64 1.0.0-19.el9 rhel-9-for-aarch64-appstream-rpms libnetfilter_queue aarch64 1.0.5-1.el9 rhel-9-for-aarch64-appstream-rpms openvswitch-selinux-extra-policy noarch 1.0-31.el9fdp fast-datapath-for-rhel-9-aarch64-rpms openvswitch2.17 aarch64 2.17.0-49.el9fdp fast-datapath-for-rhel-9-aarch64-rpms unbound-libs aarch64 1.13.1-13.el9_0 rhel-9-for-aarch64-appstream-rpms

Set the pull-secret for the crio environment.

$ sudo vi /etc/crio/openshift-pull-secret

Now let's enable crio environment.

$ sudo systemctl enable crio --now

Manually pull the Arm topolvm images we defined in the release_arm.go file above. Again note these images are available publicly and require access to the repository.

$ sudo crictl pull --auth "<YOUR AUTH TOKEN>" $ sudo crictl pull --auth "<YOUR AUTH TOKEN>" $ sudo crictl pull --auth "<YOUR AUTH TOKEN>" $ sudo crictl pull --auth "<YOUR AUTH TOKEN>" $ sudo crictl pull --auth "<YOUR AUTH TOKEN>" $ sudo crictl pull --auth "<YOUR AUTH TOKEN>"

At this point we are ready to start Microshift up.

$ sudo systemctl enable microshift --now Created symlink /etc/systemd/system/ → /usr/lib/systemd/system/microshift.service.

Once the services have been started let's go ahead and create a hidden directory called .kube and copy the kubeconfig in there.

$ mkdir ~/.kube $ sudo cat /var/lib/microshift/resources/kubeadmin/kubeconfig > ~/.kube/config

In a few minutes we can then issue a oc get pods -A and hopefully we see the following pods running.

$ oc get pods -A NAMESPACE NAME READY STATUS RESTARTS AGE openshift-dns dns-default-ph555 1/2 Running 0 6m57s openshift-dns node-resolver-brnj6 1/1 Running 0 6m57s openshift-ingress router-default-54bc9ff944-clr4r 1/1 Running 0 6m57s openshift-ovn-kubernetes ovnkube-master-t9q4w 4/4 Running 0 6m57s openshift-ovn-kubernetes ovnkube-node-f6z66 1/1 Running 0 6m57s openshift-service-ca service-ca-5bb4c5d7f7-zs2gg 1/1 Running 0 6m57s openshift-storage topolvm-controller-5d4f58ff8c-kl7v4 4/4 Running 0 6m57s openshift-storage topolvm-node-7wsh5 4/4 Running 0 6m57s

Hopefully this provides a glimpse as to what one can do on a Red Hat Enterprise Linux 9 virtual machine running on a Apple M1 processor once the Arm packages and images for Microshift become readily available. It could be a great way to do test development work before actually transferring it to the real edge device hardware that utilizes Arm.