Wednesday, November 09, 2022

Monitoring Sensors and Taking Action

I recently wrote a blog around using Microshift to run my Zigbee2MQTT workload. This blog described all the details on how to deploy Microshift and then deploy the components inside of Microshift to enable some home automation. Of course with Zigbee2MQTT there is an intuitive web interface to interact with the smart devices. However I wanted to take another approach that felt more realistic when it comes to edge use cases. I felt that in a industrial scenario there would be some code that would most likely subscribed and monitoring the MQTT queue. An action would be performed when a certain event was observed and the action itself might publish something into the MQTT queue. The rest of this blog will cover a simple scenario like I just described.

First we continue to use the same lab environment I used in my previous blog. The only difference here in the diagram below is we have now added a smart power outlet and a temperature/humidity sensor that can both be controlled remotely via the Zigbee protocol like all my other devices.

The Script

With my lab in place I decided I wanted ot write something in Perl. Some might think why use such an antiquated language like Perl and part of that is because I am old school. For my scenario I envisioned using the humidity sensor to detect when the humidity levels got too high. The threshold would then trigger an action on the event to turn on/off a dehumidifier plugged into the smart outlet. The basic process flow looks like the following diagram:

The script itself can take four different parameters:

  • --hostname: hostname or IP address of MQTT host (required)
  • --port: port for MQTT (optional but will default to 1883 if not provided)
  • --threshold: humidity value that determines when action should be taken
  • --help: prints the usage of script

The script itself is located here

When one runs the script without any flags the usage and an example will be displayed.

./mqtt-humidity.pl Usage: --hostname,-h Hostname or IP address of MQTT host --port,-p Port for MQTT (defaults to default 1883) --threshold,-t Threshold for humidity (defaults to 60) --help,-h Print this help Example: mqtt-humidity.pl -ho 10.43.26.170 -p 1883 -t 65

The Demonstration of Script

To demonstrate this script I went ahead and plugged in a light into my smart outlet which was in the off setting. I launched the script in a terminal window. Then I took the temperature/humidity sensor, cupped it in my hands and blew into my hands. The moisture in my breath is enough to temporarily raise the value. The script provides output so we can see the values changing and sure enough when I breathed into my hands with the sensor the value jumped to 81.37% which triggered the action event and turned on the light. I then set the sensor back on my desk and over the course of 5 minutes the value slowly receded. Once it dropped below the threshold value the light then turned back off. The output of my script run is below:

$ perl mqtt-humidity.pl -ho 10.43.26.170 -p 1883 -t 60 Temp C = 23.43 : Temp F = 74.174 : Humidity = 51.22 Temp C = 23.43 : Temp F = 74.174 : Humidity = 81.37 <-- Smart outlet turned on Temp C = 23.43 : Temp F = 74.174 : Humidity = 84.37 Temp C = 23.43 : Temp F = 74.174 : Humidity = 82.37 Temp C = 23.43 : Temp F = 74.174 : Humidity = 80.28 Temp C = 23.43 : Temp F = 74.174 : Humidity = 78.79 Temp C = 23.63 : Temp F = 74.534 : Humidity = 78.79 Temp C = 23.63 : Temp F = 74.534 : Humidity = 73.21 Temp C = 23.63 : Temp F = 74.534 : Humidity = 74.34 Temp C = 23.63 : Temp F = 74.534 : Humidity = 72.91 Temp C = 23.63 : Temp F = 74.534 : Humidity = 71.65 Temp C = 23.63 : Temp F = 74.534 : Humidity = 70.55 Temp C = 23.63 : Temp F = 74.534 : Humidity = 69.15 Temp C = 23.63 : Temp F = 74.534 : Humidity = 67.93 Temp C = 23.63 : Temp F = 74.534 : Humidity = 66.57 Temp C = 23.63 : Temp F = 74.534 : Humidity = 64.87 Temp C = 23.63 : Temp F = 74.534 : Humidity = 63.28 Temp C = 23.63 : Temp F = 74.534 : Humidity = 62.08 Temp C = 23.63 : Temp F = 74.534 : Humidity = 60.71 Temp C = 23.63 : Temp F = 74.534 : Humidity = 59.12 <-- Smart outlet turned off Temp C = 23.63 : Temp F = 74.534 : Humidity = 57.72 Temp C = 23.63 : Temp F = 74.534 : Humidity = 56.7 Temp C = 23.63 : Temp F = 74.534 : Humidity = 55.6 Temp C = 23.63 : Temp F = 74.534 : Humidity = 54.23 Temp C = 23.63 : Temp F = 74.534 : Humidity = 53.21 Temp C = 23.63 : Temp F = 74.534 : Humidity = 52.14 Temp C = 23.63 : Temp F = 74.534 : Humidity = 51.05 Temp C = 23.63 : Temp F = 74.534 : Humidity = 50.04 Temp C = 23.22 : Temp F = 73.796 : Humidity = 50.04 Temp C = 23.22 : Temp F = 73.796 : Humidity = 50.04 ^C

Now this was a very simple example but imagine the possibilities. For example what if this was a greenhouse that needed to keep the humidity and/or even the temperature at a certain range. If the device that reduces the humidity/temperature (dehumidifier -or- exhaust fan) in the greenhouse could take Zigbee commands directly and control the speed of operation we might be able to not only turn it on/off but also increase/decrease speed of operation. All of this ensures that whatever was growing in the greenhouse is not damaged and also ensures we are powering devices only when we need to have them powered. The bottom line is it saves businesses like the greenhouse operational costs when they are operating efficiently.

Sunday, November 06, 2022

Microshift, RHEL9 & Apple M1 Virtual Machine


I previously had written a blog around Microshift running on a virtual machine with Fedora 35 on a MacBook Pro with an M1 processor. However that blog was using Fedora and an older version of Microshift based on the 4.8 release of OpenShift. However in this blog I want to demonstrate running the virtual machine with Red Hat Enterprise Linux 9 and Microshift based on the future 4.12 OpenShift release.

Lab Environment

The following lab environment was created in order to provide this demonstration, which includes the following:

  • MacBook Pro
    • M1 Max Processor
    • 32GB of memory
    • 1.8TB SSD
  • MacOS Ventura
  • UTM Virtualization UI 4.1.0
  • 1 Virtual Machine using Apple Virtualization Framework
    • 4 cores of vCPU
    • 8GB memory
    • 256GB disk
    • Red Hat Enterprise Linux 9
    • Static ip address configured

I have already documented how to install Red Hat Enterprise Linux 9 on an M1 virtual machine and the video for it can be found here.

Microshift Enhancements:

  • OVN replaces Flannel as CNI
  • TopoLVM replaces HostPathProvisioning as CSI storage backend

Build->Deploy->Run Microshift

To get started lets ensure we have the right repositories enabled on our Red Hat Enterprise Linux 9 virtual machine. We will go ahead and register the system, disable all repositories and then enable the repositories we will need.

$ sudo subscription-manager register $ sudo subscription-manager repos --disable=* $ sudo subscription-manager repos --enable=rhel-9-for-aarch64-baseos-rpms --enable=rhel-9-for-aarch64-appstream-rpms --enable=rhel-9-for-aarch64-supplementary-rpms --enable=fast-datapath-for-rhel-9-aarch64-rpms Repository 'rhel-9-for-aarch64-baseos-rpms' is enabled for this system. Repository 'rhel-9-for-aarch64-appstream-rpms' is enabled for this system. Repository 'rhel-9-for-aarch64-supplementary-rpms' is enabled for this system. Repository 'fast-datapath-for-rhel-9-aarch64-rpms' is enabled for this system.

Now let's install some of the pre-requisite packages we will need. Notice we are not installing Golang here and that is because we need a more recent version then what is shipping with Red Hat Enterprise Linux 9.

$ sudo dnf install -y git cockpit make selinux-policy-devel rpm-build bash-completion jq gcc

Now let's fetch Golang with wget and then extract it into /usr/local. We can also make a soft link from /usr/bin/go to the actual binary for convenience.

$ cd ~/ $ wget https://go.dev/dl/go1.19.3.linux-arm64.tar.gz $ sudo rm -rf /usr/local/go && sudo tar -C /usr/local -xzf go1.19.3.linux-arm64.tar.gz $ sudo ln -s /usr/local/go/bin/go /usr/bin/go

Next we can open up some firewall rules that are required for access when running Microshift.

$ sudo firewall-cmd --permanent --zone=trusted --add-source=10.42.0.0/16 $ sudo firewall-cmd --permanent --zone=trusted --add-source=169.254.169.1 $ sudo firewall-cmd --permanent --zone=public --add-port=80/tcp $ sudo firewall-cmd --permanent --zone=public --add-port=443/tcp $ sudo firewall-cmd --permanent --zone=public --add-port=5353/udp $ sudo firewall-cmd --permanent --zone=public --add-port=30000-32767/tcp $ sudo firewall-cmd --permanent --zone=public --add-port=30000-32767/udp $ sudo firewall-cmd --permanent --zone=public --add-port=6443/tcp $ sudo firewall-cmd --reload

We also need to manually grab the following packages from https://access.redhat.com as we could not find the Red Hat Enterprise Linux 9 equivalents in our repositories.

$ ls -1 *.el8.aarch64* cri-o-1.24.3-5.rhaos4.11.gitc4567c0.el8.aarch64.rpm cri-tools-1.24.2-6.el8.aarch64.rpm openshift-clients-4.11.0-202209201358.p0.g262ac9c.assembly.stream.el8.aarch64.rpm

Once the packages finish downloading we can install them.

$ sudo dnf localinstall cri-tools-1.24.2-6.el8.aarch64.rpm cri-o-1.24.3-5.rhaos4.11.gitc4567c0.el8.aarch64.rpm openshift-clients-4.11.0-202209201358.p0.g262ac9c.assembly.stream.el8.aarch64.rpm

Next we can go ahead and clone the GitHub repository for Microshift.

$ git clone https://github.com/openshift/microshift.git ~/microshift

Update the following release_arm64.go file to the following based off of this Github issue. Note these image location are not publicly accessible until the Arm version of Microshift images because readily available.

$ cp ~/microshift/pkg/release/release_arm64.go ~/microshift/pkg/release/release_arm64.go.bak $ cat << EOF > ~/microshift/pkg/release/release_arm64.go /* Copyright © 2021 MicroShift Contributors Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with the License. You may obtain a copy of the License at http://www.apache.org/licenses/LICENSE-2.0 Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and limitations under the License. */ package release // For the amd64 architecture we use the existing and tested and // published OCP or other component upstream images func init() { Image = map[string]string{ "cli": "quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:fe65a036a65af078f6f61017ae96e141dbb203f3602ecaca7f63ec8f58a1f6c6", "coredns": "quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:b5b3d024b2586bd0bf7b1315b2866f36a9b8b0acd23f0a9c6459371234dc8429", "haproxy_router": "quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:349e73813f432203920ae9ed04fc33a4026507e26ecc23ff2ab609d5b95b4206", "kube_rbac_proxy": "quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:c19226019fe605b5ab10496fb0b7cb4712cb694a7ee1e26642d63d515ca6b7cc", "openssl": "registry.access.redhat.com/ubi8/openssl@sha256:3f781a07e59d164eba065dba7d8e7661ab2494b21199c379b65b0ff514a1b8d0", "ovn_kubernetes_microshift": "quay.io/microshift/ovn-kubernetes-singlenode@sha256:012e743363b5f15f442c238099d35a0c70343fd1d4dc15b0a57a7340a338ffdb", "pause": "k8s.gcr.io/pause:3.6", "service_ca_operator": "quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:2fe468f25881e7b5ae8118c7d54b41a7fbb132a186f0156bbe46df0fd6a2f1f8", "odf_topolvm": "quay.io/rhceph-dev/odf4-odf-topolvm-rhel8@sha256:2855918d1849c99a835eb03c53ce07170c238111fd15d2fe50cd45611fcd1ceb", "ose_csi_ext_provisioner": "quay.io/rhceph-dev/openshift-ose-csi-external-provisioner@sha256:c3b2417f8fcb8883275f0e613037f83133ccc3f91311a30688e4be520544ea4a", "ose_csi_ext_resizer": "quay.io/rhceph-dev/openshift-ose-csi-external-resizer@sha256:213f43d61b3a214a4a433c7132537be082a108d55005f2ba0777c2ea97489799", "topolvm-csi-snapshotter": "quay.io/rhceph-dev/openshift-ose-csi-external-snapshotter@sha256:734c095670d21b77f18c84670d6c9a7742be1d9151dca0da20f41858ede65ed8", "ose_csi_livenessprobe": "quay.io/rhceph-dev/openshift-ose-csi-livenessprobe@sha256:b05559aa038708ab448cfdfed2ca880726aed6cc30371fea4d6a42c972c0c728", "ose_csi_node_registrar": "quay.io/rhceph-dev/openshift-ose-csi-node-driver-registrar@sha256:fb0f5e531847db94dcadc61446b9a892f6f92ddf282e192abf2fdef6c6af78f2", } } EOF

Also since we are not using a packaged installed Golang we will comment out the Golang build requirements in the specification file.

$ sed -e '/golang/ s/^#*/#/' -i ~/microshift/packaging//rpm/microshift.spec

With the release file updated we can proceed to make the packages of Microshift.

$ cd ~/microshift $ make rpm

After the rpm packages have completed being created proceed to install them.

$ sudo dnf localinstall -y ~/microshift/_output/rpmbuild/RPMS/*/*.rpm

Note the above installation will pull in the following dependencies:

NetworkManager-ovs aarch64 1:1.36.0-5.el9_0 rhel-9-for-aarch64-appstream-rpms conntrack-tools aarch64 1.4.5-10.el9_0.1 rhel-9-for-aarch64-appstream-rpms libnetfilter_cthelper aarch64 1.0.0-22.el9 rhel-9-for-aarch64-appstream-rpms libnetfilter_cttimeout aarch64 1.0.0-19.el9 rhel-9-for-aarch64-appstream-rpms libnetfilter_queue aarch64 1.0.5-1.el9 rhel-9-for-aarch64-appstream-rpms openvswitch-selinux-extra-policy noarch 1.0-31.el9fdp fast-datapath-for-rhel-9-aarch64-rpms openvswitch2.17 aarch64 2.17.0-49.el9fdp fast-datapath-for-rhel-9-aarch64-rpms unbound-libs aarch64 1.13.1-13.el9_0 rhel-9-for-aarch64-appstream-rpms

Set the pull-secret for the crio environment.

$ sudo vi /etc/crio/openshift-pull-secret

Now let's enable crio environment.

$ sudo systemctl enable crio --now

Manually pull the Arm topolvm images we defined in the release_arm.go file above. Again note these images are available publicly and require access to the repository.

$ sudo crictl pull --auth "<YOUR AUTH TOKEN>" quay.io/rhceph-dev/odf4-odf-topolvm-rhel8@sha256:2855918d1849c99a835eb03c53ce07170c238111fd15d2fe50cd45611fcd1ceb $ sudo crictl pull --auth "<YOUR AUTH TOKEN>" quay.io/rhceph-dev/openshift-ose-csi-external-provisioner@sha256:c3b2417f8fcb8883275f0e613037f83133ccc3f91311a30688e4be520544ea4a $ sudo crictl pull --auth "<YOUR AUTH TOKEN>" quay.io/rhceph-dev/openshift-ose-csi-external-resizer@sha256:213f43d61b3a214a4a433c7132537be082a108d55005f2ba0777c2ea97489799 $ sudo crictl pull --auth "<YOUR AUTH TOKEN>" quay.io/rhceph-dev/openshift-ose-csi-external-snapshotter@sha256:734c095670d21b77f18c84670d6c9a7742be1d9151dca0da20f41858ede65ed8 $ sudo crictl pull --auth "<YOUR AUTH TOKEN>" quay.io/rhceph-dev/openshift-ose-csi-livenessprobe@sha256:b05559aa038708ab448cfdfed2ca880726aed6cc30371fea4d6a42c972c0c728 $ sudo crictl pull --auth "<YOUR AUTH TOKEN>" quay.io/rhceph-dev/openshift-ose-csi-node-driver-registrar@sha256:fb0f5e531847db94dcadc61446b9a892f6f92ddf282e192abf2fdef6c6af78f2

At this point we are ready to start Microshift up.

$ sudo systemctl enable microshift --now Created symlink /etc/systemd/system/multi-user.target.wants/microshift.service → /usr/lib/systemd/system/microshift.service.

Once the services have been started let's go ahead and create a hidden directory called .kube and copy the kubeconfig in there.

$ mkdir ~/.kube $ sudo cat /var/lib/microshift/resources/kubeadmin/kubeconfig > ~/.kube/config

In a few minutes we can then issue a oc get pods -A and hopefully we see the following pods running.

$ oc get pods -A NAMESPACE NAME READY STATUS RESTARTS AGE openshift-dns dns-default-ph555 1/2 Running 0 6m57s openshift-dns node-resolver-brnj6 1/1 Running 0 6m57s openshift-ingress router-default-54bc9ff944-clr4r 1/1 Running 0 6m57s openshift-ovn-kubernetes ovnkube-master-t9q4w 4/4 Running 0 6m57s openshift-ovn-kubernetes ovnkube-node-f6z66 1/1 Running 0 6m57s openshift-service-ca service-ca-5bb4c5d7f7-zs2gg 1/1 Running 0 6m57s openshift-storage topolvm-controller-5d4f58ff8c-kl7v4 4/4 Running 0 6m57s openshift-storage topolvm-node-7wsh5 4/4 Running 0 6m57s

Hopefully this provides a glimpse as to what one can do on a Red Hat Enterprise Linux 9 virtual machine running on a Apple M1 processor once the Arm packages and images for Microshift become readily available. It could be a great way to do test development work before actually transferring it to the real edge device hardware that utilizes Arm.