OpenShift has a lot of certificates associated to the services it runs. With that in mind it might make sense to check on those certificates every once and a while with some kind of simple report. I have had customers make this request on occasion and it got me thinking about a quick and dirty way to visualize this. The following blog show the fruits of this simple task in a simple bash script.
First lets go ahead and create the certs-expired.sh script:
$ cat << EOF > ~/certs-expired.sh
#!/bin/bash
format="%-8s%-8s%-60s%-26s%-60s\n"
printf "$format" STATE DAYS NAME EXPIRY NAMESPACE
printf "$format" ----- ---- ---- ------ ---------
oc get secrets -A -o go-template='{{range .items}}{{if eq .type "kubernetes.io/tls"}}{{.metadata.namespace}}{{" "}}{{.metadata.name}}{{" "}}{{index .data "tls.crt"}}{{"\n"}}{{end}}{{end}}' | while read namespace name cert
do
certdate=`echo $cert | base64 -d | openssl x509 -noout -enddate|cut -d= -f2`
epochcertdate=$(date -d "$certdate" +"%s")
currentdate=$(date +%s)
if ((epochcertdate > currentdate)); then
datediff=$((epochcertdate-currentdate))
state="OK"
else
state="EXPIRED"
datediff=$((currentdate-epochcertdate))
fi
days=$((datediff/86400))
certdate=`echo $cert | base64 -d | openssl x509 -noout -enddate| cut -d= -f2`
printf "$format" "$state" "$days" "$name" "$certdate" "$namespace"
done
EOF
The script makes the assumptions that the oc binary is in the current path variable and that the kubeconfig has been set. This ensures that the oc command inside the script can pull the appropriate data. If those assumptions are met we can go ahead and run the script. I chose just to issue a bash against the script but we could have also set the file with the execute permissions. When we execute the script we can see the output below:
$ bash certs-expired.sh STATE DAYS NAME EXPIRY NAMESPACE ----- ---- ---- ------ --------- OK 715 openshift-apiserver-operator-serving-cert May 5 21:33:47 2024 GMT openshift-apiserver-operator OK 3635 etcd-client May 3 21:13:54 2032 GMT openshift-apiserver OK 715 serving-cert May 5 21:33:52 2024 GMT openshift-apiserver OK 715 serving-cert May 5 21:33:59 2024 GMT openshift-authentication-operator OK 715 v4-0-config-system-serving-cert May 5 21:33:49 2024 GMT openshift-authentication OK 715 cloud-credential-operator-serving-cert May 5 21:33:50 2024 GMT openshift-cloud-credential-operator OK 715 machine-approver-tls May 5 21:33:48 2024 GMT openshift-cluster-machine-approver OK 715 node-tuning-operator-tls May 5 21:33:47 2024 GMT openshift-cluster-node-tuning-operator OK 715 samples-operator-tls May 5 21:37:44 2024 GMT openshift-cluster-samples-operator OK 715 cluster-storage-operator-serving-cert May 5 21:33:55 2024 GMT openshift-cluster-storage-operator OK 715 csi-snapshot-webhook-secret May 5 21:33:47 2024 GMT openshift-cluster-storage-operator OK 715 serving-cert May 5 21:33:54 2024 GMT openshift-cluster-storage-operator OK 715 cluster-version-operator-serving-cert May 5 21:33:52 2024 GMT openshift-cluster-version OK 15 kube-controller-manager-client-cert-key Jun 5 21:33:41 2022 GMT openshift-config-managed OK 15 kube-scheduler-client-cert-key Jun 5 21:33:34 2022 GMT openshift-config-managed OK 715 config-operator-serving-cert May 5 21:33:47 2024 GMT openshift-config-operator OK 3635 etcd-client May 3 21:13:54 2032 GMT openshift-config OK 3635 etcd-metric-client May 3 21:13:54 2032 GMT openshift-config OK 3635 etcd-metric-signer May 3 21:13:54 2032 GMT openshift-config OK 3635 etcd-signer May 3 21:13:54 2032 GMT openshift-config OK 715 serving-cert May 5 21:41:37 2024 GMT openshift-console-operator OK 715 console-serving-cert May 5 21:42:15 2024 GMT openshift-console OK 715 openshift-controller-manager-operator-serving-cert May 5 21:33:47 2024 GMT openshift-controller-manager-operator OK 715 serving-cert May 5 21:33:56 2024 GMT openshift-controller-manager OK 715 metrics-tls May 5 21:33:58 2024 GMT openshift-dns-operator OK 715 dns-default-metrics-tls May 5 21:34:59 2024 GMT openshift-dns OK 3635 etcd-client May 3 21:13:54 2032 GMT openshift-etcd-operator OK 715 etcd-operator-serving-cert May 5 21:33:57 2024 GMT openshift-etcd-operator OK 3635 etcd-client May 3 21:13:54 2032 GMT openshift-etcd OK 1080 etcd-peer-asus-vm1.kni.schmaustech.com May 5 21:51:28 2025 GMT openshift-etcd OK 1080 etcd-peer-asus1-vm2.kni.schmaustech.com May 5 21:33:23 2025 GMT openshift-etcd OK 1080 etcd-peer-asus1-vm3.kni.schmaustech.com May 5 21:33:24 2025 GMT openshift-etcd OK 1080 etcd-serving-asus-vm1.kni.schmaustech.com May 5 21:51:28 2025 GMT openshift-etcd OK 1080 etcd-serving-asus1-vm2.kni.schmaustech.com May 5 21:33:23 2025 GMT openshift-etcd OK 1080 etcd-serving-asus1-vm3.kni.schmaustech.com May 5 21:33:24 2025 GMT openshift-etcd OK 1080 etcd-serving-metrics-asus-vm1.kni.schmaustech.com May 5 21:51:27 2025 GMT openshift-etcd OK 1080 etcd-serving-metrics-asus1-vm2.kni.schmaustech.com May 5 21:33:23 2025 GMT openshift-etcd OK 1080 etcd-serving-metrics-asus1-vm3.kni.schmaustech.com May 5 21:33:24 2025 GMT openshift-etcd OK 715 serving-cert May 5 21:33:59 2024 GMT openshift-etcd OK 715 image-registry-operator-tls May 5 21:33:58 2024 GMT openshift-image-registry OK 715 metrics-tls May 5 21:33:55 2024 GMT openshift-ingress-operator OK 715 router-ca May 5 21:35:59 2024 GMT openshift-ingress-operator OK 715 router-certs-default May 5 21:36:01 2024 GMT openshift-ingress OK 715 router-metrics-certs-default May 5 21:36:00 2024 GMT openshift-ingress OK 715 openshift-insights-serving-cert May 5 21:33:51 2024 GMT openshift-insights OK 15 aggregator-client-signer Jun 6 16:21:59 2022 GMT openshift-kube-apiserver-operator OK 715 kube-apiserver-operator-serving-cert May 5 21:33:54 2024 GMT openshift-kube-apiserver-operator OK 350 kube-apiserver-to-kubelet-signer May 6 21:09:57 2023 GMT openshift-kube-apiserver-operator OK 350 kube-control-plane-signer May 6 21:09:57 2023 GMT openshift-kube-apiserver-operator OK 3635 loadbalancer-serving-signer May 3 21:09:52 2032 GMT openshift-kube-apiserver-operator OK 3635 localhost-recovery-serving-signer May 3 21:33:29 2032 GMT openshift-kube-apiserver-operator OK 3635 localhost-serving-signer May 3 21:09:50 2032 GMT openshift-kube-apiserver-operator OK 105 node-system-admin-client Sep 3 21:33:40 2022 GMT openshift-kube-apiserver-operator OK 350 node-system-admin-signer May 6 21:33:29 2023 GMT openshift-kube-apiserver-operator OK 3635 service-network-serving-signer May 3 21:09:51 2032 GMT openshift-kube-apiserver-operator OK 15 aggregator-client Jun 6 16:21:59 2022 GMT openshift-kube-apiserver OK 15 check-endpoints-client-cert-key Jun 5 21:33:46 2022 GMT openshift-kube-apiserver OK 15 control-plane-node-admin-client-cert-key Jun 5 21:33:53 2022 GMT openshift-kube-apiserver OK 3635 etcd-client May 3 21:13:54 2032 GMT openshift-kube-apiserver OK 3635 etcd-client-10 May 3 21:13:54 2032 GMT openshift-kube-apiserver OK 3635 etcd-client-11 May 3 21:13:54 2032 GMT openshift-kube-apiserver OK 3635 etcd-client-12 May 3 21:13:54 2032 GMT openshift-kube-apiserver OK 3635 etcd-client-8 May 3 21:13:54 2032 GMT openshift-kube-apiserver OK 3635 etcd-client-9 May 3 21:13:54 2032 GMT openshift-kube-apiserver OK 15 external-loadbalancer-serving-certkey Jun 5 21:33:52 2022 GMT openshift-kube-apiserver OK 15 internal-loadbalancer-serving-certkey Jun 5 21:33:34 2022 GMT openshift-kube-apiserver OK 15 kubelet-client Jun 5 21:33:34 2022 GMT openshift-kube-apiserver OK 3635 localhost-recovery-serving-certkey May 3 21:33:29 2032 GMT openshift-kube-apiserver OK 3635 localhost-recovery-serving-certkey-10 May 3 21:33:29 2032 GMT openshift-kube-apiserver OK 3635 localhost-recovery-serving-certkey-11 May 3 21:33:29 2032 GMT openshift-kube-apiserver OK 3635 localhost-recovery-serving-certkey-12 May 3 21:33:29 2032 GMT openshift-kube-apiserver OK 3635 localhost-recovery-serving-certkey-8 May 3 21:33:29 2032 GMT openshift-kube-apiserver OK 3635 localhost-recovery-serving-certkey-9 May 3 21:33:29 2032 GMT openshift-kube-apiserver OK 15 localhost-serving-cert-certkey Jun 5 21:33:34 2022 GMT openshift-kube-apiserver OK 15 service-network-serving-certkey Jun 5 21:33:33 2022 GMT openshift-kube-apiserver OK 15 csr-signer Jun 6 16:26:40 2022 GMT openshift-kube-controller-manager-operator OK 45 csr-signer-signer Jul 6 16:22:14 2022 GMT openshift-kube-controller-manager-operator OK 715 kube-controller-manager-operator-serving-cert May 5 21:33:57 2024 GMT openshift-kube-controller-manager-operator OK 15 csr-signer Jun 6 16:26:40 2022 GMT openshift-kube-controller-manager OK 15 kube-controller-manager-client-cert-key Jun 5 21:33:41 2022 GMT openshift-kube-controller-manager OK 715 serving-cert May 5 21:33:51 2024 GMT openshift-kube-controller-manager OK 715 serving-cert-2 May 5 21:33:51 2024 GMT openshift-kube-controller-manager OK 715 serving-cert-3 May 5 21:33:51 2024 GMT openshift-kube-controller-manager OK 715 serving-cert-4 May 5 21:33:51 2024 GMT openshift-kube-controller-manager OK 715 serving-cert-5 May 5 21:33:51 2024 GMT openshift-kube-controller-manager OK 715 serving-cert-6 May 5 21:33:51 2024 GMT openshift-kube-controller-manager OK 715 serving-cert-7 May 5 21:33:51 2024 GMT openshift-kube-controller-manager OK 715 kube-scheduler-operator-serving-cert May 5 21:33:50 2024 GMT openshift-kube-scheduler-operator OK 15 kube-scheduler-client-cert-key Jun 5 21:33:34 2022 GMT openshift-kube-scheduler OK 715 serving-cert May 5 21:33:59 2024 GMT openshift-kube-scheduler OK 715 serving-cert-3 May 5 21:33:59 2024 GMT openshift-kube-scheduler OK 715 serving-cert-4 May 5 21:33:59 2024 GMT openshift-kube-scheduler OK 715 serving-cert-5 May 5 21:33:59 2024 GMT openshift-kube-scheduler OK 715 serving-cert-6 May 5 21:33:59 2024 GMT openshift-kube-scheduler OK 715 serving-cert-7 May 5 21:33:59 2024 GMT openshift-kube-scheduler OK 715 serving-cert May 5 21:34:00 2024 GMT openshift-kube-storage-version-migrator-operator OK 725 diskmaker-metric-serving-cert May 15 23:33:46 2024 GMT openshift-local-storage OK 715 baremetal-operator-webhook-server-cert May 5 21:36:34 2024 GMT openshift-machine-api OK 715 cluster-autoscaler-operator-cert May 5 21:34:01 2024 GMT openshift-machine-api OK 715 cluster-baremetal-operator-tls May 5 21:33:58 2024 GMT openshift-machine-api OK 715 cluster-baremetal-webhook-server-cert May 5 21:33:48 2024 GMT openshift-machine-api OK 715 machine-api-controllers-tls May 5 21:33:47 2024 GMT openshift-machine-api OK 715 machine-api-operator-tls May 5 21:33:56 2024 GMT openshift-machine-api OK 715 machine-api-operator-webhook-cert May 5 21:33:53 2024 GMT openshift-machine-api OK 715 proxy-tls May 5 21:34:00 2024 GMT openshift-machine-config-operator OK 715 marketplace-operator-metrics May 5 21:33:50 2024 GMT openshift-marketplace OK 715 alertmanager-main-tls May 5 21:45:20 2024 GMT openshift-monitoring OK 715 cluster-monitoring-operator-tls May 5 21:33:52 2024 GMT openshift-monitoring OK 715 grafana-tls May 5 21:45:20 2024 GMT openshift-monitoring OK 715 kube-state-metrics-tls May 5 21:35:59 2024 GMT openshift-monitoring OK 715 node-exporter-tls May 5 21:35:59 2024 GMT openshift-monitoring OK 715 openshift-state-metrics-tls May 5 21:35:58 2024 GMT openshift-monitoring OK 715 prometheus-adapter-tls May 5 21:35:59 2024 GMT openshift-monitoring OK 715 prometheus-k8s-thanos-sidecar-tls May 5 21:45:22 2024 GMT openshift-monitoring OK 715 prometheus-k8s-tls May 5 21:45:21 2024 GMT openshift-monitoring OK 715 prometheus-operator-tls May 5 21:35:43 2024 GMT openshift-monitoring OK 715 telemeter-client-tls May 5 21:37:44 2024 GMT openshift-monitoring OK 715 thanos-querier-tls May 5 21:35:58 2024 GMT openshift-monitoring OK 715 metrics-daemon-secret May 5 21:33:56 2024 GMT openshift-multus OK 715 multus-admission-controller-secret May 5 21:33:48 2024 GMT openshift-multus OK 3635 etcd-client May 3 21:13:54 2032 GMT openshift-oauth-apiserver OK 715 serving-cert May 5 21:34:01 2024 GMT openshift-oauth-apiserver OK 715 catalog-operator-serving-cert May 5 21:33:47 2024 GMT openshift-operator-lifecycle-manager OK 715 olm-operator-serving-cert May 5 21:33:48 2024 GMT openshift-operator-lifecycle-manager OK 714 packageserver-service-cert May 4 21:34:44 2024 GMT openshift-operator-lifecycle-manager OK 0 pprof-cert May 21 18:30:03 2022 GMT openshift-operator-lifecycle-manager OK 3635 ovn-ca May 3 21:27:45 2032 GMT openshift-ovn-kubernetes OK 167 ovn-cert Nov 5 09:27:45 2022 GMT openshift-ovn-kubernetes OK 715 ovn-master-metrics-cert May 5 21:33:53 2024 GMT openshift-ovn-kubernetes OK 715 ovn-node-metrics-cert May 5 21:33:49 2024 GMT openshift-ovn-kubernetes OK 3635 signer-ca May 3 21:27:46 2032 GMT openshift-ovn-kubernetes OK 167 signer-cert Nov 5 09:27:46 2022 GMT openshift-ovn-kubernetes OK 715 serving-cert May 5 21:33:54 2024 GMT openshift-service-ca-operator OK 775 signing-key Jul 4 21:33:37 2024 GMT openshift-service-ca OK 725 noobaa-db-serving-cert May 15 23:42:26 2024 GMT openshift-storage OK 725 noobaa-mgmt-serving-cert May 15 23:42:26 2024 GMT openshift-storage OK 725 noobaa-operator-service-cert May 16 06:23:29 2024 GMT openshift-storage OK 725 noobaa-s3-serving-cert May 15 23:42:26 2024 GMT openshift-storage OK 725 ocs-storagecluster-cos-ceph-rgw-tls-cert May 15 23:41:32 2024 GMT openshift-storage OK 725 odf-console-serving-cert May 15 23:27:38 2024 GMT openshift-storage
The output of the script is simple. The first column contains the state of the certificate. If its okay then it just says OK and if its expired the field will say EXPIRED. The next column tells us how many days until the certificate expires and if the number is negative then the certificate is expired and has been for that many days. The third column tells us the certificates name while the fourth gives us the actual expiry date. Finally the last column provides the namespace the certificate is in.
Again just a simple script but provides an example of how we can see this type of information. However if one has a fleet of clusters then configuring a Red Hat Advanced Cluster Management Certificate Policy Controller might be a more effective method at expired certificate management.
