Posts

Guide: Installing microK8s on M600 tiny-PCs

Archi

Archi

— 2 min read

This guide covers installing microK8s on a cluster of 4x Lenovo M600 tiny-PCs, using Rancher Longhorn for Distributed storage and Portainer for cluster management. The hardware used is mentioned here.

On each node

# Install Ubuntu 20.04 Server, Minimal install
# Configure Static IP and Enable SSH

# Ensure you have working resolvable DNS for all nodes!

# Update the system
sudo apt update && sudo apt upgrade -y

# Install microK8s, latest stable version
sudo snap install microk8s --classic --channel=latest/stable

# Give your user rights to microk8s
sudo usermod -a -G microk8s $USER
sudo chown -f -R $USER ~/.kube

# Activate your new user rights
newgrp microk8s

# Check and Wait till all microk8s services are ready
microk8s status --wait-ready

# Check microk8s configuration
microk8s config

# Create an alias - if you don't have kubectl already installed
alias kubectl='microk8s kubectl'

On the manager node

# Generate command string and copy/paste/run for each additional node
microk8s add-node

# Check the nodes are being added successfully, it may take a few minutes
microk8s kubectl get nodes

# Enable the community repository/plugin
microk8s enable community

# Enable Nginx for Ingress capabilities
microk8s enable ingress

# Enable metalLB - Load Balancer
microk8s enable metallb:<ip range on your network that metalLB will use to assign to applications>

# Configure metalLB
# Create file 'metallb-ingress-service.yaml'
nano metallb-ingress-service.yaml

# Add these contents, configure the loadBalancerIP. 
# If not specified, it will pull from the IP range configured above.

apiVersion: v1
kind: Service
metadata:
  name: ingress-lb
  namespace: ingress
spec:
  selector:
    name: nginx-ingress-microk8s
  type: LoadBalancer
  loadBalancerIP: 192.168.69.26
  externalTrafficPolicy: Local
  ports:
    - name: http
      protocol: TCP
      port: 80
      targetPort: 80
    - name: https
      protocol: TCP
      port: 443
      targetPort: 443
      
# Apply the metalLB configuration
microk8s kubectl apply -f metallb-ingress-service.yaml

# Add Longhorn distributed storage (love this!!!)
# https://longhorn.io/docs/1.5.1/deploy/install/install-with-kubectl/
microk8s kubectl apply -f https://raw.githubusercontent.com/longhorn/longhorn/v1.5.1/deploy/longhorn.yaml

# If errors (likely), run this command # solution from: https://artsysops.com/2021/10/31/how-to-make-rancher-longhorn-work-with-microk8s/
microk8s kubectl edit deployment/longhorn-driver-deployer -n longhorn-system

# Add following above   name: CSI_ATTACHER_IMAGE 
- name: KUBELET_ROOT_DIR
  value: /var/snap/microk8s/common/var/lib/kubelet

# To use longhorn storage for deployments
storageClass: "longhorn"
accessMode: ReadWriteOnce

# Enable Portainer to manage everything
microk8s enable portainer

# Navigate to Nodeport service for Portainer on port 30777
https://<host-IP>:30779/

If you need to reset and start over

# On member nodes that you want to remove from the cluster
microk8s leave

# After a member node has left, remove it from the master list
# Find the name of the node that you want to remove from the cluster:
microk8s.kubectl get nodes

# Then remove it
microk8s remove-node <name-of-the-node>

# Once all member nodes are removed, reset microK8s on the manager node
# Source https://microk8s.io/docs/command-reference#heading--microk8s-reset
sudo microk8s reset

# Remove Microk8s from manager node
sudo snap remove microk8s

Once you're happy with the whole setup, add a backup solution for Longhorn.

Source: https://longhorn.io/docs/1.5.1/snapshots-and-backups/backup-and-restore/set-backup-target/

# CIFS was used for this example
# Create the user and share on the file server

# -----------------------------------------
# run this bash script on the Manager Node
USERNAME=CIFS_Username
PASSWORD=CIFS_Password

CIFS_USERNAME=`echo -n ${USERNAME} | base64`
CIFS_PASSWORD=`echo -n ${PASSWORD} | base64`

cat <<EOF >>cifs_secret.yml
apiVersion: v1
kind: Secret
metadata:
  name: cifs-secret
  namespace: longhorn-system
type: Opaque
data:
  CIFS_USERNAME: ${CIFS_USERNAME}
  CIFS_PASSWORD: ${CIFS_PASSWORD}
EOF

microk8s kubectl apply -f cifs_secret.yml
# -----------------------------------------

# Then open the Longhorn GUI and go to the Backup section
# Input the CIFS path
cifs://<server>/<share>

# Input the name of the secret created above
cifs-secret

# Go to Setting > Backup and create a test backup to ensure everything fuctions
# If successful, go to Recurring Job and create a backup schedule
# Note that the Job Name must be all lowercase and use dashes -, not underscore _

Read more