Introduction:
RKE2 is the enterprise ready, stable and secure Kubernetes distribution which is easy to install configure and manage. Most of the enterprise configurations comes out of the box from the installation like:
Prerequisites:
- Fresh 3 ubuntu 22.04 server VM (On any Cloud Provider/Local Machine/Vagrant)
Server/ Master node Setup
The server or master node vm.
Install RKE2 binaries:
curl -sfL https://get.rke2.io | sudo sh -
sudo systemctl enable rke2-server.service
If you have extra ips or domains as API endpoints add them in config file /etc/rancher/rke2/config.yaml like this-
sudo vim /etc/rancher/rke2/config.yaml
SAMPLE CONFIG -->
write-kubeconfig-mode: "0644"
tls-san:
- "foo.local"
Now start RKE2 server
sudo systemctl start rke2-server.service
if you want to see the logs of process run this in separate shell of server
sudo journalctl -u rke2-server -f
Now enable your kubernetes credentials for the user
sudo mkdir ~/.kube
sudo cp /etc/rancher/rke2/rke2.yaml ~/.kube/config
sudo chown $(whoami):$(whoami) ~/.kube/config
The last step before using kubectl and relevant cli tools, is to add the $PATH Environment Variable.
export PATH=$PATH:/var/lib/rancher/rke2/bin/
Now check if kubectl is working or not.
kubectl get nodes
NAME STATUS ROLES AGE VERSION
k8s Ready control-plane,etcd,master 28m v1.28.9+rke2r1
A token that can be used to register other server or agent nodes will be created at /var/lib/rancher/rke2/server/node-token copy the token when registering your client/worker nodes.
sudo cat /var/lib/rancher/rke2/server/node-token
Agent/worker node setup
Export the following Environment Variable before getting started with the Worker Node Configuration.
export INSTALL_RKE2_TYPE="agent"
Install and enable the worker node or agent binary.
sudo bash
curl -sfL https://get.rke2.io | sudo sh -
sudo systemctl enable rke2-agent.service
Now before we start the node we need to configure the token and master node api address. Run bellow commands to create config folder and configure the master details.
sudo mkdir -p /etc/rancher/rke2/
sudo vim /etc/rancher/rke2/config.yaml
Content for config.yaml:
server: https://<server>:9345
token: <token from server node>
Replace the server from the real master server ip or hostname and replace the correct token.
Now join the agent
systemctl start rke2-agent.service
Additional Services (ft. Control Plane)
Local storage provisioner installation:
Dynamic storage provisioning
For the dynamic provisioning we need a storage class and rancher have the answer for this lab.
kubectl apply -f https://raw.githubusercontent.com/rancher/local-path-provisioner/v0.0.23/deploy/local-path-storage.yaml
``````
```bash
kubectl apply -f https://github.com/cert-manager/cert-manager/releases/download/v1.13.0/cert-manager.yaml
You can patch this storageClass to act as default
kubectl patch storageclass local-path -p '{"metadata": {"annotations":{"storageclass.kubernetes.io/is-default-class":"true"}}}'
Create a PVC and pod
kubectl create -f https://raw.githubusercontent.com/rancher/local-path-provisioner/master/examples/pvc/pvc.yaml
kubectl create -f https://raw.githubusercontent.com/rancher/local-path-provisioner/master/examples/pod/pod.yaml
Cert manager installation:
kubectl apply -f https://github.com/cert-manager/cert-manager/releases/download/v1.13.0/cert-manager.yaml
ETCD backup restore
Snapshots are enabled by default.
The snapshot directory defaults to /var/lib/rancher/rke2/server/db/snapshots.
In RKE2, snapshots are stored on each etcd node. If you have multiple etcd or etcd + control-plane nodes, you will have multiple copies of local etcd snapshots.
You can take a snapshot manually while RKE2 is running with the etcd-snapshot subcommand. For example: rke2 etcd-snapshot save --name pre-upgrade-snapshot.
Cluster Reset
RKE2 enables a feature to reset the cluster to one member cluster by passing --cluster-reset flag, when passing this flag to rke2 server it will reset the cluster with the same data dir in place, the data directory for etcd exists in /var/lib/rancher/rke2/server/db/etcd, this flag can be passed in the events of quorum loss in the cluster.
To pass the reset flag, first you need to stop RKE2 service if its enabled via systemd:
sudo systemctl stop rke2-server
rke2 server --cluster-reset
Result: A message in the logs say that RKE2 can be restarted without the flags. Start rke2 again and it should start rke2 as a 1 member cluster.
Restoring a Snapshot to Existing Nodes
When RKE2 is restored from backup, the old data directory will be moved to /var/lib/rancher/rke2/server/db/etcd-old-%date%/. RKE2 will then attempt to restore the snapshot by creating a new data directory and start etcd with a new RKE2 cluster with one etcd member.
You must stop RKE2 service on all server nodes if it is enabled via systemd. Use the following command to do so:
sudo systemctl stop rke2-server
Next, you will initiate the restore from snapshot on the first server node with the following commands:
rke2 server \
--cluster-reset \
--cluster-reset-restore-path=<PATH-TO-SNAPSHOT>
Once the restore process is complete, start the rke2-server service on the first server node as follows:
sudo systemctl start rke2-server
Remove the rke2 db directory on the other server nodes as follows:
sudo rm -rf /var/lib/rancher/rke2/server/db
Start the rke2-server service on other server nodes with the following command:
sudo systemctl start rke2-server
Result: After a successful restore, a message in the logs says that etcd is running, and RKE2 can be restarted without the flags. Start RKE2 again, and it should run successfully and be restored from the specified snapshot.
When rke2 resets the cluster, it creates an empty file at /var/lib/rancher/rke2/server/db/reset-flag. This file is harmless to leave in place, but must be removed in order to perform subsequent resets or restores. This file is deleted when rke2 starts normally.
Auto-Deploying Manifests
One of the amazing feature in RKE2 is, any file found in /var/lib/rancher/rke2/server/manifests will automatically be deployed to Kubernetes in a manner similar to kubectl apply.
Thanks for reading till the very end.
Follow me on Twitter, LinkedIn and GitHub for more amazing blogs about Tech and More!
