I like using RKE to build, destroy, update, upgrade k8s clusters. I have come to understand that there are a lot of small but very impactful things we need to take care of it. If I forget this is what I will come back to check
- Create a separate user for RKE (let us say the user is rke) and use the ssh keyfile for authentication
- Install docker using the official docker package from docker.io as per their documentation
https://docs.docker.com/engine/install/ubuntu/ - Disable docker apt repository so that when you install updates to the system, the node won't update to the latest docker version and thus breaking your cluster. On ubuntu 18 I first checked the apt repositories list
cat /etc/apt/sources.list | grep docker
# deb-src [arch=amd64] https://download.docker.com/linux/ubuntu bionic stablerke@build-hq-45:~$ sudo apt list upradeapt list --upgradable^C
rke@build-hq-45:~$ sudo apt list upradeapt list --upgradable^Crke@build-hq-45:~$ apt list --upgradable
Listing... Done
docker-ce/bionic 5:20.10.2~3-0~ubuntu-bionic rke64 [upgradable from: 5:19.03.14~3-0~ubuntu-bionic]
docker-ce-cli/bionic 5:20.10.2~3-0~ubuntu-bionic rke64 [upgradable from: 5:19.03.14~3-0~ubuntu-bionic]
linux-generic/bionic-updates,bionic-security 4.15.0.132.119 rke64 [upgradable from: 4.15.0.130.117]
linux-headers-generic/bionic-updates,bionic-security 4.15.0.132.119 rke64 [upgradable from: 4.15.0.130.117]
linux-image-generic/bionic-updates,bionic-security 4.15.0.132.119 rke64 [upgradable from: 4.15.0.130.117]
tar/bionic-updates,bionic-security 1.29b-2ubuntu0.2 rke64 [upgradable from: 1.29b-2ubuntu0.1]
rke@build-hq-45:~$ cat /etc/apt/sources.list | grep dock
# deb-src [arch=rke64] https://download.docker.com/linux/ubuntu bionic stable
rke@build-hq-45:~$ sudo add-apt-repository --remove deb-src [arch=rke64] https://download.docker.com/linux/ubuntu bionic stable
Error: need a single repository as argument
rke@build-hq-45:~$ sudo add-apt-repository --remove 'deb-src [arch=rke64] https://download.docker.com/linux/ubuntu bionic stable'
rke@build-hq-45:~$ cat /etc/apt/sources.list | grep dock
4. sudo chmod 777 /var/run/docker.sock && sudo chown rke /var/run/docker.sock
5. sudo usermod -aG docker rke && sudo usermod -aG rke
1. Uninstall existing docker sudo apt-get remove
docker docker-engine docker.io containerd runc
2. sudo apt-get install -y \
apt-transport-https \
ca-certificates \
curl \
gnupg-agent \
software-properties-common
3. curl -fsSL https://get.docker.com -o get-docker.sh
4. chmod +x *.sh
5. sudo sh get-docker.sh
6. Uninstall again sudo apt-get remove docker docker-ce docker-engine docker.io containerd runc
The script has updated the docker apt repository and
installed the latest docker version too. We want to retain the repository
source but install a specific version of docker.
7.
sudo apt-get purge docker-ce docker-ce-cli containerd.io
8. sudo apt-get update
9. apt-cache madison docker-ce (list all versions)
10. DOCKER_VERSION="5:19.03.14~3-0~ubuntu-bionic"
[we just want docker version 19 since that is what is compatible with out rke
and k8s 1.18…]
Cleaning up of nodes
When you remove k8s or reinstall, then you might want to do it.
- `docker rm -f $(docker ps -qa)`
- `docker rmi -f $(docker images -q)`
- `docker volume rm $(docker volume ls -q)`
- `for mount in $(mount | grep tmpfs | grep '/var/lib/kubelet' | awk '{ print $3 }') /var/lib/kubelet /var/lib/rancher; do umount $mount; done`
- `sudo rm -rf /etc/ceph \
/etc/cni \
/etc/kubernetes \
/opt/cni \
/opt/rke \
/run/secrets/kubernetes.io \
/run/calico \
/run/flannel \
/var/lib/calico \
/var/lib/etcd \
/var/lib/cni \
/var/lib/kubelet \
/var/lib/rancher/rke/log \
/var/log/containers \
/var/log/pods \
/var/run/calico`
