profile for Gajendra D Ambi on Stack Exchange, a network of free, community-driven Q&A sites

Friday, January 8, 2021

A production kubernetes cluster


  1. An HA nginx cluster of a minimum of  3 VMs with tls
  2. An HA etcd cluster of a minimum of 3 VMs with tls
  3. Install k8s cluster with at least 3 master nodes using the external etcd cluster 
  4. nginx will be used to load balance amongst all 3 master nodes, All connections to k8s including the api calls will go through nginx 
  5. Use the external ceph cluster for main workloads
  6. Use the local rook-ceph cluster which uses the drives in the physical worker nodes.


  1. Restricted network
  2. Public network


nodes: 3
Install a 3 node ceph cluster.


nodes: 3
Node 1 will contain the following vms
  1. master1
  2. nginx1
  3. etcd1
  4. worker1
Node 2 will contain the following vms
  1. master2
  2. nginx2
  3. etcd2
  4. worker2
Node 3 will contain the following vms
  1. master3
  2. nginx3
  3. etcd3
  4. worker3


K8s will be installed on 3 VMs + X physical nodes as workers. All these hypervisors, VMs and k8s will be installed in restricted network. Public network ip addresses will be used for metallb loadbalancer and apps on k8s will use these public network ip addresses. The nginx (not the nginx cluster of VMs) ingress controller deployed on the k8s via the helm chart will take care of ingress.

No comments:

Post a Comment