profile for Gajendra D Ambi on Stack Exchange, a network of free, community-driven Q&A sites

Thursday, March 5, 2020

MetalLB load balancer for your Rancher 2 Kubernetes cluster

It is fairly simple to setup a kubernetes baremetal cluster with rancher 2 which comes with all the bells and whistles. I however had some touch time in figuring out why I couldnt get the metallb load balaner working.
First setup your rancher 2 baremetal cluster.
Then go to
After this deploy the metallb on your kubernetes cluster as per
then deploy a configmap with an ip pool.

kubectl run nginx --image nginx
kubectl expose deployment nginx --type=LoadBalancer --name=nginx-service

Now you will see an exteranl IP from the metallb ip pool. You delete the service and metallb will reclaim it.

Wednesday, January 29, 2020

Clustering RabbitMQ on ubuntu 18.x

I am exploring a highly available, multi site, multi network rabbitmq server for what it does best.
3 VMs
on KVM+kimchi virtualization setup.

sudo vim /etc/hosts
and update it with the details of all 3 nodes

192.x.y.z ub1
192.x.y.z ub2
192.x.y.z ub3

Install and Setup RabbitMQ on all

Copy paste the following on all 3 nodes.

sudo apt update
sudo apt upgrade
sudo apt install rabbitmq-server -y

sudo systemctl start rabbitmq-server
sudo systemctl enable rabbitmq-server


sudo rabbitmq-plugins enable rabbitmq_management
sudo systemctl restart rabbitmq-server


sudo ufw allow ssh
sudo ufw enable
sudo ufw allow 5672,15672,4369,25672/tcp
sudo ufw status


So the erlang cookie used by rabbitmq should be same on all hosts/nodes. We can copy it from node1 to others via scp but we need to change the permissions on other nodes for this cookie so that you don't get permission denied error. Run the below command on ub2 & ub3.

sudo chmod 777 /var/lib/rabbitmq/.erlang.cookie

Run the following on ub1

scp /var/lib/rabbitmq/.erlang.cookie root@ub2:/var/lib/rabbitmq/
scp /var/lib/rabbitmq/.erlang.cookie root@ub3:/var/lib/rabbitmq/

Change back the permission on the cookie to be accessible only by the owner. Run the following on ub2,ub3.

sudo chmod 600 /var/lib/rabbitmq/.erlang.cookie
Again on ub2,ub3

sudo systemctl restart rabbitmq-server
sudo rabbitmqctl stop_app
sudo rabbitmqctl join_cluster rabbit@ub1
sudo rabbitmqctl start_app

User Setup

Run the following on ub1. This will create an admin user and deletes the guest user.

sudo rabbitmqctl add_user admin admin
sudo rabbitmqctl set_user_tags admin administrator
sudo rabbitmqctl set_permissions -p / admin ".*" ".*" ".*"
sudo rabbitmqctl delete_user guest

Run the following on ub2,ub3 to confirm that the clustering is working.

sudo rabbitmqctl cluster_status
sudo rabbitmqctl list_users

HA[Queue Mirroring]

sudo rabbitmqctl set_policy ha-all "." '{"ha-mode":"all"}'

Monday, November 4, 2019

Smartphones: Psychology of product design

A picture speaks louder than 1000 words.
Above is a size comparison of iphone 5, iphone 11 pro, google pixel 4 xl. 
How did they become so big?
Why did they become so big?
When did they become so big?
Who made them so big?
It was a gradual, progressive change by the manufacturers which no user asked for. How are they
justifying it? That is the main question.
This is how they normalized big phones.
  1. Same phone size but bigger screen (by reducing the bezel) to make us get used to the big screen
  2. New phone with the previous screen but with some added bezel on top and bottom
Repeat the steps 1,2.

Most often they say that to accommodate bigger battery we need bigger phones but we only need
bigger battery if we have a bigger phone which eats a lot of battery. Now let us come to the icon
placement of applications and OS (android, iOS).
Most use their phones with one hand and mainly and only they use thumb. If that is all you use then
you probably hate using a bigger phone and want a smaller phone, so they forced users to use two
hands by placing action items of OS and application at top and bottom, mostly top because if you hold
your phone from the bottom and if the action items are on top then you use other hand to activate them (by touch)
On top you have OS items like notification, settings etc.,. At the bottom you have some other stuff.
In the middle you just have empty space. What if all of it was at the bottom? Then you can use it with
just one hand and a thumb and realize that you don’t need a huge long phone. Why aren’t app
developers placing their action items all at bottom too? Well most app developers use templates
given by OS (apple iOS , google android) makers and these templates force you to have such layout. 

Psychology of a big phone

  1. If a phone is small and you are using it with only one hand then your other hand has something else or doing something else. Then you have two choices; Phone at hand 1 and something else at hand 2 (may be TV remote or a book). The chances of you leaving your phone for something else are now 50%. When the phone is big and you have to use both hands to use it then you have only 1 choice forward in your hand. Your phone. So the chances of you not leaving your phone for something else are reduced by 50%. 
  2. A smaller phone fits into your pockets and thus it is out of your sight. A bigger phone does not fit into your pocket, especially when you are sitting down, it presses against your thighs and makes it very uncomfortable. So at work or at home when you are sitting you take it out of your pocket and keep it in your hand. So a bigger phone essentially makes you to always hold it in your hands and keep it in your sight which exponentially increases the chance of you using it now since you are always seeing, touching and holding it.
  3. Across cultures humans have always joined their palms to show submission. Whether it is the namaste of hindus, budhists, jains, sikhs, the open palm towards the sky of muslims, or joining hands with crossed fingers for christians or any other faith. A bigger smartphone requires you to join your hands in similar fashion resembling as sign of submission to a higher being.

Wednesday, October 30, 2019

Jenkins dev environment in one line

So I wanted a jenkins environment to play around with. Luckily I am using RHEL 7.x as my daily desktop at office and it helps.
OS : linux
prerequisites: docker
Just run this and you are good to go. The data btw persists across reboots of your system.

docker run \
  --rm \
  -u root \
  -p 8080:8080 \
  -v jenkins-data:/var/jenkins_home \
  -v /var/run/docker.sock:/var/run/docker.sock \
  -v "$HOME":/home \

Thursday, October 3, 2019

Getting started with operators using operator-sdk

Operators? What are they? Operators is a better way to do kubernetis native deployment of applications. You can read all about it here. I am using RHEL desktop so this is for rhel/centos linux.
There are few ways that you can do it. The hard way is be an expert in golang and write from scratch.
Easiest way for now is use the operator-sdk (by coreOS > redhat > IBM) and there is another way by the kubernetes itself.
Let us stop talking and do it.
1. Install, configure go for operator development
2. Install operator-sdk

Install configure Go

Here I am getting the golang 1.12. You can get the link for the version that you desire from here.

# download go
# extract go binaries from archive
tar -xzf go1.12.7.linux-amd64.tar.gz
# place go binaries appropriately
mv go /usr/local

We have to setup
- GOROOT - location of go binaries
- GOPATH - Go project location
edit your ~./bashrc and add the following

# go binaries location
export GOROOT=/usr/local/go
# go project location (your's varies)
export GOPATH=$HOME/Projects/Proj1
# add to system's path
export PATH=$GOPATH/bin:$GOROOT/bin:$PATH

Now save and do

source ~./bashrc 
to load the new system variables.
Now a command from your terminal 'go version' command will reveal the version of go.

Please also do

go get
cd $GOPATH/src/
go install ./...
to install the dependency manager.

Install Operator-sdk

mkdir -p $GOPATH/src/
cd $GOPATH/src/
git clone
cd operator-sdk
git checkout v0.4.0
make dep
make install

and you are good to go.
Sometimes you might notice that you will find 'dep' package was not found at your $PATH. Just do this.
mkdir -p $GOPATH/bin
cd $GOPATH/bin
curl | sh
If your $GOPATH changes then you might have to do this again.

Now we get into the operator-sdk directory and start with our project/operator.
1. create new project
2. add api
3. add controller

# make sure you are the in the right directory
cd $GOPATH/src/
# new operator project
operator-sdk new podset-operator
# add api to it
operator-sdk add api --kind=MyOperator
# add controller
operator-sdk add controller --kind=MyOperator

Now that the api and controller are in place. Now check out the pkg/apis/app/v1alpha1/myoperator_types.go here is where you define the spec. To begin with just add this
type PodSetSpec struct {
  Replicas int32 `json:"replicas"`
}type PodSetStatus struct {
  PodNames []string `json:"podNames"`
 Whenever we make change to the structure, we have to generate the k8s code by doing
operator-sdk generate k8s
Now we want to build the image and push it to a registry. Currently here I am going to use a public repo, you might want to use your own.

operator-sdk build
docker push

Now update our yaml files to use this image.

sed -i 's|REPLACE_IMAGE||g' deploy/operator.yaml

Let us deploy it now

# Setup Service Account
$ oc create -f deploy/service_account.yaml
# Setup RBAC
$ oc create -f deploy/role.yaml
$ oc create -f deploy/role_binding.yaml
# Setup the CRD
$ oc create -f deploy/crds/app_v1alpha1_myoperator_crd.yaml
# Deploy the myoperator-operator
$ oc create -f deploy/operator.yaml
# Deploy the custom resource
oc create -f deploy/crds/app_v1alpha1_myoperator_cr.yaml