Problem with CentOS 7 VM Guest tools, screen resize on VirtualBox

First of all I know that all the so called developers care only and mostly about mac, apple and its eco system (shit-stem).
The actual developers who develop for linux are those who care nothing about those who are beginners, intermediate or anything less than those who can write a custom kernel and prefer to work mostly in CLI and hate anything other than C.
Then there is windows devs who care nothing about quality but quantity (microsoft).
I am on windows unfortunately since I like gaming and most enterprises still offer windows laptop to them at office.
I want to use VMware Workstation but it isnt free. It does not have a windowed mode for VMs like virtualbox. Docker, containers, kubernetes, vagrant and all these upcoming devops tools support mainly and only virtualbox since it is cross platform and free. So I am forced to use virtualbox.
Now, I want the screen resolution of my cent os linux to be at least 1080p but in any VMs (including vmware) you will never see 1080p as the resolution. So hoping that it will come up if you get the additional tools installed you try to install it.

1 2 3 4

yum install perl gcc dkms kernel-devel kernel-headers make bzip2 yum update kernel* yum install -y kernel-devel kernel-devel-$(uname -r) ls -l /usr/src/kernels/$(uname -r)

Now mount the guest tools image. It will ask you to run and go run it. Install it. Reboot it. You will see that
Auto resize guest display and
virtual screen 1
options are all grayed out.
Now god only knows why and how. Once you make the auto resize guest display ungrayed out then you get to choose the resolutions under virtual screen 1. I usually
reboot 3-4 times,
power off,
wait for a minute.
close the virtual box application.
start the vbox.
start the vm.
It usually takes 2-3 hours to have a centOS VM ready to a stage where I Can change the resolution of it. It is so sad that oracle is willingly or unwillingly or for whatever messed up reason not allowing its engineers to create a repo where one can just download and install the guest tools for the VM from inside it via yum or rpm. Even the paid vmware workstation too doesnt have a repository for linux guests. what a shame that they still want us to mount an ISO and install from it. I think that it is pure arrogance and lack of competition. Look at intel & AMD or nvidia and Radeon graphics cards. No competition to them; nobody improves unless they have to, unless it is a threat to their well being.
YOU HAVE TO DO THIS EVERY TIME YOU UPDATE YOUR SYSTEM. :(

Install and Default python 3.x on CentOS 7

TLDR

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20

#install the centos7 repository sudo yum -y install https://centos7.iuscommunity.org/ius-release.rpm #install python3.6 sudo yum -y install python36u # install pip3 for python 3.x (since pip is for python 2.x) sudo yum -y install python36u-pip # it shows the default python 2.x python -V # this is how you run the python 3.x but it is inconvenient python3.6 -V # so set alias # when we run python the OS will not autotranslate that the alias value python3.6 alias python=python3.6 alias pip=pip3.6 # make these alias changes permanent source ~/.bashrc # install virtualenv pip install virtualenv # how to create a virtual environment for python3? # virtualenv <name of the virtual environment>

When we get our linux box it has python 2.7.x but we know that most of the development and the future sustenance is towards python 3.x. So let us do these

1. Install CentOS IUS repository
2. Install python 3.x
3. Install pip3.x
4. Make python 3 as your default python
5. Make pip3 as your default pip

Run line 2 to install the centos repository.

Run line 4 to install python 3.6

Run line 6 to install the python package manager 3.x for python 3.x

Run line 13-17 for all user accounts where you want the python 3.x and pip3.x to be the default.

Log in as that user and run 13-17.

Bonus : - Run line 18 to have the virtual environment installed for your system.

Now run python and it defaults to python 3.6. do exit() to exit out of python.

Run pip and it will default to pip3.6.

Django developing to production best practices.

So this is going to be a note, reminder and a collections of lessons to the future me and others like me who want to learn, develop, plan and deploy a django web server.

1. Start from bottom
Start with a bare minimum unless you absolutely need something which can't be achieved with the default stuff that comes with django. ex: Unless you need your data to be stored as json you don't need a full fledged database like postgre sql or mysql till you go to production. Start only with a virtualenv and django in it.

2. Stick to matured than young and trendy

ex: I switched to materialize css and later during production came to know that the django allauth package I am using does not work well with materialize css. So I switched back to the time tested bootstrap. If I had a lot of previous experience with django development and deployment then I Should have tried to be courageous and try the trendy stuff but I wanted to be sure/safe than sorry. I wanted to deploy my app on GKE (google kubernetes engine), explored amazone ECS and azure AKS too but it was adding too much of overhead for the size of developing team (just I, me and myself). There are a lot of security aspects you need to figure out and it is expensive too. It is easier to find tutorials, guides, documentation for matured, old trusted, tested and tasted by pros than the new kids on the block.

3. Start small, dream big

I want with single VM at the initial stage of my project with a plan to add more VMs and districute the responsibilities amongs these cloud servers later (app server, db server, web server etc.,). If you bite the bullet and went with a 3 tier in the beginning itself then 

1. you will have too shell out a lot more
2. performance issues will arise 
3. you have to monitor a lot more servers now and scale all of them accordingly as per their load.
4. You might even hit the network traffic limit your cloud provider has for your server (unlikely but possible)

4. One life is too short to re invent everything

Re use the tools, packages developed by pros who have spend the better half of their life and time in them. Ex: django allauth. It is a well trusted, widely used package with many stars, forks and followers on github. Build on others work, because they have done it too.

5. Deploy your app before it is perfect.

Most fall into this pit. Do not worry about the looks yet. If you application is ready with the most functionality then deploy it to know what else you want to start implementing now itself instead of developing a full fledged application and realizing that you should have to redo a lot of things since in production it is bombing badly.

6. Add only 1 element at a time

So let us say currently you have django (with default sqlite db) app ready to be deployed. Add these elements in these order to avoid graying out most of your hair (If you have any).

1. Add  gunicorn and NGINX (dont choose alternative) and make them work using a shell script.
2. Now make that script part of a systemd service (yes, use linux) so that it monitors that script, runs that script. You can also alternatively choose supervisor or upstart but they seem to add another layer of toolset to learn, maintain and another layer of failure. Try to stick with something embedded in the operating system itself. I found the systemd option to be easier to learn. I had nothing but failures with supervisor, upstart and the likes of it.
3. Now replace the sqlite with a production grade database server, preferably an SQL server which also offers nosql stuff too. I Chose psotgresql since I had some experience with it than mysql which I have not played around with at all.
4. Now add a domain (if you don't have it then buy one for production and one for development purposes) preferably something not for production but for development or for testing.
1. web browser<-->domain provider<-->cloud provider
5. Once you are able to access your site via a browser then add cloudflare to your site (to protect yourself against DDOS attack). 
1. web browser<-->cloudflare<-->domain provider<-->cloud provider
6. Now sign up for a smtp provider to send mails and integrate that with cloudflare.
7. Add https using certbot and lets encrypt. Make sure it auto renews before it expires.

7. Security

1. Search 'security settings for django' and try implement as much as possible from the official django webpage result that you get. 
2. Serach 'django vulnerability scan' and use them to scan your site for any vulnerabilities and implement all of them as much as possible
   start with https://www.ponycheckup.com.
3. Security is a journey and not a destination. So have a list of the packages that you have for your project, their versions and their specific vulnerabilities.

8. Code management

1. Add a config.py to your production and development servers and to the local repo of all your developers. Add it to gitignore. Check this. http://www.cloudishes.com/2018/09/git-and-ignoring-files.html
2. Have a master branch. Have a dev branch. Not more than 2 or 2 percentile of your total size of developers should have access to write to this. It means make it a protected branch if you are using github pro/enterprise. So only merging that can happen to master should be from dev. Anyone can create a branch from dev but before merging it should be approved by at least 1 or 2 reviewers. 
3. Once the code is on the dev branch and done testing on the developing server then push it to master remote branch.
4. In your production server, to a pull of the dev server content only and see how it goes for a day. Which means the old working code is only on the master production branch. The remote master, remote dev, developing server dev, developing server master have the same latest code. After 24 hours or 2 days or whatever time that you have decided, if you do not face any major problems then on the production server switch to master, pull down master from remote master. commit. Restart your django server's system service which now makes it run on the latest code. I know this is not a proper CI CD pipeline but as I said earlier. Add only one element at a time. 
5. Once you are comfortable with all these tools that you are already using and confident then add other devops tools if you like. Try to choose those tools which require least maintenance , capex, opex.ex: gitlab has an auto CI/CD tool online if you have your repository hosted on their site.
6. Never merge your code to dev until you have locally tested and 100% sure that it works well and with everything else in your project smoothly. Never push your code to local or remote master unless you are absolutely sure and your peers too have individually tested and okayed it. code, test, review (self and others) then push up.

9. Back it up

1. Make sure you regularly backup your master branch, database and server itself on every cycle (every sprint or on a particular day of each week or on a predefined date or event)

configuring IBM bluemix provider for terraform on windows

There is no official IBM provider plugin available for terraform for now. We can easily configure this by

1. Download the windows 64 version of the IBM providre plugin from https://github.com/IBM-Cloud/terraform-provider-ibm/releases
2. Extract the exe from that archive
3. Create a terraform.d at any place of your choosing
4. Create a plugins directory inside the above directory
5. create a windows_amd64 directory inside the above directory
6. copy the downloaded exe file into the windows_amd64  and plugins directory
7. Now copy the terraform.d file to all the available directories below
1. C:\Users\<username>\AppData
2. C:\Users\<username>\AppData\Local
3. C:\Users\<username>\AppData\Local<whatever>
4. C:\Users\<username>\AppData\Roaming

Now create a main.tf file just with the following lines

provider ibm {}

and do a terraform init and it will work.

git and ignoring files

So if you had thought that creating a .gitignore file in your git repository's root was enough then you are wrong. You will notice that that is what is the official way to ignore file. Here is the problem.

1. I started working on a project and stored all secrets in secrets.py
2. 7 more people in my team joined it and now we decided that we should not commit secrets.py anymore to dev server. 
3. Everybody added it to their .gitignore file but no go

What to do?

1 2	git rm <filepath>/<filename> git rm --cached <filepath>/<filename>

Since the file was being tracked earlier it will still be tracked. So remove that from tracking and you are good to go.