profile for Gajendra D Ambi on Stack Exchange, a network of free, community-driven Q&A sites

Friday, July 27, 2018

Kubernetes on windows


  1. I hope you have enough rights on your windows machine to get started.
  2. Turn windows features on or off > disable hyper v;enable containers
  3. Install Virtualbox
  4. Search for ‘docker toolbox for windows’ and install it
  5. The usual method of installing kubernetes cli via powershell won’t work most of the times so install chocolatey

1
Set-ExecutionPolicy Bypass -Scope Process -Force; iex ((New-Object System.Net.WebClient).DownloadString('https://chocolatey.org/install.ps1'))

     6. Install the kubernetes cli (kubectl)

1
choco install kubernetes-cli

Run minikube now on your windows to get started and good luck.

Tuesday, July 24, 2018

Learn with me Python 3.7

1.Python 3.7 Introduction
2.Python - Input Output [3.7]
3.Python - Data Types [3.7]
4.Python - Data Types - Integers [3.7]
5.Python - Data Type - numbers - float and ...[3.7]
6.Python - Data Types - Strings [3.7]
7.Python - Data Types - Lists [3.7]
8.Python - Data Types - Tuple [3.7]
9. Python - Data Types - Sets [3.7]
10.Python - Data Types - Dictionary [3.7]
11.Python - Loops (for and while) [3.7]
12.Python - Operators [3.7]
13.Python - Explore and inspect [3.7]
14.Python - Shallow Copy and Deep Copy [3.7]
15.Python - Package management with pip and VirtualEnv [3.7]
16.Python - Math [3.7]
17.Python - formatting [3.7]
18.Python - Conditional statements (if elif else) (decision making) [3.7]
19.Python - flow control (break, continue, pass) [3.7]
20.Python - Functions, *args, **kwargs, recursions [3.7]
21.Python - Closures [3.7]
22.Python - Decorators [3.7]
23.Python - Exceptions and error handling [3.7]
24.Python - Variables [3.7]
25.Python - List Comprehensions [3.7]
26.Python - Set comprehensions [3.7]
27.Python - Dictionary comprehensions [3.7]
28.Python - Resource or context management [3.7]
29.Python - OOP - class [3.7]

Python 3.7 Introduction

So why do you wanna learn python? or why should you.

  1. Because I love it
  2. Beginners language
  3. easy to learn
  4. easy to teach
  5. readable code
  6. do more with less code
  7. faster development
  8. opensource
  9. cost effective
  10. Career or job security
  11. It is the master of many trades and jack of none.....
I can go on and one. It is one of the top tool used in
  1. web development
  2. application development
  3. Artificial Intelligence, Machine Learning, Deep Learning, Data Science
  4. Healthcare
  5. Internet of things [IOT]
  6. robotics
  7. automation (IT or anything else)
  8. mathematics, physics
  9. statistics
  10. finance
  11. weather, earthquake prediction to name a few
I am pretty sure I have missed many aspects of it but you get the point. If you are someone who is planning to have a career and you want something which opens a lot of doors for you, not just one or two then python is the answer since it makes you eligible in a lot of areas. Later you can branch out or up with python in whichever stream you want/like to.



Monday, July 23, 2018

Why containers and why not VMs?! or Vice Versa


I hope the above gives you a visual representation of what the container stuff is all about. Let me also collage some major differences.

Scalability
A VM is limited to the hardware that it is running on. The maximum memory, cpu or network resource a VM can have is limited by the hardware that it is running on. Many a times there is also a limitation of the virtualization layer tech that you might be using. If it is VMware, my favorite one then you have a cap on what is the maximum cpu and memory that you can assign. It is a scale up architecture. It is a like a huge tall building with many floors. Ultimately there is a limitation (call it gravity, sustainibility or whatever) and you can't build a skyscraper tall enough to touch the moon.

Containers are scale out. you can always have a building which can be as wide as the earth allows it to be. Instead of getting big like a VM it multiplies or clones itself as and when needed and the clones disappear automagically when the load is low.
winner:container

Security
Containers are fairly new and thus they aren't as secure as a VM architecture is. It is not the fault of containers but the security itself and how the tools are developed and who is their main target audience. Most of the current IT security tools, software are all designed for a traditional datacenter and not the micro architecture. This can be countered if you develop your applications to be cloud ready or cloud native.
winner:VM

Simplicity
A tradition VM architecture is easier and faster to deploy and get started than a container based implementation unless you are starting off new and you start your development with container based architecture as your platform. The skillgap, industry readiness are also adding to this factor.
winner:VM

Availability
Max availability a VM based architecture provides a node failure. A node1 goes down with VM1 but node2 has a copy of that VM and that takes over without a downtime. What if the 3 nodes go down? or 5 nodes? Containers offer a never go down architecture  since it is a scale out architecture. You can have your copies of the container run on different nodes and you can just mention how many nodes you want to span them across and that is it. An orchestrator engine like kubernetes or docker swarm will take care of the rest.
winner:container

Portability
Let us say you have the need of AI/ML frameworks for your project. You can spin up a kubernetes cluster on AKS (azure kubernetes service) to access azure ML/DL frameworks or do it on google to access their AI/ML/DL frameworks and they can easily span over to your on premise datacenter. You can migrate your workload/container between different cloud providers including your on/off premise datacenters. You start off with azure. Tomorrow you might want to go to google or aws or move back to your on premise hardware. That is all possible with containers and this is a big win.
winner:container

Did I miss anything? Do let me know.

Sunday, July 22, 2018

Django dyanimic url for employees or users

Let us say you are creating a site and you want your users to sign up and you want them to redirected once they log in. Below are some url formats for some well known sites.
linkedin
https://www.linkedin.com/in/<username>/
facebook
https://www.facebook.com/<username>/
How can we do that?
It is extremely inefficient to have an html page created for each user and it is a very heavy load on your site.
How to achieve this?
By default django and allauth redirect the logged in user to domain/profile so let us use that to do some magic. Here is my urls.py


 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
from django.contrib import admin
from django.urls import path, include
from . import views as core_views
from stars import views as star_views

urlpatterns = [
    path('admin/', admin.site.urls),
    path('', core_views.welcome, name="welcome"),
    path('settings/', core_views.settings, name='settings'),

    # allauth
    path('accounts/', include('allauth.urls')),

    # stars
    path('profile/', star_views.profile, name="profile"),
    path('<slug:pid>/', star_views.rprofile, name='rprofile'),
]

Concentrate on the line 15,16. So I am letting the django/allauth redirect the url to domain/profile.html and then use the star_views.profile to redirect the request to star_views.rprofile and that is done like this

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
from django.shortcuts import render
from django.contrib.auth.models import User
from django.http import HttpResponseRedirect, HttpResponse
from django.shortcuts import redirect
##Create your views here.

def profile(request):
    site = 'http://127.0.0.1:8000'
    username = request.user.username
    url = f'{site}/{username}'
    return HttpResponseRedirect(url)

def rprofile(request, pid):
    u = User.objects.get(username=pid)
    if u:
        return render(request, 'profile.html')

Now every user has his own url which will be easy to share.

Django Allauth signal to trigger something

So I am still testing the waters of django and I am still like a baby swimmer with swim tires. I had some trouble in using django allauth signals to do something.
Django 2.0
Django allauth
All in a virtualenv.
So here is how you use signals trigger some action. Go to https://github.com/pennersr/django-allauth/blob/master/docs/signals.rst to checkout the signal allauth emits at various stages. I am interested in the signal which gets dispatched after the user signs up. I am using email+password to signup and signin.


 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
from django.db import models
import time
from django.contrib.auth.models import User
from allauth.account.signals import user_signed_up, password_set
from django.dispatch import receiver, Signal

# Create your models here.
@receiver(user_signed_up)
def employeeID(sender=User, **kwargs):
    old_username = kwargs['user']
    user = User.objects.get(username = old_username)
    user.username = str(time.time()).split('.')[0]
    user.save()

1-5 import modules.
8. mention the signal that we want to use.
as per the allauth documents mentioned in the https://github.com/pennersr/django-allauth/blob/master/docs/signals.rst this signal contains the following.
user_signed_up = Signal(providing_args=["request", "user"])
So the entire info received from the signal is passed on to the funtion at line 9 as **kwargs.
10. extract the signedup user's username.
11. query the database for the data of that user.
12. change the username to some random number.
13. save the changes to the database.

So, when we are using email+password only to signup and signin, allauth still creates a username='email ID' of the user (without the @email.com, so if i signed up with mail@email.com then my username will be mail.) So I thought I will give my own username and it worked.