OSI Layer : Network and Communication - A Simple Explanation

Open System Interconnection (OSI) Layer is an ISO standard conceptual model used for visualising or troubleshooting computer networks. It has 7 layers namely

1) Physical layer - is where the real bits are transmitted with ethernet cables. Data here is in binary format 0's and 1's. Data will be in the form of electrical pulses if it is coaxial or twisted cables and in the form of light if it is optical fibre cable.

2) Data Link - is responsible to establish reliable links between two directly connecting nodes via switches or bridges. It organises bits into frame and ensures hop to hop delivery. It consists of Media Access Control (MAC) and Logical Link Control (LLC). MAC maintains how devices connected to the network gain access, whereas LLC ensures error checking and packet synchronisation.  Data here is converted from packets to frames.


3) Network - is responsible for routing the information packets from source to destination via the best path available. This is done via router using routing protocols. Source and destination is determined by the ip address of the router. Data segment is converted into packets here.

4) Transport -  is responsible for information transferred between the operating system and the web server. It also decides how much data is sent and received and used for end to end connectivity. It identifies service TCP/UDP, also segmentation segments the data into small data segments, sequencing/reassembling takes care that data is arranged in correct sequence. In case if a data segment is missed in sending to the target node, error correction mechanism ensures that the missing data segment is re-transmitted to target node.

5) Session - maintains sessions/connections between operating system and third party system. It is responsible for establishing, maintaining and terminating sessions. When we visit a url on the web, we interact with application layer, application layer interacts with presentation layer, the session later then interacts with the web server.

6) Presentation - is where operating system interacts with the data. It's responsible for translation, encoding/decoding, compression/decompression and encryption/decryption of data. User interacts with application layer, the application layer interacts with the presentation layer.


7) Application - is where a user interacts directly. It provides networking services to the user using port numbers. For example Web browser (http request on port 80).


An easy way to remember the layer name is

Application    >>         All

Presentation   >>         People

Session           >>         Seem

Transport        >>         To

Network          >>         Need

Data Link       >>          Data

Physical          >>         Processing

Face Detection with Open CV - Python

OpenCV is one of the most popular library for computer vision. It uses machine learning algorithms for computer vision.

I Installed OpenCV using conda on my Mac with below command

"conda install -c conda-forge opencv"

Face detection algorithms work by reading a block of image and matching against thousands of small patterns and features to detect if it is the desired object. OpenCv uses cascades for this. Cascades are trained with a sample views of a particular objects. Read more about cascades here.

OpenCV cascade makes object detection faster because it checks based on stages. If initial tests are positive, it proceeds with further tests. If the input picture has failed for the test, the algorithm will not test it for all the patterns and features. This makes object detection faster and in real time.

Below is a simple example of face detection in action.

1. First import OpenCV and Sys 

2. Create the OpenCv cascade and initialise with face cascade.

This loads the face cascade into memory. We will use haarcascade_frontalface_default.xml which comes with OpenCV. It is an xml file which contains data to detect faces.

3. Read the input image and convert it into gray scale

4. Call the function detectMultiScale on face cascade to detect face. Pass the desired parameters like scaleFactor, minNeighbors, minSize etc. This function returns a list of rectangles which it believes is a face. It returns the x,y coordinates of the face along with the width and height.

5. Draw a rectangle around the face using the values returned from detectMultiScale function.

The source code looks like below

In order to test this program. Go to terminal and type

Python DetectFace.py <InputImage.png without the angular brackets> 

Make sure that face cascade xml is placed in the same folder as your source .py file. Face cascade xml will be supplied by the OpenCV library. The other option is to fully qualify the path during initialization of the cascade.

The output image I got looks like below

OpenCV is very interesting and promising. I hope to catch up on these in future.

Web scraping with Beautiful Soup - PYTHON

Web scraping is an art of extracting information from the webpages in an automated way. Python has a very useful library called Beautiful Soup which does the job.

Below is a attempt to learn some elements of Beautiful Soup. Dataquest has a very nice explanation on this.

Basic example of sentiment Analysis - Twitter API

I will demonstrate a basic example of sentiment analysis using in python using jupyter notebook as an editor. We will be analysing tweets from twitter using tweepy.  Tweepy is a python library for accessing twitter API.

We start by creating a new app in https://apps.twitter.com/

Once you create an app in twitter, generate the access token and set the permissions (read/write) accordingly. To accomplish this, you need to key in app name, description and a url. Url could be anything in the right format if you don't have one for yourself.

Once the consumer and access secret keys are generated. Next step would be to fire up python in your choice of editor.

I chose jupyter here. Begin by importing tweepy and textblob libraries. textblob is a python library for text processing.

Assign consumer and access key. Use OAuthHandler method of tweepy by passing consumer key and consumer secret. Then call set_access_token method by passing access token and access secret key.

Once this is done, your authentication is done. Now you can use search method and pass in a text string to search for.

I have searched tweets for 'Indian cuisine' and successfully printed the results back via python.

IMPORTANT : Never expose/share Consumer and Access secret key.

Simplified Text processing Example - Python

Sentiment Analysis can be used to understand human emotions and psychology using their feedback, comments or posts from social media network or any other survey platform.

Below is a very simple example in Python where just in few lines of code we could take a string input and derive a polarity out of it. I used textblob for this.

The output will be a polarity in the range of -1 to +1

-1 being negative sentiment and + 1 is very positive sentiment .

I tested 3 cases as below

a) "Last night food was good, but it was expensive"

The above sentiment resulted in a polarity of 0.06

b) "Last night food was good"

This one resulted in 0.35

c) "I went for running, it felt awesome"

This resulted in 1

Data Visualization: BarChart - using Matplotlib

A Bar Chart is used when we want to show how some quantity varies among some discrete set of items. It also helps to explore how the values are distributed. For example, How many centuries were scored by a player in a cricket tournament.

player = ['Sachin', 'Sehwag' , 'Dhoni', 'Virat', 'Dravid']
num_of_century = [7, 2, 3, 4, 5]

Below is a demonstration in python to generate a Bar chart using matplotlib in python.

Some very nice examples of Bar charts are available here

Data Visualisation - A Simple Example

Visualising data is one of the most import and powerful aspect of data science. It helps us with two things, one to explore the data and second to communicate the data to other people. It simply has a story to tell with the chart.

There are a lot of tools for visualising data like matplotlib library, tableau, d3.js etc.

Here is a basic example of matplotlib using which I am going to compare my savings over a period of time.

I am going to do this in python. I will be needing two lists to represent my X and Y axis. Y axis will show time period in years and X axis will show savings in INR.

I will be using some library methods to add title and label and then finally generate the chart. The chart generated is a Line Chart which is used when we want to compare a specific value over a period of time.

Below is a simple example written in python to demonstrate a Line Chart.