Linear Regression

-

What is Regression?

A measure of the relation between the continuous dependent variable(e.g. output) with one/many discrete/continuous independent variables(e.g. input).
In Machine Learning, Regression is a supervised statistical predictive model.
Other statistical models are Descriptive model , Prescriptive model etc.

Different Types of Regression model in Machine Learning

In Machine learning, There are mainly  Three types  of  Regression model.

  • Linear Regression Model 

In this type of predictive model, The relationship between the Independent and Dependent Variables can be  represented as straight line(single/multiple particularly where clusters are present).
linear regression

  • Polynomial Regression Model
In this type of predictive model, The relationship between the Independent and Dependent Variables can be  represented as polynomial curve(single/multiple particularly where clusters are present).
Polynomial Regression Model

  • Logistic Regression Model
 It is special type of classification model where this model can classify dependent variable in different classes(Binomial for 2 classes/Multinomial for more than 2 classes) where it use a linear model implicitly.

Logistic Regression Model

In this post we are going to see about the most simple REGRESSION MODEL (LINEAR) 


To listen this page please open the player

Basics of Linear Model in Machine Learning:

Here, using linear model we express the dependent variable(Y) in terms of independent variables(X1, X2, X3)  
    Y = b0 + b1*x1+b2*x2+b3*x3
Here, we have taken 3 independent variables only.
We choose the best fit straight line by minimizing the mean of sum of squared error(MSE).
From that if we minimize the summation of squared error then also we can able to minimize the above expression. This function is called Cost function.

error
From the cost function we can get the coefficients(or weights) of each independent variable by equating the first derivative of the cost function equals to Zero.

cost function


NOTE: 
If we have bivariate data, then from the above expression we will get a very beautiful result.
 Y = b0 + b1*x    [ For Bivariate Data ]
Then we will get these result, given as -
                 b1 = cov(x,y)/var(x)
                 b0 = E(y)-b1*E(x)
where E(x) and E(y) gives the mean of x and y respectively.

Reasons for using MSE(mean squared error) over MAE(mean absolute error)

  • Cost function, as being quadratic function, is a continuous differentiable function. That's why we get a global minima that ensures the best fit line.
  • On the other hand, |x| function is continuous but a non differentiable function we can not do the previous steps.

Disadvantage of using MSE(mean squared error)

  • If the Error is very large then the square of it will become a very large number. Then for our simplicity we will use different types of error estimation function

Conclusion:

From the above concept we have learnt about What Regression problem is and What is linear regression. For the next concepts please go though the other posts of Data Science.


Comments

Post a Comment

Popular posts from this blog

Distance measurement using opencv

-
Image
In Covid-19 pandemic, we all know about the importance of Social Distancing . Right now, It is the only thing that can help to reduce the number of affected people. So I have decided to make a social distancing checker using python. The main idea is to measure the distance of object in the image and if the distance between two objects are less than some predefined pixel value then 'green' dot at the center of the bounding-box will convert to 'red' dot . The output of the code will like this- social distancing checker Python code for Distance Measurement I am providing the python code to measure the distance of the object in the image or each frame of a video. Later I will discuss some key points related to the code. Key points to be Noted: Everything will be the same like object detection (discussed in Part-I and Part-II ) as I have implemented distance measurement based on object detection. We will filter out only the  'person' object   from the obje
Read more

Object Detection (Part I)

-
Image
The most fundamental computer technology related to Computer vision and Image Processing is Object Detection. It deals with detecting the   semantic object of a certain class in digital images and videos. Have a look on the images, bellow. Object detection with YOLO Detecting home objects Here, we can see that the model can detect the home objects (chair, bag, laptop, bottle, laptop, bowl, cup etc.) and also the outer-home objects ( bus, bi-cycle, human, car etc.).  Object Detection Object Detection is a supervised learning technique by which the model can detect or predict the object present in the digital images or videos. We can make our own object detecting tool using python with the help of Computer vision with deep learning model ( Here we have used YOLO model ). In this post we are going to create our own object detecting model in python.  As this is the PART-I, I will keep this very simple but FPS(Frames Per Seconds)  will be very low due to its simplicity. In the next post
Read more

Invisible Cloak using OpenCV

-
Image
Have you ever seen Harry Potter’s   Invisible Cloak   If not then let's have a look.     Was it wonderful? Do you surprised how this can be possible? If yes, then this post is going to help you. You will be able to make such invisible cloak by your own but yes, not in real way but it is all about graphics . Algorithm behind the idea First, we take the background frame ( for some second ) and store in the memory. Then, we will take the frame with cloth. Then, detect and segment the portion of the cloth of pre-determined color. For Example here we will use Red. [ known as Color Detection and Image Segmentation ] Then, that segment will be replaced by the background , we have stored in the memory, segment. At last, create the final image to make the magic. Note: All the things will be done in python due to the availability of the various libraries mainly numpy . Python Code: First of all, importing all the required libraries.   To make a vid
Read more