In this section, I will introduce you to machine learning.
You must have heard of it or read about it.
But if you are not clear that what it really is or do you know about it and bits and pieces, this section
should help you put things in perspective and build some intuition around it.
So let us first answer the question.
What is machine learning?
Machine learning is programming the computer to optimize that is to maximize or minimize the performance
criteria based on past data, even more simply, based on past data, your machine will improve its
effectiveness of performing a task.
Below is an analogy of how machine learning is similar to human learning.
Humans observe the actions and outcomes and keep them in memory.
This learning turns into a skill for a human.
Similarly, when we will give postictal to our machine, our computer will learn from that data and
start performing better, and if its job is to predict, it will start predicting better.
You can probably imagine why everyone is talking about it.
Two things which have given it further boost are the ability of today's machines to handle huge datasets.
And the second thing is the ability to collect so much data.
Big companies like Google and Facebook have so much individual data with them, even political parties
are collecting data to create effective campaigns and win elections.
Good government are collecting and using data to create effective public schemes and meaningful policies.
Within organizations, also, if you are a working professional, you will know that at lower levels,
only basic data analysis is being done and that to poor is reporting.
But at higher levels, predictive and prescriptive models are being built to help in decision making
and business strategy formulation.
Machine learning is a tool which is helping in creating this effective prediction models.
There are few terms which are being used interchangeably these days.
However, they have a minor difference between them.
The difference between machine learning and statistics is that statistics estimates and it provides
us with some mathematical models to analyze the data.
But machine learning may not be based on a statistical model.
Most of the times it is, but it is not necessary, and secondly, one major part of machine learning
is programming.
Which is why in this course, we will pick up one statistical model and then learn how to do it in popular
data programming software.
When we compare artificial intelligence and machine learning, you can think of it like machine learning
is a part of artificial intelligence, the part where intelligence is being gathered by learning from
experience of postictal.
Comparing machine learning and data mining is difficult, you can think of data mining as a tool to
be used by people to gather in process data, machine learning and also be part of this process.
But there'll be some person doing data mining to achieve a particular goal, whereas machine learning
can be a standalone organism performing a particular task effectively, let us see some examples.
Now it is said that people uses machine learning to predict fraudulent transactions and prevent them
before it happens.
Similarly, this is something every one of us would have seen.
Google is able to predict what we want to search versus what people around us are searching or is our
own search history and numerous other predictors.
There are several other use cases of machine learning, which I have categorized as the industry in
banking, telecom and retail sector, companies are using machine learning to identify the prospective
customers, good or bad customers, dissatisfaction level of each customer.
They are also using machine learning to effectively use their advertising budget so that within the
same budget they can get the maximum number of sales.
They're using it to decrease their credit risk by identifying the good customer and then assigning appropriate
credit limit to that customer.
As I told you, PayPal is using it to identify fraud, so that will lead to fewer fraud.
Machine learning can be used to identify potential traders, the customers which are going to stop using
your product or service.
And biomedical and biometrics machine learning is being used to diagnose a particular disease more effectively
or for drug discovery, that is, to identify which drug works more effectively on which particular
disease.
And biometrics, you must have realized your facial recognition software in your mobile phones or your
fingerprinting device in your mobile phones is getting better day by day.
That is due to some machine learning algorithm running behind it.
Computer and Internet have been at the forefront of machine learning, you may identify these devices,
Google Home, Amazon Echo, they all have in machine learning algorithms and these boards such as CD,
Alexa, all these have machine learning running behind it.
Your Gmail can automatically filter spam mails, Google can automatically identify the language of every
website, and so just as to translate it all, these are examples of machine learning.
Now, let us enter the world of machine learning based on statistics here.
Our agenda will be to find the relationship between input and output variables versus historic data.
For example, consider a real estate agent who wants to put a price on a particular property.
So the output variable for this person is going to be the price of the property.
Next, this agent has to immediately decide which factors may be affecting the price of the property,
some of them, which I could think of as the area covered number of bedrooms, proximity to landmark,
proximity to market and so on, mathematically, the agent wants to establish price as a function of
these variables.
There can be two motivations behind estimating this function.
One is prediction, as we saw, for a real estate agent who just wants to predict the value of the property
prediction means we are just interested in getting the value of Y and not in the relationship of Y with
each individual variable.
Second, this inference here, we want to establish the relationship between each input, variable and
output so that we know how output will change when we manipulate input.
Suppose instead of a real estate agent, there was a builder, that person would probably like to know
whether a building near a market, which is a better place or building near a school, that is a better
place.
So that guy's motivation would be inference and not just prediction.
Is a motivation.
Whether we want to infer or whether we want to predict, we will choose the model for analysis.
Because when we want to predict, we want your prediction to be accurate and we do not care if we understand
the model or not.
There are some models which will develop better, however, are difficult to interpret such models,
such prediction.
But there are some models which may not predict as accurately.
But we will easily make out the relationship of variables from these models.
That is, they have more and such models.
So inference.
So is motivation will choose the model.
Now there are two major classifications of models, some models that parametric in nature and some are
non parametric in a parametric approach, we assume a functional form of relationship between input
and output variables.
For example, if you think that there is a linear relationship, we have already assigned dysfunctional
form to the relationship.
Now we will find the best values of Izidor, even a do so that the predictive values of praise are as
close to the actual values.
But in a non parametric method, we do not assign a functional form to this relationship.
The functional form itself is estimated by the model and it could be quite complex.
For example, in this image, a three dimensional plane is first drawn by the model, which is as close
to the point and has a smooth surface, will later identify the function at each of these point, maybe
very complex, and it may be very difficult to interpret that.
What is the relationship between each input variable and the predicted output?
So as you can imagine, it is going to be easy to interpret the result of parametric model.
However, it may or may not fit the data as well.
But non parametric model will be potentially able to fit the data better.
However, it will be very difficult to interpret the relationship among the variables also known.
Parametric approaches usually require a larger amount of data to train.
So once you know what your motivation is.
You know, the type of model we use now, there are two types of learning besides the data we have.
When your data has a particular output variable and one or more input variables and model will learn
from this data that what values of input give, what output, we have seen the examples of this type
of data.
And this type of lending is called supervised learning.
However, if you do not have any outward variable, but you just have a set of variables and your model
is learning the relationship between these variables, it is called unsupervised learning.
This basically helps us find the underlying structures and patterns in our data.
Let us look at these two types of learnings from an illustration, suppose we have this data, we have
four images and corresponding levels against each.
Since these images will be our input data and these levels will be output data, this is supervised
learning.
Now, let's say that the labels are category and we.
Give this data to a learning model.
Now, if I present this image to our model and ask what category it belongs to, this becomes a classification
problem.
The model has to learn from previous data and classify the newly presented data into a particular category.
So this is a supervised learning and the problem is a classification problem.
Another example of classification setting is credit scoring.
If you want to classify a customer as high risk or low risk versus their income and savings.
I can bring my model from a bastida.
And maybe get a result like this where I have a particular value of income and savings above which I
can categorize the customer as a low risk customer or else the customer belongs to a high risk category.
So I am able to classify the customer into these two categories on a graph.
This will look something like this.
The customer's belonging to this area will be low risk customers since they will have income more than
a particular value and savings more than some other particular value.
The customers which are belonging to this region will be high risk.
Other applications of classification problem that pattern recognition, face recognition, character
recognition, medical diagnosis, web advertising.
Now, instead of classifying if we are estimating the weight of product, which is a quantitative variable.
This becomes an example of regression.
Well, the model is based on supervised learning.
Another example of immigration problem would be estimating the price of a used car.
Here we will identify the attributes of God, which may affect the final price of that used car, and
if we assume that there is a linear relationship between the price of data and the God attribute will
have something like this bicycle to some function of X and some constant.
We will use the past data to identify this leadership graphically, it will be represented like a straight
line, and these are different data points that we have from our past data.
And using this line and putting the card attribute, we can predict the price of some other used car.
Other applications of regression analysis is weather forecasting, sales forecasting, advertising,
budget allocation or product pricing.
There are several algorithms under supervised learning.
Most popular linear regression, logistic regression, artificial neural networks.
Dangerous neighbors and so on.
In unsupervised learning, we do not have any outward variable, we will just provide these five images
and ask the model to group these very similar properties, maybe pieces of duct tape.
It will make two categories.
Of apples and bananas, maybe this is of the color, it will make three categories of red, green and
yellow, or it may find some other important characteristic which I am missing.
Here is a list of some popular unsupervised learning algorithms.
What lasting one of the popular ones is gaming's, another popular unsupervised learning algorithm is
that analysis or principal component analysis which is used in dimension reduction.
And there are some other algorithms also, these algorithms will not be part of this cause.