As discussed in last video later, we are going to start our discussion now on support vector classifier
so that we can handle non perfectly separable scenarios.
And secondly, so that our model is less sensitive to individual observations.
So let me first tell you what support vector classifier is.
A support vector, it is a soft margin classifier, the meaning of this is that we will have a hyper
plane and the margins are only hyper plane, but all the observations need not be on the correct side
of this margin.
We will allow some observations to be present within the margin area and some may even go on the wrong
side of the hyper plain.
So if you look at this graph.
The purple points are supposed to be on the left side of this margin.
Whereas on the other margin, the blue points are supposed to be on the right side of this margin.
But if you look at point number eight.
This point is supposed to be on the right hand side of the upper margin.
But this is not so.
This point is presenting the wrong side of the margin of, however sensitive still to the date of the
hyperbole, it is correctly classified.
So it is on the wrong side of modern but correctly classified various point number well.
This point is a blue point, which is supposed to be on the right side of the upper margin, whereas
it is on the left side of the hyper plane itself.
So this point will be incorrectly classified by our support vector classifier.
So there are two types of errors that we are alone.
One is classification is correct, but it is on the wrong side of the margin, which is represented
by these blue circle point.
That is point number one I need.
And the second type of error that we allow is.
Allowing the point to be misclassified, that is being on the wrong side of the hyperbole.
Two point number eleven and point number three are misclassified.
Now let's see how a support vector classify it is created.
The underlying concept stays the same, that we are trying to get a hyper plane, that maximum margin,
but with one additional constraint.
The constraint are going to add is a budget of mistakes.
We know some points are going to be misclassified, so we will create a budget B, which will be the
amount of misclassification that is acceptable to us.
So the software will be trying to find a hyper plane with maximum margin while staying within this budget.
So here in that example.
If we find out the distance off point one from its margin.
So if we find a distance of this one with this mob, then.
This is the amount of edit done with this point.
Let us say this distances X1.
Now, if I look at the other Purple Point, which is live in.
The distance of this eleven point will be from this margin to eleven point, which is say to.
To put people point, we have excellent and x2.
As the misclassification error.
What the blue point.
Say, point eight.
This was the the upper lane was the margin.
So the distance from that upper lane would be the error of classification.
So that would be X3.
Similarly for point, well, we can find out export, which is the distance off point twelve from the
up and margin.
We will add all these misclassification errors and we will say that all these errors should be less
than predefined budget.
The.
In this way, we will allow some misclassification.
While trying to maximize the margins.
Another way of implementing this same concept, which is often found in software packages, is user
cost parameter.
This is effectively the same thing.
Only difference is that instead of having.
Unit costs and a, budget.
B, we will have C, times the cost and the unit budget.
So see as kind of an inverse of B.
This C will be a hyper parameter that we will choose the value of C business cross-validation so that
we get minimum test set error.
One more thing to note here is these point point number one eight.
Levin will.
Which lay within the margin or on the margin.
These all these points will be called as support vectors.
Sense now, instead of just the point on the boundary, the margin depend on the presence of these points
until.
So all the points within these margins and points length exactly on the margin, all these points will
be considered as the support vectors.
Any point outside the margin is not that relevant.
I support to classify it will change.
Ban any of these support vectors are teenaged.
Now, let us see what is the impact of increasing or decreasing the value of this parameter C.
If we decrease the value of C and make it model.
This would mean that there is less cost of misclassification.
In such a scenario, more cases will be allowed within the margins and also mostly cases will be allowed
to be misclassified.
On the other hand, if you increase, well, you'll see the cost of making a mistake is high, so there
will be fewer of the between the margins and fewer misclassified points.
But doing this will increase the sensitivity of our model to individual observations, which may result
in overfitting.
This is why we need to carefully to the value of see, a small value could lead to a lot of misclassification
and a large value could lead to overbidding.
So we will try to find the optimum value of C at which we get the best tested performance.
Here I am also sharing a link to a Web page on Stanford Web site.
It's a great tool to visualize the effect of adding and removing points in a to day predictor space.
And the effect of changing value of C.
Let me show you this, too.
So does it in a page, since we have discussed only support vector classifier, which is a linear classifier.
We will toggle this to linear classifier.
We'll be discussing the kernel based, nonlinear classifiers in the coming lectures.
So for now, this is a linear classifier.
There are two types of point, great point.
And Greenpoint.
Some of the points that we added, we can click within this two dimensional space to create a great
point.
And if you want to create a green point, you press shift and then do almost click to add a green point.
So if you add Greenpoint near the boundary, you'll see that the classifier keeps on changing.
And if you scroll down.
There is this parameter C, which is the cost parameter.
As I told you, if you have less value of cost.
There'll be more mass classifications and more number of points will be present within the margins.
So you will have large margins if the value of seats decreased.
You can see this is what is happening.
If you increase the well, you'll see fewer points will be misclassified.
But our system will become very sensitive to individual observations.
So using this Tudi tool, you can easily visualize the effect of adding different types of points anywhere
in this base.
Plus, see the effect of changing?
Well, you'll see on the margin.
Good.