Hello and welcome back to the course on Machine Learning.
In today's tutorial, we will find out how we can take our Non-Linearly separable
data set, map it to a higher dimension and get a linearly separable data set.
Invoke the support vector machine algorithm, build a decision boundary
for a data set, and then project all of that back into our original dimensions.
So quite a lot to cover. Let's get started.
First off we're going to look at a simplified example.
We're going to look at a one dimensional data set.
So normally we visualize everything in the PowerPoint with two dimensions.
to make it, you know, look pretty.
And so that we can, kind of understand how it would work in multiple dimensions.
But right now, that will be a bit too complex for us to start with.
So we're going to start with a single dimension.
So here we've got the X1 dimension.
We've got some points here.
So we've got nine data points.
And as we can see they are non linearly separable.
So in a in this dimensional in a single dimension
dimensional space, a linear separator would not be a line would be a dot.
Right.
So in a two dimensional space, a linear, a linear
separator is a line, in a three dimensional space it's a hyperplane.
But in a one dimensional space it's a single dot.
So can we separate a separate the green from the red with a single dot?
Here.
No we cannot we if we put it here then these will be separated from that.
If we put it here will be separate from that.
So this is a non-linearly separable data set.
Now how can we apply
the method of increasing the dimensionality of this space
to make it a linearly separable data set in a higher dimension?
That's what we're going to look at.
And it might seem impossible. Right.
So the first time I learned about this, for me it was like, wow, how can you take,
a non-linearly separable data set and somehow magically increase
the dimensionality and you get a, linearly separable data
set that, you know, sounded like absurd, but it is actually possible.
And that's what we're going to see right now.
So we're going to create this mapping function on the fly.
So, let's say that this point over here is around five.
So, this is zero over here.
And then somewhere here we've got five. It doesn't really matter.
It can be any number.
But just for argument's sake, let's say that this point over here is five, right?
And then it keeps going.
So our first step to build the mapping function,
there can be multiple mapping functions that you can build.
I'm just going to show you one that came to my mind.
so the first step will be to go f equals x minus five.
So to subtract five from our data set.
And that is going to what is that going to do.
That is going to move everything to the left.
So basically, now, this is what, the result looks like.
So if you take five, you subtract five from x, you get, you know,
like these ones will go into negative, these ones will stay and positive.
And then the next step would be to, square all of that.
So f is now equals to x minus five square.
So how will that all look like.
Well basically you'll have this squared function
going through your, chart.
And then all of these will be projected onto the function.
there we go.
So that's what it looks like f equals x minus five squared.
And now what we want to do is we just want to see
that it is indeed linearly separable.
So there we go.
There's our linear separator.
So in a two dimensional space as we remember a linear separator
is a straight line.
And as you can see this data set
became linearly separable in this dimension.
I know it's it's surprising and even a bit shocking.
But indeed it is is the case.
So you can see that we were able to take this line and separate all of them,
red elements or other data set from the green elements, and that's it.
And then what we would do next from here is we would project everything back
onto our original space, and we would know how
to functionally separate, the green from the red.
And that is, what happens when you map something to a higher dimension.
So now knowing this example and seeing that it works in, reality,
we can proceed to a higher dimension, you know, to start
with a two dimensional space. So let's have a look.
So there's our two dimensional space.
And basically you would apply the same principle.
So here you can apply you kind of invoke the support vector machine algorithm
because it is not a non, linearly
separable data set in this space.
But then you would apply some sort of mapping function
like right now we won't go into detail what exactly mapping function it would be.
And again there could be multiple different options and so on.
But basically based on the previous example
we now know that it's possible, like we've seen, empirical evidence
that it is possible
to do the same thing applies
to two dimensional space moving into a three dimensional space,
you would map it into a three dimensional space, and then somehow
it would become a linearly separable data set in this space.
And here we've got the new dimension, which is Z.
And in a three dimensional space,
the linear separator is no longer a line, it's a hyperplane.
And so this hyperplane separates the two,
parts of our data set in the way we want.
So the support vector machine algorithm has helped us build this hyperplane.
And then basically so we've got this result that
we just projected back into our two dimensional space.
And we've got this, circle that encompasses our,
classes or separates our classes.
And there we go.
We've got the non linear separator.
So as you can see we can still even though we I've got a
a bit of a more complex problem where we cannot,
directly apply the support vector machine algorithm as we used to.
we can still go into a, higher, dimension
and then apply the support vector machine algorithm and, like,
we won't go into detail if it's possible all the time.
And if this case is one, it's not possible.
And so what do you do there?
But the point is that, there is a solution that you can
explore the, higher dimensions so that this is not a dead end.
You can just do that. But there's a problem.
The problem with this algorithm, and the problem is that mapping
to a high dimensional space can be highly compute intensive.
So it might require a lot of computation, a lot of, processing power.
And, you know, the larger your data set, the more, of a problem this can cause.
And therefore,
this approach isn't the best because you can imagine, like,
you have a data set and then, mapping it to a higher dimension,
performing all the calculations there, and then coming back to your lower
dimension that can,
even for a computer, not just like in our minds as humans,
but just like for a computer that can,
cause a lot of, delays.
It can cause a lot of, like, processing
backlog and issues in that sense.
And, we don't want that to happen.
And therefore, we're going to explore
something else we can explore, a a different approach,
which is called in mathematics, the kernel trick.
And that's, approach is going to help us perform
very similar, gets very similar results,
but without having to go to a higher dimensional space.
And we're going to talk about that in the next
tutorial, is going to be exciting, and I can't wait to see you there.
Until then, happy analyzing.