Hi everyone, and welcome to the intuition Lecture
for Linear Discriminant Analysis, or LDA.
Now, for those of you who are coming from the previous section
on Principal Component Analysis or PCA, this may seem a bit similar,
but there is a difference between the two, and we're going to get started
by taking a look at overall what LDA entails.
It's a pretty brief and straightforward intuition lecture,
but we'll get to the main takeaways between PCA and LDA.
LDA is commonly used as a dimensionality reduction technique,
and we've heard that before with PCA.
It's used in the pre-processing step
for pattern classification and machine learning algorithms,
and its goal is to project a data set onto a lower dimensional space.
Sounds similar to PCA, but LDA differs
because in addition to finding the component axes with LDA,
we are interested in the axes that maximize the separation
between multiple classes, and that is the main takeaway.
Or the main point is where PCA.
We are with that distinction and working with the principal component analysis
with the axes, the principal components within the data.
But where as we're looking in LDA, we are looking
for the separation of those classes within the data
and to break it down further.
The goal of LDA is to project a feature space onto a small subspace
while maintaining the class discriminatory information.
And we have both PCA and LDA as linear
transformation techniques used for dimensionality reduction.
PCA is a unsupervised algorithm, but LDA is supervised
because of the relation to the dependent variable,
and we can see here from this visualization
the main operations and main differences between PCA and LDA.
PCA. Again, we're looking in that
the subspace and the dimensionality reduction technique of the data
to examine how the principal component axes are in relation.
Whereas in LDA we're looking for that class separation.
And I and I think this visualization kind of makes it the most clear
between the two.
If you want some additional information, you can always take a look
at the following link.
But here we have the PCA and LDA and the main operations related to each.
Again,
LDA supervised because of the relation to the dependent variable.
And I think when you start working through this
in the upcoming lecture, in the hands on part, it's going to make more sense.
But that's the main takeaway to focus on LDA.
And you can accomplish this by five main steps.
Similar again to PCA, the five main
steps for LDA include the following.
The computation of the d dimensional mean vectors.
The computation of the scatter matrices.
You have to also compute the eigenvectors.
Sort the eigenvectors by decreasing
eigenvalues, and use the d times k eigenvector matrix
to transform the samples onto the new subspace.
Overall, very similar to PCA, two different types of dimensionality
reduction techniques, one being unsupervised and one being supervised,
but the main distinction with LDA to take away
is that we're looking for that class separation within the data.
Overall, if you're coming from PCA, this should seem familiar
for the majority of operations.
If you're new to this, I advise you to go take a look at PCA as well.
But when you are starting to work through the next coming part,
it should make more sense.
But just keep in mind that the main takeaway for LDA
is that class separation and it is a supervised learning technique.
If you have any questions, as always, please feel free to share them
and enjoy machine learning.