Welcome to section 34 Principal Component Analysis, or PCA.
Before you get some hands on experience coming up with PCA,
in the practical exercises, we're going to take a look at the intuition behind it.
PCA is considered to be one of the most
used unsupervised algorithms, and can be seen
as the most popular dimensionality reduction algorithm.
PCA is used
for operations such as visualization, feature extraction,
noise filtering, and can be seen in algorithms used for stock market
predictions and gene analysis, just to name a few.
The goal of PCA
is to identify and detect correlation between variables.
If there's a strong correlation and it's found,
then you could reduce the dimensionality, which really what PCA is intended for.
You find the directions
of maximum variance in high dimensional data,
and then you project it into a smaller dimensional subspace while retaining
most of the information, usually again with PCA.
The goal to reduce the dimensions of a deep dimensional data set by projecting it
onto a k dimensional subspace where K is less than D,
and for a overall breakdown and wrap up of the PCA,
we can see here that the mean functions
of the PCA algorithm would be to standardize the data,
obtain the eigenvectors and eigenvalues
to then sort the eigenvalues in descending order.
Construct the projection matrix W
from the selected k eigenvectors
and to transform the original data set,
and you can explore it further if you follow that link.
But one thing I want to examine here
with PCA and I think the visualization is really helpful
if we visit the following link, it's going to take us to this page
where we can actually view it in 2D and 3D examples.
Now with PCA in a 2D, you can start to see the relationship
and how PCA is playing out amongst the variables in the data.
You can also on this site drag them
around, drag the data points around
to see the PCA.
The coordinates, adjust within the system.
But really what I think is helpful is the 3D example with the 3D example,
you can actually see the relationship, the data within this model
and comparing it to the 2D, you know, within the higher dimensional space,
obviously it can be a much easier visualization.
And I think this is helpful really to kind of grasp what PCA is doing.
And if we drag the data points around again,
just for a test, we can click on the show
PCA to reset that and we'll show it.
And we can see the PCA here.
We can actually move the model around.
Since it's not on that 2D plot, we can visualize it within the 3D example.
All right.
So to wrap PCA up short and sweet
PCA is not like linear regression, although it may look like it
because rather than attempting to predict the values, PCA is attempting
to learn about the relationship between the x and y values.
It's quantified by finding a list of principal axes.
And I think one of the best ways is to look at the visualizations.
You don't compare it to 2D and 3D dimension,
the analysis and the visualizations that we looked at before.
In addition, you know, on a side note, PCA does have a weakness.
It is highly affected by outliers in the data, but PCA is considered
to be one of the most used, and it's extremely popular to use.
And I think once you start working through the practical, it will make more sense.
If you have any questions, please let us know and enjoy machine learning.