Hello and welcome back to the course on Deep Learning.
Today we're going to wrap up with backpropagation.
All right.
So we already know
pretty much everything we need to know about what happens in the neural network.
We know that, there's a process called forward propagation
where information is entered into the input layer
and then it's propagated forward to get our Y hats, our output values.
And then we compare those to the actual values, that we have in our training set.
And then we calculate the errors.
Then the errors are back propagated through the network
in the opposite direction.
And, that allows us to train the network by adjusting the weights.
So the one key important thing to remember here
is that backpropagation is an advanced algorithm driven by very,
interesting and sophisticated mathematics,
which allows us to adjust the weights all of them at the same time.
All of the weights are adjusted simultaneously.
So if, we were doing this manually or if we are coming up
with a different type of algorithm, then even if we calculate the error.
And then we were trying to understand what effect each of the weights has
on the error, we'd have to somehow,
adjust each of the weights in the pin independently or individually.
The huge advantage
of backpropagation, and this is a key thing to remember,
is that during the process of backpropagation,
simply because of the way,
the algorithm is structured,
you are able to adjust all of the weights at the same time.
So you basically know which part of the error
each of your weights in the neural network is responsible for.
Now that is the key fundamental
underlying, principle of backpropagation.
And, this was why it picked up so rapidly in the 1980s.
And this was the major breakthrough.
And if you'd like to learn more about that and how exactly the mathematics,
works in the background, then a good article,
which we've already mentioned
is the neural networks and deep learning is actually a book by Michael Nielsen.
There you'll find, the mathematics written out
and, it'll help you understand how exactly this is possible.
But for now, for our purposes, if from an intuition point of view,
the important part is to remember that, that's what, backpropagation does.
It adjusts all of the weights at the same time.
And now we're going to just wrap everything up with a step by step
walkthrough of what happens in, the training of a neural network.
All right.
So step one, we randomly initialized the weights
to small numbers close to zero, but not zero.
we didn't really focus on the initialization of weights
during the intuition tutorials, but, the weights have to start somewhere,
and they are initialized with random values near zero.
And from there, through the process of forward propagation backpropagation,
these weights are adjusted, until the error is minimized,
until the cost function is minimized.
then step two, inputs the first observation of your data sets.
So the first row into the input layer.
Each feature is one input node.
So basically take the columns and put them into the input nodes.
step three forward propagation from left to right.
The neurons are activated
in a way that the impact of each neuron activation is limited by the weights.
The weights basically determine, how important each neuron's activation is,
then propagate the activations until getting the predicted result
Y hat in this case.
So basically you propagate from left to right.
You go all the way until you get to the end.
You get your Y hat.
Then compare the prediction result to the actual result.
Measure the generated error.
and then you do the back propagation from right to left.
The error is back.
Propagate it.
Update the weights according to how much they're responsible for the error.
Again, you are able to calculate that because of the way that back propagate
that perturbation algorithm is structured,
the learning rate decides by how much we update the weights.
Learning rate is a parameter you can control in your, neural network.
Step six.
Repeat steps 1 to 5 and update the weights after each observation.
that is called reinforcement learning.
And in our case that was stochastic gradient descent or repeat
steps 1 to 5, but update weights only after a batch of observations.
So batch learning it's either, full gradient descent
or batch gradient descent or mini batch gradient descent.
And step seven, when the whole training set pass through the artificial neural
neural network, that makes an epoch, redo more epochs.
So basically you just keep doing that and doing that and doing that,
to allowing your neural network
to train better and better and better and constantly adjust itself,
as you minimize the cost function.
So there we go.
those are the steps you need to take
to build your artificial neural networks and train it.
And, these are the steps that you will be taking together
had learned in the practical tutorials.
Wish you the best of luck, and I look forward to seeing you next time.
Until then, enjoy deep learning.