Hello and welcome back to the course on Deep Learning.
Today we're finally at step number four.
Full connection.
So what is this step all about?
Well, in this step, we're adding a whole
artificial neural network to our convolutional neural network.
So to all of the things that we've done so far which are convolution
pooling and flattening, now we're adding a whole new a.
And then on the back of that.
How intense is that.
That is just that is something that is definitely something.
And so here we've got the input layer.
We've got a fully connected layer an output layer.
And by the way the fully connected layer
in artificial neural networks we used to call them hidden layers.
And here we're calling them fully connected layers
because they are hidden layers.
But at the same time, they're a more specific type of hidden layers.
They're a fully connected layer in artificial neural networks,
hidden layers don't have to be fully connected.
Whereas in convolutional neural networks we're going to be using fully connected
layers.
And that's why, they're generally called fully connected layers.
And so basically that's whole column or vector of outputs
that we have after the flattening we're passing it into the input layer.
And here we've got a very simplified example.
just for illustration purposes.
And what the main purpose of the artificial neural network is, is to
combine our features into more attributes that predict the classes, even better.
So we already in our vector of outputs,
in the flattened and often the flattened result.
From what we've already done,
we have some features encoded in the numbers in that vector,
and they can already do probably a pretty good job at predicting,
what's what's, class we're looking at, whether it's a dog or a cat
or whether it's a tumor or not, a tumor and so on.
But at the same time, we know that we have this,
structure called artificial neural network, which is designed which,
which has a purpose of dealing with attributes
and coming up or dealing with features and coming up with new attributes
and combining attributes together to even better predict,
things that we're trying to predict.
And we know that from the previous part.
So why not leverage that?
And that's exactly what the plan here is.
So how about we pass on those values into an artificial neural network
and let it even further optimize everything that we're doing.
And so that's what we're going to be doing.
But let's look at a more realistic example.
Because this one is a bit too simple.
So here we've got a better looking artificial neural network
where we have five attributes on the inputs than we have.
in the first hidden layer we have six neurons.
In the second, or in the second fully connected layer we have eight neurons.
And then we have two outputs, one for dog and one for cat.
And so an important thing to,
talk of for us to talk about here is that why do we have two outputs?
We are kind of
used to having only one output, in our artificial neural networks.
Well, one output is for kind of when you're predicting a numerical value,
when you're when you're running a regression type of problem.
But, when you're doing classification, you need an output per class,
except for the exception is when you have just two classes,
like we have two classes here, dog and cat.
And we could have just done one output and made it a binary output
and said one is a dog and zero is a cat.
And that would have worked totally fine.
And actually, in fact, you'll see had learned
do that in the practical tutorials and that's how they'll be structured.
But at the same time, if you have more than, two categories,
for instance, dogs, cats and birds,
then you have to have a neuron for every category.
And that's why we're going to practice with two categories.
In this example,
so that we know what to expect if we ever have more than two categories.
And so what's going to be happening here.
So we've already done all the groundwork.
We've done the convolution.
We've done the pooling and the flattening.
And now the information is going to go through the artificial neural network.
So let's have a look at how all that all happens.
There's the information going through from the very start from the moment
when the image is processed and kind of a loop convolved,
then pooled flattened and then through the artificial neural network,
all four steps
and then a prediction is made and we'll see how this happens in a moment.
It'll be very, very interesting.
But for now let's just say a prediction is made.
And for instance, 80% that it's a dog, but it turns out to be a cat.
And then an error is calculated like, well,
what we used to call a cost function in a artificial neural network,
and we used a mean squared error there, or, in common,
delusional neural networks, it's called a loss function.
And we use a, cross entropy function for that.
And we'll talk about cross entropy and mean squared errors
in a separate tutorial.
And how all that happens.
But for now, let's just say we have a, lost type of function which tells us how
well our network is performing and we're trying to optimize, optimize it
or minimize that function to optimize our network.
So we the error is calculated and then it's back propagated through the network,
just like we had in artificial neural networks, is back propagated.
And the some things are adjusted in the network
to help optimize the performance.
And the things that are adjusted are, as usual,
the weights in the artificial neural network parts.
So the the blue lines that you see here, the synapses.
Then also another thing that is adjusted
is the feature detectors.
So we know that we're looking for features,
but what if we're looking for the wrong features.
What if this didn't work out because the features are incorrect.
And so the feature detectors, those remember those little matrices
that we had, that's the three by three matrices.
they are adjusted so that maybe
next time it'll be better and let's see what happens type of thing.
And but of course, it's all done, with a lot of,
science in the background of a lot of math
and it's all done through a gradient, a gradient descent with, backpropagation.
So it's all it's all not just random perturbations.
It's actually very, thought through how it's done, but,
nevertheless, the, feature detectors are adjusted,
the weights are adjusted, and this whole process happens again.
And then again, the errors back propagate.
And this keeps going on and on and on.
And that's how our network is optimized.
That's how our network trains on the data.
So the and the important important thing here is that
the data goes through the whole network
from the very start to the very end, then the errors compared.
so the error is calculated and then it's back propagated.
So same story
as with artificial neural networks, just a a bit longer because of that whole
for the first three steps that we already had.
And now let's have a look at the the interesting part,
the really interesting part.
How do these two classes work.
Because or how do these two output neurons work.
Because before we've always kind of had one output neuron.
What happens when we have two.
How does how does this situation of classification of images, play out?
Well, let's start with the top neuron.
First we're going to start with the dog.
how do we the main purpose, what we need to do
first is we need to understand what weights to assign
to all of these synapses that connect to the dog, so that we know
which of the previous neurons are actually important for the dog.
And let's see how that is done.
So let's say hypothetically, we've got these numbers in our, previous,
layer of previous fully connected in the final fully connected layer.
And again, these numbers can be absolutely anything.
They don't have to be that they they can be any numbers, but
just for argument's sake, we're going to agree that
we are looking specifically at numbers between 0 and 1.
so it's easier for us to argue these things and understand.
And one means that that neuron was very confident
that it found a certain feature.
And zero is going to mean that,
that neuron didn't find the features looking for.
So because at the end of the day, these neurons,
like anything
else on this from on this left side is just looking at features at an image.
This is already very, very processed, but still it's detecting a certain feature
or combination of features on the image right before we in the convolve step,
we had kind of recognizable features in the pool set.
They're less recognizable.
Then they become even less recognizable, visible
in the flattened image, and then they get combined and so on.
But nevertheless, this we're talking about here,
certain features that are present image or their combination.
So a one which is being passed and this is important is being passed to
both the dog and the cat at the same time to both the output neurons.
So one means that for us, for our argument
it means that, this, this neuron has is firing up.
It's, it's really rapidly detecting that feature that,
you know, might be an eyebrow it might be detecting this eyebrow
for again, for simplicity's sake, is detecting this eyebrow
and is communicating that to the dog neuron to the cartoon
neuron saying I can see my eyebrow, I can see my eyebrow.
And then it's up to the dog
and the cat neuron to, understand what that means for them.
Right.
And so in this case, which neurons are firing up,
these three neurons are firing up the eyebrow and let's say the nose
is saying, I can see, I can see a big nose and I can see floppy ears.
So it and it's saying that to the dog and to the cat and then what, the dog.
And then what happens is we know that this is a dog.
So the dog neuron knows that the answer is it is actually a dog.
because at the end we're comparing to the picture
or to the label on the picture, and it knows the dog.
So basically the dog neuron is going to say So
I should be triggered in this case.
So these are my neurons.
They're telling the signal that they're sending to both to me to the dog
and to the cat is actually a indication for me that it is a dog.
And throughout these lots and lots
and lots of iterations, if this happens many times, the dog will learn that
these neurons do indeed fire up when the feature belongs to a dog.
On the other hand,
the cat neuron will know that it's not a cat, and it will know that
this feature is firing up and this neuron is telling me it can see
floppy ears, floppy ears, floppy ears, but at the same time, it's not a cat.
So basically, to me, that's a signal that I should ignore this neuron as like.
And the more that happens,
the more the cat neuron is going to ignore this neuron about the floppy ears.
And so basically, that that's
how through lots and lots of iterations if this happens often.
So this is just one example.
But if this happens often
maybe a one, maybe 0.8, 0.9, maybe sometimes it won't fire.
But overall on average this neuron is lighting up
very often when it is indeed a dog.
The dog neuron will start attributing higher importance to this neuron.
And so there we go. That's that's how we're going to signify it.
We're going to say that
these three neurons through this iterative process
with matter, with many, many, many, many samples and many, many epochs, remember.
So a sample is a row in your data set.
An epoch is when you go through your whole data set again and again and again.
There are lots and lots of iterations.
This dog neuron learned that this, eyebrow neuron
and this big nose neuron and this, floppy ear neuron,
they all seem to really,
contribute very well to the,
classification of what it's looking for and which is a dog.
So that's how it works.
And again, these ears and nose and eyebrows, those are very,
very, approximate or like very far fetched examples
because by this stage, in this whole convolution convolutional neural network,
it is completely unrecognizable what they're looking for.
But at the same time, it is something in the features of dogs
or cats or whatever you're classifying.
And then so let's move on to the next one.
Now we're going to look at the cat neuron.
But these we're going to remember that these weights are you know they have
that we've sorted out the dog.
So the dog is kind of like
pretty much ignoring all these other neurons one, two, three, 4 or 5.
But it's really paying attention to what these three neurons are saying.
Now, what is the cat listening to?
Well, whenever it is actually a cat, right.
the this is this is an example of a situation when it's actually a cat.
So, you'll see that this, these three neurons, 0.9, 0.9 and one,
they're saying something, they're saying something to both the dog and the cat.
And this is again important to remember.
So this output signal goes both ways.
It's the same. Right.
It's it's saying one to the dog is saying one to the cat,
but then it's up to the dog, to the cat to decide to whether to,
take into account that signal and learn from it or not.
And both the dog and the cat can see that this is a photo.
I should have put a photo of a cat here, but basically imagine a photo of a cat.
Both the dog and the cat can see that this is actually a cat.
So basically the dog is like, oh, okay.
So these whiskers and these pointy triangle ears
and this small size, I guess, or I don't know, oh,
maybe these, this type, you know, how cats have, these things in the eyes?
Their eyes are like little.
They're not circles. They're,
lines or something.
I got, like, cat eyes. Basically. These cat eyes.
They're definitely not working for me.
They're not helping me out. Predict.
Because every time these neurons light up,
the prediction is not what I'm looking for.
On the other hand, the cat is like, that's interesting.
Every time these this one lights up, it's or most of the time
it lights up, it matches my expectation.
It matches what I'm looking for.
Okay, I'm going to listen to this guy more than this one.
This one.
Same thing every time it lights up.
Or most of the times it lights up.
I happen to get a good
I happen to be rewarded for my, prediction because I get it right.
It's a cat. Okay, so I'm going to listen to him more.
You know, this one useless to me, because he's not actually,
you know, like, he's, he's not even lighting up.
It's a cat, but it's he's not lighting up. So the opposite is happening.
And this one as well.
It's a cat, but he's not lighting up, so I'm not going to listen to him.
But this one, when he.
When, What?
What is this? The eyes. The cat eyes light up.
We can see. I can see that it's a cat.
It matches most of the time. So I'm going to learn from that.
And I'm going to listen to these three guys, more often than not.
And so basically the cat is listening to these three,
and it's ignoring the other five.
And that is how these final neurons
learn which neurons in the,
final fully connected layer to listen to.
So the output neurons learn which of the fully or
which of the final fully connected layer neurons to listen to.
And that's how they understand.
And basically that's how the features are propagated
through the network and conveyed to the output.
And so even though these, features, of course, don't have that much meaning
to them, like floppy ears or whiskers,
at the same time, they do have some distinctive
they are a distinctive feature of that specific class.
And that's how the network is trained, because we also during remember,
during the backpropagation process, we also adjust the feature detectors.
So if a feature is useless to the output, it's going to it's going
to probably be disregarded because this doesn't happen over 1 or 2 days.
This happens through thousands and thousands of iterations.
So, with time, a feature that is useless to
the network is going to be disregarded and replacement feature is useful.
And so at the end of the day, in this final layer of neurons,
you are likely to have lots of features or combinations of features from the image
that are indeed representative, or descriptive of dogs and cats.
And so then
once your network is trained up, then we this is how it's applied.
So this is the next step.
Like we've already trained up our network when this happened.
This is what happens when the, this network is applied.
So let's say we pass on an image of a dog,
the values are propagated through a network.
We get certain values.
And so this time the dog and the cat neurons don't know.
They don't have the image of the dog here.
They don't know that it's a dog or a cat.
They have no idea what it is, but they have learned
to listen to what is being shown here.
Right?
They have learned to listen to the dog ear and listen to these three neurons.
The cat neuron listens to these three.
And so the dog neuron looks at one, two, three and says, oh, these are pretty high.
So my probability is going to be high.
That is a dog.
The cat neuron looks at these three and says, okay these
this one is pretty high but these are pretty low.
Interesting.
So my probability is going to be 0.05.
And and then and that's and that's where you get your prediction.
So then your first choice for this neural network is dog.
Second choice is cat. And that's pretty much it.
So the answer is dog.
And same thing happens when you pass an image of a cat.
you get new values and you can see that even though this one's higher,
these ones are low.
And for the cat, this one's higher, this one's high, and this one's a bit low.
So the probability here might not be as great as previously,
but still you can see that it's a cat of 79%.
And so therefore the neural network is going to vote that it's a cat.
And so basically or the neural network is going to conclude that it's a cat
voting is a term that is used for these guys.
So these neurons in the final fully connected layer, they get to vote.
And these are their votes.
And again we are just for argument's sake putting values between 0 and 1 here.
These could be any values.
But they get to vote.
And then these weights are the importance of their votes.
So this is these these purple weights are how the dog neuron views their votes,
how much importance it assigns to these neurons and to those votes.
And this is how much importance the cat, neuron assigns to these,
those to the votes of these neurons.
And so these neurons vote the dog in the cats based on their learned weights.
They decide who to listen to, and then they make their predictions.
And then the whole neural network concludes that this is, in this
case, a cat.
And then that's, and then that's your conclusion, and that's
how you get images like this where you have, a cheetah
and then you have a, cheetah class with, you know, like a high, high probability.
So this is, you know, the probability that the network has predicted
and these are lost,
but these still exist because there's still kind of like a small chance
that other neurons are also listening to their voters and they're saying,
oh, maybe it's actually a leopard and the bullet train very, very probably.
Here. Scissors.
You know, this one one.
But hand gloss was very close. Second.
And then finally, best, stethoscope because you could see, like these guys,
this, this neuron, the scissors neuron, the output services, this neuron listen to
its voters.
And it had the predominant vote overall.
But then the hand class had a good outcome as well.
So there we go.
That's how the full connection works and how this is all this
all plays out together.
I hope you enjoyed today's tutorial.
We're going to summarize all of this in the summary as well,
and I'll see you next time.
Until then, enjoy deep learning.