Hello and welcome back to the course on Deep Learning.
In the previous tutorial, we found out what Convolutional
Neural Networks are all about.
And today we're going to dive into step one.
Can evolution.
So this is the convolution function.
I know we try to stay away from mathematics and keep things intuitive,
but I couldn't help but share this formula for you because it is so simple.
A convolution is basically a combined integration of two functions.
And it shows you how one function modifies,
the other, modifies the shape of the other.
And if you've done any signal processing or electrical engineering
or a profession where signal processing is required,
you would have inevitably come across the convolution function.
It is quite popular.
Now once again, we're going to, keep the mathematics lights or keep them separate.
And if you'd like to get into the math behind the convolutional neural networks,
a great additional read is, Introduction to Convolutional Neural Networks
by Jensen Wu, who is a professor at the Nanjing University in China.
This paper was published literally days ago, like 5 or 6 days ago.
And it is oriented specifically at people who are starting out,
at beginners who are getting to know convolutional neural networks.
So, the mathematics there should be accessible.
I actually emailed, a professor Shan seen Wu, and,
yeah, he said his whole goal is to
make, break, the complex things down
so that people who are new to this field can understand. And.
also, he mentioned that he's got some materials available on his homepage.
So if you in the URL,
if you just remove the last two parts and you just go to, like
to that part, that's his homepage and you'll be able to find
more additional tutorials and materials which haven't been published as papers.
But he uses them in his tutorials.
So, you might find those useful.
So browse around there if you'd like to get an introduction
into the mathematics behind convolutional neural networks and kind of,
build a solid. Base around that.
Area, but we're going to move on and, we're going to talk about the convolution.
So what is a convolution in intuitive terms
here on the left we've got an input image as we discussed.
that's how we're going to look at images.
Just ones and zeros to simplify things.
And you can see the smiley face there.
There we've got a feature detector.
So feature detectors a three by three matrix doesn't have to be three by three.
No it doesn't.
AlexNet. I think it uses, seven.
By seven.
and then some other, one of those other.
Famous ones uses like five by five feature detectors.
they can be different, but usually you'll see that there are three by three.
And, They are, you know, reasons to make them three by three.
So we're going to stick to the conventional way.
Having a three by three feature detector.
also the feature detectors called
these are important terms because you might come across them.
They're many different terms, for the feature detector.
But the most common ones are feature detector.
Or you might hear it being called kernel, or you might hear it being called filter.
So in this course we're going to be using either filter or a feature detector
interchangeably.
But just bear in mind that it has those names.
And A convolution operation is signified by an X
in a circle, just as you saw in the formulas before.
And here what happens is, on an intuitive level,
or just to think of it in terms of what is actually happening in the background
rather than the mathematics, well, you take this feature detector,
or filter and you put it on your image like you see on the left.
So you cover the, for instance, in this case
the top left corner, nine pixels in the top left corner.
And you basically multiply
each value by each value, so respective values.
So, the top zero by the top left go value by the top left value.
Then basically it's position number.
One, one by position number one.
One position by a number or zero, one by zero, one zero, two by zero two and so on.
So just it's, element wise multiplication of these matrices
and then you add up the results.
So in this case nothing matches up.
So always always either a zero by zero zero by one.
So the result is zero.
Here you can see that one.
Of them matched up the one on the left.
Matched up. And therefore we got a one here.
Nothing matched up. Nothing matched up, nothing matched up.
Then we move on to the next row.
So, and the step at which we're moving
this whole, filter is called the stride.
So here we have a stride of one pixel.
So here you can see again something matched up the bottom right corner.
Matched up.
Against stride.
But Bottom one in the middle matched up.
here. Top right, one matched up.
Then nothing. Mentioned. The stride is one.
you can change the stride.
you can make it one, two.
you can make it three. Whatever you like.
the conventionally the one that works well is usually a two.
So that's what people stick to.
And we'll we'll talk about what the stride is, towards the end of this tutorial.
So here we've got, we're matching up.
So we just keep our eye here.
You can see we've got a two because two of them matched up.
And so on and so on, so on.
there we go. There's another one that matched up.
There we go.
And there we're done.
So what's what have we created?
Right.
a couple of important things here.
the image on the right is called a feature map.
Also has several terms.
It also can be called sometimes, it convolved feature.
So when you apply a convolution operation operator to something,
it doesn't become convoluted.
It becomes convolved.
And yeah, I use sometimes I like I.
Think to myself in the.
Wrong way, but it's the correct term is convolved.
it's a kind of old feature.
Or it can also be called the activation map,
but we're going to be calling it a feature map in this course.
so it can be called it.
Any one of those things.
And what have we done here.
Well, as you can see, we've reduced the size of the image.
That's number one.
And that's the important thing I wanted to mention about,
your input image and the feature detect and the stride.
Right.
If you have a stride of one, you can see the image reduced a bit.
But if you have a Strided two, the image is going to reduce more.
So the feature map is going to be even smaller.
And that's an, a very important, function of the feature
detector of this whole convolution step is to make the image smaller.
because that will be it'll be easier to process it.
and. it'll be just faster.
It will and,
It'll be just faster because imagine like here
we've got a what, a seven by seven image.
But imagine if you have a proper photo, right.
or you have a, like a 256 by 256 pixel image.
That's it's a huge number of pixels by 256, squared.
or like let's say you have a 300 by 300 pixels.
So, so we don't get confused with the RGB 256.
Let's just say we have a 300 by 300, image in terms of size in pixels.
Then you have 300 squared number of pixels.
That's a huge number.
and therefore feature detectors, will reduce the size of the image.
And therefore stride of two is actually beneficial.
But then the question is do we lose information
or are we losing information when we're applying the feature detector.
Well, some information we are losing
of course, because we have less values in our resulting matrix.
But at the same time, the purpose of the feature detector is to detect certain
features, certain parts of the image that are integral.
And so, for instance, if you think about it this way,
like the feature detector has a certain pattern on it,
the highest number in your feature map is when that pattern matches up.
In fact, the highest number you can get is in.
Another simplified example is when, the feature is that it matches exactly.
And you can see with that number four we have in our feature map.
That's exactly.
So if you look over here, that's exactly where this feature detector,
because there's only four ones in it matched perfectly.
So you can see this this part over here.
So the feature was detected here.
And as we discussed at the very start of this section
that features is
how we see things, is how we recognize things.
We don't look at every single.
Pixel, so to speak,
in what we see on an image or in real life.
We don't look at every single pixel.
We look at features, we look at the the nose, the hat, the the feather.
the, the, eyes under
or the little black marks under the cheetah's, eyes to distinguish
it between a cheetah and a leopard or the shape of the train.
We don't, to distinguish between a bullet train, a normal train, and so on.
So we don't look at everything,
we look at features, and that's what we're preserving.
And that's what the feature map helps us preserve.
Actually, that's what it it's, allows us to bring forward
and get rid of all of the unnecessary things that even as humans,
we don't processes so much information.
going into your eyes at the, at any given time, like gigabytes of information,
if you look at every single, dot, if not terabytes of information
going into your eyes per second, and still we're able
to process that because we get rid of what is unnecessary.
We only focus on the important features of features that are important to us.
And, that is exactly what the feature map does.
So now moving on.
This is our input image.
And you we create a feature map.
So the front one let's say the front one is the one we just created.
But then how come there's many of them.
But we create multiple feature maps.
because we use different filters.
Right.
And that's another way.
That we preserve lots of the information.
So we don't just have one feature map, we look for certain features
and then, or basically the network decides through its training.
And this is something we'll discuss towards the end of this section.
Through its training, it decides which, features
are important for certain types or certain categories, and.
It looks for them, and therefore.
We'll have different filters. And we'll talk about filters just now.
But basically it'll apply these filters.
So to get this feature map it applied a filter like the one we saw.
But then to get this feature map but apply a different filter
to get this feature map to apply a different filter and so on.
and. So
basically it just creates these feature maps.
And actually that's why personally I think the term feature detector
is better than filter.
So remember over here
we have this filter which we also can call a feature detector.
Well actually the word feature detector I think is better suited.
And the reason for that is that's what the purpose is, right?
We don't want to just we don't want to just filter out our image.
But even though that's the whole
that's the same same just a question of terminology.
But basically we want to detect features. All right.
In this in this layer we're going to our in this.
Feature map we've detected where certain features are in the image.
In this feature map we've detected where certain
other features are where a certain specific feature is located.
And this feature map we've detected where
a certain other feature is located on the image.
So that's, that's what we were doing.
And let's have a look at a couple of examples.
So, here, we're using and this is, from GitHub, dawg.
They're documentation.
It's a free, like a kind of tool, like paint.
And you can.
Use it to adjust your images or work with your images, but basically they have
some valuable examples in their documentation.
And here they have a picture of the Taj Mahal.
And you can choose which filter you want to apply.
So if you download this program and you upload a photo into it,
and then you can actually, start a convolution matrix
and apply filters, and you will see that, these things,
these convolution matrices are actually applied in image processing and.
Design and so on.
So let's have a look at what we get, what we get.
So so if we apply this filter five in the.
Middle of minus one minus one minus one minus.
One, you can see that it's sharpens the image.
And yeah.
So this is, it's quite intuitive if you think of it.
So five is the pixel the main pixel like in the middle of the,
of the filter or the feature detector.
And then minus one, minus one and minus.
One just to kind of like reduces the pixels around the inner in an intuitive.
sense.
then blur.
So basically it takes, c equal significant
gives equal significance to all of the, pixels around the one in the center.
And therefore it combines them together and you get a blur edge enhance.
So here you can see that.
minus one and one and then.
You get zeros. Right.
So you did delete remove the pixels around, the main one in the middle.
And you only keep this. One at a minus one.
And it gives you an edge.
And this was a bit harder to. Understand.
How it works.
Like probably 100, just to think of it intuitively.
edge detect. Right.
So this one probably makes more sense. Right?
You, take the middle one, you reduce the middle one.
The probably like the strength of the middle pixel.
And then you look for the ones you look for.
these ones you,
increase the strength.
Of the ones around them.
So you have. The ones. There.
yeah. So that's that.
Gives you, like, an edge detection, and you can see what you get there and.
Boss, another one. So.
the, the key here is that it's asymmetrical.
And you can see the image becomes asymmetrical as well.
So you.
Got like that kind of, feeling that it's standing out.
Towards you.
And that's what you get when you have like minuses here.
And pluses here.
Again, this is very this is getting a bit technical now.
But at least we can get some kind of intuitive understanding.
Let's just go quickly through them again.
So there's sharpen. There's blur.
There's edge enhance. There's edge detect.
There's emboss.
And so as you can see these are great examples of the same image.
But we're getting feature maps.
So we use different
feature detectors to get different feature maps of the same image.
And therefore now we have lots of the lots of this versions of this image.
where in each one we've tried to detect certain
things, these terms, they're not applicable to us.
They're we can say like emboss is probably not applicable to us
in terms of convolutional neural networks, but edge detect, that's important.
We want to detect the edges.
Edge enhance probably not blur sharpen.
So certain things like edge detectors probably the most important one
for our type of, work.
And in terms of understanding like computers,
they will decide for themselves.
The neural network will decide for itself what's important, what's not.
And it probably won't be even, recognizable to the human eye.
You won't be able to understand what those features
mean, but the computer will decide.
And that's the beauty that, of neural networks,
that they can process so many different.
Things and understand without.
Even having that intuition, without having that,
explanation why they will understand which features are.
Important to them, whether we have a name for them or not.
That's a whole that's an irrelevant question for the artificial neural
network.
And my favorite one, here's a image of Geoffrey Hinton.
photo of Geoffrey Hinton.
passed through, the, one of these filters.
All right, so that brings us to the end of today's tutorial.
I hope you enjoyed learning about convolution.
The key takeaway is that, convolution, the the primary purpose of a convolution
is to find features in your image using the feature detector,
put them into a feature map, and by having them in a feature map,
it still preserves the spatial relationships,
between pixels, which is very important for us to, you know,
because if they're completely jumbled up, then we've, we've lost the pattern.
And at the same time, it's important to understand that most of the time,
the features a neural network will detect and use to recognize certain images
and classes will mean nothing to humans, but nevertheless, they work.
And that's what convolution is.
And I look forward to seeing you on next tutorial.
Until then, enjoy deep learning.