Welcome to part two of data pre processing.
We've done the first couple of steps of the workflow and we've ended about here in the last lesson where
we just cropped the image.
Now what we're gonna do is we're gonna figure out how to resize our image.
We're gonna make our image smaller namely we're gonna size it down to around 20 pixels on the long end
and then we're gonna add some padding around it so that by the end we have a 28 by 28 pixel image.
So let's have a think about the resizing.
If I've got my website loaded up here and I draw something on this canvas See I draw a long and narrow
digit like this.
So I want to draw like it to hit and I click on check answer that I've got my long and narrow image
here.
But if I draw a viewer short and wide to then it will look something like this.
That's the output of the current code that we've written.
So if you look at these two images and you had to think about how to shrink these two images how would
you go about it.
How would you get the largest possible number two on the shrunken image after resizing.
So before we're gonna dive in and write our code let's figure this out the way that the endless data
is structured is that the digit at the center of the image has to fit into a 20 pixel by 20 pixels square.
So what this means is that the long edge of say a tall and narrow digit should be around 20 pixels.
We would want the height of this tall and narrow number to to be 20 pixels but for a short and wide
digit like this one here we would want the width to be 20 pixels.
In other words in order to get the largest possible image into that 20 by 20 pixels square we would
have to make sure that the long edge is going to be 20 pixels in length.
Now what about the short edge.
How many pixels should the short edge be.
So what I want to do is I want to go through a quick example to nail down the logic before we write
our code.
So say we have this number two right here that we've drawn on the canvas.
It's 100 pixels tall and 40 pixels wide.
And what we want to do is we want to shrink it down to 20 pixels.
So how wide should this image be.
After shrinking it down.
Now I know that's not a terribly complicated question.
And just by looking at these numbers you can probably tell what the answer should be.
But what would the code look like to work this out to work out the new width.
What we need to actually know is the ratio of the current height to the target height.
I'm going to call this a scale factor.
Our scale factor in this case would be one hundred divided by 20.
So in this case the scale factor is equal to five.
And what this means is that the new width should be equal to 40 the old width divided by the scale factor.
All right.
So that's the logic that we're going to incorporate into our code but how do we actually scale images
using open CV for that we're going to look at the documentation in particular we're to look at this
page here on geometric transformations of images in fact the very first transformation on this page
is already about scaling and the function here is called resize and it takes a couple of different inputs
one two three four five six different inputs to be exact.
So what are the parameters here.
What are the things that we need to pay attention to.
The first two we've already encountered we have to provide an input and we have to store the output
somewhere.
But what about these ones here D size effects an F Why.
What are these.
In reading this documentation him the key thing to realize is that there are actually two ways you can
use this resize function.
Either you provide these scale factors so this affects an F Y or you provide a value for D size if you
provide the scale factors so you give a value for F X and if y then these sizes worked out automatically
and if you do it the other way round if you provide the target size if you will then the scale factors
are worked out automatically by this function.
And it says here when you want something to be worked at automatically then you just give it the value
zero.
Now let's look at the code snippet below on the documentation what we see here is that they're actually
giving a value for these sites.
So they're creating some variable designs and they're sending an equal to three hundred by three hundred.
So what they're going to do in this code snippet is resize this picture here to three hundred by three
hundred.
The scale factors are worked out automatically by leaving them as zero.
So if I click try then the end result here is a three hundred three hundred size picture.
So in our code let's do the same.
Let's work out what the D size should be and then implement this resize function and to do that we first
need a width and a height so coming into our code what is the width and the height gonna be.
So for the height c var height it's gonna be equal to the number of rows in our image.
Right so image dot rows it's gonna be equal to our height.
Forget a semicolon here.
What about the width in this case.
It's gonna be the number of columns in the image so far width is equal to image dot calls now at this
point I'd like to pose a challenge to you.
Can you write the code to resize this image.
Now remember the long edge has to be 20 pixels long and the short edge needs to be worked out and you're
gonna need to round the short edge to the nearest integer so you're gonna have to do the calculations
slightly differently for the tall and narrow images and the short and wide images you get to make sure
they both scale down properly and the long edge of either of those two is going to be 20 pixels.
Now I recommend you use this documentation right here and you have a bit of a Google on how to round
numbers in JavaScript.
I'll give you a few seconds to pause the video before I show you the solution.
Have a go
all right.
Ready.
Here's my solution.
There's a couple of different ways you could have written your code.
But here's my simple solution for you.
I'm going to be using an if else statement if my height is greater than the width.
In other words I have a tall and narrow image.
What I would like to do is I would like to set my new target height equal to 20 and I'm going to calculate
my scale factor.
So const scale factor is equal to the old height.
So that's image dot rows divided by my target height.
That's equal to height which I've just said equal to 20.
And my new width my target width if you will.
It's gonna be equal to math dot round in parentheses the old width so image dot calls the number of
columns divided by my scale factor so I'm doing this calculation and then rounding it to the nearest
integer.
And with that I have my new height which is equal to 20 so that's my long edge and my short edge is
gonna be worked out from the old width divided by the scale factor that I've worked out on the line
above.
Now what if I've got a short and narrow image in that case I'm going to have this ls class trigger and
here I'm know that my width should be equal to 20.
So that's the long edge but my scale factor in this case is gonna be equal to image dot columns in this
case divided by the width and that of course leaves the height to be equal to the rounded value.
Now I thought round of the old height image dot rows divided by the scale factor.
There we go.
Now what we can do is create this size object here.
So just like in the documentation we can create a new size object with something like let your size
is equal to new c v dot size.
Capital S there and for the pixel values for the width and the height we're gonna use the width and
the height that we've just worked out above.
So it's gonna be with come on height now that we've got our new signs that we want we can call this
resize method here and we can do that with CV dot resize and our input is once again our image our output
is going gonna be our image.
So we're going to update it and then we're gonna provide this D size argument here.
And for us I didn't call it D size.
I called it new size.
I think that's more descriptive.
What about f x and f y.
Well we're gonna provide the value 0 for both of those because they're gonna be worked out automatically
and then for the interpretation that last parameter we're gonna go with the same thing they're using
here in the code snippet CV dot into area brilliant.
So now that we've got that let's save and let's come back in here and refresh our site and let's draw
something like a very very narrow number two.
And now let's draw a very very wide.
Number two as you can see my changes aren't being applied at all.
So I'm just gonna go to more tools clear browsing data and just make sure that the newest version of
my web site is indeed being served up here.
So let's try again refresh and try.
OK.
So there we go.
Now you see a tiny tiny 20 pixel tall number to appear here.
And if I make a very wide number two then you get something a bit more square looking wonderful.
So that works if you can't see this.
I can zoom in to uh maybe 300 percent or 500 percent and you can see that in this corner here that we've
indeed scaled this down to 20 pixels.
So it's absolutely tiny and you're probably only going to be able to see it very well on your own screen.
We've resized our image.
Now it's time to add the padding.
Now what will adding the padding do it's gonna make our image nice and square.
No matter what aspect ratio it was previously in particular for that very very narrow number to where
we've got that long edge on the y axis what adding the padding will do is it will square it out so that
it's going to be 28 by 28 pixels.
And what this means for us is that we have to work out how much padding to actually add.
So we have to work out actually four values the top the bottom the left and the right.
So let me ask you a question.
I've got the padding on the left right here marked in green.
How would you work out the length of the Green Line.
Well it's going to be four plus the length of this segment right here.
Right.
What's that equal to.
Well it's going to be 20 which is this distance right here minus the width of the actual digit divided
by two because there is one segment only right on this side and this segment here belongs to the padding
on the right hand side so the full equation for the padding on the left side would be four plus 20 minus
the width of the image divided by two.
So let's head back into v as code and add four constants here.
Let's start out with the left one the left one we said was gonna be equal to four plus twenty minus
the width divided by two.
And the same was true for the right hand side right.
So I'm just going to copy the line pasted below.
Change this to right and then I'm gonna paste that line again and I'm going to work out the top value
so at top it's gonna be four plus twenty minus what not the width right the height divided by two and
for the bottom value it's gonna be pretty much the same as the top value right.
I'll just change the name of the constant.
Now this looks like it could work but what if the width is equal to five pixels if the width is equal
to five pixels then we get twenty minus five divided by two.
So that's 15 divided by two and we have seven point five.
We have a fractional pixel so let's make a small modification here let's around the left value up and
let's round the right value down so that we have integers for the putting value on the left and the
right in order to round something up.
We're gonna use that math functionality right.
And math has a function here called Seal for sealing which rounds up and as you can imagine math also
has a function called flaw which rounds down so ceiling four up and floor four down and we'll do the
same sort of alternation on the top and the bottom.
Now that we've calculated these constants let's well let's put them to the console.
Let's take a look at what these values actually are.
So when I had a log statement here so we can look at them every time we click the button and I'm going
to use some string interpolation here.
So when I use that back tick and the dollar signs and the curly braces in order to write this log statement.
So the top is going to be equal to dollar sign.
Curly braces top and then I'm just going to add the bottom the Left the Right.
So we have them all in the log statement.
Now that's all done.
Let's save our changes come back refresh and let's take a look.
So if I draw this tall and narrow number two then at the top and the bottom I'm only adding four pixels
of padding.
But on the left on the right I'm adding 10.
That makes sense right.
And if I draw something like this then I should get a very different look.
So top and bottom I'm adding 10 and 9 pixels.
So here you can see that rounding in action and on the left and on the right I'm adding four because
that's my long edge.
Would you like to venture a guess what the values are going to come out at.
If I draw a number one well in that case I'm going to be adding a lot of padding on the left and the
right because it's such a narrow digit.
All right fantastic.
So we've worked out these values we know what the padding should be.
But how do we actually add padding using open CV.
So when I throw this over to you as a challenge once again I'd like to ask you to find the correct function
in the open CV documentation and implement it in your code.
I'll give you a few seconds to pause the video and have a Google make sure you look for the javascript
part of the open CV documentation and yeah just have a go show the solution in a few seconds
ready.
Here's the solution.
The page where the function is listed is actually under basic operations on images.
I can actually search this page for padding and you'll find that there is a section here called Making
borders for images parentheses padding and the way it works is given by this code snippet right here.
If I click try it you can see that a border was added to this image and this border can act as a padding
value.
Let's look at the parameters for this copy.
Make border function.
We've got a source which is our input.
We've got the top bottom left and right which we've just calculated.
That's the border width and the number of pixels corresponding to the directions.
Then we've got a border type.
So we want a constant colored border.
But of course we don't want a red border like they've have in the code sample here.
What we actually want is a black border.
Now where does the color actually come in here.
If we have a look at this documentation it's not that obvious at first.
But if you look at the code sample you can see this line right here.
Two hundred and fifty five 0 0.
Two hundred and fifty five and that's stored in this value called S and it comes in right here at the
end after the constant border.
So part of this challenge was actually figuring out that this here this thing called scalar is actually
the color.
This isn't our GDP a.
So red green blue Alpha format.
So the reason why this border is red is because it's got the value 255 for red.
And the reason you can see it so well is because it's opaque it's not transparent.
So that's why there's a two hundred and fifty five value here.
So this thing here is actually the color.
Now what would the scalar look like for us.
We don't want a red color after all.
Right.
We want a black color.
So in that case this value would be zero.
And while this value would be zero as well so let's call this function in V.S. coat CV.
Don't copy make border and the parameters where our image for the input or image for the output and
then it was top bottom left and right for the type of board it was the dot for the underscore constant
and for the color we wanted that scalar right.
So let's create that scalar let's say const in all caps black for a scalar it's going to be a new CV
dot scalar and it was 0 0 0 and zero semicolon at the end.
And now we can use our color that we've created right here.
Let's save our code head back into our Web site refresh.
And now let's see if we get a square output here.
So put a five here and one.
We've got the padding successfully added and we've got a nice square image.
And if I draw one all the same.
Right.
So now we've got that consistent sizing.
Thanks to the padding.
No matter what we draw on our canvas what's the size of these anyways let's verify what the dimensions
are of these little canvases here that we've got on here as a sample output.
If I right click on this thing and I click inspect then I get the width and the height of all these
small little elements.
And what we see here is that they're all equal to 28 by 28 pixels which is perfect.
That's exactly what we want.
And that concludes this lesson in the next lesson.
We're going to tackle the last part of this track here.
We're going to calculate the center of mass which is very exciting that we're going to shift our image
and then we're finally going to create our tensor and that will finish up all the pre processing steps
and get our output from our canvas to be pre processed exactly in the same way as the endless dataset.
So for all of that and more I'll see you in the next lesson.
Take care.