Now we have a fairly good idea about our x and y variables our x variable is present in the form of
2D array of 28 and 220 pixel intensities where each individual pixel intensity lies between 0 and 255.
And since we are going to use gradient descent to compile our model we need to normalize this pixel
intensities by normalizing I mean we have to restrict this pixel intensities between 0 and 1 a very
simple way to do this is by dividing all the pixel intensities by 255.
So Zillow will remain 0 and 255 which stands for completely white pixel becomes 1.
And so on.
So to normalize we can just be very low at X strength for let's say by 255.
And similarly we have to normalize our test data set as well.
So for tests also we are dividing all the pixel intensities by 255 this normalization is different from
the normalization we generally do for machine learning algorithms since here we know that all these
values are on an absolute scale of 0 to 255.
We can radically divided by 255 but for the general machine learning databases we don't know the absolute
scale.
So we generally subtract the mean from these numbers and divided by their standard deviations but that
process is not needed here.
Since we know that the pixel in densities lies between 0 and 255 so here we can directly divide it by
255.
And one thing you can notice is that we are not dividing it by 255.
We are dividing it by two fifty five point zero.
That because we want the final output in the form of floating numbers between 0 and 1.
If we divide it by just integer values of 255.
So since the intensities are integer value there might be some cases with some python version where
we get the output as integer since we won the whole grade scale between 0 and 1.
We have to use to fifty five point zero with three cent python version.
You don't have to do this but to make sure that the code is compatible with all other Python versions
it's better to do it with a floating number so that the final output is in the form of floating number
between 0 and 1.
Just figured this we're calling over normalized datasets as Xander screen underscore and an X underscore
tests underscore and
as I told you earlier our trained dataset is of 6000 observations and no test dataset is of another
10000 observations.
We will further divide our green data set and two screening and validation sets we will use the first
5000 observations as our validation test.
And next five posing as a training dataset.
So to do that we can just do.
Using this simple operations we are saving over 0 to 5000 data sets and 2 x validation.
And from five thousand one to 60000 and too extreme.
Similarly we have to do this for a world wide dataset also we are saving first 5000 observations and
2 x validation.
And next fifty five thousand observations into victory.
And our x test will remain the same.
So we are just saving our normalized data and do X test data so just run this.
Now we have three datasets.
First is the validation set of 5000.
Then the training set of fifty five thousand.
And then add another dataset of 10000 observations in our has dataset
we will be using green data set to train our model.
We will be using validation set to optimize the performance of our model.
And then after tuning all the hyper parameters we will be using test data set to evaluate the performance
of automotive to view the values of this dataset.
You can just call the data.
You can see now the values are between 0 and 1.
Just look at the first when you
here you can see there are some values which are between 0 and 1 and now what it has normalized in the
next lecture will look at different methods that are available to create neural network using kid us.
Thank you.