Now, we have a fairly good idea about our X and Y variables, our X variable is present in the form
of 2D array of 28 and 28 pixel intensities, where each individual pixel intensity lies between zero
and 255.
And since we are going to use gradient descent to compile lower model, we need to normalize this pixel
intensities.
By normalizing, I mean, we have to restrict this pixel intensities between zero and one.
A very simple way to do this is by dividing all the pixel intensities by 255.
So zero will remain zero and 255, which sends four completely white pixel becomes one.
And so on.
So to normalize, we can just be way too extreme for each data set by 255.
And similarly, we have to normalize our test data set as well.
So four tests also.
We are dividing all the pixel intensities by 255.
This normalization is different from the normalization we generally do for machine learning algorithms.
Since here, we know that all of these values are on an absolute scale of zero to 255.
We can vertically divided by 250 feet.
But for general machine learning databases, we don't know the absolute scale.
So we generally subtract the mean from these numbers and divided by their standard deviations.
But that process is not needed here, since we know that the pixel intensities lies between zero and
255.
So here we can directly divide it by two fifty five.
And one thing you can notice is that we are not dividing it by 250 feet.
We are dividing it by two fifty five point zero.
That because we want the final output in the form of floating numbers between zero and one four, divided
by just integer values of 255.
So since the intensities are integer value, there might be some cases with some Python version where
we get the output as integer since we want the whole greyscale between zero and one.
We have to use two fifty five point zero with three cent Python version.
You don't have to do this but to make sure that the code is compatible with all other Python versions.
It's better to do it with a floating number so that the final output is in the form of floating number
between zero and one.
Just figured this.
We're calling over normalize data sets, as Xander underscore, Crane, underscore N and X, underscore
tests underscore N.
As I told you earlier, our train dataset is off.
Six thousand observations.
Another test dataset is of another 10000 observations.
We will further divide over Krien dataset and do training and validation sets.
We will use the first 5000 observations as our validation test.
And next, fifty five thousand as ever creaming dataset.
So to do that, we can just do I did using this simple operations.
We are saving over zero to 5000 datasets and two X validation.
And from 5000 vun to 60000.
And to extreme, similarly, we have to do this for a world wide dataset.
Also, we are saving first 5000 observations and two X validation.
And next 55000 observations.
And two, Vytorin.
And our X test will remain the same.
So we are just saving our normalized data and do X test data.
So just run this.
Now we have two datasets.
First is the validation set of 5000.
Then the training site of 55000.
And then add another dataset of 10000 observations in our dataset.
We will be using green dataset to train our model.
We will be using validation set to optimize the performance of automotive.
And then after tuning all the hyper parameters, we will be using test data set.
To evaluate the performance of automotive.
To view the values of this dataset, you can just call the dataset.
You can see now the values are between zero and one.
Just look at the first when you.
Here you can see there are some values which are between zero and one.
And now it is normalized in the next lecture.
We'll look at different methods that are available to create neural network using get us.
Thank you.