Very often we find that some values in our data are missing.
Now, this may happen when you are collecting data from people asking them to fill a physical form.
And they do not answer all questions.
This may happen because some of the data stored got corrupted.
There may be many other reasons.
But this results into a serious problem as we cannot run a machine learning algorithm on missing values.
Now, this leaves us with only two options.
Either we remove the entire rule, which has some missing value, when we have large dataset and only
few of the observations of missing value, then we can delete those few observations and still have.
Substantial number of observations preparing and test that model.
But if the number of missing values is high, then the moving observations is not advisable in such
a case.
We should replace these blank places with some harmless values, harmless meaning that they should not
impact the model much.
Such neutral values are values close to the center of that variable.
So we can choose mean or median value to report in these blanks.
Here are some most common ways to do missing value imputation.
First is too important.
Zero.
Note that a missing value is not zero.
You can simply put a zero where you have a missing value.
But this rarely makes business sense.
So if zero makes sense in any particular scenario, we can do that.
Second option is to impute with centers as this will not have much impact on the model.
For example, if we have high rates of people as a variable and some of the values are missing in the
data, putting zero does not make any sense.
So we can put mean or median height of people in the missing field.
If you have a categorical variable with some values missing, you can assign that category which has
maximum frequency in the missing fields.
So suppose we have gender data and 90 percent of them are males.
Then in the missing fields, all the variable we can safely put maybe.
The third option is, again, putting in mean value, but instead of using the population mean if he
can make old relevant segments from the data we can use this segment mean for that segments observation.
For example, if we have rainfall data of different cities, but the data is missing for some cities
instead of using population mean we can take the mean of neighboring cities or cities belonging to similar
region or cities belonging to same state to fill the blanks.
This makes sense since rainfall does not ready a lot among neighboring cities.
Therefore, we should use business knowledge so that we can select one of these methods wisely.
When we find EDTA in any software, these softwares do give us the number of blank values in each variables.
We will learn how to fill the blank values in this software.
We are going to use.