Now we are beginning this section on feature engineering in this section.
We are going to learn the following things will first discuss water features and what is feature engineering.
Then we will see the rationale that as we explored the reason for doing feature engineering in times
it is data.
Then in the practical sessions, we will learn how to develop basic daytime based input features like
build features and sliding window based features.
So let us understand what our features.
In time, cities forecasting problem, we usually have this type of data in one column.
We will have daytime values and in the other column we have the values of the variable that we are monitoring.
And we want to forecast, for example, the first column will have bid values.
And the second column has the same values.
We have to transform this type of data to this format where we have input variables and one output variable
to the not model.
For example, instead of having only a date here, we should probably input whether it is a weekday
or weekend.
This can be one input variable.
We can input seasons such as summer or winter as another input, these inputs are called features.
And the task of creating or inventing new features from the time to these data is called feature engineering.
The idea is that our software does not understand much.
The idea is that our software does not understand much when it sees a Dead Sea.
Third, then it would make more sense from the data.
If we delayed that third, then was a weekend or not.
Or whether it rained or not on that day or whether people received salary in that week or not, et cetera.
Such features, which might actually impact the output variable, are to be identified and given as
input.
And this is the aim of doing feature engineering in the practical part of this course.
We are going to explore how to find these three classes of features.
First is daytime features.
Date and time values have several components.
We try to separate those components for each observation.
For example, from the date we can find day of the week, we can find season of the year.
These components are present as information in the date only.
Second, puzzle features are called lag features.
These are values at previous time steps.
For example, by forecasting this month's sale.
It makes sense to consider.
Last month is also third class of features, is called window features.
These are a somebody of values over a fixed window of prior time stapes.
For example, we can use the average seal of previous three months instead of last month to help predict
this month is average values can help us remove noise in the data.
And if carefully done, we can also remove seasonality effects from the data.
So in some situations, window features are very helpful.
Now, let's look at an example.
To further understand these three types of features, in the first table, you'll see times these data
that is in one column.
You have the date value.
And in the second column, you have the footfall in your store.
That is the number of customers visiting your store.
So on TNT, Jan, of 2020, you had 853 customers visiting your store on 11th Day.
It was 376 and so on.
Now, to convert this time to this data into a format which is usable for our model, we convert this
date into features.
The first is an example of big time feature we're corresponding to that particular date.
We are assigning whether that be was a weekend or not.
We are doing this because we think that probably footfall depends on whether the day is a weekday or
weekend.
So we tried to extract that information from the days that we have.
In the second column, I have a lag feature.
So we are looking at the footfall.
Exactly seven days ago.
Why we are doing this, because exactly seven days ago, it was the same day as today.
So if it is Thursday, today, seven days ago will also be Thursday.
And the idea behind that could be that seal on a particular day could be similar.
So seal on this Saturday, it will be similar to SEAL on last Saturday.
So that is why we create one more feature in which we are looking at the land value of the variable
to be predicted in the third column.
We are looking at a window feature.
We are looking at the last seven days.
This is the window at which we are looking.
We are looking at the last seven days and finding the average footfall for last seven days.
How does ERPs probably averaging the footfall for last seven days?
Removes the noise and seasonality within a week.
So average value in the last seven days can probably help us predict the footfall on the presently.
So these are the three types of features, daytime features like features and window features that we
will learn how to extract using our software.
Now, within window features, we will explore two types of window features.
One is a rolling window and the other is expanding window.
The ruling window is when we've fixed the width of the window and we slide it forward as we move forward.
For example, if I fixed the window with two last seven days on 10 January, I will look at the last
seven days.
That is from second down to nine.
An average that this 985 is the average value of footfall from second down to 19.
This 972 is average for full value from third down to 10.
So the window, it stays the same.
It is lost seven days only, but the window is sliding.
That is on 10 then.
The window was from second down to nine, 10 on 11th, 10.
The window is from third down to 10.
So each step that we take, we also move our window accordingly.
The other option is an expanding window in which window size keeps on increasing.
So in this scenario, if we are looking at maximum footfall value tilted at window of observation will
keep on expanding by one day each day.
So on TNT, we are looking at all the footfall values built and then so suppose the maximum value of
footfall built and then was eleven ninety five.
On 11th can also sense the value of football on Tenjin was less than this value.
This value of maximum footfall stays the same.
But on twelve, ten deep, thirteen hundred seventy six, footfall value will also be included in the
window.
And now the thirteen hundred seventy six value becomes the largest footfall value.
So from 2010 onwards, maximum for full value will be thirteen hundred and seventy six.
And this will continue until the footfall value becomes more than 13.
And it's 76.
So the concept is, even if we are increasing the date, we do not shift the window, we just expanded.
So on 11 that the window will include eleven, then on twelve, then it will include eleven done and
will then board and so on.
It will keep on including the new date and do it window to this as expanding window.
Before we start doing this, practically, I want to answer this one question that students often ask
how do I know which features and how many features I should use for my problem?
My suggestion is that you should understand the problem deeply and try to identify as many features
as you can.
Even if it intuitively makes less sense, I would suggest that you should create that feature and run
the analysis.
You will later see that most of our models also tell us which of the variables are useful and which
are not.
My point is that including a used less variable does less harm than excluding a useful variable.
So do avoid absurd variables, but do not exclude a business relevant variable because it seems counterintuitive
to you.
That's all.
Now, let us see the practical implementation of these concept.