Hello my friends, and welcome to this new part.
On Natural Language Processing.
I'm super excited to start this part because this is the branch of machine
learning with which you can build chat bots and machine translations.
So of course, this is not
what we're going to do in this part because this is really advanced NLP.
So we'll just cover the basics with sentiment analysis, which consists
of training a machine to understand some text
and predict a certain outcome for this text.
So in our case study here, these text will be reviews of a restaurant.
And we'll have to train a machine
to understand if each review is positive or negative.
So very simple, very classic.
But the best way to be introduced to NLP.
All right.
So before we start let's make sure everyone here is on the same page.
This is the folder containing all the codes and data sets
and of which I give you the link right before this tutorial in the article.
So make sure to connect to that link.
And now there we go.
We can and support seven natural language processing.
So in this part you will only find one section.
That's because we only do one case study of NLP about sentiment analysis.
However, you will see that you can try
diverse machine learning models to tackle the problem.
Indeed, the essential
part of our implementation will be to build the bag of Words model.
But then once it is built, we can try several classification models.
Why classification models?
That's because we'll have to predict, you know, a binary outcome 1 or 0 one,
meaning the review is positive and zero, meaning the review is negative.
So you'll have actually the flexibility to try several machinery models.
And this will actually be your final exercise of this section.
So there we go. Let's do this.
Let's enter section 36.
Now natural language processing.
And as usual we're going to start with Python in which you will find two files.
The implementation natural language processing dot Ipynb,
which you can open with either Google Collaboratory or Jupyter Notebook.
And our data set restaurant reviews dot this time.
Net CSV, but SVM and this will be a good opportunity
for me to train you on how to import a TSV data set.
TSV mean Tab separated value instead of comma separated value like in a CSV.
So basically the only difference
is that in the previous data sets we worked with, well,
you know, the features and the dependent variable were separated by commas.
And in this one, well, instead of being separated by commas,
the reviews and the dependent variable will be separated by a tab.
And this makes sense, right?
Because in the reviews we already have commas
and therefore they would create nonsense features.
But let me show you what this data set looks like.
So as you can see,
there are only two columns, the first one containing all the reviews.
So for example, this is the first one.
Well, not this place.
A second one trust is not good.
another one, great touch, etc..
So you have in total let's see, 1000 reviews.
Right.
So we're going to train our machine learning
to actually understand text and predict if the text are positive or negative.
With 1000 texts.
All right.
And then the second column is of course if the review is positive or negative.
So one means that it is positive meaning the customer liked it.
And zero means that the review is negative.
And of course we have the real outcomes in order to train our machine
learning model to understand if each of these text is positive or negative.
So that's purely in the end, you know, a classification problem.
But the essential part of it is that will train the machine to understand
these text first and then to predict if they are positive or negative.
All right.
So very simple case study very simple data set.
That means we are ready to start the implementation
of natural language processing.
So as you prefer
feel free to open it with either Google Colaboratory or Jupyter Notebook.
I'm opening it
with Google Collab as usual, so feel free to do the same if you'd like.
And now the Notebook is loading.
In a second it will be laying out all right loading leading out.
Perfect. And this is the implementation.
And as usual this is in read only mode.
And we want to re-implement this from scratch.
Therefore we're going
to create a copy right away so that we can modify the code inside.
So we're going to click save a Copy and drive here.
This will create a copy, after which we will be able
to modify the code and re-implement this from scratch.
And speaking of re-implementing it from scratch.
Well, let's delete all the code cells because we will re-implement them.
So let's click this trash button here and each of the code cells,
but not the text, so that we can keep that well highlighted structure
and see where we're going at each time of the implementation.
All right. So almost done.
It's actually an implementation in about ten steps.
But you will recognize some of the steps as steps we did before.
You'll see I'm going to show you in a second.
All right.
So let's have a look at the structure of this implementation.
We will start first by importing the libraries as usual.
Because indeed we will need several libraries to preprocess
our texts and train our future machine learning model.
Then we will import the data set.
So that's actually the data preprocessing phase.
But not only that, the data preprocessing
phase will also contain the next two cells cleaning the text.
Indeed, we will have to simplify the text as much as we can
in order to ease the learning process of the machine learning model.
You know, we'll have to remove all the punctuations.
We'll have to put all the letters in lowercase.
Then we'll have to apply stemming.
You know, we'll have to make very clean text to alleviate
the learning process of the future classification model will build.
So that's a compulsory process.
When doing NLP you have to preprocess the text basically.
Then we'll create the bag of words model
which is at the heart of sentiment analysis.
And then there you go.
That's where you will recognize everything.
Once we have the bag of Words
model, we basically have a data set ready to be trained, right?
We have a dataset ready to be trained by a machine learning model.
And that's why then we will just apply the classic process of training a model.
First, we will split the data set into the training set and test it
so that we can indeed have a set where we train the model
to understand text and predict if the text are positive or negative,
and the test set so that we can evaluate the performance
on mutex on which the model wasn't trained.
And then there we go, we train.
So I chose a Naive Bayes model on the training set.
But you will see that your
final exercise at the end will be to try the other classification models
and see if you can beat the accuracy I will get in this implementation.
Then we will predict the test result and finally
we will make the confusion matrix and get the final accuracy.
So that's our structure.
That's our NLP journey.
So now as soon as you're ready let's start in the next tutorial
with the simple data preprocessing phase.
I can't wait to start.
See you in the next tutorial and until then, enjoy machine learning!