In this lesson we're going to create our own word cloud. 1
Also I'll show you how to get hold of entire novels through the NLTK resources. 2
Let's add a quick little section heading here with a markdown cell that reads "Creating a Word Cloud". 3
Now, whenever you're using an unknown package or a new package that you haven't seen before, it's always 4
a good idea to pull up the documentation of this package. 5
In our case, the package that we're going to be using is called wordcloud and it was originally made 6
by a chap called Andreas Mueller. 7
But since then he's had approximately 33 people contribute and improve and add their work to make this 8
package better. 9
As you can see the entire code base is up on GitHub for anybody to look at 10
and there's some basic instructions here as well, 11
but the more detailed API documentation can be found here. On this Web site you can see a bit more information 12
on how to use the wordcloud as well as some more examples. 13
I've added the links to both these sites into the lesson resources. 14
So without further ado let's get started with a basic example. 15
Now having downloaded and installed the package in the previous lesson it's time to add it to our notebook 16
imports. 17
So here we can say "from wordcloud import WordCloud". Lets hit Shift+Enter and scroll all the way down. 18
Now we're gonna be using that example email again to generate our first word cloud. 19
The way to do it is to use "WordCloud().generate()" and then 20
we just have to supply the piece of text that we'd like to use to generate the word cloud from. 21
So in our case it was the email body. I'm going to actually save this in a variable called "word_cloud", 22
so I'll say "word_ 23
cloud = WordCloud().generete(email_body)". 24
This is step one. 25
Now matplotlib is going to come into play. So matplotlib was stored inside a variable called "plt" 26
and here we're going to use the "imshow" method, "imshow()" and we can supply, 27
well you guessed it, we're going to supply our "word_cloud" to "imshow" and now let's use "plt.show()" and 28
see what. We get we get something like this. Now I don't know about you but I can see we can make some 29
improvements on this right away. The quality isn't that great, the letters look a little bit jagged and 30
we've got these axes on both sides, on the x and on the y. These axes are really easy to remove actually. 31
If we use "plt.axis()" and then single quotes with the string 'off' and hit Shift+Enter, 32
the axes disappear, but we still kind of have these jagged edges on some of these letters. It doesn't 33
look that clean. 34
So what we can do is come up to the "imshow" method, put in a comma after "word_cloud" and add an additional 35
argument here. 36
This one's going to be called "interpolation" and we can set that equal to some sort of value, 37
right. At first we had a default value for interpolation which was "none", 38
so there was no interpolation going on, but we can do some interpolation to smooth out these edges. 39
I'm going to go with "bilinear" and now we have a much cleaner picture. Of course, the only reason I know 40
to set "interpolation" to "bilinear" is because I've had a look at the quick documentation here and I 41
scrolled down to the interpolation parameter and I've tried out a couple of these. 42
So there is quite a big number to choose from. 43
The default is "none", but I did find that I could get some improvement in the look of my word cloud by 44
playing around with this and I found that "bilinear" works well for me. Now I think as a minimalist basic 45
example, this works really, really well 46
and you'll find something very, very similar on the documentation page actually. 47
So let's take this to the next level. 48
Let's make this a little bit more interesting. To make our next word clouds a little bit more interesting, 49
I want to show you how to download entire novel from the natural language tool kit. NLTK actually 50
has a whole bunch of resources that we can use if we go back up to where we were downloading our tokenizer 51
and our stop words, you'll recall both of these were downloaded to the "nltk_data" 52
folder on our hard drive. All we need to do is add two additional lines of code here, namely "nltk. 53
download('gutenberg')" and "nltk.download(' 54
shakespeare')". If I hit Shift+Enter on this cell it will download some additional corpora for us that 55
we can make use of in our word cloud. Check it out. So it got the zip files and it unzipped them 56
and here they are in my "corpora" folder. 57
I've got "shakespeare" right here with a number of his plays including Hamlet, Julius Caesar, Macbeth and 58
so on. I think "r_and_j" stands for Romeo and Juliet, 59
another classic. And Gutenberg has also a whole bunch of books, so you can read these for free if you 60
like. 61
This one here looks like Alice in Wonderland by Lewis Carroll. 62
There we go. 63
Opening them up in my Atom text editor I can see the whole text of the entire novel nicely formatted. The 64
one we're gonna use is Moby Dick by Herman Melville. 65
If you actually wanted to read this book, you have to scroll down quite a bit past all the acknowledgements 66
and praises and so on to, uh, eventually Chapter 1 which is on line 500 with the famous opening sentences. 67
But okay, so far so good. 68
Now we've got access to the text of a whole bunch of novels, 69
now it's up to us to put these into a word cloud. So I'm going to insert a few more cells at the bottom of 70
our notebook and I'm going to get hold of one of these works. The way I'm going to get hold of one 71
of these is like this, I'm going to have to go "nltk.corpus.gutenberg. 72
.words". 73
So in this case I'm getting a hold of one of the text files in the Gutenberg folder. The one I'm interested 74
in is going to be Moby Dick, so I'll have single quotes in the parentheses and I'll type out the exact 75
spelling of this text file, so it's 'melville-moby_dick.txt'. 76
I'm including the file extension. The string that I'm passing into my "words" method here has to match 77
this file name exactly. Now I'm going to create a variable called "example_corpus" and that's 78
where I'm going. to store my Moby Dick novel. 79
If you're wondering how many words are in this entire novel by the way you can pull it up with the built 80
in length function, "len( 81
example_corpus)". 82
Let's see how much work was put into Moby Dick. 83
It's not War and Peace but 260000 words is still no slouch. 84
Now another thing that might be quite interesting actually is to see what type of variable or type of 85
object we're dealing with. 86
So in this case our "example_corpus" is something called a Stream Backed Corpus View. 87
The reason I'm drawing attention to this is because you might be forgiven for thinking that what you're 88
getting back here is just a big long string, but you're actually dealing with a different kind of object 89
here. 90
This thing that we're working with actually has a whole bunch of tokens. 91
If you wanted to get the list of words, you have to kind of join them together, 92
you have to join all the tokens together. 93
Let me show you what I mean. 94
The first thing I'll do is I'm going to create a list, I'm going to call it "word_list" and set that equal to 95
square brackets, 96
and then inside these square brackets I'm going to use Python's list comprehension to join all the tokens 97
together. 98
So it'll be two single quotes, then dot, "join(word)", 99
and yeah you guessed that there's a loop coming "for word in example_corpus". This here will join all of 100
our words together. 101
So if you're wondering what this looked like before, "example_corpus", Shift+Enter looks like so and the 102
"word_list" that I'm getting after running my list comprehension and joining all the words together will 103
look like so. Very similar looking I know, but remember we're going from tokens to a list of words which 104
we can then use to join together into a single string. 105
So if I say "novel_as_string = ' '. 106
join(word_list)". This here will actually take all the words out of the list and put 107
them into a single string, so "novel_as_string" will look like 108
so. Now if you're wondering why we're going through this trouble, it's because we have to do some pre- 109
processing to feed our text into our word cloud. 110
Remember, this "generate" function here, 111
it's kind of expecting a very simple string to build the word cloud from, we can't give it a corpus 112
straight out of the NLTK toolkit. 113
And I also don't want to give it a list of tokens or a list of individual words. I'm planning to give 114
my word cloud a simple string. So why don't we try it out for now? 115
I'm going to copy this, come down here, paste it in, instead of having email body inside my "generate" method, 116
I'm going to have "novel_as_string" and see what we get. Fantastic! So that's working. We've successfully extracted 117
all the words from a corpus in the NLTK resources and fed them into our word cloud package. 118
Now all we have to do is make this thing look a bit better and style it. 119
I'm planning to arrange all these words into the shape of a whale. 120
Let's see if we can do this.