To create our word cloud for our ham messages and our spam messages, 1
we're gonna use two different masks. 2
I've got thumbs up for the non-spam messages and I've got thumbs down for the spam messages. 3
These are gonna be our masks for our respective word clouds. 4
The dataset backing our word clouds will come from some of the previous work we've done. So we've got 5
a flattened list of non-spam words and we've got a flattened list of spam words. 6
This is the data that we're going to feed into our word clouds. 7
So let's tackle our ham messages first. 8
I'm going to add a markdown cell here and write "Word Cloud of Ham and Spam Messages". 9
Next I'm going to add some constants. 10
I'm going to add the thumbs up and thumbs down files, so I'll paste the relative path for the skull file 11
and just rename these to "THUMBS_UP_FILE" and "THUMBS_DOWN_FILE". 12
Of course the icon names have to change as well. 13
So this one well call "thumbs-up.png", and this one we'll call "thumbs-down. 14
png". Shift+Enter, 15
scroll down and 16
now what I'm gonna do is I'm going to copy this code here for our whale, come down here and paste it in. 17
There's a few modifications that we can make here that will save us a bit of time. 18
So first off we're gonna change the image to say thumbs up, 19
then what we'll have to do is we'll have to generate the text as a string for the word cloud, 20
right. 21
In this case, I'm going to store all of this text in a variable called "ham_str" and that will 22
be equal to "' '.join(flat_list_ham)". 23
So we've got a list of non-spam words and we're going to join them all together into a single string. Before 24
I run this, 25
I have to do one last thing. I have to change this "novel_as_string" to our "ham_str" variable. 26
That way I'm not generating another word cloud for Moby Dick, 27
instead I'm generating it for non-spam emails. 28
And here's our output. 29
We still have the Moby Dick styling, but we can change that very easily with a different colour map and 30
I think otherwise it looks pretty good. In terms of colour map, 31
let's go for 'winter' and, I don't know, maybe a bit more granularity on the words. 32
Now, 33
I think this looks pretty good actually, but there's probably one thing that you'll notice that might 34
bother you a little bit. 35
"People", for example is spelled without the "e" at the end. 36
"Change" is spelled without the "e" at the end. "Provide is spelled without the "e". 37
We can only guess what this word "tri" 38
is supposed to be. The reason for this is is that the data that we fed into our word cloud are the 39
stemmed words. 40
That's why we're seeing our word cloud presented like this. 41
Our flattened list of non-spam keywords is coming from our nested list of non-spam keywords, which is 42
coming from our nested list, which we've created up here when we applied our "clean_msg_ 43
no_html" method. 44
This is where we did this stemming if you recall. 45
If you wanted to see what this word cloud would look like without the stemmed words, then you'd have 46
to copy this code, paste it in, use the original words and comment out 47
this line right here. If you refresh this cell and then go to "Kernel" > "Restart & Run All", then you'll 48
run all the cells in the entire notebook and after waiting a little while, you can scroll down and look 49
at the output for the word cloud without the stemmed words. 50
There we go. 51
This is starting to look a lot better. 52
The stemmed words are going to be verym very useful for our Bayes' classifier but they're actually not so 53
useful for our word cloud. 54
So if you've changed it like I did then it's important to make a mental note to change this back and 55
use the stemmed words before we train our model. Now I've got one final challenge for you on the word 56
cloud front. 57
I'd like you to pull up the word cloud documentation and figure out how to use the custom font that 58
I've included in the lesson resources instead of this default font here and then create a word cloud 59
of all the words in the spam messages. 60
I've included two font files. 61
The "OpenSansCondensed-Bold" font file and I've included the "OpenSansCondensed-Light" font file. 62
If you look at this word cloud here, what kind of font do you think would work best with it? 63
What characteristics should a font have in order to make a word cloud look a lot more attractive? Because 64
in my humble opinion this default font here doesn't work as well as it should. 65
What kind of improvement on the font front would you make to this visualization? 66
I can think of two improvements right off the bat. 67
First off, the words should probably be written in all caps rather than lowercase. 68
And second, the letters of the font should probably be closer together and bold. 69
These are the first two things that I would try to make this look a lot more convincing. 70
So I hope you will give this a shot when solving the challenge. 71
Pause the video and have a go. All good? 72
Here's the solution. 73
We've already added the relative path to the thumbs down file so we can already use it. 74
I'm going to copy this code right here 75
and paste it in below. 76
I'll swap up thumbs up for thumbs down, this line stays the same, as does this one. Our string of course 77
will need to change, so this will be the spam string and this will come from the flattened spam list. 78
And when it comes to generating the cloud we will use the spam string instead of the ham string. For a 79
bit of visual differentiation, 80
I'm going to go with something reddish from the color map. 81
In fact, I'm going to try this "gist_heat" color map that we've got here. 82
So instead of "winter" I'll go for "gist_heat" and I'll keep some of the other things the 83
same. 84
Now when I said check the word cloud documentation, what I wanted you to pick up on was the fact that 85
you could supply a font here as an argument and you can do so by giving the word cloud a font path. The 86
font path is the font that will be used in the word cloud. 87
And this has to be a OTF or a TFF file and a TFF file is exactly what we've got. Going back up, 88
I'll add my relative path here for my font file. So I'll call this one "CUSTOM_FONT_ 89
FILE" and this will be my "OpenSansCondensed-Bold.ttf" file. 90
You can try both of these and see which one looks better, but I think I prefer the bold one. Just gotta 91
make sure there's no typos here in this file name. Let me hit Shift+Enter on this, scroll back down, 92
and now I can supply a "font_path" and set that equal to the custom font file. 93
The other thing that you might have had to have a play with is the maximum font size. So in getting it to 94
look the way you want it to, 95
there is a mention of a font size parameter in the quick documentation as well. Despite the rather strange 96
word wrapping at the moment, 97
you can see it right here, "max_font_size" by default is equal to None, but we can give it another value 98
as well, so "max_font_size" equals, say I don't know, I think 300 worked well last 99
time I looked at this. Now let me hit Shift+Enter and see what this looks like. 100
My computer is really struggling. 101
I think I've got too many things open. 102
Okay. 103
So here we go. 104
This is the result. 105
I think the font looks pretty good, but I think we need to up the number of words being displayed in 106
this image. 107
Also we're still in lower case, so let's change these two things. 108
I'm going to be pretty radical and change the maximum number of words to 2000 from 500 and when it 109
comes to generating my word cloud I'm going to convert my string to uppercase letters. 110
So "spam_str.upper()" will convert all of my letters to uppercase. Now I can refresh and 111
see if we're happy with the result. 112
I think the red color definitely worked really well but with these last two changes I think the design 113
should really come together. 114
Okay so that's interesting. 115
The largest word has randomly been given a white color. 116
This is probably not what I want, so I'm going to refresh this and see if I'm more lucky 117
the second time round. Much better. 118
Brilliant. 119
So I quite like the look of this but I'd be quite curious what sort of designs you've come up with. 120
And speaking of your own designs, a good question at this point is: Where can you find both fonts and 121
masks for your own projects, for your own word clouds? When it comes to fonts, 122
one of my favorite places to go to is Google Fonts, so Google Fonts are free and open source and you 123
can just browse and download the font files that you like from here. 124
This is a brilliant resource. When it comes to masks and icons, 125
you've got a lot of choice as well. 126
One place that you could check out is the icon set on fontawesome.com. 127
Here you can download various icons that you can then use as masks for your PNG files. 128
Just remember to double check the licensing on all of these image files that you're interested in using. 129
In any case font awesome allows you to filter on free icons which is quite handy. 130
So if we check out this accessible icon here, then you can download this icon as an SVG file, a vector 131
which you can then scale as large as you like with some image processing software. 132
Now with those tools in hand I think you're all set to create something fantastic. 133
If you do please post it in the comments section below this video, 134
I'd love to see what you've made. In the next lessons, 135
we're going to generate the vocabulary and the dictionary for our Bayes' classifier. It's about time I get 136
some food, so I'll see in the next lesson. 137
Take care.