Let's talk about web scraping.
One of the things we can do with Python.
What is web scraping?
Well, web scraping simply means programmatically using Python or any language grabbing data from a
website.
So for example, I can grab data from Amazon, from Airbnb.
From maybe a recipe website, any website can be scraped.
So essentially when we talk about web scraping, we mean downloading a Web page, grabbing the data
from that web page, cleaning it up somehow so that it's useful for us and then using that data.
You see, this is how websites work.
We have a browser that we use like a chrome and we have a server.
A server is simply a computer somewhere in the world that has three files that are needed to display
a web page.
We have an HTML file, which is the content of the website.
You can think of that as text and images of a website.
We have CSS, which gives us the colors and pretty styling that websites have.
And then we have JavaScript which allows us to have different behaviors in our website, like submitting
forms or having dropdown menus and all a browser does like chrome.
Say, Hey, if Andre visits this website, let's say Fifa.com browser is going to know when I type in
Fifa.com that says Hey server, can you give me the fifa.com?
HTML, CSS and JavaScript files.
And the server says Yep, no problem, here they are.
And then the page gets displayed onto our browser.
So these three files, well, the data in them is accessible to us, right?
It gets displayed on a web page.
So these files are now something that we can actually check out.
For example, if we go to the Fifa.com website, you see over here that well, we have a pretty page
with a lot of information.
That's great.
And if I right click here and do view page source, this is the HTML page of the FIFA website.
As you can see, it's pretty long.
But it has all the data that we see in this page.
Now the CSS and JavaScript files are linked to it.
But the main data that is the text may be the images.
Are all here.
So, for example, we have latest news.
We have top ten goals.
From the Women's World Cup.
Well, let's find out.
Let's say top ten goals.
From the Women's World Cup.
You see, that's what we have.
We have the text right here.
And Web scraping is simply this grabbing this data.
And then removing all the parts that we don't care about and just keeping the parts that we care about.
And web scraping works because this information is public.
In order for a website to get displayed, it has to be public.
Maybe you're still wondering why web scraping might be useful.
Well, there are many, many cases for it.
For example, I'll link to this resource where it talks about some of the big uses for web scraping.
Why would we want to scrape a website?
And you can see here, there's many reasons.
Maybe you want to get satellites leads.
That is, you want to collect all the emails that are on a web page.
Maybe a contact form.
Maybe you have a yoga office software product.
Maybe go to every single yoga studios web page and find the contact information.
Scrape that data so you don't have to manually do it.
Maybe you want to find all the jobs and open positions on a website, and instead of manually clicking
through different pages, you can scrape that data.
And there's many, many use cases in our project that we're going to build for this session.
We're actually going to scrape Hacker News, a very popular website for many programmers, and we're
going to turn this into something even more useful for us to read, analyze and stay current with our
industry.
But we'll get to that in a bit.
Let's take a break here and learn a little bit more about Web scraping in the next video.