Make a Soup
Here’s another overview of the different topics that we’re going to talk about, and we need to start off by, again, scraping the site. So this is what you learned in Part 2, which is just using
requests to get that specific query result and save it to the
response object. I’m going to execute this first cell here—this was our scraping step in this case—and now we’re going to start to parse the results that we got back from there.
00:50 So for this, we’re using a library called Beautiful Soup, which is a standard for doing web scraping with Python. It’s very powerful and pretty intuitive, so it’s definitely a good library to know. There’s some other ones out there as well but Beautiful Soup is the defacto standard for web scraping.
01:09 So, I’ll go ahead and import this. I also have this installed in the virtual environment. And then, you are ready to create a soup! Which is Beautiful Soup’s way of parsing through the HTML content so that it then is accessible through intuitive methods and attributes on that object.
So, this is going to show you exactly the same code. This is what
requests scrapes from the web, and then Beautiful Soup—once you parse it—just also can represent it in a bit more nicer formatted way. But otherwise, here, you’re just looking at the same content that
requests scraped earlier. However, the
soup object that it is now has a bunch of very, very useful methods and ways of interacting with it to pick out the information. We’re going to look at those next.
Become a Member to join the conversation.