Inspect Your Data Source (Recap)

00:00 Let’s do a quick recap on Part 1, inspecting your website. This is actually a very important part of the web scraping process because you really need to understand how is the website structured and how do you interact with it before you know how to scrape it.

00:15 What you learned in here is that you first want to go and explore the website as a normal user, which means just heading to the URL with your browser and clicking around the website, essentially seeing how would you interact with it if you weren’t going to be using it with the script to scrape the information, but just using your hands and your eyes, essentially.

00:37 Next, you learned how to decipher the information that is hidden inside of URLs—the parts that a browser uses to communicate with the site, but that you, as a normal user, usually don’t have to worry about.

00:51 But there’s information in there, and you learned about query parameters and how these are structured and how you can add information in here. Change it even up in the URL and the website is going to change.

01:03 And the other way around—if you change something on the website and submit the information, then the URL is going to change. This connection is important, so keep it in mind.

01:12 Your way of interacting with websites here are the query parameters.

01:18 Then, finally, you looked at using developer tools to even deeper inspect the website that you’re working with. You learned how to open them up through a menu item or right-clicking, and then you can hover over different pieces of the website and see the corresponding code in HTML. Or the other way around—click on the HTML, and then see the page structure that relates to it light up. And like this, you can expand, collapse, figure out what are the actual pieces of information in the HTML structure that contain the information that you want, and that you’re going to want to target in your scraping and parsing later on.

01:58 Okay! So, that’s enough working with the browser for now. It’s time to write some Python code. We’re going to move over to Part 2 where you will learn about scraping the website.