Join us and get access to hundreds of tutorials and a community of expert Pythonistas.

Unlock This Lesson

This lesson is for members only. Join us and get access to hundreds of tutorials and a community of expert Pythonistas.

Unlock This Lesson

Hint: You can adjust the default video playback speed in your account settings.
Hint: You can set the default subtitles language in your account settings.
Sorry! Looks like there’s an issue with video playback 🙁 This might be due to a temporary outage or because of a configuration issue with your browser. Please see our video player troubleshooting guide to resolve the issue.

Find Elements by ID

Martin Breuss

Web Scraping With Beautiful Soup and Python
Martin Breuss 02:53

Recommended Tutorial Course Slides (.pdf) Sample Code (.zip)
Give Feedback

00:00 Now to the actual step of finding an element by ID. Let’s head back over to the Notebook.

00:07 So, we’ve seen before that the response is all of this content, essentially, and we haven’t sifted through anything yet, but now the first step we’re going to do is find an element by ID. Now, what was the ID? Remember that this is an iterative process—this exploring of what is the information that you want to get from the site, of the content. And also just like figuring out where is it located.

00:32 And this is where your knowledge of the page comes in very handy.

00:36 This is why you explored it before. You already know that there’s some sort of column here that contains all that information, and this has an ID, and that’s exactly what we’re looking for now.

00:48 Keep in mind that this is always available to you. You can keep coming back to your browser and just inspect using the developer tools to figure out what is the exact thing that you’re looking for. Right?

00:58 So, we did this looking for it, see if I can find it like this. Yeah. So, remember? We found this <td> with the id="resultsCol" (results column).

01:11 So, this is going to be the column that contains all the information that you’re currently looking for, and this is the ID that this <td> HTML element has.

01:20 So, I can go ahead and just say, okay, "resultsCol"—this is what I want to pick out. And on the soup object, you have a method called .find() that you can pass in an id and then the name of the id. So, in this case, this is 'resultsCol', and you can just run this, execute this code.

01:39 And then the result is going to be not the full HTML from before anymore. So if you look at this, this has everything—the <head>, a bunch of JavaScript that’s included here as well.

01:51 So, this has all of the content of the page,

01:54 but here we sifted through it and picked out only the <td> element with the id "resultsCol" and everything that it contains.

02:03 So this is similar to here, when I expand this—everything that’s inside of this HTML element is what we now have access to under the name results.

02:16 So, go ahead and try this out some more. Look for another element in here—here’s one, for example, with an id—and just locate the element by ID. Practice this a bit.

02:30 You can make additional cells over in the Jupyter Notebook here and just run some code so that you understand what the syntax is to find an element by ID.

02:41 And since this is still pretty big, we want to drill down a bit further inside of this results container. We’re going to do that by class name in the next lesson.

Become a Member to join the conversation.