Beautiful Soup: Build a Web Scraper With Python

Web Scraping With Beautiful Soup and Python

Martin Breuss
Martin Breuss 25 Lessons 1h 42m data-science intermediate tools web-scraping
Start Now Rate and Review

The incredible amount of data on the Internet is a rich resource for any field of research or personal interest. To effectively harvest that data, you’ll need to become skilled at web scraping. The Python libraries requests and Beautiful Soup are powerful tools for the job. If you like to learn with hands-on examples and you have a basic understanding of Python and HTML, then this course is for you.

In this course, you’ll learn how to:

  • Use requests and Beautiful Soup for scraping and parsing data from the Web
  • Walk through a web scraping pipeline from start to finish
  • Build a script that fetches job offers from the Web and displays relevant information in your console

What’s Included:

Downloadable Resources:

Related Learning Paths:

Start Now

Introduction to Web Scraping

6 Lessons 16m

1. Web Scraping With Beautiful Soup and Python (Overview) 02:45

2. Introduction to Web Scraping 02:53

3. The Challenge of Variety 01:17

4. The Challenge of Durability 01:55

5. APIs: An Alternative to Web Scraping 04:54

6. A Message From the Future 02:57

Inspect Your Data Source

5 Lessons 19m

1. Inspect Your Data Source (Introduction) 01:11

2. Explore the Website 03:48

3. Decipher the Information in URLs 06:29

4. Inspect the Site Using Developer Tools 05:43

5. Inspect Your Data Source (Recap) 02:11

Scrape HTML Content From a Page

5 Lessons 20m

1. Scrape HTML Content From a Page (Introduction) 02:01

2. Static Websites 07:07

3. Hidden Websites 02:38

4. Dynamic Websites 05:21

5. Scrape HTML Content From a Page (Recap) 03:07

Parse HTML Code With Beautiful Soup

7 Lessons 31m

1. Parse HTML Code With Beautiful Soup (Introduction) 01:52

2. Make a Soup 03:03

3. Find Elements by ID 02:53

4. Find Elements by HTML Class Name 04:53

5. Extract Text From HTML Elements 04:37

6. Extract Attributes From HTML Elements 08:16

7. Parse HTML Code With Beautiful Soup (Recap) 06:20

Additional Practice and Summary

2 Lessons 14m

1. Your Turn: Build a Pipeline 05:33

2. Web Scraping With Beautiful Soup and Python (Summary) 08:39

Start Now

About Martin Breuss

Martin Breuss Martin Breuss

Martin likes automation, goofy jokes, and snakes, all of which fit into the Python community. He enjoys learning and exploring and is up for talking about it, too. He writes and records content for Real Python and CodingNomads.

» More about Martin

Each tutorial at Real Python is created by a team of developers so that it meets our high quality standards. The team members who worked on this tutorial are:

Christopher Bailey

Christopher
Sadie Parker

Sadie
Tappan Moore

Tappan
Geir Arne Hjelle

Geir Arne
Joanna Jablonski

Joanna
Aldren Santos

Aldren
Jaya Zhané

Jaya
Mike Driscoll

Mike
Bartosz Zaczyński

Bartosz
Jacob Schmitt

Jacob

Participant Comments

jramirez857 on Jan. 2, 2022

Great course! It is beginner level as others have mentioned but the fact the site had changed was a good challenge to test my knowledge on the topics covered in the video.

Oliver on Aug. 24, 2021

Thank you, Martin! Great course for a beginner, nicely explained. Just enough to start trying myself.

Mark on July 6, 2021

Outstanding course, Martin! I’ve done nothing with web scraping, and this course gave me a real foundation. Past comments suggested this as beginner, and it sure did help this beginner. I like your teaching style, too.

« Browse All Courses