Web Scraping in Python

Exercises Course: Introduction to Web Scraping With Python

Martin Breuss
Martin Breuss 23 Lessons 1h intermediate data-science tools web-scraping
Start Now Rate and Review

Web scraping is the process of collecting and parsing raw data from the Web, and the Python community has come up with some pretty powerful web scraping tools.

The Internet hosts the greatest source of information on the planet. Many disciplines, such as data science, business intelligence, and investigative reporting, can benefit enormously from collecting and analyzing data from websites.

In this course, you’ll practice:

  • Parsing website data using string methods and regular expressions
  • Parsing website data using an HTML parser
  • Interacting with forms and other website components

What’s Included:

Downloadable Resources:

Related Learning Paths:

Start Now

Scraping and Parsing With urllib

9 Lessons 29m

  1. 1. Exercises Course: Introduction to Web Scraping With Python (Overview)02:47
  2. 2. Scraping and Parsing Text From Websites Exercise01:26
  3. 3. Checking Out the Website in Your Browser02:44
  4. 4. Scraping the HTML03:02
  5. 5. Brainstorming a Solution in the Python REPL05:17
  6. 6. Taking Notes About Each Subtask02:22
  7. 7. Removing the Text Before the Info03:33
  8. 8. Slicing Off the Text After the Info03:07
  9. 9. Generalizing and Cleaning Up the Code05:03

Using an HTML Parser

7 Lessons 15m

  1. 1. Using an HTML Parser Exercise01:51
  2. 2. Inspecting the Site With Your Browser01:23
  3. 3. Making a Plan and Reusing Code01:51
  4. 4. Setting Up a venv and Installing Beautiful Soup03:16
  5. 5. Exploring the Soup in Interactive Mode03:36
  6. 6. Constructing the Scraping Script02:20
  7. 7. Confirming the Script Works as Expected01:40

Interacting With Forms via Mechanical Soup

7 Lessons 15m

  1. 1. Interacting With HTML Forms Exercise01:12
  2. 2. Exploring the Task in the Web Interface01:03
  3. 3. Installing Mechanical Soup and Making a Plan01:39
  4. 4. Solving the Task in Your REPL04:40
  5. 5. Converting the Solution Into a Script03:09
  6. 6. Remembering Where to Use Mechanical Soup00:41
  7. 7. Exercises Course: Introduction to Web Scraping With Python (Summary)02:50
Start Now

About Martin Breuss

Martin likes automation, goofy jokes, and snakes, all of which fit into the Python community. He enjoys learning and exploring and is up for talking about it, too. He writes and records content for Real Python and CodingNomads.

» More about Martin

Each tutorial at Real Python is created by a team of developers so that it meets our high quality standards. The team members who worked on this tutorial are:

Brenda

Christopher

Tappan

David

Geir Arne

Joanna

Aldren

Jacob

Philipp

Kate

« Browse All Courses