Start Here
Learn Python

Start Learning
Your Learning Plan →Your personalized Python learning plan Python Tutorials →In-depth articles and video courses Learning Paths →Guided study plans for accelerated learning

Practice
Quizzes & Exercises →Check your learning progress Browse Topics →Focus on a specific area or skill level Reference →Concise definitions for common Python terms

Get Help
Mentor AI →NewYour personal Python mentor Office Hours →Live Q&A calls with Python experts Community Chat →Learn with other Pythonistas

Go Deeper
Live Courses →Live, instructor-led Python courses Books →Round out your knowledge and learn offline Podcast →Hear what’s new in the world of Python

Unlock All Content and Become a Python Expert →Get unlimited access to all Real Python has to offer

Unlock All Content →Become a Python Expert
More
Learner Stories Python Newsletter Python Job Board Meet the Team Become a Contributor

Search

/

Join
Sign‑In

Loading exercise...

Exercise: Fetch a Profile Page by Name

Martin Breuss

Exercises Course: Introduction to Web Scraping With Python Martin Breuss 08:00

Recommended Tutorial Course Slides (.pdf) Sample Code (.zip)

Ask a Question

Contents
Discussion

Become a Member to join the conversation.

Course Contents

Overview

13%

Scraping and Parsing With urllib

Exercises Course: Introduction to Web Scraping With Python (Overview) 02:47

Scraping and Parsing Text From Websites Exercise 01:26

Checking Out the Website in Your Browser 02:44

Scraping the HTML 03:02

Exercise: Fetch a Profile Page by Name 08:00

Brainstorming a Solution in the Python REPL 05:17

Taking Notes About Each Subtask 02:22

Removing the Text Before the Info 03:33

Slicing Off the Text After the Info 03:07

Exercise: Get a Page's Title With String Methods 08:00

Generalizing and Cleaning Up the Code 05:03

Exercise: Collect Hometowns From Several Profiles 10:30

Using an HTML Parser

Using an HTML Parser Exercise 01:51

Inspecting the Site With Your Browser 01:23

Making a Plan and Reusing Code 01:51

Setting Up a venv and Installing Beautiful Soup 03:16

Exploring the Soup in Interactive Mode 03:36

Exercise: Pick Out the Profile Links With Beautiful Soup 08:00

Constructing the Scraping Script 02:20

Exercise: Build Full Image URLs From a Profile Page 08:00

Confirming the Script Works as Expected 01:40

Exercise: Scrape the Image URLs of a Profile 08:30

Interacting With Forms via Mechanical Soup

Interacting With HTML Forms Exercise 01:12

Exploring the Task in the Web Interface 01:03

Installing Mechanical Soup and Making a Plan 01:39

Solving the Task in Your REPL 04:40

Exercise: List a Login Form's Fields With MechanicalSoup 08:00

Converting the Solution Into a Script 03:09

Exercise: Log In to the Agora With MechanicalSoup 08:30

Remembering Where to Use Mechanical Soup 00:41

Introduction to Web Scraping With Python (Quiz) 06:00

Exercises Course: Introduction to Web Scraping With Python (Summary) 02:50

© 2012–2026 Real Python ⋅ Privacy Policy