Exercise: Parse HTML Table and Save CSV

In the tutorial “Grabbing your data,” the command presented is as follows:

tables = pd.read_html(response.read())

This, however, creates a huge OSError message. This is due to a new version of pandas. The solution is:

import io

html_data = response.read().decode("utf-8")  # Read HTML content and convert into a string
html_stream = io.StringIO(html_data)  # Package the string into a virtual data object (StringIO)
tables = pd.read_html(html_stream)  # Hand over to Pandas

@Gerhard Mehler thanks for reporting this, and your fix is right.

Passing HTML content straight to pd.read_html() was deprecated in pandas 2.1. On pandas 2.x it still worked, but with a warning:

FutureWarning: Passing literal html to 'read_html' is deprecated and will be removed
in a future version. To read from a literal string, wrap it in a 'StringIO' object.

In pandas 3.0 it was removed, so the argument now gets treated as a file path or URL. That’s where your OSError comes from, a FileNotFoundError with the entire page of HTML echoed back as the “filename,” which is why the message is so enormous.

Your io.StringIO version is the documented replacement. One shortcut: since response.read() already gives you bytes, you can skip the decode step and hand pandas an io.BytesIO instead.

import io

with urllib.request.urlopen(request) as response:
    tables = pd.read_html(io.BytesIO(response.read()))

Both get you to the same place. The StringIO route is the one to remember for when you already have HTML sitting in a string.

I’ll get Grabbing Your Data updated, since it still shows the old call. Thanks for flagging it and for posting a working solution 🙂