Real Python Podcast Episode #201 Title Artwork

Episode 201: Decoupling Systems to Get Closer to the Data

The Real Python Podcast

Apr 19, 2024 1h 9m

RSS Apple Spotify YouTube
Podcast Addict Overcast Pocket Casts CastBox Stitcher Download MP3
Christopher Bailey
Christopher Bailey
Host
Phillip Cloud
Phillip Cloud
Guest

What are the benefits of using a decoupled data processing system? How do you write reusable queries for a variety of backend data platforms? This week on the show, Phillip Cloud, the lead maintainer of Ibis, will discuss this portable Python dataframe library.

Episode Sponsor:

Phillip contrasts Ibis’s workflow with other Python dataframe libraries. We discuss how “getting close to the data” speeds things up and conserves memory.

He describes the different approaches Ibis provides for querying data and how to select a specific backend. We discuss ways to get started with the library and how to access example data sets to experiment with the platform.

Phillip discovered Ibis while looking for a tool that allowed him to reuse SQL queries written for a specific data platform on a different one. He recounts how he got involved with the Ibis project, sharing his background in open source and learning how to contribute to a first project.

This episode is sponsored by Mailtrap.

Topics:

  • 00:00:00 – Introduction
  • 00:02:18 – How did you get started with Ibis?
  • 00:08:10 – First contribution to open source
  • 00:13:46 – Comparing Ibis to other dataframe libraries
  • 00:20:09 – Sponsor: Mailtrap
  • 00:20:43 – What goes into the selection of backend?
  • 00:27:07 – Database connections vs SQL compilers
  • 00:30:03 – Raw SQL approach
  • 00:34:06 – Dataframe approach
  • 00:38:31 – What does “getting close to the data” mean?
  • 00:41:52 – Video Course Spotlight
  • 00:43:24 – Phillip in the cloud - YouTube channel
  • 00:44:56 – Access to sample data sets
  • 00:50:11 – Additional resources
  • 00:52:50 – What are some of the backends Ibis supports?
  • 00:54:13 – Entry points to the platform
  • 00:55:00 – How are you supported?
  • 00:57:10 – Exporting a SQL query
  • 00:59:23 – What are you excited about in the world of Python?
  • 01:04:28 – What do you want to learn next?
  • 01:07:12 – How can people follow your work online?
  • 01:08:00 – Thanks and goodbye

Show Links:

Level Up Your Python Skills With These Courses:

Build Your Python Project Documentation With MkDocs

Building Python Project Documentation With MkDocs

Python Folium: Create Web Maps From Your Data

Creating Web Maps From Your Data With Python Folium

Python's raise: Effectively Raising Exceptions in Your Code

Using raise for Effective Exceptions

← Previous All Episodes Next →