Real Python Podcast Episode #167 Title Artwork

Episode 167: Exploring pandas 2.0 & Targets for Apache Arrow

The Real Python Podcast

Aug 04, 2023 1h 14m

What are the new ways to describe your data in pandas 2.0? Will the addition of Apache Arrow to the data back end foster the growth of data interoperability? This week on the show, we talk with pandas core developer Marc Garcia about the release of pandas 2.0.

Marc shares his background and work on pandas. We discuss the history of data representation in pandas and the need to move beyond NumPy. We also talk about how Apache Arrow only solves some of the issues.

We dig into the potential of an Apache Arrow back end and how it could offer interoperability between data platforms. We also cover the moderate adoption and backward-compatibility concerns. Marc also shares his thoughts on making pandas more extensible.


  • 00:00:00 – Introduction
  • 00:02:07 – Getting involved with the pandas project
  • 00:03:48 – Continued growth of the platform
  • 00:06:49 – Parallel branch development
  • 00:09:19 – The introduction of Apache Arrow
  • 00:18:53 – Working with NumPy data in pandas
  • 00:30:18 – Arrow data types and strings
  • 00:41:23 – Video Course Spotlight
  • 00:42:37 – Interoperability of Arrow data back end
  • 00:50:36 – Could pandas be more extensible?
  • 01:00:49 – Python DataFrame Summit 2023
  • 01:08:12 – What are you excited about in the world of Python?
  • 01:11:13 – What do you want to learn next?
  • 01:12:12 – How can people follow your work online?
  • 01:13:46 – Thanks and Goodbye

Show Links: