Real Python Podcast Episode #172 Title Artwork

Episode 172: Measuring Multiple Facets of Python Performance With Scalene

The Real Python Podcast

Sep 15, 2023 1h 3m

When choosing a tool for profiling Python code performance, should it focus on the CPU, GPU, memory, or individual lines of code? What if it looked at all those factors and didn’t alter code performance while measuring it? This week on the show, we talk about Scalene with Emery Berger, Professor of Computer Science at the University of Massachusetts Amherst.

Emery talks about his background in memory management and his collaboration on Hoard, a scalable memory manager system used in Mac OS X. We discuss the need for improving code performance on modern computer architecture. He highlights this idea by contrasting the familiar limitations of Moore’s law with the lesser-known rule of Dennard scaling.

Working with his students in the university lab, they developed Scalene. Scalene is a high-performance CPU, GPU, and memory profiler. It can look at code from the individual function or line-by-line level and compare time spent in Python vs C code. Emery talks about the recent Scalene feature of AI-powered optimization proposals and covers a couple of examples. He also shares a collection of additional Python code-assistant tools from their lab.

Course Spotlight: What Does if name == “main” Mean in Python?

In this video course, you’ll learn all about Python’s name-main idiom. You’ll learn what it does in Python, how it works, when to use it, when to avoid it, and how to refer to it.

Topics:

00:00:00 – Introduction
00:02:13 – College of Information and Computer Sciences
00:03:25 – Memory management systems background
00:05:15 – Dennard Scaling vs Moore’s Law
00:10:12 – Starting work on Python profiling
00:15:00 – Deciding on a statistical profiler
00:17:05 – Wanting to trace memory
00:21:21 – Finding memory issues
00:23:59 – Line-by-line analysis
00:25:56 – Video Course Spotlight
00:27:14 – Measuring profiler performance
00:30:30 – Memory leak detection
00:34:31 – When should you run a profiler?
00:37:27 – Considerations for measuring cloud performance
00:39:12 – Working with Jupyter and Conda
00:42:18 – Common issues and AI solutions
00:45:50 – Using a profiler to learn a codebase
00:50:48 – Examples of AI-powered optimizations
00:55:50 – What are you excited about in the world of Python?
00:58:30 – What do you want to learn next?
01:01:48 – How can people follow your work online?
01:02:56 – Thanks and goodbye

Show Links:

Level Up Your Python Skills With These Courses:

Debugging in Python With pdb

Testing Your Code With pytest

What Does if __name__ == "__main__" Do in Python?

What Does if __name__ == "__main__" Mean in Python?

← Previous All Episodes Next →