Real Python Podcast Episode #123 Title Artwork

Episode 123: Creating a Python Code Completer & More Abstract Syntax Tree Projects

The Real Python Podcast

Sep 02, 2022 1h 13m

How does a code completion tool work? What is an Abstract Syntax Tree, and how is it created in Python? How does an AST help you write programs and projects that inspect and modify your Python code? This week on the show, Meredydd Luff, co-founder of Anvil, shares his PyCon talk, “Building a Python Code Completer.”

Episode Sponsor:

Meredydd talks about his experience building a code completion engine for the Anvil platform. The conversation leads us to discuss how Python parses the code that you type. We examine tokenization, abstract syntax trees, and how parsing has changed in Python.

We cover additional projects that you can explore once you have a tool that inspects the Python code you’re writing. Join us as we dive into multiple rabbit holes of research and exploration.

Course Spotlight: Python Assignment Expressions and Using the Walrus Operator

In this course, you’ll learn about assignment expressions and the walrus operator. The biggest change in Python 3.8 was the inclusion of the := operator, which you can use to assign variables in the middle of expressions. You’ll see several examples of how to take advantage of this feature.

Topics:

00:00:00 – Introduction
00:02:11 – Building a Python Code Completer - Talk
00:08:48 – How is recursion being used?
00:10:28 – Abstract Syntax Trees, Parsers, and Tokens
00:14:58 – Sponsor: CData Software
00:15:39 – Is parsing into tokens a separate step?
00:21:45 – What makes the new PEG parser unique?
00:25:32 – Using the AST doesn’t have to be scary.
00:27:33 – More on the PEG parser
00:29:54 – Keeping scope and holding completions
00:39:43 – Video Course Spotlight
00:41:15 – Was this your first expirement in creating a code completer?
00:46:46 – How can adding type hints assist a completer?
00:50:53 – Projects you can build using AST
00:53:02 – Linter project
00:54:15 – Automatic code formatter
00:54:59 – Code testing and the pytest project
00:56:22 – Security tools
00:57:11 – Friendlier tracebacks
00:59:57 – Code completion vs GitHub Copilot
01:04:14 – Code completion in a text editor vs IDE
01:08:53 – What are you excited about in the world of Python?
01:09:54 – What do you want to learn next?
01:11:20 – How can people follow your work?
01:12:31 – Thanks and goodbye

Show Links:

Level Up Your Python Skills With These Courses:

How to Write Beautiful Python Code With PEP 8

Writing Beautiful Pythonic Code With PEP 8

The Walrus Operator: Python's Assignment Expressions

Python Assignment Expressions and Using the Walrus Operator

Testing Your Code With pytest

← Previous All Episodes Next →