Anthony Shaw’s CPython Internals Book Is…
Your Guided Tour Through the Python 3.9 Interpreter
Unlock the Inner Workings of the Python Language, Compile the Python Interpreter From Source Code,
And Participate in the Development of CPython:
Are there certain parts of Python that just seem like magic?
Like, how dictionaries are so much faster than looping over a list to find an item…
How does a generator remember the state of the variables each time it yields a value—and why do you never have to allocate memory like in other languages?
It turns out, CPython, the most popular Python runtime is written in human-readable C and Python code…
The CPython interpreter abstracts away the complexities of the underlying C platform and your operating system.
It makes threading cross-platform and straightforward.
It takes the pain of memory management in C and makes it simple.
In short, CPython gives you the platform to write scalable and performant applications using the comfortable and powerful syntax of Python…
However, these abstractions aren’t perfect,
and they’re often leaky:
That’s why at some stage in your progression as a Python developer, you can benefit greatly from understanding how CPython works internally.
By “peeking behind the scenes” of the Python runtime, you’ll deepen your skills and become a true Pythonista…
You’ll get one step closer to truly mastering Python, enabling you to write beautiful & idiomatic code that’s also fast and efficient.
Once you see how Python works at the interpreter level, you can optimize your applications and fully leverage the power of Python
Few developers take the time to understand their tools at such a deep level—it’s a surefire way to impress colleagues and interviewers alike.
This book explains the concepts, ideas, and technicalities of CPython in an approachable and hands-on fashion:
Goals—
By the End of the Book You’ll Be Able To:
Read and navigate the CPython 3.9 interpreter source code. You’ll deeply comprehend and appreciate the inner workings of concepts like lists, dictionaries, and generators.
Make changes to the Python syntax and compile your own version of CPython, from scratch. You’ll customize the Python core data types with new functionality and run CPython’s automated test suite.
Master CPython’s memory management capabilities and scale your Python code with parallelism and concurrency.
Debug C and Python code like a true professional. Profile and benchmark the performance of your Python code and the runtime.
Participate in the development of CPython and know how to contribute to future versions of the Python interpreter and standard library. How great would it feel to give back to the community as a “Python Core Developer?”
CPython Internals:
Your Guide to the Python 3 Interpreter
With this book you’ll cover the critical concepts behind the internals of CPython and how they work with visual explanations as you go along.
Each page in the book has been carefully laid out with beautiful typography, syntax highlighting for code examples, and instructional illustrations so you can process and memorize the information effectively:
Take your time for each chapter and make sure you try out the examples and hands-on “code along” sections. Once you’ve completed the book, you’ll be able to use your newfound knowledge to:
-
Write custom extensions for Python, written in the C programming language (the book includes an “Intro to C for Pythonistas” chapter)
-
Use your deep knowledge of the CPython interpreter to improve your own Python applications
-
Contribute to the CPython project and start your journey towards becoming a Python Core Developer
Moreover, you’ll feel a sense of achievement that you fully grasp the core concepts of Python that make you a better and more effective Python programmer.
CPython Internals Book
$54.99
[ Or, get a print copy on Amazon. ]
About the Author
Anthony Shaw is a CPython contributor and a PyCon speaker. He has a B.Sc. in Computer Science and over 8 years of experience working with Python. Anthony is also a Fellow of the Python Software Foundation and member of the Open-Source Apache Foundation.
Table of Contents
& Sample Chapters
All of the material is completely up to date for Python 3.9, the latest version of the Python programming language.
In a nutshell, here’s what you’ll learn with CPython Internals: Your Guide to the Python Interpreter:
Chapter 1: Introduction How to make the most of this book so you can learn effectively and quickly get up to speed with CPython development.
Chapter 2: Getting the CPython Source Code The CPython source distribution comes with a whole range of tools, libraries, and components. You’ll start exploring them in this chapter.
Chapter 3: Setting Up Your Development Environment Throughout this book, you’ll be working with C and Python code. In this chapter you’ll see how to configure your development environment to support both languages.
Chapter 4: Compiling CPython Now that you have CPython downloaded, set up a development environment and configured it, you can compile the CPython source code into an executable interpreter.
Chapter 5: The Python Language and Grammar Understand why CPython is written in C and not Python and then dig into the Python Language Specification and it’s Parser Generator setup.
Chapter 6: Configuration and Input Now that you’ve seen the Python grammar, its time to explore how code gets input into a state that can be executed.
Chapter 7: Lexing and Parsing With Syntax Trees In the previous chapter, you explored how Python text is read from various sources. It needs to be converted into a structure that the compiler can use. This stage is “parsing.” In this chapter, you will explore how the text is parsed into logical structures that can be compiled.
Chapter 8: The Python Compiler In this chapter, you will focus on the compilation of an AST module into a code object that the CPython interpreter can execute. These code objects contain lists of discrete operations in the form of bytecode.
Chapter 9: The Evaluation Loop Execution of code in CPython happens within a central loop, called the “evaluation loop.” In this chapter you’ll get familiar with the evaluation loop as the “brain” of CPython.
Chapter 10: Memory Management By the end of this chapter, you will understand how CPython allocates memory on the Operating System, how object memory is allocated and freed, and how CPython manages memory leaks.
Chapter 11: Parallelism and Concurrency CPython offers many approaches to Parallelism and Concurrency. Your choice depends on several factors. There are also overlapping use cases across models as CPython has evolved. This major chapter of the book will make you familiar with the relevant concepts.
Chapter 12: Objects and Types Learn how common built-in types like strings, lists, tuples, and dictionaries are implemented in CPython. You’ll also implement a custom built-in operator on Python strings in this chapter.
Chapter 13: The Python Standard Library There are two types of modules in the CPython standard library: those written in pure Python that provide a utility, and those written in C with Python wrappers. You will explore both types in this chapter.
Chapter 14: The CPython Test Suite Learn how to run CPython’s extensive test suite covering the core interpreter, the standard library, the tooling, and distribution on Windows, Linux, and macOS. If you want to get involved in the CPython project, writing or extending unit tests is a great place to start.
Chapter 15: Debugging In this chapter, you will learn how to attach a debugger to the CPython interpreter and how to use the debugger to see inside a running CPython process.
Chapter 16: Benchmarking, Profiling, and Tracing In this chapter, you have explored benchmarking, profiling, and tracing using a number of tools designed for CPython. With the right tooling, you can find bottlenecks, compare performance of multiple builds and identify improvements.
Chapter 17: Conclusion & Further Reading How you can use your newfound knowledge to extend the functionality of Python and to contribute to the CPython project.
Appendix: Introduction to C for Python Programmers This introduction is intended to get an experienced Python programmer up to speed with the basics of the C language and how it’s used in the CPython source code.
Here’s What’s “In The Box”
To recap, let’s take another look at what you’ll receive with this offer:
-
DRM-free PDF book, optimized for reading on tablets and computers (383 pages)
-
DRM-free ePub book
-
DRM-free Kindle book
-
Downloadable sample code & examples
-
Free updates
CPython Internals Book
$54.99
[ Or, get a print copy on Amazon. ]
What Pythonistas Say About the Book
“Why do I want to share Anthony’s CPython Internals with you? It’s the book that I wish existed years ago when I started my Python journey.
More importantly, I believe we, as members of the Python community, have a unique opportunity – to put our expertise to work to help solve the complex real-world problems facing us.
Python’s strengths include its readability and a welcoming community dedicated to education.
Anthony embraces these strengths when explaining CPython, encouraging you to read the source, and sharing the building blocks of the language with you.
I’m confident after reading this book your skills will grow and you will be able solve even more complex problems that can improve our world.”
— Carol Willing, CPython Core Developer & Member of the CPython Steering Council
“A comprehensive walkthrough of the Python internals, a topic which surprisingly has almost no good resource, in an easy-to-understand manner for both beginners as well as advanced Python users.”
— Abhishek Sharma, Data Scientist
“What impresses me the most about Anthony’s book is how it puts all the steps for making changes to the CPython code base in an easy to follow sequence. It really feels like a ‘missing manual’ of sorts.
Diving into the C underpinnings of Python was a lot of fun and it cleared up some longstanding questions marks for me. I found the chapter about CPython’s memory allocator especially enlightening.
CPython Internals is a great (and unique) resource for anybody looking to take their knowledge of Python to a deeper level.”
— Dan Bader, Author of Python Tricks & Editor-in-Chief at Real Python
“The ‘Parallelism and Concurrency’ chapter is one of my favorites. I had been looking to get an in depth understanding around this topic and I found your book extremely helpful.
Of course, after going over that chapter I couldn’t resist the rest. I am eagerly looking forward to have my own printed copy once it's out!
I had gone through your ‘Guide to CPython Source code’ article previously which got me interested in finding out more about the internals.
There are a ton of books on Python which teach the language, but I haven’t really come across anything that would go about explaining the internals to those curious minded.
And while I teach Python to my daughter currently, I have this book added in her must-read list. She’s currently studying Information Systems at Georgia State University.”
— Milan Patel, Vice President at (a major investment bank)
“I can recommend CPython Internals to anyone who wants to get going with hacking on CPython”
— Guido van Rossum, Creator of Python
1 Year “Try It Yourself” Guarantee—
100% Money-Back Refund Policy
The last thing we want is for you to pass on this book because you’re not sure it’ll work for you. We know it will, so here’s our offer to you:
Put us to the test. Take “CPython Internals: Your Guide to the Python 3 Interpreter” for a spin today.
Spend just a few days with the material and start building your Python skills.
Then take up to a full year to grow them…
And if for some reason you don’t agree that we’ve given you a faster, easier way to absorb the knowledge you need to rise head-and-shoulders above other developers, we’ll cheerfully refund your money, no questions and no hassles, any time within the next year.
Either way, all the risk is on us.
CPython Internals Book
$54.99
[ Or, get a print copy on Amazon. ]
Frequently Asked Questions
Got any sample chapters? Sure do! Click here to download a PDF sample. Enjoy!
What if I don’t like the book? We want happy customers only. If you’re not happy with the digital version of the book, send us an email and tell us where it left you hanging. We’ll refund you in full—even 12 months later.
Can I order a print version? Yes, CPython Internals is available on Amazon as a printed book (paperback). Click here to order a printed copy.
How skilled do I need to be in Python to use this book? This book is aimed at intermediate to advanced Python developers. Every effort has been taken to break down code examples, but some intermediate Python techniques will be used throughout the book. If you’re not sure you’re ready, making your way through the chapters of the book is a great way to challenge yourself and identify and fix weak spots in your Python knowledge.
Do I need to know C to use this book? You do not need to be proficient in C to use this book. If you are new to C, check out Appendix 1: Introduction to C for Python Programmers at the back of the book for a quick introduction.
How long will it take to finish this book? “I don’t recommend rushing this book, try reading a chapter at a time, trying the examples after each chapter and exploring the code simultaneously. Once you’ve finished the book, it will make a great reference guide for you to come back to in time. The chapters are fairly self-contained too, so you’ll be able to jump around and focus on the sections that interest you the most.” (Anthony Shaw)
Won’t the CPython specifics I’ll learn be out of date quickly? “Python has been around for over 30 years. Some parts of the CPython code haven’t been touched since they were originally written. Many of the principles in this book have been the same for ten or more years. In fact, whilst writing this book, I discovered many lines of code written by Guido van Rossum (the author of Python) and untouched since CPython version 1.0.
The skills you’ll learn in this book will help you read and understand current and future versions of CPython. Change is constant, and your expertise is something you can develop along the way. Some of the concepts in this book are brand-new; some are even experimental.
While writing this book, I came across issues in the source code and bugs in CPython. Then, they got fixed or improved. That’s part of the wonder of CPython as a flourishing open-source project.” (Anthony Shaw)
How will I receive updates? We announce updates via email as they come out. Plus, you’ll always find the latest version of the book available for download in your realpython.com account.
I have another question! Please email us at info@realpython.com and we’ll get back to you as soon as possible.
CPython Internals Book
$54.99