Python Performance Optimization: A Practical Guide

Python performance optimization is the practice of making your code run faster. You find the bottleneck first, then fix only what’s slow, starting with algorithms and data structures before reaching for concurrency or C extensions.

You’ve written a Python script that works perfectly until you run it on real data and it crawls, or quietly consumes far more memory than it should. The results are correct, and the tests pass, so the code isn’t broken. It’s just inefficient.

The trap most developers fall into here is guessing. You stare at your code, decide which part looks slow, and rewrite it. Sometimes you get lucky. Often, you burn an afternoon speeding up a function that was never the problem. In this tutorial, you’ll learn to measure first and choose the right fix for a slowdown so you can stop guessing and start diagnosing.

By the end of this tutorial, you’ll understand that:

• Measuring with timeit confirms a bottleneck you already suspect, while a profiler finds one in an unfamiliar codebase.
• Switching a membership test from a list to a set turns an O(n) scan into a lookup that’s constant time on average.
• Python’s built-ins and standard library often push per-item work down into C, so Counter counts faster than a handwritten loop.
• functools.cache stores results by argument, so it’s only safe when a function’s output depends solely on its arguments.
• The fixes here have a natural order, starting with the cheap, safe ones and escalating to more invasive ones only when the simpler fixes aren’t enough.

One thing before you start: this is a conceptual guide. The mindset comes first, and the tools follow. Once you understand the order in which to tackle a performance problem, the specific techniques become much easier to apply.

Before you dig in, these two related tutorials are worth bookmarking:

• Common Python Data Structures (Guide)
• Speed Up Your Python Program With Concurrency

With those on hand, you’ll start by pinning down what optimization actually means.

What Is Python Performance Optimization?

Before you can fix a performance problem, you need to understand what you’re actually trying to do. This section starts with a clear definition and a mental model to guide you through the rest of the tutorial.

The Goal of Optimization

Here’s the part that trips people up: optimization isn’t the same as fixing bugs. Your code already works, so the question isn’t whether it produces the right answer but how efficiently it gets there. You pursue that efficiency deliberately, not by tinkering at random.

The Doctor Analogy

Think about how a good doctor works. You walk in feeling unwell, and they don’t immediately hand you a prescription. They run tests first, then treat the specific problem the results reveal. Prescribing medication before a diagnosis would be reckless.

When you optimize without measuring, you’re doing exactly that: prescribing a cure for a disease you haven’t diagnosed. The bottleneck—the specific part of your program that consumes the most time or memory—is the disease.

Keep this picture in mind because it anchors everything that follows: diagnose, confirm, then treat. Profilers do the diagnosing, and Real Python has tutorials devoted to them. This tutorial picks up at the confirmation step and focuses on treatment.

Now that you have a clear definition and the right mental model, you’re ready to put them to work. In the next section, you’ll first confirm your suspicion, then choose the fix that matches what you find.

Measure First With timeit

Before you fix anything, you need to know whether it’s actually slow and by how much. Locating a bottleneck in a large, unfamiliar program is a job for a profiler like cProfile in the standard library or the third-party line_profiler. To go deeper on both, read Profiling in Python: How to Find Performance Bottlenecks and Python Timer Functions: Three Ways to Monitor Your Code.

This tutorial isn’t about that process. It picks up one step later, once you already suspect a specific function is the problem and want to confirm it. This is where timeit comes in. It doesn’t hunt for bottlenecks. It measures one so you can confirm your suspicion with a real number instead of a guess.

Suppose you’re processing a batch of customer orders, and you suspect that the function for calculating each order’s total is holding up the job. Timing it over half a million calls gives you the two numbers you need:

import timeit

def calculate_order_total(items):
    total = 0
    for item in items:
        total = total + item["price"] * item["quantity"]
    return total

order = [
    {"price": 19.99, "quantity": 3},
    {"price": 5.50, "quantity": 10},
    {"price": 42.00, "quantity": 1},
] * 100

runs = 500_000

print(f"{calculate_order_total(order):.2f}")

order_time = timeit.timeit(
    lambda: calculate_order_total(order), number=runs
)
print(f"total for {runs:,} runs: {order_time:.4f} seconds")
print(f"per call: {order_time / runs * 1_000_000:.1f} microseconds")

$ python step-1.py
15697.00
total for 500,000 runs: 4.4478 seconds
per call: 8.9 microseconds