caching

Caching is the practice of keeping copies of data, or the results of an expensive computation, in faster storage so that future requests for the same items are served quickly. A request that finds its data in the cache is a cache hit, and one that does not is a cache miss, which falls back to the slower original source. The fraction of requests served from the cache is the hit ratio, and this flow decides which path each request takes:

A request checks the cache. A hit returns fast. A miss fetches from the backing store, stores the copy, then returns it.
A Hit Returns Fast, a Miss Fetches Then Populates the Cache

Caching works because real workloads tend to reuse the same data, a pattern called locality of reference. Because a cache holds only a copy, that copy can drift out of step with the source when the source changes, leaving stale data. Knowing when to refresh or discard an entry, a problem called cache invalidation, is one of the hardest parts of the technique.

A cache has limited room, so once it fills, a replacement policy chooses which entry to evict:

  • Least recently used (LRU) drops the entry left untouched the longest.
  • Least frequently used (LFU) drops the entry with the fewest accesses.
  • First in, first out (FIFO) drops the oldest entry regardless of use.

Because each rule weighs a different signal, the same stream of requests can leave the three caches holding completely different entries. Step through the requests below and watch FIFO, LRU, and LFU each pick a different entry to evict, then finish with different hit ratios:

Interactive diagram — enable JavaScript to view.

Caches sit at nearly every layer of a computing system, from the L1, L2, and L3 caches on a processor to the operating system’s page cache, web caches and content delivery networks, and DNS resolvers. At the application level, storing a function’s return value against its arguments is called memoization, which Python provides through functools.

Caching in Python Using the LRU Cache Strategy

Tutorial

Caching in Python Using the LRU Cache Strategy

Caching is an essential optimization technique. In this tutorial, you'll learn how to use Python's @lru_cache decorator to cache the results of your functions using the LRU cache strategy. This is a powerful technique you can use to leverage the power of caching in your implementations.

intermediate python

For additional information on related topics, take a look at the following resources:

By Martin Breuss • Updated July 21, 2026
Computer Science Glossary Share Feedback