`mmap`

The Python mmap module allows you to map files or devices into memory, allowing you to read and modify them in place.

This module can help you improve performance, especially when working with large files, as it enables file I/O operations to be performed directly in memory without the need for additional copying.

Here’s a quick example:

Python
      
        
      
    
>>> import mmap

>>> # Create a memory-mapped file
>>> with open("example.txt", mode="rb") as file:
...     with mmap.mmap(file.fileno(), length=0, access=mmap.ACCESS_READ) as mm:
...             print(mm[:5])
...
b'Hello'

For this example to work, you need a file called example.txt containing the string "Hello, World!".

Key Features

Maps files or devices into memory
Allows in-place modification of files
Supports both read and write operations
Can be used to share memory between processes

Frequently Used Classes and Functions

Object	Type	Description
`mmap.mmap`	Class	Creates a memory-mapped file or device
`mmap.ACCESS_READ`	Constant	Provides read-only access to a memory-mapped file
`mmap.ACCESS_WRITE`	Constant	Provides write-through access to a memory-mapped file
`mmap.ACCESS_COPY`	Constant	Provides copy-on-write access to a memory-mapped file

Examples

Create a memory-mapped file and read from it:

Python
      
        
      
    
>>> with open("example.txt", mode="rb") as file:
...     with mmap.mmap(file.fileno(), length=0, access=mmap.ACCESS_READ) as mm:
...             print(mm.read())
...
b'Hello, World!\n'

Common Use Cases

Efficiently reading and writing large files
Modifying files in place without additional copying
Sharing memory between processes for inter-process communication

Real-World Example

Suppose you have a large log file that you need to process and update specific entries without loading the entire file into memory. You can use mmap to map the file and make modifications directly:

Python
      
        
      
    
>>> import mmap

>>> with open("large_log.txt", mode="r+b") as file:
...     with mmap.mmap(file.fileno(), length=0, access=mmap.ACCESS_COPY) as mm:
...         if b"ERROR" in mm:
...             start = mm.find(b"ERROR")
...             mm[start:start+5] = b"FIXED"
...

In this example, mmap allows you to find and replace occurrences of "ERROR" with "FIXED" directly in the file, showcasing its utility in handling large files efficiently.

Note that for this code to work, the replacement content must be the same length as the target content. If the replacement isn’t exactly the same length as the target, Python will raise a ValueError exception.

Tutorial

Python mmap: Improved File I/O With Memory Mapping

In this tutorial, you'll learn how to use Python's mmap module to improve your code's performance when you're working with files. You'll get a quick overview of the different types of memory before diving into how and why memory mapping with mmap can make your file I/O operations faster.

intermediate python

For additional information on related topics, take a look at the following resources: