Python's Counter: The Pythonic Way to Count Objects

Constructing Counters

There are a few ways for you to create Counter instances. However, if your goal is to count several objects at once, then you need to use a sequence or iterable to initialize the counter. For example, here’s how you can rewrite the Mississippi example using Counter:

>>> from collections import Counter

>>> # Use a string as an argument
>>> Counter("mississippi")
Counter({'i': 4, 's': 4, 'p': 2, 'm': 1})

>>> # Use a list as an argument
>>> Counter(list("mississippi"))
Counter({'i': 4, 's': 4, 'p': 2, 'm': 1})

Counter iterates over "mississippi" and produces a dictionary with the letters as keys and their frequency as values. In the first example, you use a string as an argument to Counter. You can also use lists, tuples, or any iterables with repeated objects, as you see in the second example.

There are other ways to create Counter instances. However, they don’t strictly imply counting. For example, you can use a dictionary containing keys and counts like this:

>>> from collections import Counter

>>> Counter({"i": 4, "s": 4, "p": 2, "m": 1})
Counter({'i': 4, 's': 4, 'p': 2, 'm': 1})

The counter now has an initial group of key-count pairs. This way to create a Counter instance is useful when you need to provide initial counts of an existing group of objects.

You can also produce similar results by using keyword arguments when you call the class’s constructor:

>>> from collections import Counter

>>> Counter(i=4, s=4, p=2, m=1)
Counter({'i': 4, 's': 4, 'p': 2, 'm': 1})

Again, you can use this approach to create a Counter object with a specific initial state for its keys and counts pairs.

In practice, if you’re using Counter to count things from scratch, then you don’t need to initialize the counts since they have a zero value by default. Another possibility might be to initialize the counts to 1. In that case, you can do something like this:

>>> from collections import Counter

>>> Counter(set("mississippi"))
Counter({'p': 1, 's': 1, 'm': 1, 'i': 1})

Python sets store unique objects, so the call to set() in this example throws out the repeated letters. After this, you end up with one instance of each letter in the original iterable.

Counter inherits the interface of regular dictionaries. However, it doesn’t provide a working implementation of .fromkeys() to prevent ambiguities, such as Counter.fromkeys("mississippi", 2). In this specific example, each letter would have a default count of 2 despite of its current number of occurrences in the input iterable.

There are no restrictions on the objects you can store in the keys and values of a counter. The keys can store hashable objects, whereas the values can store any objects. However, to work as counters, the values should be integer numbers representing counts.

Here’s an example of a Counter instance that holds negative and zero counts:

>>> from collections import Counter

>>> inventory = Counter(
...     apple=10,
...     orange=15,
...     banana=0,
...     tomato=-15
... )

In this example, you may ask, “Why do I have -15 tomatoes?” Well, that could be an internal convention to signal that you have a client’s order for 15 tomatoes, and you don’t have any in your current inventory. Who knows? Counter allows you to do this, and you can probably find a few use cases for the feature.

Updating Object Counts

Once you have a Counter instance in place, you can use .update() to update it with new objects and counts. Rather than replacing values like its dict counterpart, the .update() implementation provided by Counter adds existing counts together. It also creates new key-count pairs when necessary.

You can use .update() with both iterables and mappings of counts as arguments. If you use an iterable, the method counts its items and updates the counter accordingly:

>>> from collections import Counter

>>> letters = Counter({"i": 4, "s": 4, "p": 2, "m": 1})

>>> letters.update("missouri")
>>> letters
Counter({'i': 6, 's': 6, 'p': 2, 'm': 2, 'o': 1, 'u': 1, 'r': 1})

Now you have 6 instances of i, 6 instances of s, and so on. You also have some new key-count pairs, such as 'o': 1, 'u': 1, and 'r': 1. Note that the iterable needs to be a sequence of items rather than a sequence of (key, count) pairs.

>>> from collections import Counter

>>> letters = Counter({"i": "4", "s": "4", "p": "2", "m": "1"})

>>> letters.update("missouri")
Traceback (most recent call last):
  ...
TypeError: can only concatenate str (not "int") to str

The second way to use .update() is to provide another counter or a mapping of counts as an argument. In that case, you can do something like this:

>>> from collections import Counter
>>> sales = Counter(apple=25, orange=15, banana=12)

>>> # Use a counter
>>> monday_sales = Counter(apple=10, orange=8, banana=3)
>>> sales.update(monday_sales)
>>> sales
Counter({'apple': 35, 'orange': 23, 'banana': 15})

>>> # Use a dictionary of counts
>>> tuesday_sales = {"apple": 4, "orange": 7, "tomato": 4}
>>> sales.update(tuesday_sales)
>>> sales
Counter({'apple': 39, 'orange': 30, 'banana': 15, 'tomato': 4})

In the first example, you update an existing counter, sales, using another counter, monday_sales. Note how .update() adds the count from both counters.

Next, you use a regular dictionary containing items and counts. In this case, .update() adds the counts of existing keys and creates the missing key-count pairs.

Accessing the Counter’s Content

As you already know, Counter has almost the same interface as dict. You can perform nearly the same actions with counters as you can with standard dictionaries. For example, you can access their values using dictionary-like key access ([key]). You can also iterate over the keys, values, and items using the usual techniques and methods:

>>> from collections import Counter

>>> letters = Counter("mississippi")
>>> letters["p"]
2
>>> letters["s"]
4

>>> for letter in letters:
...     print(letter, letters[letter])
...
m 1
i 4
s 4
p 2

>>> for letter in letters.keys():
...     print(letter, letters[letter])
...
m 1
i 4
s 4
p 2

>>> for count in letters.values():
...     print(count)
...
1
4
4
2

>>> for letter, count in letters.items():
...     print(letter, count)
...
m 1
i 4
s 4
p 2

In these examples, you access and iterate over the keys (letters) and values (counts) of your counter using the familiar dictionary interface, which includes methods such as .keys(), .values(), and .items().

A final point to note about Counter is that if you try to access a missing key, then you get zero instead of a KeyError:

>>> from collections import Counter

>>> letters = Counter("mississippi")
>>> letters["a"]
0

Since the letter "a" doesn’t appear in the string "mississippi", the counter returns 0 when you try to access the count for that letter.

Finding Most Common Objects

If you need to list a group of objects according to their frequency, or the number of times they appear, then you can use .most_common(). This method returns a list of (object, count) sorted by the objects’ current count. Objects with equal counts come in the order they first appear.

If you supply an integer number n as an argument to .most_common(), then you get the n most common objects. If you omit n or set it to None, then .most_common() returns all the objects in the counter:

>>> from collections import Counter
>>> sales = Counter(banana=15, tomato=4, apple=39, orange=30)

>>> # The most common object
>>> sales.most_common(1)
[('apple', 39)]

>>> # The two most common objects
>>> sales.most_common(2)
[('apple', 39), ('orange', 30)]

>>> # All objects sorted by count
>>> sales.most_common()
[('apple', 39), ('orange', 30), ('banana', 15), ('tomato', 4)]

>>> sales.most_common(None)
[('apple', 39), ('orange', 30), ('banana', 15), ('tomato', 4)]

>>> sales.most_common(20)
[('apple', 39), ('orange', 30), ('banana', 15), ('tomato', 4)]

In these examples, you use .most_common() to retrieve the most frequent objects in sales. With no argument or with None, the method returns all the objects. If the argument to .most_common() is greater than the current counter’s length, then you get all the objects again.

You can also get the least-common objects by slicing the result of .most_common():

>>> from collections import Counter
>>> sales = Counter(banana=15, tomato=4, apple=39, orange=30)

>>> # All objects in reverse order
>>> sales.most_common()[::-1]
[('tomato', 4), ('banana', 15), ('orange', 30), ('apple', 39)]

>>> # The two least-common objects
>>> sales.most_common()[:-3:-1]
[('tomato', 4), ('banana', 15)]

The first slicing, [::-1], returns all the objects in sales in reverse order according to their respective counts. The slicing [:-3:-1] extracts the last two objects from the result of .most_common(). You can tweak the number of least-common objects you get by changing the second offset value in the slicing operator. For example, to get the three least-frequent objects, you can change -3 to -4, and so on.

If you want .most_common() to work correctly, then make sure that the values in your counters are sortable. This is something to keep in mind because, as mentioned, you can store any data types in a counter.

Counting Letters in a Text File

Say you have a file that contains some text. You need to count the number of times each letter appears in the text. For example, say you have a file called pyzen.txt with the following content:

The Zen of Python, by Tim Peters

Beautiful is better than ugly.
Explicit is better than implicit.
Simple is better than complex.
Complex is better than complicated.
Flat is better than nested.
Sparse is better than dense.
Readability counts.
Special cases aren't special enough to break the rules.
Although practicality beats purity.
Errors should never pass silently.
Unless explicitly silenced.
In the face of ambiguity, refuse the temptation to guess.
There should be one-- and preferably only one --obvious way to do it.
Although that way may not be obvious at first unless you're Dutch.
Now is better than never.
Although never is often better than *right* now.
If the implementation is hard to explain, it's a bad idea.
If the implementation is easy to explain, it may be a good idea.
Namespaces are one honking great idea -- let's do more of those!

Yes, this is The Zen of Python, a list of guiding principles that define the core philosophy behind Python’s design. To count the number of times each letter appears in this text, you can take advantage of Counter and write a function like this:

 1# letters.py
 2
 3from collections import Counter
 4
 5def count_letters(filename):
 6    letter_counter = Counter()
 7    with open(filename) as file:
 8        for line in file:
 9            line_letters = [
10                char for char in line.lower() if char.isalpha()
11            ]
12            letter_counter.update(Counter(line_letters))
13    return letter_counter

Here’s how this code works:

• Line 5 defines count_letters(). This function takes a string-based file path as an argument.
• Line 6 creates an empty counter for counting the letters in the target text.
• Line 7 opens the input file for reading and creates an iterator over the file content.
• Line 8 starts a loop that iterates through the file content line by line.
• Lines 9 to 11 define a list comprehension to exclude nonletter characters from the current line using .isalpha(). The comprehension lowercases the letters before filtering them to prevent having separate lowercase and uppercase counts.
• Line 12 calls .update() on the letters counter to update the counts of each letter.

To use count_letters(), you can do something like this:

>>> from letters import count_letters
>>> letter_counter = count_letters("pyzen.txt")

>>> for letter, count in letter_counter.items():
...     print(letter, "->", count)
...
t -> 79
h -> 31
e -> 92
z -> 1
  ...
k -> 2
v -> 5
w -> 4

>>> for letter, count in letter_counter.most_common(5):
...     print(letter, "->", count)
...
e -> 92
t -> 79
i -> 53
a -> 53
s -> 46

Great! Your code counts the frequency of every letter in a given text file. Linguists often use letter frequency for language identification. In English, for example, studies on the average letter frequency have revealed that the five most common letters are “e,” “t,” “a,” “o,” and “i.” Wow! That almost matches your results!

Plotting Categorical Data With ASCII Bar Charts

Statistics is another field in which you can use Counter. For example, when you’re working with categorical data, you might want to create bar charts to visualize the number of observations per category. Bar charts are especially handy for plotting this type of data.

Now say you want to create a function that allows you to create ASCII bar chart on your terminal. To do that, you can use the following code:

# bar_chart.py

from collections import Counter

def print_ascii_bar_chart(data, symbol="#"):
    counter = Counter(data).most_common()
    chart = {category: symbol * frequency for category, frequency in counter}
    max_len = max(len(category) for category in chart)
    for category, frequency in chart.items():
        padding = (max_len - len(category)) * " "
        print(f"{category}{padding} |{frequency}")

In this example, print_ascii_bar_chart() takes some categorical data, counts the number of times each unique category appears in the data (frequency), and generates an ASCII bar chart that reflects that frequency.

Here’s how you can use this function:

>>> from bar_chart import print_ascii_bar_chart

>>> letters = "mississippimississippimississippimississippi"
>>> print_ascii_bar_chart(letters)
i |################
s |################
p |########
m |####

>>> from collections import Counter
>>> sales = Counter(banana=15, tomato=4, apple=39, orange=30)

>>> print_ascii_bar_chart(sales, symbol="+")
apple  |+++++++++++++++++++++++++++++++++++++++
orange |++++++++++++++++++++++++++++++
banana |+++++++++++++++
tomato |++++

The first call to print_ascii_bar_chart() plots the frequency of each letter in the input string. The second call plots sales per fruit. In this case, you use a counter as input. Also, note that you can use symbol to change the character for the bars.

When you’re creating bar charts, using horizontal bars allows you to have enough room for the category labels. Another helpful feature of bar charts is the possibility of sorting the data according to their frequency. In this example, you sort the data using .most_common().

Plotting Categorical Data With Matplotlib

It’s nice to know how to create ASCII bar charts from scratch using Python. However, in the Python ecosystem, you can find several tools for plotting data. One of those tools is Matplotlib.

Matplotlib is a third-party library for creating statical, animated, and interactive visualizations in Python. You can install the library from PyPI using pip as usual:

$ python -m pip install matplotlib

This command installs Matplotlib in your Python environment. Once you’ve installed the library, you can use it to create your bar charts and more. Here’s how you can create a minimal bar chart with Matplotlib:

>>> from collections import Counter
>>> import matplotlib.pyplot as plt

>>> sales = Counter(banana=15, tomato=4, apple=39, orange=30).most_common()
>>> x, y = zip(*sales)
>>> x
('apple', 'orange', 'banana', 'tomato')
>>> y
(39, 30, 15, 4)

>>> plt.bar(x, y)
<BarContainer object of 4 artists>
>>> plt.show()

Here, you first do the required imports. Then you create a counter with some initial data about fruit sales and use .most_common() to sort the data.

You use zip() to unzip the content of sales into two variables:

1. x holds a list of fruits.
2. y holds the corresponding units sold per fruit.

Then you create a bar chart using plt.bar(). When you run plt.show(), you get a window like the following on your screen:

In this chart, the horizontal axis shows the name of each unique fruit. Meanwhile, the vertical axis indicates the number of units sold per fruit.

Finding the Mode of a Sample

In statistics, the mode is the most frequent value (or values) in a sample of data. For example, if you have the sample [2, 1, 2, 2, 3, 5, 3], then the mode is 2 because it appears most frequently.

In some cases, the mode isn’t a unique value. Consider the sample [2, 1, 2, 2, 3, 5, 3, 3]. Here you have two modes, 2 and 3, because both appear the same number of times.

You’ll often use the mode to describe categorical data. For example, the mode is useful when you need to know which category is the most common in your data.

To find the mode with Python, you need to count the number of occurrences of each value in your sample. Then you have to find the most frequent value (or values). In other words, the value with the highest number of occurrences. That sounds like something you can do using Counter and .most_common().

Here’s a function that computes the mode of a sample:

# mode.py

from collections import Counter

def mode(data):
    counter = Counter(data)
    _, top_count = counter.most_common(1)[0]
    return [point for point, count in counter.items() if count == top_count]

Inside mode(), you first count the number of times each observation appears in the input data. Then you use .most_common(1) to get the frequency of the most common observation. Since .most_common() returns a list of tuples in the form (point, count), you need to retrieve the tuple at index 0, which is the most common in the list. Then you unpack the tuple into two variables:

1. _ holds the most common object. Using an underscore to name a variable suggests that you don’t need to use that variable in your code, but you need it as a placeholder.
2. top_count holds the frequency of the most common object in data.

The list comprehension compares the count of each object with the count of the most common one, top_count. This allows you to identify multiple modes in a given sample.

To use this function, you can do something like this:

>>> from collections import Counter
>>> from mode import mode

>>> # Single mode, numerical data
>>> mode([2, 1, 2, 2, 3, 5, 3])
[2]

>>> # Multiple modes, numerical data
>>> mode([2, 1, 2, 2, 3, 5, 3, 3])
[2, 3]

>>> # Single mode, categorical data
>>> data = [
...     "apple",
...     "orange",
...     "apple",
...     "apple",
...     "orange",
...     "banana",
...     "banana",
...     "banana",
...     "apple",
... ]

>>> mode(data)
['apple']

>>> # Multiple modes, categorical data
>>> mode(Counter(apple=4, orange=4, banana=2))
['apple', 'orange']

Your mode() works! It finds the mode of numerical and categorical data. It also works with single-mode and multimode samples. Most of the time, your data will come in a sequence of values. However, the final example shows that you can also use a counter to provide the input data.

Counting Files by Type

Another interesting example involving Counter is to count the files in a given directory, grouping them by file extension or file type. To do that, you can take advantage of pathlib:

>>> import pathlib
>>> from collections import Counter

>>> entries = pathlib.Path("Pictures/").iterdir()
>>> extensions = [entry.suffix for entry in entries if entry.is_file()]
['.gif', '.png', '.jpeg', '.png', '.png', ..., '.png']

>>> Counter(extensions)
Counter({'.png': 50, '.jpg': 11, '.gif': 10, '.jpeg': 9, '.mp4': 9})

In this example, you first create an iterator over the entries in a given directory using Path.iterdir(). Then you use a list comprehension to build a list containing the extensions (.suffix) of all the files in the target directory. Finally, you count the number of files using the file extension as the grouping criteria.

If you run this code on your computer, then you’ll get a different output depending on the content of your Pictures/ directory, if it exists at all. So, you’ll probably need to use another input directory for this code to work.

Restoring Elements From a Counter

The first multiset feature of Counter that you’re going to learn about is .elements(). This method returns an iterator over the elements in a multiset (Counter instance), repeating each of them as many times as its count says:

>>> from collections import Counter

>>> for letter in Counter("mississippi").elements():
...     print(letter)
...
m
i
i
i
i
s
s
s
s
p
p

The net effect of calling .elements() on a counter is to restore the original data you used to create the counter itself. The method returns the elements—letters in this example—in the same order they first appear in the underlying counter. Since Python 3.7, Counter remembers the insertion order of its keys as a feature inherited from dict.

The docstring of .elements() in the source code file provides an interesting example of using this method to compute a number from its prime factors. Since a given prime factor may occur more than once, you might end up with a multiset. For example, you can express the number 1836 as the product of its prime factors like this:

1836 = 2 × 2 × 3 × 3 × 3 × 17 = 2² × 3³ × 17¹

You can write this expression as a multiset like {2, 2, 3, 3, 3, 17}. Using a Python’s Counter, you’ll have Counter({2: 2, 3: 3, 17: 1}). Once you have this counter in place, you can compute the original number using its prime factors:

>>> from collections import Counter

>>> # Prime factors of 1836
>>> prime_factors = Counter({2: 2, 3: 3, 17: 1})
>>> product = 1

>>> for factor in prime_factors.elements():
...     product *= factor
...

>>> product
1836

The loop iterates over the elements in prime_factors and multiplies them to compute the original number, 1836. If you’re using Python 3.8 or beyond, then you can use prod() from math to get a similar result. This function calculates the product of all elements in the input iterable:

>>> import math
>>> from collections import Counter

>>> prime_factors = Counter({2: 2, 3: 3, 17: 1})
>>> math.prod(prime_factors.elements())
1836

In this example, the call to .elements() restores the prime factors. Then math.prod() computes 1836 from them in one go, which saves you from writing a loop and from having a few intermediate variables.

Using .elements() provides a way to restore the original input data. Its only drawback is that, in most cases, the order of items in the input won’t match the order in the output:

>>> from collections import Counter

>>> "".join(Counter("mississippi").elements())
'miiiisssspp'

In this example, the resulting string doesn’t spell the original word, mississippi. However, it has the same content in terms of letters.

Subtracting the Elements’ Multiplicity

Sometimes you need to subtract the multiplicity (count) of the elements in a multiset or counter. In that case, you can use .subtract(). As its name implies, this method subtracts the counts supplied in an iterable or mapping from the counts in the target counter.

Say you have a multiset with your current fruit inventory and you need to keep it up to date. Then you can run some of the following operations:

>>> from collections import Counter

>>> inventory = Counter(apple=39, orange=30, banana=15)

>>> # Use a counter
>>> wastage = Counter(apple=6, orange=5, banana=8)
>>> inventory.subtract(wastage)
>>> inventory
Counter({'apple': 33, 'orange': 25, 'banana': 7})

>>> # Use a mapping of counts
>>> order_1 = {"apple": 12, "orange": 12}
>>> inventory.subtract(order_1)
>>> inventory
Counter({'apple': 21, 'orange': 13, 'banana': 7})

>>> # Use an iterable
>>> order_2 = ["apple", "apple", "apple", "apple", "banana", "banana"]
>>> inventory.subtract(order_2)
>>> inventory
Counter({'apple': 17, 'orange': 13, 'banana': 5})

Here, you use several ways to provide the input data to .subtract(). In all cases, you update the counts of each unique object by subtracting the counts provided in the input data. You can think of .subtract() as the counterpart of .update().

Doing Arithmetic With Elements’ Multiplicity

With .subtract() and .update(), you can combine counters by subtracting and adding corresponding element counts. Alternatively, Python provides handy operators for addition (+) and subtraction (-) of element counts, as well as operators for intersection (&) and union (|). The intersection operator returns the minimum of corresponding counts, while the union operator returns the maximum of counts.

Here are a few examples of how all these operators work:

>>> from collections import Counter

>>> # Fruit sold per day
>>> sales_day1 = Counter(apple=4, orange=9, banana=4)
>>> sales_day2 = Counter(apple=10, orange=8, banana=6)

>>> # Total sales
>>> sales_day1 + sales_day2
Counter({'orange': 17, 'apple': 14, 'banana': 10})

>>> # Sales increment
>>> sales_day2 - sales_day1
Counter({'apple': 6, 'banana': 2})

>>> # Minimum sales
>>> sales_day1 & sales_day2
Counter({'orange': 8, 'apple': 4, 'banana': 4})

>>> # Maximum sales
>>> sales_day1 | sales_day2
Counter({'apple': 10, 'orange': 9, 'banana': 6})

Here, you first add two counters together using the addition operator (+). The resulting counter contains the same keys (elements), while their respective values (multiplicities) hold the sum of counts from both involved counters.

The second example shows how the subtraction operator (-) works. Note that negative and zero counts result in not including the key-count pair in the resulting counter. So, you don’t see orange in the output because 8 - 9 = -1.

The intersection operator (&) extracts the objects with lower counts from both counters, whereas the union operator (|) returns the objects with higher counts from both involved counters.

Counter also supports some unary operations. For example, you can get the items with positive and negative counts using the plus (+) and minus (-) signs, respectively:

>>> from collections import Counter

>>> counter = Counter(a=2, b=-4, c=0)

>>> +counter
Counter({'a': 2})

>>> -counter
Counter({'b': 4})

When you use the plus sign (+) as a unary operator on an existing counter, you get all the objects whose counts are greater than zero. On the other hand, if you use the minus sign (-), you get the objects with negative counts. Note that the result excludes objects whose counts are equal to zero in both cases.