Splitting Iterables Into Batches

00:00 In this lesson, you’ll fix the lazy grouper for data that doesn’t divide evenly, first with a classic itertools recipe, and then with a function that does the whole job for you.

00:12 Ask better_grouper() for groups of four, and something goes missing. You get two groups, and nine and ten have vanished. zip() stops as soon as its shortest input runs out.

00:24 To fix that, import itertools. You’ll use the short name it from now on. Take a list of five numbers, and a list of three letters.

00:35 zip() gives you three pairs and drops the rest. zip_longest() keeps going until the longest input runs out, and it fills the gaps with None. Swap zip() for zip_longest() in the lazy grouper, and you get the grouper recipe.

00:52 It’s the same shared-iterator trick, plus a fillvalue for the last group.

01:00 Now nine and ten survive, padded with None. A version of this recipe lives in the itertools docs, and the recipes section there is a great source of ideas.

01:11 Here’s the difference between the two approaches. The grouper recipe always hands you full groups, so the last one gets padded with None. That’s useful sometimes, but often you have to strip the padding back out.

01:23 Since Python 3.12, itertools has a function called batched(). It returns the same full groups, and the last batch is just shorter. Now try out batched() in the REPL.

01:34 Batches of four give you two full batches and a short one with nine and ten. No padding needed. If a short batch would be a bug in your program, then pass strict=True, which arrived in Python 3.13.

01:49 Now batched() raises a ValueError. With batches of five, everything fits, so strict mode is happy.

01:59 In this lesson, you met zip_longest() and the grouper recipe, and then batched(), the built-in way to split any iterable into chunks. Like the recipe, batched() never calls len(), so it works on infinite iterators, too.

02:12 You’ll come back to batched() at the end of the course to pick relay teams. Next up, though, it’s time to brute-force a classic interview puzzle.