Join us and get access to hundreds of tutorials and a community of expert Pythonistas.

Unlock This Lesson

This lesson is for members only. Join us and get access to hundreds of tutorials and a community of expert Pythonistas.

Unlock This Lesson

Hint: You can adjust the default video playback speed in your account settings.
Hint: You can set the default subtitles language in your account settings.
Sorry! Looks like there’s an issue with video playback 🙁 This might be due to a temporary outage or because of a configuration issue with your browser. Please see our video player troubleshooting guide to resolve the issue.

Asynchronous Generators in Python

Give Feedback

In this lesson you’ll learn how to create an asynchronous generator:

async def square_odds(start, stop):
    for odd in odds(start, stop):
        await asyncio.sleep(2)
        yield odd ** 2

You’ll also see how to loop over values asynchronously using an async for loop.

00:00 One of the last theoretical things I want to talk about is asynchronous generators. Asynchronous generators are basically an amalgam of this odds() function and randn(), meaning it’s a generator in that it produces values, but it’s asynchronous in that when it produces the values, the values get produced asynchronously, which means the first value may be 1 second, the second value may come out 10 seconds later, the third value may come out 7.3 seconds later.

00:27 So the values come out at different times, okay? The best way to show this is just an example, so I’m going to say async. It’s a function, it’s an asynchronous generator, so I’m going to use async. def means I’m going to create a function, and I’m going to call this thing square_odds(),

00:47 because I’m going to take some odd value and then I’m going to square it. I’m going to pass in some start value, some stop value, and I’m going to leverage this existing generator up here.

01:00 I’m going to say for odd in odds() and I’m going to pass in my start, my stop. Now, I’m going to wait some amount of time asynchronously, so I’m going to say await. This is what makes this function asynchronous, is that I’m using await.

01:27 And I’m going to sleep(), and let’s say I’m going to sleep() 2 seconds. But what I’m doing right here is I’m pretending that I’m talking to a database or a file system or a web server—this is what I’m actually simulating here in this step right there.

01:44 So, imagine me calling a website or getting back some value from a database—that’s what that could be. And then I’m just going to return the value, okay? So I’m going to say yieldI’m going to yield out, because I’m using a generator, so I’m using yield.

02:00 So I’m using yield like I did in the generator, but I’m using await like I did in this coroutine, this asynchronous function. So that’s why this is like a mixture of the other two.

02:11 And I’m just going to yield that odd ** 2 (squared), because it’s square_odds().

02:19 Okay, now that I have this square_odds() asynchronous generator, let’s go ahead and execute it, or cause it to run. So normally, when you have a generator you would just use what’s called a for in loop. Because this is an asynchronous generator, you’re going to use an async for in loop.

02:36 So you’re going to say async for, let’s say so (square odd) in square_odds(). I’m going to pass in some start value, let’s say 11, and some stop value, say 17,

02:51 colon (:), and then let’s say print('so:', so).

02:59 So this allows me to loop over something that’s asynchronous, which means I could be calling an API, talking to a database, writing to a file system—you don’t really know how long it’s going to take.

03:11 And so when you’re looping over something that’s asynchronous, creating this async for is basically the perfect solution for that. So let’s see if this actually works.

03:22 This was my output from the initial generator.

03:28 That was from the coroutines. And then this is my square_odds(). So 11 squared, 12 squared—or sorry, 11, 13, 15, and then 17.

03:40 So that’s the square of all those, and then it finished. So yes, you can see I was able to loop asynchronously over those values as they came in one after the other.

03:53 So, this kind of wraps up the theoretical part of the lecture. What’s going to happen next is I’m going to go and we’re going to build a very tiny application that kind of puts all this stuff together.

karncx on Aug. 30, 2019

Why does the async for take number of steps * 2 seconds? Why does it not save time like asyncio.gather?

tevakrief on Sept. 12, 2019

Hello ! Same question as karncx, why does it take 2s each time ?

Thanks !

Hilman on Feb. 8, 2020

My understanding is that asyncio.gather is the one that responsible for the coroutine which basically means, do the tasks together. As for the asynchronous generator, it just show an example of one async task (a generator in this case).

You can try to run multiple asynchronous coroutine-ly. Maybe change the code to something like this:

async def square_odds(start, stop):
    for odd in odds(start, stop):
        await asyncio.sleep(2)  # talking to database simulation
        yield odd ** 2

async def for_async(func_id):
    async for so in square_odds(11, 17):
        print(f"({func_id}) so: {so}")

await asyncio.gather(*(for_async(f"for_loop {i}") for i in range(3)))

Become a Member to join the conversation.