In this lesson you will see how to put everything you have learned about generators, async generators, coroutines, and asyncio to build a real world async application that calls an HTTP API and manipulates data from it.
A Real-World asyncio Example
00:00 What I want to do now is put everything that we’ve learned together—generators, coroutines, asynchronous generators—and sort of build something that is a little more real-world, that actually uses real IO, real input/output, something like calling a database or a website.
00:30 It basically looks like a dictionary in Python. So, we’re going to call a website, we’re going to get back what’s called JSON—which looks like a dictionary—and then we’re just going to process it. Okay.
00:52 And they actually have some actual physical equipment that produces random numbers based on vacuums. It has to do with physics. I don’t want to get into how it works, but at the end, the purpose of the website is to produce actual, real random numbers, instead of in a computer they’re what’s called pseudorandom.
But these are, like, real random numbers—or at least theoretically. Okay, so this is a website. And what happens is I can say, “Give me 10 numbers, 10 random numbers.” And
uint16, it means 16-bit, so that means it goes from 0 to 65,000.
So, “Any number between 0 and 65,000, give me 10 of these.” And I hit Enter, you can see it spinning around, and then it gave me some other numbers. And I can make this into a
20, and now I have 20 numbers here.
This is JSON right here—these little brackets right here, and then these are the keys, and these are the values. So once again, if you’re used to Python and dictionaries, this should look very similar. And Python has a
json package, or a module, that you can bring in that will actually convert JSON into an actual dictionary.
02:06 Okay, so this is what we want to do. I’m going to do this. And because this involves IO, or input/output, the async package is perfect for this. I’m going to come back here and let’s start off and let’s create our imports.
import asyncio, and then I’m going to
json is going to allow us, like I said, to take JSON that comes from a website and then turn it into a dictionary so that Python can convert it.
I’m going to
import time, because we want to see how long something takes, and I’m going to
import aiohttp. This is an HTTP package, which means I’m going to call a website and I’m going to call it asynchronously.
So if you want to do this, if you want to call an HTTP endpoint, or a website, and if you want it to be nonblocking, you have to use the
aiohttp. Now, there are other packages out here, but this is one built into Python. Actually, I take that back.
'__main__'. I’m going to say
if __name__ == '__main__':. Okay, what I’d like to do is I’d like to time this and see how long this is going to take, so let’s create a
start time and we’ll do what we kind of did before
Oh, wait, we have to do an
await on this because it’s in an
async. Okay. So let’s just try this, and this will prove that our program is kind of working at least up to this point. So
and it took about
1.00 second, which is exactly what I had right here. So this shows this is working. Great. What I would like to do is I want
main() to create a session, an HTTP session, and then send that off to somewhere.
05:57 This worker is going to be asynchronous too, which means when you call it, it’ll create a coroutine. And so I could create a whole bunch of these coroutines as little workers, and then have them work—that you might think are running in parallel, but they’re… It is running asynchronously, and really it’s not parallel—at least from Python’s perspective, because once again, Python just has a single thread, so it’s single-threaded. But once it leaves Python and the request is out in the real world, then those requests could happen almost in parallel. Okay?
def, I’ll call this thing
worker(), like this, and this is the
name of the worker,
n is how many numbers I want to get back from my random number generator on the web.
n is how many numbers that I want. And then
session is, this is the HTTP session that we’re going to be using. Okay, so I’m going to stop right there, so
pass—that makes it correct syntactically. So let’s go back here.
Okay. So, what I’d like to do now is maybe let’s do one worker, so let’s make one request for this worker, and let’s just check what the results are. I’m going to say, maybe
response = await worker(), and I’m going to pass in a
name, let’s say
'bob' is the name of the request. And the
n, this means how many numbers am I going to get back? Let’s say I want to get back
3. And then I’m going to pass in the
Okay, so this looks pretty good. Now I need this
worker() to do something. I need it to, like, take in this worker’s name, and then how many numbers I’m going to get from this website, and then this is the HTTP session. Okay. So let’s come back here.
So I can just copy this, come back here and say
url is equal to
f''—that. So, that looks pretty good. The only thing I want to do is I’ve hard-coded in
20 here, and I want
20 to just be
So, “Give me back
n results.” Okay, so there’s my
url. That looks pretty good. Now let’s go ahead and make a request, so I’m going to say
response = await, because what I’m about to do is asynchronous IO.
and I’m going to do
method='GET'. So, if you don’t know much about HTTP, this could be a little confusing here, but when you do this—when you make this request right here—this is called a
GET request. There are other ones called
PUT and things like that, but let’s not worry too much about that.
10:59 So just to recap, this line right here is the actual line that sends the request out of the computer and it finds its way to this computer—so it’s going to go from my computer to this computer, okay?
This response is going to come back and it’s going to be an HTTP response, which could be a little confusing, and this is going to turn that
response and just show me just the text from this. Let’s take this text and it will be returned—here’s my
value here, and I could just return that.
So I have my
value, now let’s go ahead and return that. So I’m going to say
return value like that, and let’s just see what happens. So, this
value right here gets returned, and so I’m calling
worker() and it should go into
value should basically go into
response and I can print that out.
Let’s see what that does. If I run this, it says the worker’s name is
bob, and the response is this thing. Now, you’re probably wondering like, “What is this thing? This thing here, it looks like a dictionary.” It’s actually not a dictionary, and I can prove that to you by typing in
response and then
type() of the
Let’s save that, come back here, run it again. Once again,
worker-bob is running and you can see this thing here is not actually a dictionary—it’s a
str (string). I want to turn this string, which is JSON, into an actual dictionary.
So now I have something that Python actually knows about, and I want to return this key inside this dictionary. How do I do that? I just say
value and I pass in the key, which is—let’s look at it again, it’s called
Okay, let’s sum that up. Let’s come back here, run this again. And you can see now my response is just those three values. Now this whole program is called
summer. You’re probably like, “Why is it called
summer?” Because I want to return the sum of those three values, so I’m just going to take
sum() of those three values, like that.
Save it, come back here, and let’s see what that is. And yes, 105,000. So yeah, I was able to actually create this—from here, I was able to asynchronously call this website, get back the data, get its text, turn it into JSON—or, turn it into a dictionary, and then take the dictionary, grab its key, and then this key ends up being a list, and so I’m taking the
sum() of a
list and that ends up being a single value.
14:54 Okay. So, that all is pretty neat and awesome. But how long did this thing take? Let’s look at this thing again. It took about 1 second. But what if I did a whole bunch of these? Let’s say, what if I did 30 of these?
Become a Member to join the conversation.