Join us and get access to hundreds of tutorials and a community of expert Pythonistas.

Unlock This Lesson

This lesson is for members only. Join us and get access to hundreds of tutorials and a community of expert Pythonistas.

Unlock This Lesson

Hint: You can adjust the default video playback speed in your account settings.
Hint: You can set the default subtitles language in your account settings.
Sorry! Looks like there’s an issue with video playback 🙁 This might be due to a temporary outage or because of a configuration issue with your browser. Please see our video player troubleshooting guide to resolve the issue.

Query String Parameters

Give Feedback

What are query strings? How do I use them? In this lesson you’ll learn exactly that. The .get() function accepts a params argument, which can be a dictionary, a list of tuples or raw bytes. The following example uses the GitHub Search API looking for the requests library:

>>> requests.get(
...     'https://api.github.com/search/repositories',
...     params={'q': 'requests+language:python'}
... )
<Response [200]>

00:00 Hey, welcome to Section 2. We’re going to get started next with talking about query string parameters, then we’ll continue on to request headers, other HTTP methods, the message body, and then inspecting your request. So, what are query string parameters? Well, they’re the part of the URL which assigns values to specify parameters.

00:19 They’re commonly added to the base of the URL, like in this example where we’re searching repositories. It starts with the question mark (?), and then you can see a query added.

00:29 Other examples—if you remember us returning back our .content from the Root Rest API, here are some of those pairs. One was commit_search_url, and here you can see it talking about searching within the commits.

00:41 Another would be to search for users, and the user_search_url starts with its parameters here, just after users. Query string parameters can be composed of a series of arguments.

00:53 The series is separated by a plus sign (+). So, let’s try it out. Back in our editor here, we’re going to create a new file. I’m simply going to call it search.

01:09 So, we’re going to pass a query string into GitHub’s Search API here. So, let’s start by importing requests. Give myself some room.

01:24 All right. We’re going to make a response variable and do a GET request into it. In this particular case, we’d put in the url as a first parameter, and then we can add additional parameters. Let’s start with that.

01:42 So in this case, we’re going to say params (parameters). We’ll do them as a dictionary. It’s only one, but… The key will be 'q', the value is we’re searching for 'requests', and we’ll add a second argument of 'language:python'.

02:00 We’ll get rid of these extra lines, and down at the bottom here we’ll inspect some attributes from the repository. We’ll make another variable with the JSON response.

02:12 Take our response object, have it come back as JSON, and then we’ll make a repository object that will be from that json_response.

02:24 Since it’s a dictionary, we’ll pull the 'items' key and its very first value. Like a Google search—are you lucky?—we’re going to find the very first item in our search.

02:38 And from that repository object that’s being returned back, we’ll use a couple of keys here—one being the "name", and then the second being we’ll pull back the "description" and print them out using f-strings. Again, just a note: you will need Python 3.6 or higher to use these.

02:58 So, let’s save and let’s run it.

03:06 It’s searching right now. All right! Here is what it returned back. The repository name is requests! Exactly what we were looking for, so we got a good hit. And it printed out the description, the description being Python HTTP Requests for Humans™.

03:22 We could modify it a little bit, try to search a different repository.

03:27 Let’s say we’re searching for 'schedule' and maybe we know the 'user' as a parameter. Let’s save that and try again.

03:47 So, that repository is named schedule. Python job scheduling for humans. Great!

agerbes on March 14, 2020

Dear Christopher Bailey,

thank you very much for the great tutorial.

I have a special question.

Using the request library I print out the result of the url = ‘www.dropbox.com/h?preview=ophelp.txt

I would like to store the content of this url into it’s name = ‘ophelp’.

How can I use the request library to get the name searching for ‘preview=’ and getting ‘ophelp’?

I thought about using the query string parameters as you shown in your tutorial. But I am not sure how to search for it exactly.

Thank you Anja

Chris Bailey RP Team on March 15, 2020

Hi @agerbes Anja, I’m not sure I completely understand your question. It looks like you want to search for a file on dropbox, and get a preview of the text. I believe searches would be limited to a particular users files. I haven’t used the dropbox api, but I looked it over a bit and you would need to use a different url to use requests with it. The url would be 'https://api.dropbox.com/'. I did find some of the resources available from their site. I hope they can help you with what you are trying to accomplish. This link has the http documentation for using the dropbox api, including the available end points: Dropbox for HTTP

This link is to their Python package for doing development. It looks pretty well documented also: Dropbox for Python Developers. I hope some of this helps.

jc123k on April 9, 2020

The speed of these videos is quite fast, IMHO. Speed and depth of explanations not really beginner-friendly.

Dan Bader RP Team on April 9, 2020

Thanks for your feedback there @jc123k. This course is aimed at an intermediate skill level, so it isn’t intended to be a beginner’s tutorial. But thank you for the feedback, maybe there’s room for doing a beginner-focused course or tutorial on this topic in the future :)

DaveyD on May 3, 2020

The response I got does not relate to the webpage. Where does the data come from?

Response from search.py:

Repository name: grequests Repository description: Requests + Gevent = <3

Response from Url(api.github.com/search/repositories):

{ “message”: “Validation Failed”, “errors”: [ { “resource”: “Search”, “field”: “q”, “code”: “missing” } ], “documentation_url”: “developer.github.com/v3/search” }

DaveyD on May 3, 2020

Sorry to correct what I said this is the response from the repo:

{ “message”: “Validation Failed”, “errors”: [ { “message”: “The listed users and repositories cannot be searched either because the resources do not exist or you do not have permission to view them.”, “resource”: “Search”, “field”: “q”, “code”: “invalid” } ], “documentation_url”: “developer.github.com/v3/search/” }

DaveyD on May 3, 2020

Got it. Did some research on string query and worked out what to use in url: api.github.com/search/repositories?q=schedule+user:dbader

mikesult on May 4, 2020

DaveyD, I also had the same response from search.py

Repository name: grequests Repository description: Requests + Gevent = <3

if I omitted the ‘+language:python’ part of the query then ‘requests’ is positioned at index 0 of ‘items’. (params={‘q’: ‘requests’}). it seems true for today anyway.

Repository name: requests Repository description: A simple, yet elegant HTTP library.

I’m guessing that just means the repository structure at github has changed a little since the video was made. Since we are looking for index 0 of the items both results are successful queries. Just not what we expected.

mikesult on May 4, 2020

Thinking more about why grequests came out at index 0. some questions:

  1. When using multiple params are they ‘or’ed together or ‘and’ed together? I was thinkng that you added ‘+language:python’ as an ‘and’ type of filter in case there were other repos named ‘requests’ from another language and you didn’t want them in the response.

  2. Since grequests was a match I infer that query ‘requests’ will match any name that contains ‘requests’? i.e. fiddlerequests, frequests lorequests. Or maybe that match was because it is a python language repo? I’m confused as to why grequests matched?

  3. Is json.items() an alphabetically ordering?, i.e. if there was a repo name arequests would that bump grequests down.?

Chris Bailey RP Team on May 5, 2020

Hi @DaveyD and @mikesult, Wow. A lot has changed in the last year, not only for the results of this search but for the “requests” package. It is now under the Python Software Foundation the PSF.

You are both right that existing search now returns ‘grequests’ as the first item. And you are correct, if you remove the ‘+language:python’ part it will now return ‘requests’ as a first item. Not sure why it is not under ‘Python’ for the language, weird.

I was thinking if you want to see more of the search results you could add a simple for loop.

# at the end of the file change it to
json_response = response.json()
for i in range(len(json_response['items')):
    repository = json_response['items'][i]
    print(f'{i}: Respository name: {repository["name"]}')
    print(f'{i}: Respository description: {repository["description"]}')

This would give you list of the returned search items.

To answer your other questions

1: The ‘+’ works like an ‘AND’ statement, so it will reduce your results. In this case narrowing to only the keyword language, with Python. You can try this search out directly from the search bar on the actual site github.com itself. Another keywords as I show later is ‘user:’ and I just tried out another ‘topic:’

2: You are right that it will look within the name of the repository but it also looks at (searches) the description text for the word. It’s truly doing a search.

3: It is a ranking, not alpha. So it depends on how github orders the results.

I hope this helps. Any site will have their own scheme for how this will work. And it will be a moving target for a video to capture results of a search for the moment in time.

Tonya Sims on July 23, 2020

Hi Chris and Real Python community! Your tutorial is amazing and I’m learning a lot. Quick question about the query string for the first example. When I go to this url in my browser: api.github.com/search/respositories?q=requests+language:python

I get this:

{ “message”: “Not Found”, “documentation_url”: “developer.github.com/v3” }

I read through the comments here and just want to verify that it won’t work anymore because Github moved it to another repository? Is that correct or am I just misunderstanding?

Tonya Sims on July 23, 2020

Hi again, about my previous comment. When I run the code for:

response = requests.get( “api.github.com/search/repositories”, params={‘q’:’requests+language:python’}, )

I’m getting JSON data back, so maybe the way I constructed the URL is incorrect: api.github.com/search/respositories?q=requests+language:python

Chris Bailey RP Team on July 23, 2020

Hi @Tonya,

As you noticed in my earlier comments, the URL scheme seems to be changed a bit for where to find “requests”. Here is a URL scheme that seems to work now. https://api.github.com/search/repositories?q=requests+org:psf . It looks like the “full name” is now psf/requests. The types of things you can specify in the query string seem to really vary over time. “org” for organizations, language, etc. You can also get an idea if you want to see the types of URL formats, from this URL: https://api.github.com, it will return JSON of the end points to explore. And the documentation is also here.

Have fun exploring!

mikelisfbay on July 30, 2020

Hi

I found the root cause of the problem. It is due to percent encoding scheme performed by the requests package on “+” and “:”.

The solution is to pass the string directly.

1. Original solution with the dictionary fromatted payload

payload = {‘q’: ‘requests+language:python’, ‘sort’: ‘stars’, ‘order’: ‘desc’} #added two more fields to get the item 0 correctly.

response = requests.get(‘api.github.com/search/repositories’, params=payload)

print(response.url)

Output: (No good) api.github.com/search/repositories?q=requests%2Blanguage%3Apython&sort=stars&order=desc Repository name: httpbin Repository description: HTTP Request & Response Service, written in Python + Flask.

2. New solution with the direct string formatted payload

payload_str = “q=requests+language:python&sort=stars&order=desc”

response = requests.get(‘api.github.com/search/repositories’, params=payload_str)

print(response.url)

Output: (This is what we want to see) api.github.com/search/repositories?q=requests+language:python&sort=stars&order=desc Repository name: requests Repository description: A simple, yet elegant HTTP library.

Perfect result!!!

3. How to convert a dictionary to a string. (the practical solution)

stackoverflow.com/questions/23496750/how-to-prevent-python-requests-from-percent-encoding-my-urls

payload = {‘format’: ‘json’, ‘key’: ‘site:dummy+type:example+group:wheel’}

payload_str = “&”.join(“%s=%s” % (k,v) for k,v in payload.items())

‘format=json&key=site:dummy+type:example+group:wheel’

r = requests.get(url, params=payload_str) print(r.url)

The above solution is contributed by a user named furas on Stackoverflow.

Become a Member to join the conversation.