Exploring HTTPS With Python

Creating an Example Application

Suppose you’re the leader of a cool Python club called the Secret Squirrels. The Squirrels, being secret, require a secret message to attend their meetings. As the leader, you choose the secret message, which changes for each meeting. Sometimes, though, it’s hard for you to meet with all the members before the meeting to tell them the secret message! You decide to set up a secret server where members can just see the secret message for themselves.

You’ve followed some tutorials on Real Python and decide to use some dependencies you know:

• Flask to build a web application
• uWSGI as a production server
• requests to exercise your server

To install all of these dependencies, you can use pip:

$ pip install flask uwsgi requests

With your dependencies installed, you begin to write your application. In a file called server.py, you create a Flask application:

# server.py
from flask import Flask

SECRET_MESSAGE = "fluffy tail"
app = Flask(__name__)

@app.route("/")
def get_secret_message():
    return SECRET_MESSAGE

This Flask application will display the secret message whenever someone visits the / path of your server. With that out of the way, you deploy your application on your secret server and run it:

$ uwsgi --http-socket 127.0.0.1:5683 --mount /=server:app

This command starts up a server using the Flask application above. You start it on a weird port because you don’t want people to be able to find it, and pat yourself on the back for being so sneaky! You can confirm that it’s working by visiting http://localhost:5683 in your browser.

Since everyone in the Secret Squirrels knows Python, you decide to help them out. You write a script called client.py that will help them get the secret message:

# client.py
import os
import requests

def get_secret_message():
    url = os.environ["SECRET_URL"]
    response = requests.get(url)
    print(f"The secret message is: {response.text}")

if __name__ == "__main__":
    get_secret_message()

This code will print out the secret message as long as they have the SECRET_URL environment variable set. In this case, the SECRET_URL is 127.0.0.1:5683. So, your plan is to give each club member the secret URL and tell them to keep it secret and safe.

While this might seem okay, rest assured it’s not! In fact, even if you were to put a username and password on this site, it still wouldn’t be safe. But even if your team somehow did manage to keep the URL safe, your secret message still wouldn’t be secure. To demonstrate why you’ll need to know a little bit about monitoring network traffic. To do this, you’ll be using a tool called Wireshark.

Setting Up Wireshark

Wireshark is a widely used tool for network and protocol analysis. What this means is that it can help you see what’s happening over network connections. Installing and setting up Wireshark is optional for this tutorial, but feel free if you’d like to follow along. The download page has several installers available:

• macOS 10.12 and later
• Windows installer 64-bit
• Windows installer 32-bit

If you’re using Windows or Mac, then you should be able to download the appropriate installer and follow the prompts. In the end, you should have a running Wireshark.

If you’re on a Debian-based Linux environment, then the installation is a bit harder, but still possible. You can install Wireshark with the following commands:

$ sudo add-apt-repository ppa:wireshark-dev/stable
$ sudo apt-get update
$ sudo apt-get install wireshark
$ sudo wireshark

You should be met with a screen that looks something like this:

With Wireshark running, it’s time to analyze some traffic!

Seeing That Your Data Is Not Safe

The way your current client and server are running is not secure. HTTP will send everything in the clear for anyone to see. What this means is that even if someone doesn’t have your SECRET_URL, they can still see everything you do as long as they can monitor traffic on any device between you and the server.

This should be relatively scary for you. After all, you don’t want other people showing up for your Secret Squirrel meetings! You can prove that this is happening. First, start up your server if you don’t still have it running:

$ uwsgi --http-socket 127.0.0.1:5683 --mount /=server:app

This will start up your Flask application on port 5683. Next, you’ll start a packet capture in Wireshark. This packet capture will help you see all the traffic going to and from the server. Begin by selecting the Loopback:lo interface on Wireshark:

You can see that the Loopback:lo portion is highlighted. This instructs Wireshark to monitor this port for traffic. You can do better and specify which port and protocol you’d like to capture. You can type port 5683 in the capture filter and http in the display filter:

The green box indicates that Wireshark is happy with the filter you typed. Now you can begin the capture by clicking on the fin in the top left:

Clicking this button will spawn a new window in Wireshark:

This new window is fairly plain, but the message at the bottom says <live capture in progress>, which indicates that it’s working. Don’t worry that nothing is being displayed, as that’s normal. In order for Wireshark to report anything, there has to be some activity on your server. To get some data, try running your client:

$ SECRET_URL="http://127.0.0.1:5683" python client.py
The secret message is: fluffy tail

After executing the client.py code from above, you should now see some entries in Wireshark. If all has gone well, then you’ll see two entries that look something like this:

These two entries represent the two parts of the communication that occurred. The first one is the client’s request to your server. When you click on the first entry, you’ll see a plethora of information:

That’s a lot of information! At the top, you still have your HTTP request and response. Once you select one of these entries, you’ll see the middle and bottom row populate with information.

The middle row provides you with a breakdown of the protocols that Wireshark was able to identify for the selected request. This breakdown allows you to explore what actually happened in your HTTP Request. Here’s a quick summary of the information Wireshark describes in the middle row from top to bottom:

1. Physical Layer: This row describes the physical interface used to send the request. In your case, this is probably Interface ID 0 (lo) for your loopback interface.
2. Ethernet Information: This row shows you the Layer-2 Protocol, which includes source and destination MAC addresses.
3. IPv4: This row displays source and destination IP addresses (127.0.0.1).
4. TCP: This row includes the required TCP handshake in order to create a reliable pipe of data.
5. HTTP: This row displays information about the HTTP request itself.

When you expand the Hypertext Transfer Protocol layer, you can see all the information that makes up an HTTP Request:

This image shows you your script’s HTTP request:

• Method: GET
• Path: /
• Version: 1.1
• Headers: Host: 127.0.0.1:5683, Connection: keep-alive, and others
• Body: No body

The last row you’ll see is a hex dump of the data. You may notice in this hex dump that you can actually see the parts of your HTTP request. That’s because your HTTP request was sent in the open. But what about the reply? If you click on the HTTP response, then you’ll see a similar view:

Again, you have the same three sections. If you look carefully at the hex dump, then you’ll see the secret message in plain text! This is a big problem for the Secret Squirrels. What this means is that anyone with some technical know-how can very easily see this traffic if they’re interested. So, how do you solve this problem? The answer is cryptography.

Understanding Cryptography Basics

Cryptography is a way to secure communications from eavesdroppers, or adversaries. Another way to state this is that you’re taking normal information, called plaintext, and converting it to scrambled text, called ciphertext.

Cryptography can be intimidating at first, but the fundamental concepts are pretty accessible. In fact, you’ve probably already practiced cryptography before. If you’ve ever had a secret language with your friends and used it to pass notes in class, then you’ve practiced cryptography. (If you haven’t done that, then don’t worry—you’re about to.)

Somehow, you need to take the string "fluffy tail" and convert it into something unintelligible. One way to do this is to map certain characters onto different characters. An effective way to do this is to shift the characters back one spot in the alphabet. Doing so would look something like this:

This image shows you how to translate from the original alphabet to the new alphabet and back. So, if you had the message ABC, then you would actually send the message ZAB. If you apply this to "fluffy tail", then assuming that spaces stay the same, you get ekteex szhk. While it’s not perfect, it’ll probably look like gibberish to anyone that sees it.

Congratulations! You’ve created what is known in cryptography as a cipher, which describes how to convert plaintext to ciphertext and back. Your cipher, in this case, is described in English. This particular type of cipher is called a substitution cipher. Fundamentally this is the same type of cipher used in the Enigma Machine, albeit a much simpler version.

Now, if you wanted to get a message out to the Secret Squirrels, then you would first need to tell them how many letters to shift and then give them the encoded message. In Python, this might look something like the following:

CIPHER = {"a": "z", "A": "Z", "b": "a"} # And so on

def encrypt(plaintext: str):
    return "".join(CIPHER.get(letter, letter) for letter in plaintext)

Here, you’ve created a function called encrypt(), which will take plaintext and convert it to ciphertext. Imagine you have a dictionary CIPHER that has all of the characters mapped out. Similarly, you could create a decrypt():

DECIPHER = {v: k for k, v in CIPHER.items()}

def decrypt(ciphertext: str):
    return "".join(DECIPHER.get(letter, letter) for letter in ciphertext)

This function is the opposite of encrypt(). It will take ciphertext and convert it to plaintext. In this form of the cipher, you have a special key that users need to know in order to encrypt and decrypt messages. For the above example, that key is 1. That is, the cipher indicates you should shift each letter back by one character. The key is very important to keep secret because anyone with the key can easily decrypt your message.

In the modern era, cryptography is a lot more advanced. It relies on complex mathematical theory and computer science to be secure. While the math behind these ciphers is outside of the scope of this tutorial, the underlying concepts are still the same. You have a cipher that describes how to take plaintext and convert it into ciphertext.

The only real difference between your substitution cipher and modern ciphers is that modern ciphers are mathematically proven to be impractical to break by an eavesdropper. Now, let’s see how to use your new ciphers.

Using Cryptography in Python HTTPS Applications

Luckily for you, you don’t have to be an expert in mathematics or computer science to use cryptography. Python also has a secrets module that can help you generate cryptographically-secure random data. In this tutorial, you’ll learn about a Python library that’s aptly named cryptography. It’s available on PyPI, so you can install it with pip:

$ pip install cryptography

This will install cryptography into your virtual environment. With cryptography installed, you can now encrypt and decrypt things in a way that’s mathematically secure by using the Fernet method.

Recall that your secret key in your cipher was 1. In the same vein, you need to create a key for Fernet to work correctly:

>>> from cryptography.fernet import Fernet
>>> key = Fernet.generate_key()
>>> key
b'8jtTR9QcD-k3RO9Pcd5ePgmTu_itJQt9WKQPzqjrcoM='

In this code, you’ve imported Fernet and generated a key. The key is just a bunch of bytes, but it’s incredibly important that you keep this key secret and safe. Just like the substitution example above, anyone with this key can easily decrypt your messages.

This key behaves much like the earlier key. It’s required to make the transition to ciphertext and back to plaintext. Now it’s time for the fun part! You can encrypt a message like this:

>>> my_cipher = Fernet(key)
>>> ciphertext = my_cipher.encrypt(b"fluffy tail")
>>> ciphertext
b'gAAAAABdlW033LxsrnmA2P0WzaS-wk1UKXA1IdyDpmHcV6yrE7H_ApmSK8KpCW-6jaODFaeTeDRKJMMsa_526koApx1suJ4_dQ=='

In this code, you’ve created a Fernet object called my_cipher, which you can then use to encrypt your message. Notice that your secret message "fluffy tail" needs to be a bytes object in order to encrypt it. After the encryption, you can see that the ciphertext is a long stream of bytes.

Thanks to Fernet, this ciphertext can’t be manipulated or read without the key! This type of encryption requires that both the server and the client have access to the key. When both sides require the same key, this is called symmetric encryption. In the next section, you’ll see how to put this symmetric encryption to use to keep your data safe.

Seeing That Your Data Is Safe

Now that you understand some of the basics of cryptography in Python, you can apply this knowledge to your server. Create a new file called symmetric_server.py:

# symmetric_server.py
import os
from flask import Flask
from cryptography.fernet import Fernet

SECRET_KEY = os.environb[b"SECRET_KEY"]
SECRET_MESSAGE = b"fluffy tail"
app = Flask(__name__)

my_cipher = Fernet(SECRET_KEY)

@app.route("/")
def get_secret_message():
    return my_cipher.encrypt(SECRET_MESSAGE)

This code combines your original server code with the Fernet object you used in the previous section. The key is now read as a bytes object from the environment using os.environb. With the server out of the way, you can now focus on the client. Paste the following into symmetric_client.py:

# symmetric_client.py
import os
import requests
from cryptography.fernet import Fernet

SECRET_KEY = os.environb[b"SECRET_KEY"]
my_cipher = Fernet(SECRET_KEY)

def get_secret_message():
    response = requests.get("http://127.0.0.1:5683")

    decrypted_message = my_cipher.decrypt(response.content)
    print(f"The codeword is: {decrypted_message}")

if __name__ == "__main__":
    get_secret_message()

Once again, this is modified code to combine your earlier client with the Fernet encryption mechanism. get_secret_message() does the following:

1. Make the request to your server.
2. Take the raw bytes from the response.
3. Attempt to decrypt the raw bytes.
4. Print the decrypted message.

If you run both the server and the client, then you’ll see that you’re successfully encrypting and decrypting your secret message:

$ uwsgi --http-socket 127.0.0.1:5683 \
    --env SECRET_KEY="8jtTR9QcD-k3RO9Pcd5ePgmTu_itJQt9WKQPzqjrcoM=" \
    --mount /=symmetric_server:app

In this call, you start the server on port 5683 again. This time, you pass in a SECRET_KEY which must be at least a 32-length base64 encoded string. With your server restarted, you may now query it:

$ SECRET_KEY="8jtTR9QcD-k3RO9Pcd5ePgmTu_itJQt9WKQPzqjrcoM=" python symmetric_client.py
The secret message is: b'fluffy tail'

Woohoo! Your were able to encrypt and decrypt your message. If you try running this with an invalid SECRET_KEY, then you’ll get an error:

$ SECRET_KEY="AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA=" python symmetric_client.py
Traceback (most recent call last):
  File ".../cryptography/fernet.py", line 104, in _verify_signature
    h.verify(data[-32:])
  File ".../cryptography/hazmat/primitives/hmac.py", line 66, in verify
    ctx.verify(signature)
  File ".../cryptography/hazmat/backends/openssl/hmac.py", line 74, in verify
    raise InvalidSignature("Signature did not match digest.")
cryptography.exceptions.InvalidSignature: Signature did not match digest.

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "symmetric_client.py", line 16, in <module>
    get_secret_message()
  File "symmetric_client.py", line 11, in get_secret_message
    decrypted_message = my_cipher.decrypt(response.content)
  File ".../cryptography/fernet.py", line 75, in decrypt
    return self._decrypt_data(data, timestamp, ttl)
  File ".../cryptography/fernet.py", line 117, in _decrypt_data
    self._verify_signature(data)
  File ".../cryptography/fernet.py", line 106, in _verify_signature
    raise InvalidToken
cryptography.fernet.InvalidToken

So, you know the encryption and decryption are working. But is it secure? Well, yes, it is. To prove this, you can go back to Wireshark and start a new capture with the same filters as before. After you have the capture setup, run the client code again:

$ SECRET_KEY="8jtTR9QcD-k3RO9Pcd5ePgmTu_itJQt9WKQPzqjrcoM=" python symmetric_client.py
The secret message is: b'fluffy tail'

You’ve made another successful HTTP request and response, and once again, you see these messages in Wireshark. Since the secret message only gets transferred in the response, you can click on that to look at the data:

In the middle row of this picture, you can see the data that was actually transferred:

gAAAAABdlXSesekh9LYGDpZE4jkxm4Ai6rZQg2iHaxyDXkPWz1O74AB37V_a4vabF13fEr4kwmCe98Wlr8Zo1XNm-WjAVtSgFQ==

Awesome! This means that the data was encrypted and that eavesdroppers have no clue what the message content actually is. Not only that, but it also means that they could spend an insanely long amount of time trying to brute-force crack this data, and they would almost never be successful.

Your data is safe! But wait a minute—you never had to know anything about a key when you were using Python HTTPS applications before. That’s because HTTPS doesn’t use symmetric encryption exclusively. As it turns out, sharing secrets is a hard problem.

To prove this concept, navigate to http://127.0.0.1:5683 in your browser, and you’ll see the encrypted response text. This is because your browser doesn’t know anything about your secret cipher key. So how do Python HTTPS applications really work? That’s where asymmetric encryption comes into play.

Becoming a Certificate Authority

A Certificate Authority is nothing more than a very important public and private key pair. To become a CA, you just need to generate a public and private key pair.

Your initial public and private key pair will be a self-signed certificate. You’re generating the initial secret, so if you’re actually going to become a CA, then it’s incredibly important that this private key is safe. If someone gets access to the CA’s public and private key pair, then they can generate a completely valid certificate, and there’s nothing you can do to detect the problem except to stop trusting your CA.

With that warning out of the way, you can generate the certificate in no time. For starters, you’ll need to generate a private key. Paste the following into a file called pki_helpers.py:

 1# pki_helpers.py
 2from cryptography.hazmat.backends import default_backend
 3from cryptography.hazmat.primitives import serialization
 4from cryptography.hazmat.primitives.asymmetric import rsa
 5
 6def generate_private_key(filename: str, passphrase: str):
 7    private_key = rsa.generate_private_key(
 8        public_exponent=65537, key_size=2048, backend=default_backend()
 9    )
10
11    utf8_pass = passphrase.encode("utf-8")
12    algorithm = serialization.BestAvailableEncryption(utf8_pass)
13
14    with open(filename, "wb") as keyfile:
15        keyfile.write(
16            private_key.private_bytes(
17                encoding=serialization.Encoding.PEM,
18                format=serialization.PrivateFormat.TraditionalOpenSSL,
19                encryption_algorithm=algorithm,
20            )
21        )
22
23    return private_key

generate_private_key() generates a private key using RSA. Here’s a breakdown of the code:

• Lines 2 to 4 import the libraries required for the function to work.
• Lines 7 to 9 use RSA to generate a private key. The magic numbers 65537 and 2048 are just two possible values. You can read more about why or just trust that these numbers are useful.
• Lines 11 to 12 set up the encryption algorithm to be used on your private key.
• Lines 14 to 21 write your private key to disk at the specified filename. This file is encrypted using the password provided.

The next step in becoming your own CA is to generate a self-signed public key. You can bypass the certificate signing request (CSR) and immediately build a public key. Paste the following into pki_helpers.py:

 1# pki_helpers.py
 2from datetime import datetime, timedelta
 3from cryptography import x509
 4from cryptography.x509.oid import NameOID
 5from cryptography.hazmat.primitives import hashes
 6
 7def generate_public_key(private_key, filename, **kwargs):
 8    subject = x509.Name(
 9        [
10            x509.NameAttribute(NameOID.COUNTRY_NAME, kwargs["country"]),
11            x509.NameAttribute(
12                NameOID.STATE_OR_PROVINCE_NAME, kwargs["state"]
13            ),
14            x509.NameAttribute(NameOID.LOCALITY_NAME, kwargs["locality"]),
15            x509.NameAttribute(NameOID.ORGANIZATION_NAME, kwargs["org"]),
16            x509.NameAttribute(NameOID.COMMON_NAME, kwargs["hostname"]),
17        ]
18    )
19
20    # Because this is self signed, the issuer is always the subject
21    issuer = subject
22
23    # This certificate is valid from now until 30 days
24    valid_from = datetime.utcnow()
25    valid_to = valid_from + timedelta(days=30)
26
27    # Used to build the certificate
28    builder = (
29        x509.CertificateBuilder()
30        .subject_name(subject)
31        .issuer_name(issuer)
32        .public_key(private_key.public_key())
33        .serial_number(x509.random_serial_number())
34        .not_valid_before(valid_from)
35        .not_valid_after(valid_to)
36        .add_extension(x509.BasicConstraints(ca=True,
37            path_length=None), critical=True)
38    )
39
40    # Sign the certificate with the private key
41    public_key = builder.sign(
42        private_key, hashes.SHA256(), default_backend()
43    )
44
45    with open(filename, "wb") as certfile:
46        certfile.write(public_key.public_bytes(serialization.Encoding.PEM))
47
48    return public_key

Here you have a new function generate_public_key() that will generate a self-signed public key. Here’s how this code works:

• Lines 2 to 5 are imports required for the function to work.
• Lines 8 to 18 build up information about the subject of the certificate.
• Line 21 uses the same issuer and subject since this is a self-signed certificate.
• Lines 24 to 25 indicate the time range during which this public key is valid. In this case, it’s 30 days.
• Lines 28 to 38 add all required information to a public key builder object, which then needs to be signed.
• Lines 41 to 43 sign the public key with the private key.
• Lines 45 to 46 write the public key out to filename.

Using these two functions, you can generate your private and public key pair quite quickly in Python:

>>> from pki_helpers import generate_private_key, generate_public_key
>>> private_key = generate_private_key("ca-private-key.pem", "secret_password")
>>> private_key
<cryptography.hazmat.backends.openssl.rsa._RSAPrivateKey object at 0x7ffbb292bf90>
>>> generate_public_key(
...   private_key,
...   filename="ca-public-key.pem",
...   country="US",
...   state="Maryland",
...   locality="Baltimore",
...   org="My CA Company",
...   hostname="my-ca.com",
... )
<Certificate(subject=<Name(C=US,ST=Maryland,L=Baltimore,O=My CA Company,CN=logan-ca.com)>, ...)>

After importing your helper functions from pki_helpers, you first generate your private key and save it to the file ca-private-key.pem. You then pass that private key into generate_public_key() to generate your public key. In your directory you should now have two files:

$ ls ca*
ca-private-key.pem ca-public-key.pem

Congratulations! You now have the ability to be a Certificate Authority.

Trusting Your Server

The first step to your server becoming trusted is for you to generate a Certificate Signing Request (CSR). In the real world, the CSR would be sent to an actual Certificate Authority like Verisign or Let’s Encrypt. In this example, you’ll use the CA you just created.

Paste the code for generating a CSR into the pki_helpers.py file from above:

 1# pki_helpers.py
 2def generate_csr(private_key, filename, **kwargs):
 3    subject = x509.Name(
 4        [
 5            x509.NameAttribute(NameOID.COUNTRY_NAME, kwargs["country"]),
 6            x509.NameAttribute(
 7                NameOID.STATE_OR_PROVINCE_NAME, kwargs["state"]
 8            ),
 9            x509.NameAttribute(NameOID.LOCALITY_NAME, kwargs["locality"]),
10            x509.NameAttribute(NameOID.ORGANIZATION_NAME, kwargs["org"]),
11            x509.NameAttribute(NameOID.COMMON_NAME, kwargs["hostname"]),
12        ]
13    )
14
15    # Generate any alternative dns names
16    alt_names = []
17    for name in kwargs.get("alt_names", []):
18        alt_names.append(x509.DNSName(name))
19    san = x509.SubjectAlternativeName(alt_names)
20
21    builder = (
22        x509.CertificateSigningRequestBuilder()
23        .subject_name(subject)
24        .add_extension(san, critical=False)
25    )
26
27    csr = builder.sign(private_key, hashes.SHA256(), default_backend())
28
29    with open(filename, "wb") as csrfile:
30        csrfile.write(csr.public_bytes(serialization.Encoding.PEM))
31
32    return csr

For the most part this code is identical to how you generated your original public key. The main differences are outlined below:

• Lines 16 to 19 set up alternate DNS names, which will be valid for your certificate.
• Lines 21 to 25 generate a different builder object, but the same fundamental principle applies as before. You’re building all the required attributes for your CSR.
• Line 27 signs your CSR with a private key.
• Lines 29 to 30 write your CSR to disk in PEM format.

You’ll notice that, in order to create a CSR, you’ll need a private key first. Luckily, you can use the same generate_private_key() from when you created your CA’s private key. Using the above function and the previous methods defined, you can do the following:

>>> from pki_helpers import generate_csr, generate_private_key
>>> server_private_key = generate_private_key(
...   "server-private-key.pem", "serverpassword"
... )
>>> server_private_key
<cryptography.hazmat.backends.openssl.rsa._RSAPrivateKey object at 0x7f6adafa3050>
>>> generate_csr(
...   server_private_key,
...   filename="server-csr.pem",
...   country="US",
...   state="Maryland",
...   locality="Baltimore",
...   org="My Company",
...   alt_names=["localhost"],
...   hostname="my-site.com",
... )
<cryptography.hazmat.backends.openssl.x509._CertificateSigningRequest object at 0x7f6ad5372210>

After you run these steps in a console, you should end up with two new files:

1. server-private-key.pem: your server’s private key
2. server-csr.pem: your server’s CSR

You can view your new CSR and private key from the console:

$ ls server*.pem
server-csr.pem  server-private-key.pem

With these two documents in hand, you can now begin the process of signing your keys. Typically, lots of verification would happen in this step. In the real world, the CA would make sure that you owned my-site.com and ask you to prove it in various ways.

Since you are the CA in this case, you can forego that headache and create your very own verified public key. To do that, you’ll add another function to your pki_helpers.py file:

 1# pki_helpers.py
 2def sign_csr(csr, ca_public_key, ca_private_key, new_filename):
 3    valid_from = datetime.utcnow()
 4    valid_until = valid_from + timedelta(days=30)
 5
 6    builder = (
 7        x509.CertificateBuilder()
 8        .subject_name(csr.subject)
 9        .issuer_name(ca_public_key.subject)
10        .public_key(csr.public_key())
11        .serial_number(x509.random_serial_number())
12        .not_valid_before(valid_from)
13        .not_valid_after(valid_until)
14    )
15
16    for extension in csr.extensions:
17        builder = builder.add_extension(extension.value, extension.critical)
18
19    public_key = builder.sign(
20        private_key=ca_private_key,
21        algorithm=hashes.SHA256(),
22        backend=default_backend(),
23    )
24
25    with open(new_filename, "wb") as keyfile:
26        keyfile.write(public_key.public_bytes(serialization.Encoding.PEM))

This code looks very similar to generate_public_key() from the generate_ca.py file. In fact, they’re nearly identical. The major differences are as follows:

• Lines 8 to 9 base the subject name on the CSR, while the issuer is based on the Certificate Authority.
• Line 10 gets the public key from the CSR this time. The line in generate_public_key() that specifies this is a CA, at the end of the builder definition, has been removed.
• Lines 16 to 17 copy any extensions that were set on the CSR.
• Line 20 signs the public key with the CA’s private key.

The next step is to fire up the Python console and use sign_csr(). You’ll need to load your CSR and your CA’s private and public key. Begin by loading your CSR:

>>> from cryptography import x509
>>> from cryptography.hazmat.backends import default_backend
>>> csr_file = open("server-csr.pem", "rb")
>>> csr = x509.load_pem_x509_csr(csr_file.read(), default_backend())
>>> csr
<cryptography.hazmat.backends.openssl.x509._CertificateSigningRequest object at 0x7f68ae289150>

In this section of code, you’re opening up your server-csr.pem file and using x509.load_pem_x509_csr() to create your csr object. Next up, you’ll need to load your CA’s public key:

>>> ca_public_key_file = open("ca-public-key.pem", "rb")
>>> ca_public_key = x509.load_pem_x509_certificate(
...   ca_public_key_file.read(), default_backend()
... )
>>> ca_public_key
<Certificate(subject=<Name(C=US,ST=Maryland,L=Baltimore,O=My CA Company,CN=logan-ca.com)>, ...)>

Once again, you’ve created a ca_public_key object which can be used by sign_csr(). The x509 module had the handy load_pem_x509_certificate() to help. The final step is to load your CA’s private key:

>>> from getpass import getpass
>>> from cryptography.hazmat.primitives import serialization
>>> ca_private_key_file = open("ca-private-key.pem", "rb")
>>> ca_private_key = serialization.load_pem_private_key(
...   ca_private_key_file.read(),
...   getpass().encode("utf-8"),
...   default_backend(),
... )
Password:
>>> private_key
<cryptography.hazmat.backends.openssl.rsa._RSAPrivateKey object at 0x7f68a85ade50>

This code will load up your private key. Recall from earlier that your private key was encrypted using the password you specified. With these three components, you can now sign your CSR and generate a verified public key:

>>> from pki_helpers import sign_csr
>>> sign_csr(csr, ca_public_key, ca_private_key, "server-public-key.pem")

After running this, you should have three server key files in your directory:

$ ls server*.pem
server-csr.pem  server-private-key.pem  server-public-key.pem

Whew! That was quite a lot of work. The good news is that now that you have your private and public key pair, you don’t have to change any server code to start using it.

Using your original server.py file, run the following command to start your brand new Python HTTPS application:

$ uwsgi \
    --master \
    --https localhost:5683,\
            logan-site.com-public-key.pem,\
            logan-site.com-private-key.pem \
    --mount /=server:app

Congratulations! You now have a Python HTTPS-enabled server running with your very own private-public key pair, which was signed by your very own Certificate Authority!

Now, all that’s left to do is query your server. First, you’ll need to make some changes to the client.py code:

# client.py
import os
import requests

def get_secret_message():
    response = requests.get("https://localhost:5683")
    print(f"The secret message is {response.text}")

if __name__ == "__main__":
    get_secret_message()

The only change from the previous code is from http to https. If you try to run this code, then you’ll be met with an error:

$ python client.py
...
requests.exceptions.SSLError: \
    HTTPSConnectionPool(host='localhost', port=5683): \
    Max retries exceeded with url: / (Caused by \
    SSLError(SSLCertVerificationError(1, \
    '[SSL: CERTIFICATE_VERIFY_FAILED] \
    certificate verify failed: unable to get local issuer \
    certificate (_ssl.c:1076)')))

That’s quite the nasty error message! The important part here is the message certificate verify failed: unable to get local issuer. These words should be more familiar to you now. Essentially, it’s saying the following:

localhost:5683 gave me a certificate. I checked the issuer of the certificate it gave me, and according to all the Certificate Authorities I know about, that issuer is not one of them.

If you attempt to navigate to your website with your browser, then you’ll get a similar message:

If you want to avoid this message, then you have to tell requests about your Certificate Authority! All you need to do is point requests at the ca-public-key.pem file that you generated earlier:

# client.py
def get_secret_message():
    response = requests.get("http://localhost:5683", verify="ca-public-key.pem")
    print(f"The secret message is {response.text}")

After doing that, you should be able to run the following successfully:

$ python client.py
The secret message is fluffy tail

Nice! You’ve made a fully-functioning Python HTTPS server and queried it successfully. You and the Secret Squirrels now have messages that you can trade back and forth happily and securely!