Writing a Simple Server
Here are resources for more information about tools covered in this lesson:
00:00 In the previous lesson, I introduced you to HTTPS. In this lesson, I’m going to show you how to build a simple server inside of Flask that tries to hide its content from the spying public.
00:13 Here’s the scenario. You want to put a secret message up on the web, you want members of your club able to view the messages, and you want to make sure that no one else can.
00:24
I’m going to start out by showing you how to build a simple web application using Flask. To do this, you’re going to want to do pip install flask, preferably in a clean virtual environment.
00:36
This is a simple Flask application. The key to it is line 8 and 9. The decorator, called @app.route, registers the "/" path with this function.
00:48
When you hit the Flask server and use / as your path, this method gets called. Whatever’s returned from this method is sent back as the HTTP response. In this case, I’ll be taking the message defined in the global variable message and returning it. Flask has a built-in development server. Generally, you don’t want to use this in production. If you’re just testing something out, you can call app.run(), and you have a web server.
01:17 You’ll recall that the web by default is on port 80. Port 80 is a restricted port. In fact, in most operating systems, any number under 1024 is restricted and you have to have superuser privileges to run on that port. Flask, like most development engines, use a different port for its development server.
01:37
Because I’m trying to hide this message, I’ve changed the default to a different port number, port 5684. So unless the members of our club know to hit this URL on this port, they’re not going to be able to get the message. In the lower window, I’m going to run the application.
01:55
The Flask development server outputs some information to you, reminds you This is a development server—not to use it in production—and tells you that it’s listening to localhost, IP address 127.0.0.1 on port 5684.
02:11
Now, I’m going to use curl to hit that server.
02:17
What comes back is the message shhhh, this is a secret.
02:22
Although I thought I was being very clever by hiding the web server on a nonstandard port, this isn’t real security. In fact, it’s very easy to figure out what processes are listening to what ports on a machine. If you’ve got a Unix-based operating system, such as Linux or Mac OS, or you’re running the Windows Subsystem for Linux on a Windows application, you’ll have access to the lsof command.
02:47
The lsof command will tell you all sorts of information, including what services are connected to what ports, and what ports are being used. If you’re on a pure Windows installation, the netstat command will give you similar sorts of information. lsof and netstat only look at the processes local to the machine, but even if you trust the people who are on your machine, there are other ways to find out what’s running on your box.
03:12
In order for your web server to be available to your friends, it has to be public. This means a port scanner, such as nmap, could find your process across the internet.
03:23
nmap is included in some Linux packages, but is not included with Mac or Windows. In order to use this tool, you’ll need to visit nmap.org and install a binary from there.
03:34
Let me show you how easy it is to find your processes. I’m going to restart the server. Flask is running on localhost port 5684. And in the upper window, I’m going to run the lsof command. lsof stands for list open files. On Unix machines, everything is treated as a file, including sockets.
03:57
The -i parameter here tells the lsof command I’m only interested in those sockets on localhost and not other open files.
04:07
Here are the results. You can see the processes owned by my username, that it’s a Python command, and then it’s listening via TCP on localhost, port 5684.
04:19 Well, there you have it. My secret’s already out.
04:23
This can also be detected through a port scanner on another machine. nmap is such a port scanner. The -A tells nmap to give back some information about the operating system that it discovers on the other side. -p says that I’m only interested in scanning certain ports. In this case, I want to scan for TCP content between ports 5600 and 5700.
04:46
You can scan all ports without a problem—it just takes a while, so I’ve restricted the window here so it doesn’t take as long. And finally, you give it an IP address—in this case, localhost. nmap starts up.
05:00
The key here is it finds port 5684 running HTTP. It’s the Werkzeug httpd (Werkzeug HTTP daemon), which is what Flask runs. It even tells you what version of Python I’m operating. In the lower window, you can see the effects of the port scanner hitting the Flask server.
05:19 There are a whole bunch of requests logged.
05:23
nmap, in order to try and figure out what kind of server’s being run and what information is available, is asking for all sorts of content.
05:33
Most of these are returning 404 because the server doesn’t have them there.
05:47
By looking at this content, nmap can tell you all sorts of information about the server that’s running on the other end. So, running it on port 5684 isn’t really a secret, and isn’t secure.
06:01 If someone ran a port scanner on your box, they’d be able to see that you are listening to 5684. Once they saw that it was a web server, they could then use other tools to look at the content. Network tools such as packet analyzers, allow you to examine the entire exchange between the browser and the server.
06:18 A popular open-source packet analyzer is called Wireshark. It’s available here. Binary packages are available for a variety of operating systems. Linux, Windows, and Mac are all there. Let me show you what you can do with Wireshark.
06:33
This is the first screen on Wireshark. What you’re seeing here is a listing of all the network interfaces on my machine. The Flask server is running on localhost, which is the loopback instance. So, I can select Loopback, and then I can enter a capture filter.
06:50
The capture filter tells Wireshark to capture information only from certain areas—in this case, port 5684.
07:01
This filters content, so you’re only getting a small amount of data at a time—the data that you’re interested in inspecting. When I push Enter, it’ll start capturing what’s on the loopback interface, i.e. localhost on port 5684.
07:18
In a different window on my machine, I’m running Flask. And in another window, I’m about to hit that server with curl. All of this information that shows up in Wireshark is what happened when curl hit the Flask web server on port 5684.
07:34 A lot of information here is stuff that you’re not interested in, so you can filter it by saying you only want HTTP content protocol information in the display. I’ve hit Enter, and now you can see the two pieces of HTTP.
07:50
The first is the GET from curl and the second is the response from the web server.
07:58
Starting with the GET, in the middle window, you can see all the pieces that are put together. Networks can be thought of as a stack of operations.
08:08 This listing here, are the different levels of the stack. In the top layer of the stack is the HTTP protocol. Below that is TCP, below that is IP, and below that is the hardware information.
08:23
By opening up the HTTP content I can see what was sent to the server. GET / using HTTP version 1.1 the Host that was being hit, the User-Agent string, the Accept contents, and then some information.
08:41
In the bottom window, you can see all of the content that was sent up. This is a hex dump of the same information showing up in the middle window. It’s a little hard to read because it’s all jumbled together, but as you can see, you’ve got the GET, the Host, and the agent information, and finally, the Accept header. So, that’s the request.
09:02
How about the response? By clicking on the corresponding response on the top, it changes over. Once again in the middle, you see the different areas of the stack. This time, HTTP response 200 came back, the Content-Type, its length, and the information that came back.
09:20 It shows that there were 22 bytes worth of content that came back from the server. The next layer down can actually show you what was sent. Uh-oh. Our secret isn’t very secret, is it? Once again, inside of the hex space, you can see this broken down byte by byte.
09:39 As you can see, if somebody’s got something like Wireshark running, you are able to see all the contents of what’s happening on the network. An obscure port number is not enough to hide your content from anyone.
09:51 The secret message is actually publicly exposed. When you’re done capturing stuff from Wireshark, you hit the stop button. You now have the option of saving that information to the disc, so that if you wish to inspect it later, you can.
10:05 I think I firmly established that an obscure port number is not enough to keep something secret. In the next lesson, I’ll talk about cryptography and how you can use that to help actually keep your secrets secret.
Christopher Trudeau RP Team on Aug. 16, 2024
Hi alphafox,
I’d need a bit more info to be able to help you out. As you’re commenting on this lesson specifically, I believe you’re having trouble with the clear text version of the Flask server.
Before even trying to connect to it, does the server itself run in your VSCode terminal? Are you running it in the terminal or are you attempting to use the “play” button? Do you see the server start up and show the message like in the video telling you it is listening on localhost port 5684?
If all that is working, what happens when you try to use curl to connect to the server? Are you seeing an error message, or nothing at all in response?
What operating system are you running? Which version of Python?
You mention adding OpenSSH, but that doesn’t come along until a later lesson, is it that server you’re having trouble with?
alphafox28js on Aug. 17, 2024
Good morning sir!
Here are the steps I have had to take thus far to resolve the issue at hand: www.vulongtran.com/how-to-use-visual-studio-to-run-curl-commands
Must have Flask running, keep terminal open, open curl, search for language model active and select Http, keep curl open, and use for testing.
Now I am working on getting the SECRET_KEY = os.environ[b"SECRET_KEY"]
my_cipher = Fernet(SECRET_KEY) for the os.evironb[b""] is not wanting to work. VSCode can be frustating at times.
alphafox28js on Aug. 17, 2024
I would also like to add as a side-note, that I really enjoy the history behind the lessons. We are unable to move forward if we do not understand where it is we are coming from. ;)
alphafox28js on Aug. 18, 2024
If i may ask; How do you have wireshark setup, I am seeing an “Adapter for loopback traffic capture”, but no Loopback as what you have depicted in your screenshare.
Inside VSCode, this is returned. Ironically, not an issue outside of VSCode.
$ nmap -A -p T:5600-5700 localhost
bash: nmap: command not found
I am also unable to view the traffic via wireshark while running. If you could provide some additional guidance possibly, it would be helpful.
Christopher Trudeau RP Team on Aug. 18, 2024
Hi Alphafox28js,
I’m not a Windows guy and only occasionally use VS Code when giving demos, I think you’ve passed the point where I can help.
Both curl and nmap are commands that can be installed on other operating systems, but come from a Unix world and I suspect you’ll have a lot less trouble if you head in that direction.
VS Code is a great editor, but it’s terminal capabilities are quite limited.
If you want to play in this space, I’d highly suggest one of the following: 1) install the Windows Linux Subsystem. This is a fully working version of Linux that operates inside a window in Windows. You’ll be far less likely to run into problems with tools like curl, nmap, and OpenSSH in that environment.
2) Install Cygwin. This is a port of most Unix commands into Windows. You get a shell that runs a port of bash and you can install Unix commands into it.
3) I haven’t tried it myself, but git-bash is a Cygwin like thing, same idea, you have a shell that can run in Windows with the Unix commands you need.
Of the three, if you’re game, I’d go with the Windows Linux Subsystem. It is a little heaftier than the other two and has a bit of a learning curve, but once you’ve played with it you’ll be comfortable in both Windows and Unix-like systems which can be valuable to your programming career.
Good luck with it. …ct
Christopher Trudeau RP Team on Aug. 18, 2024
Hi Alphafox28js,
Sorry, lost track of the other question: Wireshark. Again, this is something that is built on Unix systems and can occasionally get challenging in a Windows environment. There is a chance your built-in firewall is getting in your way.
Take a look at this tutorial:
www.lifewire.com/wireshark-tutorial-4143298
and see if it helps. Sorry I can’t be more specific. Because this code is dealing with lower-level parts of your OS, the OS itself can get picky.
alphafox28js on Aug. 18, 2024
Hi Christopher,
I appreciate the insight into my questions very much.
I may consider making that jump to Windows Linux System as that is an option listed in VSCode Terminal(WLS), well, at least when I get this self-project I am working on completed that is.
If I understand correctly, it is to be assumed that for IoT programs/web-type app builds, you are stating that Linux/Unix has superiority over Windows for such interactions, whereas for programming and functionality, tasks, block-code, etc, Windows VScode will suffice.
Christopher Trudeau RP Team on Aug. 19, 2024
Hi alphafox28js,
OS choice can sometimes border on religion, so take everything I say with a grain of salt. Windows was originally just a desktop OS, and although it now has all sorts of features it comes from a desktop first design philosophy. Configuring it is usually done through the GUI, and although the introduction of the PowerShell has made it more scriptable and administrative friendly, it is what it is.
Unix has been around since the 70s and was the original OS for servers. GUI interfaces on it came second, and so configuration and the underlying philosophy have been server first. Unix systems have always been multi-user systems, so its design from the get-go worried about things like how your use of the processor effected my use of the processor on the same box. This has lead it to be a more secure OS because it was worried about program interactions from the beginning.
Although Windows does now do the server thing, Unix still rules in this environment. Even on Azure, Microsoft is running more copies of Linux than Windows.
If you are planning on working on servers at all, learning Linux or another Unix variant will be worth your while. You’re right in that IoT and web servers are definitely more likely to be running some sort of Unix.
In the case of this course, the tools being used are ones that are written for Unix and then ported to Windows. As such, you’ll likely find it easier to get them working in a Unix environment. None of the means you can’t use Windows, just that you’re more likely to have a bumpy road.
As for VS Code, I know very little about what its terminal can and can’t do. This isn’t VS Code specific, PyCharm the other popular IDE has similar restrictions, as does IDLE, Python’s built-in environment. A very simple example is the bell sound. This is an ASCII character and if you print it, the terminal should go “boop”. It doesn’t work in IDLE. Something they’ve done isn’t fully VT-100 terminal compatible. This likely extends to things like running TUIs. I wouldn’t be surprised if you have to run a TUI in PowerShell or even the command prompt to get it to work in Windows, the VS Code terminal likely doesn’t implement the entire ANSI code sequence set.
As the tools I was talking about were built for Unix systems, running WSL and using a bash shell isn’t going to have these kinds of problems.
Computers are simultaneously fun and frustrating. Happy coding!
Become a Member to join the conversation.
alphafox28js on Aug. 15, 2024
Hello, I am unable to reach the server from within vscode and python. Is there a work around for this? I added OpenSSH Server/Client into my System Options. Unsure of next steps as to if this is a config error in vscode or not