Demonstrating SMTP

00:00 In the previous lesson, I gave an overview of the course. In this lesson, I’ll show you SMTP, which is responsible for sending email. Doing the whole email thing is actually broken down into two separate parts.

00:12 The first is SMTP, which stands for Simple Mail Transfer Protocol. This is the protocol you use to send an email to somebody. The second part is retrieving your mail.

00:24 This is separate because the original mechanism had you logging into your server and fetching the mail locally. Modern clients do this remotely for you, connecting to servers to fetch the content.

00:34 The two most common protocols for remotely fetching your email are IMAP and POP. There were several versions of POP, so you’ll likely see the latter referred to as POP3. This course focuses on the first of those two parts, sending the email.

00:50 SMTP is actually one of the oldest protocols on the internet. It was made official only a few years after the internet was invented, and the predecessor it is based upon is even older than that.

01:01 A result of this age is that it’s a little creaky. It wasn’t originally designed with security in mind, so the core mechanic is overly simple. The protocol is text-based.

01:12 You log into the server and issue a few commands to send an email. The core commands you need are HELO, with one L, which says which server you’re connecting to, MAIL FROM, to say who is sending the note, RCPT TO, to say who is getting it, and then the DATA value to input the actual message. You’ll note there is nothing in there about authentication. Like I said, very old school back when the internet was a few dozen machines and everybody trusted everybody.

01:41 To send an email, you connect to the server and specify who is getting your note to send the message. In its original form, you connected to the server that had the person’s inbox you were trying to address.

01:52 Now mail servers are also relays, so you can connect to your own server and it will pass the note along to the appropriate place based on the email address.

02:01 Lots has been done over the years to shore up the protocol. Two key changes are authentication has been added, and most servers also run over SSL so that the connections are more secure and can’t be snooped upon. The end result of this is that running your own server is a lot more complicated than it used to be. You can still do it, but if you don’t configure it right, other servers won’t listen to you and will assume everything you’re sending is spam.

02:25 Of course, you don’t need your own server. You can use Google’s or several other providers, and I’ll show you how to do that later.

02:33 Using Google or whomever is what you’ll really want to do when you need to send an email. But to see what’s going on in the guts, having a local server to test with makes sense.

02:43 I’ll be using aiosmtpd to show you the ins and outs. It’s available as a third-party Python package, which you use with pip install or whatever your favorite tool is.

02:55 As always, the best practice when installing third-party packages is to use a virtual environment. Okay, if you’ve got that going, I’m going to head to the terminal and demonstrate SMTP. I have split my terminal in two.

03:09 On the bottom, I’m going to run the aiosmtpd server. You do that by invoking it as a module.

03:19 I’ve given it two arguments. -n is the really important one. aiosmtpd is a real email server. I’m just using it as a test, though. If you were going to run it in actuality, you’d want to connect as the nobody account.

03:34 So if someone breaks in through a flaw on the server, they wouldn’t have access to all of your files. The -n here says the nobody account isn’t necessary, which avoids you needing to be an administrator to run the program.

03:46 The second argument is -d, which stands for debug. I’ll be leaving that off later, but for now, I want a bit of extra info as I go along so you can see what’s happening.

03:57 You can actually add multiple -ds to get even more debug information. I’m going to stick with one, though. The first bit of debug is here on the screen, and it tells you that the server is running and listening on localhost port 8025.

04:11 The real port for SMTP is 25. Ports under 1024 in Unix are restricted to root. So using 8025 allows this program to run as anybody. There are other flags to set the port if you want, but 8025 is good for playing around. Now in my other terminal, I’m going to start a client to talk to our server.

04:34 The nc command comes with macOS and is available on other Unix machines. I used to use telnet, but Apple decided to stop shipping it as it isn’t the most secure thing on the planet.

04:44 Using nc with a hostname and port does the same as telnet, so I’m not sure how that fixed the problem.

04:51 Anyhow, when I ran the command in the lower server window, you see that a connection got created.

04:58 The 50890 is the port being used to talk back to the client. In the top window, you see that the data came back, which tells you you’re talking to the Python SMTP server version 1.4.6.

05:11 Now that I’m connected, I’m going to type my first command.

05:15 It’s only polite to say hello.

05:18 Don’t ask me why they only used one L. Bits were expensive in the old days and lots of stuff got abbreviated. The argument to HELO is the name of the server I’m connecting to, example.com in this case.

05:29 A mail server might be responsible for multiple domains, so this is being specific about which one I’m connecting to. In the server window, you’ll notice the debug that says it got the binary content which I typed.

05:42 Let’s start sending an email.

05:48 The MAIL FROM command says who I am. Evidently I’m Bob. I think my mother might be surprised by that.

05:59 The RCPT TO command indicates who I want to send the mail to. Sticking with tradition, Bob is sending mail to Alice. In olden times, before mail relays, Alice would have had to have been on that same machine as I was logging into.

06:14 Now, that doesn’t matter. Your server will send it along for you. For Alice, I’m using an example.org address instead of a .com to indicate that you don’t have to belong to the same place for this to work.

06:27 The DATA command is what starts the message. The response to the DATA command tells you how to indicate that you’re done sending data.

06:35 CR means carriage return and LF means line feed. That’s hitting Enter for the most part. I’ll get back to that in a second. Now, let me enter my message.

06:53 A few things to note here. First, I started with a mail header, the subject line. You can tell it’s a header because it starts with a word and a colon. The headers are optional and there are a bunch of them.

07:05 The subject header is how the recipient distinguishes the subject line from the body of the message. After the header comes the body, my greeting to Alice in this case.

07:14 Then at the bottom there, you’ll notice the period and the Control M. That’s a side effect of the terminal I’m using. Control M is the carriage return line feed character.

07:25 Depending on your terminal, hitting Enter after the dot may be sufficient or it may only be sending a carriage return. Here, I had to be explicit to indicate that I was done sending data.

07:36 For the sake of explanation, I paused the screen. Now, let me unpause it. On the server, you see the message that is being sent. Notice that the server added a header, the X-Peer, which indicates where this came from.

07:49 In the client window, the response 250 OK indicates that everything is good and the message was received by the server.

07:57 And that’s it. Doing this in the real world is a little more complicated, but not by much. Every time you have sent an email in your life, this is what has been going on behind the scenes.

08:09 Typing all that out is exhausting. Let’s build some software to do it for us.