Introducing Pipes

00:00 In this lesson, you will get to grips with pipes. You can think of a pipe as some sort of a buffer that sits between two processes. So it buffers the output from one process and then feeds it as inputs into the next process.

00:16 Let me show you an example of that.

00:19 The example I’m about to show you is a Linux example, so it works on my machine because I am on Linux. So ls is a list. So it’s basically asking for all the files that are in the folder called usr/bin.

00:35 I’ll press Enter. You’re going to have to look really quickly because it goes really fast. There’s a whole bunch of files in that directory amongst which mugshot and muon as we will refer to later again. But the thing is, I didn’t want to actually necessarily see all these files.

00:52 What if I just want to see if there are any Python files in this directory? Now this is where a second command comes in. It’s called grep. grep stands for “global regular expression print”.

01:06 And the way it works is you say grep and then I’m going to look for the word python and then it’s going to look for that in some sort of file-like object.

01:18 Now, I don’t have a file-like object. Well, I don’t have a file, but what I want is I would like the output from the previous command, which is everything you can see on screen, plus everything else you cannot see on the screen, all the other files, I would like grep python to be applied to that output.

01:36 And that is of course, exactly what a pipe does. So what does that look like in Linux? You take the first command, ls usr/bin, and then you do the pipe symbol, and then grep python and that is it.

01:55 And you hit Enter and there you go. There are four files that have the word python in the list. That is the list of files of usr/bin.

02:06 So that’s the example. Let’s have a look at exactly what happened.

02:12 We had our first command ls usr/bin and we had an output from that file. There was a bunch of files in there. There were some files that had the word python there, and then there were files that called mugshot and muon as I referred to earlier, just as a reference point.

02:29 Because I have no idea what these files are, but that was it. That was our first command and its standard output. Now I have this second command grep python, which then is going to be applied to some sort of file-like object.

02:43 And that object is its stdin. Now, I want that to be the output of the previous command. So I’ve set up a pipe between the stdout of command one and the stdin of command two, and let’s say some sort of a memory buffer.

02:59 And when you then run the whole thing together, you get the output from the chain of commands if you like. And it gave me the four files with the word python in it that I was looking for.

03:11 Okay, so now that you fully understand what’s happening, let’s have a look at how this works in Python using subprocess. You’ll do that in the next lesson.